Object recommendation method, apparatus, device, medium, and computer program product

Abstract

The application discloses an object recommendation method and device, equipment, medium and computer program product, and relates to the technical field of Internet. The method comprises the following steps: performing cutout segmentation processing on a target object in a candidate image to obtain a first image corresponding to the target object, wherein the candidate image and the first image are images of a first style; converting the first image from the first style to a second style matched with the candidate image to obtain a second image, wherein the first style and the second style are two different image styles, and the second image is used as a recommended object material of an advertisement video; obtaining a background image, wherein the background image is used as a background material of the advertisement video; generating a dynamic display effect of the second image on the background image to obtain the advertisement video; and pushing the advertisement video to an advertisement medium, wherein the advertisement medium is used for recommending the target object by publishing the advertisement video. The method reduces the generation cost of the advertisement used for recommendation and improves the recommendation efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to an object recommendation method and device, equipment, medium and computer program product. BACKGROUND

[0002] When a manufacturer needs to promote a related product, the manufacturer often chooses to place an advertisement of the product through various channels to achieve a publicity effect.

[0003] In order to better promote the product in the young market, the manufacturer chooses to make an animation advertisement to promote the product. In the related art, the production process of the animation advertisement is generally manually made by a professional designer or an artist, that is, the product is designed as an animation image, and the corresponding product advertisement content is drawn frame by frame in the form of animation, so as to generate an animation video corresponding to the product.

[0004] However, the production process of the above-mentioned advertisement content consumes a large amount of time cost and human resource cost, thereby reducing the recommendation efficiency. SUMMARY

[0005] The object recommendation method, device, equipment, medium and computer program product provided in the embodiments of the present application can quickly generate stylized recommendation content, thereby improving the recommendation efficiency when recommending an object. The technical solution is as follows:

[0006] In one aspect, an object recommendation method is provided, and the method comprises:

[0007] performing cutout segmentation processing on a target object in a candidate image to obtain a first image corresponding to the target object, the candidate image and the first image being images of a first style;

[0008] converting the first image from the first style to a second style matched with the candidate image to obtain a second image, the first style and the second style being two different image styles, and the second image being used as a recommended object material of an advertisement video;

[0009] obtain a background image, the background image being used as a background material of the advertisement video;

[0010] generate a dynamic display effect of the second image on the background image to obtain the advertisement video;

[0011] push the advertisement video to an advertisement medium, the advertisement medium being used to recommend the target object by publishing the advertisement video.

[0012] In another aspect, an object recommendation device is provided, and the device comprises:

[0013] a segmentation module configured to perform matting segmentation processing on a target object in a candidate image to obtain a first image corresponding to the target object, the candidate image and the first image being images of a first style;

[0014] a first processing module configured to convert the first image from the first style to a second style matching the candidate image to obtain a second image, the first style and the second style being two different image styles, the second image being used as a recommended object material of an advertisement video;

[0015] an acquisition module configured to acquire a background image, the background image being used as a background material of the advertisement video;

[0016] a second processing module configured to generate a dynamic display effect of the second image on the background image to obtain the advertisement video;

[0017] a pushing module configured to push the advertisement video to an advertisement medium, the advertisement medium being configured to recommend the target object by publishing the advertisement video.

[0018] In another aspect, a computer device is provided, the terminal including a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the object recommendation method of any of the embodiments of the present application.

[0019] In another aspect, a computer readable storage medium is provided, the computer readable storage medium storing at least one program code, the program code being loaded and executed by a processor to implement the object recommendation method of any of the embodiments of the present application.

[0020] In another aspect, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the object recommendation method of any of the above embodiments.

[0021] The technical solutions provided by the present application at least include the following beneficial effects:

[0022] When a target object needs to be recommended, the target object obtained by matting is converted from a first style to a second style to obtain a second image in the second style, a dynamic display effect of the target object in the second style is generated in the background image to obtain an advertisement video of the target object in the second style, and the advertisement video is used for recommending the target object, thereby enriching the recommendation method when the target object is recommended, reducing the generation cost of the advertisement used for recommendation, and improving the recommendation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0025] Figure 2 is a flowchart of an object recommendation method provided by an exemplary embodiment of the present application;

[0026] Figure 3 is a schematic diagram of an advertisement video generation process provided by an exemplary embodiment of the present application;

[0027] Figure 4 is a flowchart of an object recommendation method provided by an exemplary embodiment of the present application;

[0028] Figure 5 is a schematic diagram of an image segmentation effect provided by an exemplary embodiment of the present application;

[0029] Figure 6 is a schematic diagram of a CartoonGAN network structure provided by an exemplary embodiment of the present application;

[0030] Figure 7 is a flowchart of an object recommendation method provided by an exemplary embodiment of the present application;

[0031] Figure 8 is a schematic diagram of style conversion of a target object provided by an exemplary embodiment of the present application;

[0032] Figure 9 is a schematic diagram of a model structure of a LaMa algorithm provided by an exemplary embodiment of the present application;

[0033] Figure 10 is a schematic diagram of an advertisement video generation process provided by an exemplary embodiment of the present application;

[0034] Figure 11 is a flowchart of an object recommendation method provided by an example embodiment of the present application;

[0035] Figure 12 is a structural block diagram of an object recommendation device provided by an example embodiment of the present application;

[0036] Figure 13 is a structural block diagram of an object recommendation device provided by an example embodiment of the present application;

[0037] Figure 14 is a structural diagram of a server provided by an example embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0039] First, the terms involved in the embodiments of the present application are briefly introduced:

[0040] Artificial intelligence: is to use digital computer or digital computer controlled machine simulation, extension and expansion of human intelligence, perception of environment, acquisition of knowledge and use of knowledge to obtain the best results of theory, method, technology and application system. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0041] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning, automatic driving, intelligent transportation and other several major directions.

[0042] Machine Learning (ML): is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory, etc. It is a specialized field that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure, and continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0043] Computer Vision (CV): Computer vision is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further process images to make them more suitable for human observation or transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and attempts to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, Optical Character Recognition (OCR), video processing, video semantic understanding, video content, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, map construction, etc.

[0044] For illustration, please refer to Figure 1 which shows a schematic diagram of an implementation environment provided by an example embodiment of the present application. The implementation environment includes a computer system including a terminal device, a server 120, and a communication network 130. The terminal device includes a first terminal 111 and a second terminal 112.

[0045] The terminal device includes a mobile phone, a tablet computer, a desktop computer, a portable notebook computer, a smart home appliance, a vehicle-mounted terminal, a smart billboard, and various forms of devices. The first terminal 111 is a terminal used by a content provider, and the first terminal 111 runs a first application. The content provider delivers a to-be-recommended content to an advertising medium through the first application. The second terminal 112 is a terminal used by a content receiver, and the second terminal 112 runs a second application. The content receiver receives a recommended content in the advertising medium through the second application. Optionally, the first application and the second application can be based on the same application of the advertising medium, or can be based on different applications of the advertising medium. Illustratively, the first application and the second application can be traditional application software, can be cloud application software, can be implemented as a small program or an application module in a host application program, or can be a certain web platform, which is not limited herein. Optionally, the first application and the second application can be at least one of an e-commerce application, a video application, a social application, a reading application, a search application, and an application capable of providing a content recommendation function.

[0046] The server 120 is configured to provide a backend service for the first application and / or the second application. Illustratively, the server 120 provides an object recommendation function for the first terminal 111. Illustratively, the first terminal 111 sends a candidate image containing a target object to the server 120, where the target object is an object that the content deliverer wants to promote, for example, the target object can be a to-be-sold commodity provided by the content deliverer. The server 120 performs matting on the target object in the candidate image to obtain a first image, converts the first image from a first style to a second style, and thus obtains a second image. Then, a background image is obtained, and a dynamic display effect of the second image on the background image is generated to obtain an advertising video. The advertising video is pushed to an advertising medium, and the advertising medium provides the advertising video to the second terminal 112 through the second application to recommend the target object to the content receiver.

[0047] It is worth noting that the server 120 can be a physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0048] The cloud technology refers to a hosting technology of unifying a series of resources such as hardware, software, network, etc. in a wide area network or a local area network to realize data calculation, storage, processing and sharing. The cloud technology is a general term of network technology, information technology, integration technology, management platform technology, application technology and the like applied based on a cloud computing business model, can form a resource pool, and is used on demand, flexibly and conveniently. The cloud computing technology will become an important support. The background service of a technical network system needs a large amount of calculation and storage resources, such as a video website, a picture website and more portal websites. With the high development and application of the Internet industry, in the future, each item is likely to have its own identification mark and needs to be transmitted to the background system for logical processing. Different levels of data will be processed separately, and the data of various industries all need strong system support, which can only be realized through cloud computing.

[0049] In some embodiments, the server 120 described above can also be implemented as a node in a blockchain system. The blockchain is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0050] Illustratively, the terminal devices (including the first terminal 111 and the second terminal 112) and the server 120 are connected through a communication network 130, where the communication network 130 can be a wired network or a wireless network, which is not limited herein.

[0051] Please refer to Figure 2 which shows an object recommendation method according to an embodiment of the present application. In the embodiment of the present application, the method is applied to a server as shown in Figure 1 The method comprises the following steps.

[0052] In step 201, the target object in the candidate image is subjected to a matting segmentation process to obtain a first image corresponding to the target object.

[0053] The candidate image and the first image are images of a first style.

[0054] Optionally, the candidate image can be uploaded by the content provider through the first terminal, or can be obtained from a database by the server.

[0055] Optionally, the target object can be an object that needs to be promoted by the content provider. For example, the target object can be a commodity to be promoted uploaded by a merchant; or the target object can be a scenic spot to be promoted uploaded by a merchant. Optionally, the target object can be an object in the content to be recommended uploaded by the content provider. For example, the target object can be a person or an object in a photo published by the content provider.

[0056] In some embodiments, the processing procedure of the above-mentioned matting segmentation processing of the candidate image is implemented by the server, that is, the server performs matting segmentation processing on the obtained candidate image to obtain the first image containing the target object. In other embodiments, the server directly receives the first image sent by the first terminal, that is, the matting processing procedure of the candidate image is implemented by the first terminal.

[0057] Illustratively, when the candidate image is subjected to matting segmentation processing by the server, the server identifies the target object in the candidate image, detects the edge of the target object in response to identifying the target object in the candidate image, and segments the target object from the candidate image according to the edge of the target object to obtain the first image.

[0058] In the embodiments of the present application, the image corresponds to an image style, and the same image content has different display effects under different image styles. Optionally, the image style can include a real style and a cartoon style, wherein the real style is used to indicate the style of the image content collected by the image collection device, and the cartoon style is used to indicate the style of the image content obtained by drawing by hand or drawing software.

[0059] In some embodiments, the above-mentioned cartoon style can be further divided into different sub-styles according to different drawing methods, for example, a pencil drawing style, an ink style, an oil painting style, a comic style, and a graffiti style.

[0060] In some embodiments, the first image corresponds to a first style, which can be a real style.

[0061] Step 202, converting the first image from the first style to a second style matching the candidate image to obtain a second image.

[0062] The first style and the second style are two different image styles, and the second image is used as a recommended object material of the advertisement video. In some embodiments, the second style can be a cartoon style, that is, the first image in the real style is subjected to style conversion to obtain the second image in the cartoon style. In other embodiments, the image in the cartoon style can also be converted into an image in the real style. Here, only the conversion from the real style to the cartoon style is illustratively described, and the specific style is not limited.

[0063] Optionally, when the first image is converted from the first style to the second style, at least one of the following style conversion methods can be used:

[0064] First, the first image is converted from the first style to the second style by a specified image style conversion algorithm.

[0065] Illustratively, the image style conversion algorithm is at least one of an image transformation algorithm, an image enhancement algorithm, a texture analysis algorithm, an image segmentation algorithm, an edge detection algorithm, and the like, and notably, the image style conversion algorithm is a non-neural network implemented image processing algorithm. In one example, the image style conversion process can be implemented as follows: performing bilateral filtering on the first image by specifying a filter, thereby implementing blurring processing on the first image while preserving edge features in the first image; performing edge detection on the image by a Sobel operator, and then performing enhancement processing on the detected edges; performing color saturation enhancement on the edge-enhanced image, thereby outputting a second image of a second style.

[0066] Secondly, converting the first image from the first style to the second style by a specified image style conversion network.

[0067] Illustratively, the image style conversion network is a neural network capable of providing conversion of an image from a first style to a second style, i.e., implementing style transfer of the first image based on a neural network.

[0068] Optionally, the image style conversion network can be at least one of a generative adversarial network (GAN), a visual geometry group network (VGGNet), a convolutional neural network (CNN), and the like.

[0069] Illustratively, the second style is an image style matching the candidate image. In some embodiments, since objects of some types have better display effects in cartoon style, for example, when the target object is an animal, a children's product, a stationery, and the like, i.e., the target object under these types has a greater interest tendency for cartoon style content when making recommendations, therefore, the second style can be determined according to the object type of the target object in the candidate image. That is, determining the object type corresponding to the target object in the candidate image, determining the second style matching the object type from at least two candidate styles, converting the first image from the first style to the second style to obtain the second image. In one example, when it is determined that the object type of the target object belongs to a youth-oriented type, then the comic style in the candidate style is determined as the second style, and when it is determined that the object type of the target object belongs to a child-oriented type, then the cartoon style in the candidate style is determined as the second style, and the first image is converted from the first style to the second style to obtain the second image.

[0070] In some embodiments, the second style matching the candidate image can be determined according to whether the color difference condition of the candidate image meets the conversion requirement of different candidate styles. That is, based on the pixel value difference between adjacent pixels in the candidate image, a color change parameter of the candidate image is determined, the color change parameter is used to indicate the contrast between pixels in the candidate image, the second style matching the color change parameter is determined from at least two candidate styles, the color change parameter of the candidate image is determined based on the contrast between adjacent pixels in the candidate image; the second style matching the color change parameter is determined from at least two candidate styles; the first image is converted from the first style to the second style to obtain a second image.

[0071] In one example, when the candidate styles include a painting style and a comic style, in response to the color change parameter reaching a specified threshold, the painting style is determined as the second style; in response to the color change parameter being lower than the specified threshold, the comic style is determined as the second style. That is, when the color change parameter of the candidate image indicates that the contrast of color change is high, it means that the candidate image has bright color change, and is therefore suitable for conversion to an optimized style with strong block contrast. When the color change parameter of the candidate image indicates that the contrast of color change is low, it means that the candidate image is composed of fewer colors and has weak contrast between color changes, and is therefore suitable for conversion to a comic style with black and white style.

[0072] Illustratively, when the color change parameter of the candidate image is determined by the contrast between adjacent pixels in the candidate image, the server calculates the contrast between the pixel and the adjacent pixels pixel by pixel, thereby obtaining the color change parameter that can represent the overall color change of the candidate image. In one example, for the ith pixel, the pixel values between the ith pixel and its surrounding eight pixels are respectively subtracted to obtain eight pixel difference values, each pixel difference value is compared with a preset pixel threshold, and if it is higher than the preset pixel threshold, the contrast data corresponding to the pixel is accumulated. For example, the pixel values corresponding to the ith pixel and its surrounding eight pixels are {0, 0, 0, 255, 255, 255, 255, 255, 255}, wherein the pixel value of the ith pixel located in the middle position is 255, and the difference values between the ith pixel and other pixels are 255, 255, 255, 0, 0, 0, 0, 0, 0, respectively. The preset pixel threshold is 150, and the contrast data corresponding to the ith pixel is 3 obtained by accumulation. After determining the contrast data corresponding to each pixel in the candidate image, the color change parameter corresponding to the candidate image can be obtained by accumulation or averaging.

[0073] In another example, to reduce the amount of calculation in the color change parameter determination process, the candidate image can be divided into image blocks in a preset block form, the pixel values between different image blocks are compared by taking the pixel mean value of the pixels in the image block as the pixel value representing the pixel block, to obtain the color change parameter.

[0074] In another example, to improve the determination efficiency of the color change parameter, an image color recognition network trained in advance is used to identify the color features of the candidate image, to output the color change parameter. Optionally, the image color recognition network can be at least one of a CNN, a densely connected network (DenseNet), a VGGNet, etc. Illustratively, the image color recognition network extracts color features of different regions in the candidate image, to obtain color feature representations corresponding to the different regions, determines the color difference between the different regions according to the distance between the color feature representations, and finally outputs a color change parameter that can indicate the entire candidate image.

[0075] Step 203: Obtain a background image.

[0076] The background image is used as background material for the advertisement video.

[0077] In some embodiments, the background image can be a background image corresponding to the target object itself. Illustratively, the first terminal uploads a candidate image to the server, the candidate image being a complete image containing the target object and background content, performs matting on the target object in the candidate image to obtain a first image, and supplements the image content of the remaining part after the target object is removed from the candidate image, to obtain the background image.

[0078] In some embodiments, the background image can also be an image determined by the server from a candidate background image, or a complete background image uploaded by the first terminal for use as a background.

[0079] In some other embodiments, the target object is not subjected to matting before being subjected to style conversion, i.e., the complete candidate image is subjected to style conversion to obtain a complete image in a second style, the target object in the complete image in the second style is subjected to matting, and the background part in the second style is subjected to image completion to obtain the background image.

[0080] Step 204: Generate a dynamic display effect of the second image on the background image, to obtain an advertisement video.

[0081] Illustratively, the dynamic display effect can be generated based on a dynamic display scheme, where the dynamic display scheme is used to indicate a generation manner of generating the advertisement video according to the second image and the background image. The dynamic display scheme indicates at least one dynamic display effect of the second image on the background image during the generation of the advertisement video.

[0082] Optionally, the dynamic display effect includes at least one visual effect of displacement change effect, zoom change effect, depth-of-field effect, rotation change effect, foreground occlusion effect, etc. In one example, the dynamic display scheme indicates that the generation of the advertisement video is realized by a position conversion of the target object in the background image.

[0083] Optionally, the dynamic display effect combination indicated by the dynamic display scheme can be pre-stored in the server. Illustratively, the server stores at least one candidate dynamic display scheme, and the dynamic display scheme used to generate the advertisement video is determined from the at least two candidate dynamic display schemes. Alternatively, the dynamic display effect combination indicated by the dynamic display scheme can be indicated by the first terminal, i.e., the first terminal indicates to the server which dynamic display effects need to be displayed during the generation of the advertisement video.

[0084] In some embodiments, to realize that the generated advertisement video has a three-dimensional visual effect, the dynamic display scheme indicates that the target object is moved from a first position to a second position in a target three-dimensional coordinate system according to a specified movement trajectory, to obtain at least two motion image frames corresponding to the target object in the movement process, where the first position and the second position are positions in the target three-dimensional coordinate system, and the motion image frame is an image obtained by mapping the target object to the background image according to the position of the target object in the target three-dimensional coordinate system; a special effect processing operation is performed on the at least two motion image frames to obtain at least two effect image frames, where the ith motion image frame corresponds to the ith effect image frame; and the at least two effect image frames are spliced to obtain the advertisement video.

[0085] Illustratively, the specified movement trajectory is based on the target three-dimensional coordinate system, i.e., the specified movement trajectory is a trajectory generated by the movement of the target object in the target three-dimensional coordinate system. Illustratively, the target three-dimensional coordinate system is a coordinate system having two coordinate axes parallel to the plane of the background image and one coordinate axis perpendicular to the plane of the background image.

[0086] When controlling a target object to move along a specified movement trajectory in a target three-dimensional coordinate system, the target object during the movement process is mapped onto the two-dimensional plane where the background image is located, so that a motion image frame of the target object during the movement process can be obtained. Schematically, when the movement trajectory of the target object in the target three-dimensional coordinate system includes a displacement on the coordinate system perpendicular to the plane where the background image is located, the performance when the effects before and after the movement are mapped onto the plane where the background image is located is that the size of the target object is scaled.

[0087] In one example, as Figure 3 shown, it shows a schematic diagram of the advertisement video generation process provided by an exemplary embodiment of the present application. In the target three-dimensional coordinate system 310, the x-axis and the y-axis are coordinate axes parallel to the plane where the background image 301 is located, and the z-axis is the coordinate axis perpendicular to the plane where the background image 301 is located. The target object 302 located at the first position moves to the second position according to the specified movement trajectory 303, and the server renders the position change situation mapped on the plane where the background image 301 is located according to the movement trajectory 303 as the advertisement video 330, that is, the advertisement video 330 is obtained through multiple mapped motion image frames 320.

[0088] Schematically, the above special effect processing operations include at least one of operations such as shadow processing, depth of field processing, scaling processing, rotation processing, foreground occlusion processing, etc.

[0089] Among them, shadow processing indicates adding a shadow effect to the target object in the motion image frame. In some embodiments, when the above specified movement trajectory includes the target object moving on the coordinate system perpendicular to the plane where the background image is located in the target three-dimensional coordinate system, that is, the visual effect that the target object moves from far to near (or from near to far) can be shown between the motion image frames. In the motion image frame, a changing shadow effect can be added to the target object. For example, when the movement process of the target object shows a visual effect from far to near, a shadow effect from none to existing or from light to deep can be added to the target object.

[0090] Depth of field processing indicates dividing the motion image frame into a foreground part and a background part according to the position of the target object in the motion image frame, and blurring the foreground part and / or the background part, so as to achieve the visual effect of depth of field. In some embodiments, the depth of field effect corresponding to the motion image frame can change with the change of the image frame. For example, there are n motion image frames, and the n motion image frames are arranged in chronological order, and the depth of field effect corresponding to the motion image frame deepens in chronological order. Optionally, the depth of field effect can also become shallower over time, or alternate between deep and shallow, which is not specifically limited herein.

[0091] The scaling processing indicates adjusting the size of the target object in the motion image frames; the rotation processing indicates rotating the target object in the motion image frames based on a specified rotation point or rotation axis.

[0092] The foreground occlusion processing indicates adjusting the occlusion relationship between the target object and other objects in the motion image frames, so as to achieve the visual effect of "broken window". Illustratively, a foreground object that exists occlusion to the target object in the motion image frames is identified, and the occlusion relationship between the target object and the foreground object is modified in at least one of the at least two motion image frames. For example, 20 motion image frames are intercepted, and a foreground object that exists occlusion to the target object is identified, and the occlusion of the foreground object to the target object is adjusted to the effect of the occlusion of the target object to the foreground object in the last 10 motion image frames.

[0093] In step 205, the advertisement video is pushed to the advertisement media.

[0094] The advertisement media is used to recommend the target object by publishing the advertisement video.

[0095] In some embodiments, the advertisement video generated by the candidate image can be a candidate recommendation item corresponding to the target object, that is, when the target object corresponding recommendation content is pushed to the second terminal, the advertisement video can be one of the candidate items of the recommendation content.

[0096] In some embodiments, the server generates at least two second style advertisement videos according to the candidate image, wherein the at least two second styles are different image styles, different advertisement videos can be obtained according to different second styles, and different second style advertisement videos are used as different candidate recommendation items for the recommendation of the target object. For example, a cartoon style advertisement video, an oil painting style advertisement video, a pencil drawing style advertisement video, etc. are generated respectively.

[0097] In some embodiments, the candidate image, the first image and the second image can also be used as candidate recommendation items of the target object.

[0098] In some embodiments, when the advertising media stores a plurality of candidate recommended items corresponding to the target object, the recommended content can be selected from the candidate recommended items according to the account information corresponding to the content receiving account in the second terminal for pushing. Illustratively, a recommended content acquisition request indicated by the content receiving account is received, the recommended content acquisition request indicates content recommendation to the content receiving account, and the recommended content acquisition request includes the account information of the content receiving account; the target object is determined from the candidate recommended objects based on the recommended content acquisition request; at least two candidate advertising videos corresponding to the target object are obtained, and the i th candidate advertising video corresponds to the i th second style; the style inclination degree of the content receiving account corresponding to the at least two candidate advertising videos is determined based on the account information; the candidate object with the highest style inclination degree in the at least two candidate advertising videos is determined as the target advertising video; and the target advertising video is pushed to the content receiving account.

[0099] In one example, the age information of the content receiving party is included in the account information, and the recommended content is determined according to the age range corresponding to the age information. Illustratively, the target age range to which the content receiving account belongs is determined according to the age information; and the style inclination degree corresponding to the target age range and the at least two second styles is obtained.

[0100] In another example, the pushing content is determined according to the historical operation data indicated by the account information. Illustratively, in response to the preference degree of the content receiving party to the content of the second style reaching a preset preference threshold indicated by the historical operation data, the advertising video of the second style is pushed to the second terminal. It is worth noting that before obtaining the account information of the content receiving party, the server needs to obtain sufficient authorization from the content receiving party through the second terminal and indicate the specific use of the obtained account information.

[0101] In some embodiments, in order to further strengthen the recommendation effect of the target object of the second style when recommending, after the second image obtained by style conversion of the target object, the second image can be used to generate an object pointer in addition to generating an advertising video, and the object pointer and the advertising video have an association relationship. When the advertising video is pushed to the second terminal, the second terminal displays a graphical interface for playing the advertising video, and in the graphical interface, the second terminal uses the object pointer as a pattern material for indicating the position of the mouse, that is, the second terminal replaces the original mouse pattern with the above-mentioned object pointer, thereby further strengthening the recommendation effect of the target object.

[0102] In summary, the object recommendation method provided in the embodiments of the present application, when the target object needs to be recommended, the target object obtained by matting is converted from the first style to the second style, so as to obtain the second image in the second style, and then a dynamic display effect of the target object in the second style in the background image is generated, so as to obtain the advertisement video of the target object in the second style. The advertisement video is used for recommending the target object, which enriches the recommendation method when the target object is recommended, reduces the generation cost of the advertisement for recommendation, and improves the recommendation efficiency.

[0103] Please refer to Figure 4 which shows the object recommendation method shown in an embodiment of the present application. In the embodiment of the present application, a generative adversarial network is generated to convert the first image from the first style to the second style, so as to obtain the second image. The method comprises:

[0104] Step 401, performing matting segmentation processing on the target object in the candidate image to obtain the first image corresponding to the target object.

[0105] In some embodiments, the first image described above can be an image obtained by preprocessing the candidate image uploaded by the content provider through the first terminal, wherein the preprocessing indicates that the target object is segmented from the candidate image, that is, only the target object is style converted. Illustratively, the preprocessing process can include segmenting the target object in the candidate image, thereby obtaining the first image. That is, the candidate image is obtained, which is the content provided by the object recommendation account for recommending the target object; in response to detecting the target object from the candidate image, the target object in the candidate image is segmented to obtain the first image.

[0106] Optionally, the target object can be an object in the candidate image, or a text object in the candidate image.

[0107] In one example, as Figure 5 shown, it shows a schematic diagram of image segmentation effect provided by an exemplary embodiment of the present application. The candidate image 510 includes not only the target commodity 511, but also the text content 512 for recommending the target commodity 511. The candidate image 510 is segmented to obtain the segmentation result 520, wherein the segmentation result 520 distinguishes the target commodity 511, the text content 512 and the remaining background part by different area display modes, so as to accurately crop the image content corresponding to the target commodity 511 from the candidate image 510.

[0108] Step 402, inputting the first image into the generative network to output the second image in the second style.

[0109] The generation network is configured to map the first image in the first style to the style curve of the second style to generate a second image, i.e., the second image is generated by mapping the first image in the first style to the style curve of the second style. Illustratively, the generation network maps a pixel value corresponding to a first pixel in the first image to the style curve of the second style to obtain a target pixel value, and generates the second image by taking the target pixel value as a second pixel, where the position of the second pixel in the second image corresponds to the position of the first pixel in the first image.

[0110] Illustratively, the generation network is trained by a style transfer network, which in some embodiments can be a generative adversarial network (GAN). The style transfer network includes a generator network to be trained and a discriminator network to be trained.

[0111] In some embodiments, the first style is a real style and the second style is a cartoon style. In this case, the style transfer network can be a generative adversarial network for photo cartoonization (CartoonGAN).

[0112] Illustratively, please refer to Figure 6Fig. 6 shows a schematic diagram of a network structure of the CartoonGAN 600 according to an example embodiment of the present application. In the generation network 610, the convolution stage 611 is first used, which includes a convolution layer (Conv), a normalization layer (Norm) and a linear rectification function (ReLU). The size of the convolution stage 611 is the same as that of the first input image 601, i.e., HxW. Then, the image feature data processed by the convolution stage 611 is compressed and encoded by two down-convolution blocks 612. The two down-convolution blocks 612 are used to extract the content features of the image for subsequent conversion. Each of the two down-convolution blocks 612 includes two Convs, one Norm and one ReLU, and the sizes of the two down-convolution blocks 612 are (H / 2)x(W / 2) and (H / 4)x(W / 4), respectively. Then, eight residual blocks 613 are used to construct the image content and manifold features. Each of the eight residual blocks 613 has the same layout, which is “ConvNormReLUConvNormES”. The ES indicates an element-wise sum, and the manifold feature indicates a curve in the Euclidean space. Finally, the features are reconstructed by two up-convolution blocks 614, and the first output image 602 is finally output by the output convolution layer 615. The two up-convolution blocks 614 have the same structure as the down-convolution blocks, and the sizes of the two up-convolution blocks 614 are (H / 2)x(W / 2) and HxW, respectively.

[0113] The CartoonGAN 600 further includes a discriminator network 620, which is complementary to the generation network 610 and is used to determine whether the input image is a real cartoon image. The discriminator network 620 relies on the local features of the image to determine the image style, and therefore can be designed as a shallow network. After the second input image 603 is input to the flat layer 621, the resolution is reduced by two stride convolution blocks 622, and the basic local features are encoded. Then, the classification response is obtained by using the feature construction block 623 and the output convolution layer 624, and the discrimination result 604 corresponding to the second input image 603 is output.

[0114] During the training process of the CartoonGAN, the discriminator network is trained to distinguish the images in the cartoon manifold from other images and provide an adversarial loss for the generation network to push the training of the generation network, so that the generation network can convert the images of the first style to the images of the second style. During the application of the CartoonGAN, only the generation network is used to generate the images of the second style.

[0115] Illustratively, in the training process of CartoonGAN, the loss function includes two parts, the first part is the adversarial loss, and the second part is the content loss. The weight between the two losses can be balanced by setting the weight relationship between the two losses. When the proportion of the content loss is larger, the output image of the second style can retain more content information from the first image. Conversely, the output image has less texture information and more obvious edge features.

[0116] In one example, a set of training images is obtained, the set of training images including first type training images and second type training images, the first type training images being images of a first style, and the second type training images being images of a second style; the first type training images are input into a to-be-trained generation network to obtain generated images; the first type training images and the second type training images are jointly input into a to-be-trained discrimination network to obtain a discrimination result; an adversarial loss value is determined based on the generated images and the discrimination result, and a content loss value is determined based on the generated images and the discrimination result; the to-be-trained generation network and the to-be-trained discrimination network are trained based on the adversarial loss value and the content loss value to obtain a trained generation network and a trained discrimination network, wherein the generation network is used to convert a first image from the first style to the second style to obtain a second image.

[0117] Specifically, the network parameters of the generation network and the discrimination network are initialized, n input images are randomly extracted from the first type training images and input into the generation network to generate n sample images, the generation network is fixed, and the discrimination network is trained based on the n sample images and the set of training images to enable the discrimination network to distinguish as much as possible whether the input images are real cartoon images (i.e., neither real images nor cartoon images generated from real images), the network parameters of the generation network are updated after the discrimination network is updated for k times, the updated generation network needs to enable the discrimination network to distinguish as little as possible between real cartoon images and cartoon images generated from real images, and after multiple update iterations, the discrimination network cannot distinguish whether the input cartoon image is the second type training image or the cartoon image generated by the generation network, i.e., the training is completed, and the obtained generation network is used for the generation process of the second image.

[0118] In some embodiments, the server can train different candidate networks according to different candidate styles to generate second images of different second styles. Different candidate networks correspond to different candidate style curves, i.e., a style curve corresponding to the second style is determined from at least two candidate style curves, wherein the i-th candidate style curve corresponds to the i-th candidate style.

[0119] Illustratively, the server receives a style conversion request from the first terminal before performing the style conversion on the first image, the style conversion request is used to request to convert the first image of the target object into an advertisement video, and the style conversion request includes a style identifier, the style identifier is used to indicate the second style, wherein the second style is an image style in the at least two candidate styles; the server determines the first style corresponding to the first image; in response to determining that the first style corresponding to the first image is different from the second style, the server determines the generation network corresponding to the second style from the at least two candidate networks, wherein the i-th candidate network corresponds to the i-th candidate style.

[0120] In some embodiments, to avoid waste of server resources, the server needs to determine whether the first image is suitable for style conversion processing before converting the first image. In one example, when the image style of the first image uploaded by the first terminal is the second style, the first image does not need to be converted in style, therefore, the server determines the image style of the first image through the filtering module, and in response to determining that the first style corresponding to the first image is an image style different from the second style, the server inputs the first image into the generation network.

[0121] In some embodiments, when the server provides image conversion services of multiple candidate styles, the style conversion request indicated by the first terminal can include a style identifier used to determine the second style from the multiple candidate styles, and the server can determine the generation network to be used for style conversion from the pre-trained candidate network according to the style identifier.

[0122] Step 403, obtaining a background image.

[0123] The background image is used as background material of the advertisement video.

[0124] In some embodiments, the background image can be a background image corresponding to the target object. Illustratively, the first terminal uploads a candidate image to the server, the candidate image is a complete image containing the target object and background content, the target object in the candidate image is extracted to obtain the first image, and the remaining image content after the target object is removed from the candidate image is supplemented to obtain the background image.

[0125] Step 404, generating a dynamic display effect of the second image on the background image to obtain the advertisement video.

[0126] In the embodiments of the present application, the generation of the advertisement video is realized by converting the position of the target object in the background image.

[0127] In some embodiments, to realize the generated advertisement video with a three-dimensional visual effect, a target object is moved from a first position to a second position in a target three-dimensional coordinate system according to a specified movement track, to obtain at least two motion image frames corresponding to the target object in the movement process, wherein the first position and the second position are positions in the target three-dimensional coordinate system, and the motion image frame is an image obtained by mapping the target object to a background image according to the position of the target object in the target three-dimensional coordinate system; the at least two motion image frames are subjected to special effect processing operations to obtain at least two effect image frames, wherein the ith motion image frame corresponds to the ith effect image frame; and the at least two effect image frames are spliced to obtain the advertisement video.

[0128] In step 405, the advertisement video is pushed to the advertisement medium.

[0129] The advertisement medium recommends the target object through the advertisement video.

[0130] The advertisement medium recommends the target object through the advertisement video.

[0131] In some embodiments, the advertisement video generated by the first image can be a candidate recommended item corresponding to the target object, that is, when the target object corresponding recommendation content is pushed to the second terminal, the advertisement video can be used as one of the candidate items of the recommendation content.

[0132] In some embodiments, the server generates a plurality of second style advertisement videos according to different generation networks, wherein the plurality of second styles are mutually different image styles, and different advertisement videos can be obtained according to different second styles, and the advertisement videos of different second styles are used as different candidate recommended items for the recommendation of the target object. For example, a cartoon style advertisement video, an oil painting style advertisement video, a pencil drawing style advertisement video, and the like are generated respectively.

[0133] In summary, the object recommendation method provided by the embodiments of the present application, when the target object needs to be recommended, the target object obtained by matting is converted from a first style to a second style to obtain a second image of the target object in the second style, and then a dynamic display effect is generated in a background image for the target object in the second style, to obtain an advertisement video of the target object in the second style, which is used for recommending the target object, thereby enriching the recommendation method when the target object is recommended, reducing the generation cost of the advertisement used for recommendation, and improving the recommendation efficiency.

[0134] In the embodiment of the present application, the neural network is used to convert the target object from the first style to the second style, thereby improving the style conversion efficiency of the target object during style conversion. Meanwhile, the neural network can be trained by specifying the training data of the style to obtain a generation network capable of generating different second styles, thereby enriching the diversity of the generated advertising videos.

[0135] Please refer to Figure 7 which shows an object recommendation method according to an embodiment of the present application. In the embodiment of the present application, the first image for style conversion is an image including only the target object after segmentation of the candidate image. After style conversion of the target object, background completion is needed to generate an advertising video. The method comprises the following steps:

[0136] Step 701: obtaining a candidate image.

[0137] Illustratively, the candidate image is an image uploaded by a first terminal. The first terminal indicates to perform style conversion on the target object in the candidate image and generate a corresponding advertising video, that is, the candidate image is content provided by an object recommendation account for recommendation of the target object. In some embodiments, the candidate image can be recommendation content used by a content provider when promoting the target object, that is, even if the candidate image is not processed, the candidate image can be directly applied to the recommendation process of the target object.

[0138] Step 702: segmenting the target object and the background part in the candidate image to obtain a first image corresponding to the target object and a third image corresponding to the background part.

[0139] In some embodiments, the detection process of the target object in the candidate image by the server can be realized by a neural network. For example, the server inputs the candidate image into an object detection network to identify at least one candidate object existing in the candidate image, and determines the target object from the at least one candidate object.

[0140] Alternatively, the server can determine the target object from the at least one candidate object according to the matching degree between the candidate object and a pre-stored standard object image. The standard object image can be an image pre-uploaded by the first terminal, that is, the first terminal sends the standard object image to the server, and then the first terminal can send a plurality of candidate images to the terminal. The server determines the target object from the candidate images according to the standard object image, thereby realizing batch processing of the candidate images and improving the processing efficiency of the plurality of candidate images of the target object. Alternatively, the server returns the identified candidate objects to the first terminal, and the content provider selects at least one target object from the candidate objects.

[0141] In some embodiments, the first terminal can frame the target object in the candidate image before uploading the candidate image to the server, and the server can segment the target object according to the target object framed by the first terminal, so as to obtain the first image including only the target object.

[0142] Illustratively, after the target object is segmented from the candidate image, the remaining background part is the third image. In some embodiments, when the candidate image includes text content, the text content in the candidate image can also be selected to be removed to obtain the third image including only the background part.

[0143] In step 703, the first image is converted from the first style to the second style to obtain a second image.

[0144] In the embodiments of the present application, the first image is input into the pre-trained generation network corresponding to the second style to output the second image in the second style. Illustratively, the generation network is used to map the first image in the first style to the style curve of the second style to generate the second image. In one example, as shown in Figure 8 Fig. 8 shows a schematic diagram of style conversion of a target object from a real style to a comic style. After the first image 810 of the target object in the real style is input into the generation network 801, the second image 820 of the target object in the comic style is output, in which the edges of the target object in the comic style are more obvious.

[0145] In some embodiments, since the first image is obtained by segmenting the candidate image, the first image can be an irregular image. Therefore, before the first image is input into the generation network, the first image can be preprocessed to adapt to the size of the image input into the generation network, and then the preprocessed first image is input into the generation network for style conversion.

[0146] In some embodiments, the above processing process can be filling a blank background to the first image to fill the first image to a specified size image. The blank background can be transparent or a specified color, which is not limited herein. In one example, the first image including the target object is superimposed and fused with a transparent image of a specified size to obtain the processed first image.

[0147] In some embodiments, to process the first image into an image suitable for the input generation network, before the above padding processing is performed on the first image, a size scaling operation can also be performed on the first image, so that the target object in the first image after padding can have a suitable display size, avoiding the problem that the first image after padding contains too much blank background content or the target object in the first image after padding is not fully displayed.

[0148] In the embodiments of the present application, only the target object part in the candidate image is subjected to style conversion, that is, the target object is converted from the first style to the second style, while the background part of the candidate image is still used to generate the advertising video, that is, the target object in the advertising video is of the second style and the background part is of the first style, so as to realize the combination of different styles of content and obtain an advertising video with various styles for the recommendation process of the target object, thereby enriching the diversity of the recommended content of the target object in the promotion process.

[0149] In some other embodiments, the background part in the candidate image can also be subjected to style conversion, that is, the finally generated advertising video is composed of the target object of the second style and the background of the second style. When the background part is also subjected to style conversion, the candidate image can be directly input to the generation network as the first image, so as to output the second image of the second style including the target object and the background part.

[0150] In step 704, the background of the third image is completed by the background completion network to obtain a background image.

[0151] In some embodiments, when the target object in the candidate image is segmented, the remaining part, that is, the background part, is illustratively subjected to target object detection on the candidate image. After the target object is detected, the edge of the target object is identified, and the candidate image is segmented into the first image and the third image according to the edge of the target object.

[0152] Illustratively, when the background image is an image segmented from the candidate image, the background image needs to be subjected to image completion (Image Inpainting) for the generation of the advertising video.

[0153] In some embodiments, the background image is completed using a large mask inpainting (LaMa) algorithm. Illustratively, the model structure corresponding to the LaMa algorithm is as shown in Figure 9As shown, the to-be-repaired image 901 and the mask image 902, first, the mask image 902 is negated and multiplied by the to-be-repaired image 901, the multiplied to-be-repaired image with the mask and the mask image 902 are superimposed based on the channel to obtain the intermediate processing image 903, the intermediate processing image 903 is input into the repair network 900, the repair network 900 first performs downsampling processing 910, then performs Fast Fourier Convolution (FFC) processing 920, and then performs upsampling processing 930 to output the repaired image 904. In the FFC, the input features are divided into two subparts 921 based on the channel, different subparts are input into different processing branches, wherein the first processing branch 922 is used to extract local information, and the second processing branch 923 is used to extract global information, wherein the FFC is used to extract global features in the second processing branch 923, and finally the extracted global features and local information are cross-fused and spliced based on the channel to obtain the output result. The above mask image 902 can be a segmentation result when image segmentation is performed.

[0154] In some embodiments, when the candidate image is an image including only the target object, that is, the candidate image does not need to be segmented to obtain the first image, the background image can be an image selected from the candidate background image. Optionally, the above-mentioned background image determined from the candidate background image can be indicated by the first terminal; or the server can determine the above-mentioned background image according to the type of the target object, match the above-mentioned background image from the candidate background image according to the type of the target object; or the server can obtain it by at least one candidate material from the material library in real time. Wherein, the type of the target object can be indicated by the first terminal, or the server can determine it after identifying the semantic information of the target object corresponding to the first image.

[0155] In some embodiments, when the background image is determined from the candidate background image, at least two background images are obtained to obtain a plurality of corresponding advertising videos, that is, different backgrounds are configured for the target object to generate different videos for recommending the target object; in some other embodiments, at least two background images can be obtained to generate advertising videos, that is, different backgrounds are configured for the target object to generate the same video.

[0156] In some embodiments, when the advertising video is generated by a plurality of different background images, the size normalization operation needs to be performed on the background image. Optionally, the above-mentioned size normalization operation can be scaling processing on different background images according to the specified size, or scaling processing on different background images according to the specified size.

[0157] Step 705: Generate a dynamic display effect of the second image on the background image to obtain the advertising video.

[0158] In this embodiment of the application, the dynamic display scheme indicates that the advertising video is generated by changing the position of the target object in the background image.

[0159] In some embodiments, to achieve a three-dimensional visual effect in the generated advertising video, the target object is moved from a first position to a second position in the target three-dimensional coordinate system according to a specified trajectory, resulting in at least two motion image frames corresponding to the target object during the movement. The first and second positions are positions in the target three-dimensional coordinate system, and the motion image frame is an image obtained by mapping the target object onto a background image based on its position in the target three-dimensional coordinate system. Special effects processing is performed on the at least two motion image frames to obtain at least two effect image frames, where the i-th motion image frame corresponds to the i-th effect image frame. The at least two effect image frames are then stitched together to obtain the advertising video.

[0160] In one instance, such as Figure 10 As shown, it illustrates a schematic diagram of an advertising video generation process provided by an exemplary embodiment of this application. The process involves obtaining a candidate image 1010 uploaded by a second terminal, segmenting the candidate image 1010 into a target object to obtain a first image 1020 containing the target object, and completing the remaining image 1030 to obtain a background image 1040. The first image 1020 is then style-transformed to obtain a second image 1050 containing the target object in a second style. An advertising video 1060 is generated using the background image 1040 and the second image 1050.

[0161] In some embodiments, the aforementioned advertising video may be an advertising video obtained by processing a candidate image frame, that is, moving the target object within a background content to obtain the advertising video; or, the aforementioned advertising video may be an advertising video obtained by processing multiple candidate image frames, that is, moving the target object between multiple background contents to obtain the advertising video, thereby enriching the ways of generating advertising videos.

[0162] In some embodiments, to further enrich the advertising video for recommending the target object, the visual effect of the advertising video is improved by combining multiple styles, so as to improve the recommendation effect of the advertising video. Illustratively, the background image segmented from the candidate image is taken as a first background image of a first style, the first background image is converted from the first style to a second style to obtain a second background image; the target object is converted from a first position to a second position in the first background image based on the second image to generate a first video corresponding to the target object; the target object is converted from a first position to a second position in the second background image based on the first image to generate a second video corresponding to the target object; the first video and the second video are segmented and clipped according to a specified video clipping manner to respectively obtain a first video segment corresponding to the first video and a second video segment corresponding to the second video; and the first video segment and the second video segment are cross-spliced to obtain the advertising video. That is, the advertising video is obtained by matching the target object of the first style with the background of the second style, and the target object of the second style with the background of the first style, so as to enrich the display form of the advertising video.

[0163] In one example, the above-mentioned manner of clipping the first video and the second video can be to alternately clip the first video and the second video to obtain the advertising video, for example, the first video is clipped at 1-2 seconds, 3-4 seconds and 5-6 seconds, the second video is clipped at 2-3 seconds, 4-5 seconds and 6-7 seconds, and the segments obtained by clipping are spliced in time sequence to obtain the advertising video.

[0164] In some embodiments, when generating the advertising video, the video can also be configured with audio. Illustratively, a target audio is determined from the candidate audio, and the target audio is encapsulated with the rendered video to obtain the advertising video.

[0165] In some embodiments, when the advertising video corresponds to audio, the above-mentioned manner of clipping the first video and the second video can also be to clip and splice the first video and the second video according to the rhythm change corresponding to the audio. Illustratively, the candidate audio corresponds to pre-configured rhythm change data, after the target audio is determined, the rhythm change data corresponding to the target audio is obtained, and the first video and the second video are clipped and spliced according to the rhythm change data to obtain the advertising video with the picture style change consistent with the above-mentioned rhythm change data. For example, the rhythm change frequency indicated by the rhythm change data and the picture style change frequency in the video are in a positive correlation relationship.

[0166] Step 706, pushing the advertising video to an advertising medium.

[0167] The advertising medium is used to recommend the target object by publishing the advertising video.

[0168] The advertisement media recommend the target object through the advertisement video.

[0169] In some embodiments, the advertisement video generated by the first image can be a candidate recommendation item corresponding to the target object, that is, when the target object corresponding recommendation content is pushed to the second terminal, the advertisement video can be one of the candidate items of the recommendation content.

[0170] In some embodiments, the server generates a plurality of second style advertisement videos according to different generation networks, wherein the plurality of second styles are different image styles, different advertisement videos can be obtained according to different second styles, and the advertisement videos of different second styles are used as different candidate recommendation items for the recommendation of the target object. For example, a cartoon style advertisement video, an oil painting style advertisement video, a pencil drawing style advertisement video, etc. are generated respectively.

[0171] As shown in the schematic diagram Figure 11 As shown in the schematic diagram

[0172] In summary, the object recommendation method provided by the embodiments of the present application, when the target object needs to be recommended, the target object obtained by matting is converted from the first style to the second style, and then the advertisement video of the target object in the second style is generated by converting the position of the target object in the background image in the second style, and the advertisement video is used for recommending the target object, which enriches the recommendation method when the target object is recommended, reduces the generation cost of the recommendation content, and improves the content generation efficiency.

[0173] In the embodiments of the present application, the candidate image provided by the second terminal is segmented, the target object obtained by segmentation is style converted, and then the background part is completed, so as to generate the advertisement video, which enriches the generation method of the advertisement video.

[0174] In some optional embodiments, the object recommendation method can be applied not only to content recommendation scenarios in the Internet, such as video application, social platform application, search application, etc., but also to content recommendation scenarios in a vehicle-mounted scenario. In an example, in the vehicle-mounted scenario, when a user performs a point of interest (POI) query in a map application through a vehicle-mounted device, a server obtains an entity image corresponding to the point of interest, the entity image being an image obtained by photographing a natural geographical feature or a humanistic facility in a real environment, the server performs style conversion on the entity image from a real style to a cartoon style, generates a recommended video through the obtained cartoon style image, and pushes the generated recommended video to the vehicle-mounted device for playing, so as to recommend the point of interest corresponding to the entity image, thereby enriching the diversity of content recommendation in the vehicle-mounted scenario.

[0175] Reference is made to Figure 12 which shows a structure block diagram of an object recommendation device provided by an example embodiment of the present application, the device comprising the following modules:

[0176] A segmentation module 1210 is configured to perform cutout segmentation processing on a target object in a candidate image to obtain a first image corresponding to the target object, the candidate image and the first image being images of a first style;

[0177] A first processing module 1220 is configured to convert the first image from the first style to a second style matching the candidate image to obtain a second image, the first style and the second style being two different image styles, and the second image being used as recommended object material of an advertisement video;

[0178] An acquisition module 1230 is configured to acquire a background image, the background image being used as background material of the advertisement video;

[0179] A second processing module 1240 is configured to generate a dynamic display effect of the second image on the background image to obtain the advertisement video;

[0180] A pushing module 1250 is configured to push the advertisement video to an advertisement medium, the advertisement medium being configured to recommend the target object by publishing the advertisement video.

[0181] In some optional embodiments, as shown in Figure 13 the first processing module 1220 further comprises:

[0182] A first determination unit 1221 is configured to determine an object type corresponding to the target object in the candidate image;

[0183] The first determining unit 1221 is further configured to determine the second style matching the object type from at least two candidate styles.

[0184] The conversion unit 1222 is configured to convert the first image from the first style to the second style to obtain the second image.

[0185] In some optional embodiments, the first determining unit 1221 is further configured to determine a color variation parameter of the candidate image based on a pixel value difference between adjacent pixels in the candidate image, where the color variation parameter is used to indicate a contrast between pixels in the candidate image.

[0186] The first determining unit 1221 is further configured to determine the second style matching the color variation parameter from at least two candidate styles.

[0187] The conversion unit 1222 is further configured to convert the first image from the first style to the second style to obtain the second image.

[0188] In some optional embodiments, the candidate styles include an oil painting style and a comic style.

[0189] The first determining unit 1221 is further configured to determine the oil painting style as the second style in response to the color variation parameter reaching a specified threshold.

[0190] The first determining unit 1221 is further configured to determine the comic style as the second style in response to the color variation parameter being lower than the specified threshold.

[0191] In some optional embodiments, the conversion unit 1222 is further configured to map a pixel value corresponding to a first pixel in the first image to a style curve of the second style to obtain a target pixel value.

[0192] The conversion unit 1222 is further configured to generate the second image by taking the target pixel value as a second pixel, where a position of the second pixel in the second image corresponds to a position of the first pixel in the first image.

[0193] In some optional embodiments, the segmentation module 1210 is further configured to segment the target object and a background part in the candidate image to obtain the first image corresponding to the target object and a third image corresponding to the background part.

[0194] The second processing module 1240 further includes:

[0195] The patching unit 1241 is configured to perform background completion on the third image by using a background completion network to obtain the background image.

[0196] In some optional embodiments, the background image obtained by the candidate image segmentation is a first background image of the first style;

[0197] The first processing module 1220 is further configured to convert the first background image from the first style to the second style to obtain a second background image.

[0198] The second processing module 1240 further includes:

[0199] The generation unit 1242 is configured to convert the target object from the first position in the first background image to the second position based on the second image to generate a first video corresponding to the target object;

[0200] The generation unit 1242 is further configured to convert the target object from the first position in the second background image to the second position based on the first image to generate a second video corresponding to the target object.

[0201] The generation unit 1242 is further configured to segmentally clip the first video and the second video according to a specified video clipping manner to respectively obtain a first video segment corresponding to the first video and a second video segment corresponding to the second video.

[0202] The generation unit 1242 is further configured to cross splice the first video segment and the second video segment to obtain the advertisement video.

[0203] In some optional embodiments, the generation unit 1242 is further configured to move the target object from the first position to the second position in a target three-dimensional coordinate system according to a specified movement track to obtain at least two motion image frames corresponding to the target object in a movement process, wherein the motion image frame is an image obtained by mapping the target object in the target three-dimensional coordinate system to the background image.

[0204] The generation unit 1242 is further configured to perform special effect processing on the at least two motion image frames to obtain at least two effect image frames, wherein an i-th motion image frame corresponds to an i-th effect image frame, and the special effect processing includes at least one of shadow processing, depth-of-field processing, scaling processing, rotation processing, and foreground occlusion processing.

[0205] The generation unit 1242 is further configured to splice the at least two effect image frames to obtain the advertisement video.

[0206] In some optional embodiments, the apparatus further includes:

[0207] The receiving module 1260 is configured to receive a recommended content acquisition request indicated by a content receiving account, the recommended content acquisition request indicating content recommendation to the content receiving account, and the recommended content acquisition request including account information of the content receiving account.

[0208] The pushing module 1250 further includes:

[0209] The second determining unit 1251 is configured to determine the target object from candidate recommendation objects based on the recommended content acquisition request.

[0210] The acquisition unit 1252 is configured to acquire at least two candidate advertisement videos corresponding to the target object, and an i-th candidate advertisement video corresponds to an i-th second style.

[0211] The second determining unit 1251 is further configured to determine, based on the account information, a style inclination degree of the content receiving account with respect to the at least two candidate advertisement videos.

[0212] The second determining unit 1251 is further configured to determine, as a target advertisement video, a candidate object with the highest style inclination degree among the at least two candidate advertisement videos.

[0213] The pushing unit 1253 is configured to push the target advertisement video to the content receiving account.

[0214] In some optional embodiments, the account information includes age information corresponding to the content receiving account.

[0215] The second determining unit 1251 is further configured to determine, according to the age information, a target age range to which the content receiving account belongs.

[0216] The acquisition unit 1252 is further configured to acquire a style inclination degree corresponding to the target age range and at least two second styles.

[0217] In some optional embodiments, the generation unit 1242 is further configured to generate an object pointer corresponding to the target object according to the second image.

[0218] The generation unit 1242 is further configured to establish an association between the object pointer and the advertisement video.

[0219] In the graphical interface in which the advertisement video is played, the object pointer is used as a pattern material for indicating a mouse position.

[0220] ​To sum up, the object recommendation device provided by the embodiments of the present application, when the target object needs to be recommended, the second image of the target object converted from the first style to the second style is obtained, and the position of the target object in the second image is converted to generate an advertisement video of the target object in the second style. The advertisement video is used for recommending the target object, which enriches the recommendation method when the target object is recommended, reduces the generation cost of the recommended advertisement, and improves the content generation efficiency.

[0221] It should be noted that: the object recommendation device provided by the above embodiments is only exemplified by the division of the above functional modules. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the object recommendation device and the object recommendation method provided by the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0222] Figure 14 The structure of the server provided by an example embodiment of the present application is shown. Specifically, it includes the following structures.

[0223] The server 1400 includes a central processing unit (CPU) 1401, a system memory 1404 including a random access memory (RAM) 1402 and a read-only memory (ROM) 1403, and a system bus 1405 connecting the system memory 1404 and the central processing unit 1401. The server 1400 also includes a mass storage device 1406 for storing an operating system 1413, application programs 1414, and other program modules 1415.

[0224] The mass storage device 1406 is connected to the central processing unit 1401 through a mass storage controller (not shown) connected to the system bus 1405. The mass storage device 1406 and its associated computer readable medium provide non-volatile storage for the server 1400. That is, the mass storage device 1406 can include a computer readable medium (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0225] Without loss of generality, the computer readable medium can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other solid state memory technology, CD-ROM, Digital Versatile Disc (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. It should be understood by those skilled in the art that computer storage media does not limit to the above-mentioned several kinds. The system memory 1404 and the mass storage device 1406 mentioned above can be collectively referred to as memory.

[0226] According to various embodiments of the present application, the server 1400 can also run on a remote computer connected to the network through a network connection such as the Internet. That is, the server 1400 can be connected to the network 1412 through the network interface unit 1411 connected to the system bus 1405, or can be connected to other types of networks or remote computer systems (not shown) using the network interface unit 1411.

[0227] The above-mentioned memory further includes one or more programs, one or more programs are stored in the memory and are configured to be executed by the CPU.

[0228] Embodiments of the present application also provide a computer device including a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the object recommendation method provided by the above-mentioned method embodiments. Optionally, the computer device can be a terminal or a server.

[0229] Embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the object recommendation method provided by the above-mentioned method embodiments.

[0230] The embodiment of the present application further provides a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the object recommendation method in any of the above embodiments.

[0231] Optionally, the computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a solid state disk (SSD), an optical disk, or the like. The random access memory can include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0232] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing related hardware to complete, and the program can be stored in a computer readable storage medium. The above-mentioned storage medium can be a read only memory, a magnetic disk or an optical disk.

[0233] The above-mentioned is only optional embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An object recommendation method, characterized in that, The method includes: The target object in the candidate image is segmented to obtain a first image corresponding to the target object. The candidate image and the first image are images of a first style. The first image is converted from the first style to a second style that matches the candidate image to obtain a second image. The first style and the second style are two different image styles. The second image is used as the object material for the advertising video. The second style is obtained by matching the color difference condition of the candidate image with the conversion requirements of different candidate styles. The color difference condition is determined based on the pixel value difference between adjacent pixels in the candidate image. Obtain a background image, which is used as the background material for the advertising video; The second image is dynamically displayed on the background image to obtain the advertising video; The advertising video is pushed to an advertising media outlet, which then recommends the video to the target audience.

2. The method according to claim 1, characterized in that, The step of converting the first image from the first style to a second style that matches the candidate image to obtain a second image includes: Determine the object type corresponding to the target object in the candidate image; Determine the second style that matches the object type from at least two candidate styles; The first image is converted from the first style to the second style to obtain the second image.

3. The method according to claim 1, characterized in that, The step of converting the first image from the first style to a second style that matches the candidate image to obtain a second image includes: Based on the pixel value difference between adjacent pixels in the candidate image, a color change parameter of the candidate image is determined, which is used to indicate the contrast between pixels in the candidate image; The second style that matches the color change parameters is determined from at least two candidate styles; The first image is converted from the first style to the second style to obtain the second image.

4. The method according to claim 3, characterized in that, The candidate styles include oil painting style and comic style; Determining the second style that matches the color change parameter from at least two candidate styles includes: In response to the color change parameter reaching a specified threshold, the oil painting style is determined to be the second style; In response to the color change parameter being lower than the specified threshold, the comic style is determined to be the second style.

5. The method according to any one of claims 2 to 4, characterized in that, The step of converting the first image from the first style to the second style to obtain the second image includes: The pixel value corresponding to the first pixel in the first image is mapped to the style curve of the second style to obtain the target pixel value; The target pixel value is used as the second pixel to generate the second image, and the position of the second pixel in the second image corresponds to the position of the first pixel in the first image.

6. The method according to any one of claims 1 to 4, characterized in that, The step of performing image segmentation on the target object in the candidate image to obtain a first image corresponding to the target object includes: The target object and background portion in the candidate image are segmented to obtain a first image corresponding to the target object and a third image corresponding to the background portion; The acquisition of the background image includes: The background image is obtained by performing background completion on the third image using a background completion network.

7. The method according to claim 6, characterized in that, The background image obtained by segmenting the candidate image is a first background image of the first style; The process of generating the dynamic display effect of the second image on the background image to obtain the advertising video includes: The first background image is converted from the first style to the second style to obtain the second background image; Based on the second image, the target object is moved from a first position in the first background image to a second position, and a first video corresponding to the target object is generated. Based on the first image, the target object is moved from the first position in the second background image to the second position, and a second video corresponding to the target object is generated. The first video and the second video are segmented and edited according to a specified video cropping method to obtain a first video segment corresponding to the first video and a second video segment corresponding to the second video. The first video segment and the second video segment are cross-stitched together to obtain the advertising video.

8. The method according to any one of claims 1 to 4, characterized in that, The process of generating the dynamic display effect of the second image on the background image to obtain the advertising video includes: The target object is moved from its first position in the target three-dimensional coordinate system to its second position according to a specified trajectory, thereby obtaining at least two motion image frames corresponding to the target object during the movement. The motion image frames are images obtained by mapping the target object onto the background image based on the target object's position in the target three-dimensional coordinate system. Special effects processing operations are performed on the at least two motion image frames to obtain at least two effect image frames, wherein the i-th motion image frame corresponds to the i-th effect image frame, and the special effects processing operations include at least one of shadow processing, depth of field processing, scaling processing, rotation processing, and foreground occlusion processing. The advertisement video is obtained by stitching together the at least two effect image frames.

9. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The content receiving account receives a recommended content retrieval request, which instructs the content receiving account to recommend content, and the recommended content retrieval request includes the account information of the content receiving account. The target object is determined from the candidate recommendation objects based on the recommended content retrieval request; Obtain at least two candidate ad videos corresponding to the target object, wherein the i-th candidate ad video corresponds to the i-th second style; Based on the account information, determine the style preference of the content receiving account for each of the at least two candidate ad videos; The candidate with the highest style preference among the at least two candidate ad videos is recommended and determined as the target ad video; The target advertising video is pushed to the content receiving account.

10. The method according to claim 9, characterized in that, The account information includes the age information corresponding to the content receiving account; The step of determining the style preference of the content receiving account for each of the at least two candidate ad videos based on the account information includes: The target age group to which the content receiving account belongs is determined based on the age information; Obtain the style preference degree corresponding to the target age group and at least two second styles.

11. The method according to any one of claims 1 to 4, characterized in that, After converting the first image from the first style to the second style to obtain the second image, the process further includes: Generate an object pointer corresponding to the target object based on the second image; Establish the association between the object pointer and the advertisement video; In the graphical interface where the advertising video is played, the object pointer is used as a graphic element to indicate the mouse position.

12. An object recommendation device, characterized in that, The device includes: The segmentation module is used to perform image segmentation processing on the target object in the candidate image to obtain a first image corresponding to the target object. The candidate image and the first image are images of a first style. A first processing module is used to convert the first image from the first style to a second style that matches the candidate image to obtain a second image. The first style and the second style are two different image styles. The second image is used as recommended material for advertising videos. The second style is obtained by matching the color difference condition of the candidate image with the conversion requirements of different candidate styles. The color difference condition is determined based on the pixel value difference between adjacent pixels in the candidate image. The acquisition module is used to acquire a background image, which is used as the background material of the advertising video; The second processing module is used to generate a dynamic display effect of the second image on the background image, thereby obtaining the advertising video; The push module is used to push the advertising video to the advertising media, which is used to recommend the target audience by publishing the advertising video.

13. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program that is loaded and executed by the processor to implement the object recommendation method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the object recommendation method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the object recommendation method as described in any one of claims 1 to 11.

Images