Information recommendation method and device and related products

Abstract

The embodiment of the invention provides an information recommendation method and device and a related product, and the method comprises the steps: playing video data in response to a playing instruction for the video data, and recognizing a material putting region for putting materials in a video frame of the video data through a video content understanding model; determining a target material matched with the material putting area in each to-be-put material through a material putting model, putting the target material in the material putting area, and displaying a material component corresponding to the target material; and in response to a trigger instruction for the material component, skipping to a material recommendation page corresponding to the target material, and displaying material information of the target material in the material recommendation page.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to an information recommendation method, apparatus, and related products. Background Art

[0002] In order to enrich the content of video data, materials can be placed in the video data. In some technologies, it is necessary to pre-reserve material placement positions in the video data. After the video data is produced, the materials are inserted at the reserved positions through video processing technology. This scheme of reserving material placement positions requires reserving the positions during the production of the video data, and the production process of the video data is restricted by the material placement positions, reducing the production efficiency of the video data. Summary of the Invention

[0003] Embodiments of the present disclosure provide an information recommendation method, apparatus, and related products, which do not require reserving material placement positions during the production of video data, and the production process of the video data is not restricted by the material placement positions, improving the production efficiency of the video data.

[0004] In a first aspect, embodiments of the present disclosure provide an information recommendation method, including: In response to a play instruction for video data, play the video data, and through a video content understanding model, identify a material placement area for placing materials in the video frames of the video data; Through a material placement model, determine a target material that matches the material placement area among the materials to be placed, place the target material at the material placement area, and display a material component corresponding to the target material; In response to a trigger instruction for the material component, jump to a material recommendation page corresponding to the target material, and display material information of the target material on the material recommendation page.

[0005] In a second aspect, embodiments of the present disclosure provide an information recommendation apparatus, including: An identification unit, configured to play the video data in response to a play instruction for the video data, and through a video content understanding model, identify a material placement area for placing materials in the video frames of the video data; A placement unit, configured to determine a target material that matches the material placement area among the materials to be placed through a material placement model, place the target material at the material placement area, and display a material component corresponding to the target material; A jump unit, configured to jump to a material recommendation page corresponding to the target material in response to a trigger instruction for the material component, and display material information of the target material on the material recommendation page.

[0006] In a third aspect, embodiments of the present disclosure provide an electronic device, including: a processor; and a memory configured to store computer-executable instructions, where the computer-executable instructions, when executed, cause the processor to implement the method described in the first aspect above.

[0007] In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium for storing computer-executable instructions, where the computer-executable instructions, when executed by a processor, implement the method described in the first aspect above.

[0008] In a fifth aspect, embodiments of the present disclosure provide a computer program product, where the computer program product includes a computer program, and the computer program, when executed by a processor, implements the method described in the first aspect above.

[0009] In one or more embodiments of the present disclosure, first, in response to a play instruction for video data, play the video data, and through a video content understanding model, identify a material placement area for placing materials in the video frames of the video data; then, through a material placement model, determine a target material that matches the material placement area among each material to be placed, place the target material at the material placement area, and display a material component corresponding to the target material; next, in response to a trigger instruction for the material component, jump to a material recommendation page corresponding to the target material, and display material information of the target material on the material recommendation page. It can be seen that through the embodiments of the present application, it is possible to automatically identify the material placement area to be placed in the video frame and place the target material at the identified material placement area, without reserving a position for material placement during the production of video data. The video data production process is not restricted by the material placement position, improving the production efficiency of video data. In addition, by responding to the trigger instruction for the material component, it is possible to jump to the material recommendation page corresponding to the target material and display the material information of the target material on the material recommendation page, so that the user can directly and quickly view the material information of the material of interest, significantly improving the efficiency of the user obtaining material information and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in one or more embodiments of the present disclosure or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings; Figure 1 It is a schematic flowchart of an information recommendation method provided by an embodiment of the present disclosure; Figure 2Schematic diagram of a video frame of video data provided by an embodiment of the present disclosure; Figure 3a Schematic diagram of a blank area in non-subject screen content provided by an embodiment of the present disclosure; Figure 3b Schematic diagram of a material placement area in non-subject screen content provided by an embodiment of the present disclosure; Figure 4a Schematic diagram of a target material placed in the material placement area provided by an embodiment of the present disclosure; Figure 4b Schematic diagram of a target material placed in the material placement area provided by another embodiment of the present disclosure; Figure 5a Schematic diagram of a material recommendation page corresponding to a target material provided by an embodiment of the present disclosure; Figure 5b Schematic diagram of a material recommendation page corresponding to a target material provided by another embodiment of the present disclosure; Figure 5c Schematic diagram of a material recommendation page corresponding to a target material provided by yet another embodiment of the present disclosure; Figure 6 Schematic diagram of the structure of an information recommendation device provided by an embodiment of the present disclosure; Figure 7 Schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0011] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present disclosure, the following will clearly and completely describe the technical solutions in one or more embodiments of the present disclosure with reference to the accompanying drawings in one or more embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on one or more embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0012] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, relevant parties should be informed of the type, scope of use, usage scenarios, etc. of the information involved in the present disclosure in an appropriate manner and obtain the authorization of the relevant parties in accordance with relevant laws and regulations.

[0013] For example, when a user's active request is received, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application program, server, or storage medium that performs the operations of the present disclosure's technical solution based on the prompt message.

[0014] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving the user's active request may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry selection controls for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0015] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation manner of the present disclosure. Other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0016] The embodiments of the present disclosure provide an information recommendation method, apparatus, and related products, which can automatically identify the material placement area in the video frame and place the target material at the identified material placement area, without reserving a material placement position during the production of video data, and the video data production process is not restricted by the material placement position, improving the production efficiency of video data.

[0017] To facilitate the understanding of the various embodiments of the present disclosure, the following explains each of the nouns involved.

[0018] Material: Refers to the image of the product to be recommended, including but not limited to the physical photo of the product to be recommended, the usage scenario diagram of the product to be recommended, the detailed close-up diagram of the product to be recommended, the rendered effect diagram of the product to be recommended, the packaging display diagram of the product to be recommended, and the related demonstration video or GIF animation of the product to be recommended, etc. The product to be recommended includes but not limited to physical products, virtual products, etc. Among them, physical products include but not limited to products with physical properties such as water cups, computers, skin care products, teapots, table lamps, trash cans, refrigerators, etc., and virtual products include but not limited to products without physical properties such as application programs, software, online games, coupons, consumption vouchers, online courses, etc.

[0019] Material recommendation page: Includes but not limited to the product recommendation page corresponding to the product represented by the material.

[0020] Material information: Includes but not limited to the product information of the product represented by the material, including but not limited to the purchase information of the product and the detailed information of the product.

[0021] Figure 1The flowchart of the information recommendation method provided by an embodiment of the present disclosure is shown as Figure 1 follows. The process includes: Step S102: In response to a play instruction for video data, play the video data, and identify a material placement area for placing materials in the video frames of the video data through a video content understanding model; Step S104: Determine a target material that matches the material placement area from each material to be placed through a material placement model, place the target material at the material placement area, and display a material component corresponding to the target material; Step S106: In response to a trigger instruction for the material component, jump to a material recommendation page corresponding to the target material, and display material information of the target material on the material recommendation page.

[0022] In this embodiment, first, in response to a play instruction for video data, play the video data, and identify a material placement area for placing materials in the video frames of the video data through a video content understanding model; then, determine a target material that matches the material placement area from each material to be placed through a material placement model, place the target material at the material placement area, and display a material component corresponding to the target material; next, in response to a trigger instruction for the material component, jump to a material recommendation page corresponding to the target material, and display material information of the target material on the material recommendation page. It can be seen that through the embodiment of the present application, it is possible to automatically identify the material placement area in the video frame and place the target material at the identified material placement area, without reserving a material placement position during the production of video data, and the production process of video data is not restricted by the material placement position, improving the production efficiency of video data. In addition, by responding to the trigger instruction for the material component, it is possible to jump to the material recommendation page corresponding to the target material and display the material information of the target material on the material recommendation page, so that users can directly and quickly view the material information of the materials they are interested in, significantly improving the efficiency of users obtaining material information and enhancing the user experience.

[0023] Figure 1 The method process in

[0024] In the above step S102, in response to a play instruction for video data, the video data is played. The terminal device can detect a triggering operation by the user on the play control for the video data, generate a play instruction for the video data based on this triggering operation, and then play the video data in response to this play instruction. The video data includes any one of short plays, short videos, movies, TV series, video data recorded by the user, and video data received by the user. Any video data that can be played can apply the information recommendation method in this embodiment.

[0025] In the above step S102, a video content understanding model is also used to identify, in the video frames of the video data, a material placement area for placing materials. Among them, the video content understanding model can be a visual recognition model trained using AI (Artificial Intelligence) technology and capable of performing content understanding on the video data. The material placement area for placing materials can be identified in the video frames when the video data is being played through the video content understanding model.

[0026] In one embodiment, identifying, in the video frames of the video data, a material placement area for placing materials through the video content understanding model includes: Through the video content understanding model, identifying the picture content related to the plot of the video data in the video frame as the main picture content, and regarding the picture content in the video frame other than the main picture content as non-main picture content; Through the video content understanding model, selecting, from the non-main picture content, an area whose size information meets the size requirement as the material placement area.

[0027] In this embodiment, a video content understanding model is pre-trained. Through the video content understanding model, content understanding can be performed on the video data, and the picture content related to the plot in the video frames when the video data is being played can be identified as the main picture content, and the picture content in the video frame other than the main picture content is regarded as non-main picture content. Among them, the main picture content is the picture content that directly promotes the plot in the video data, and the main picture content includes at least one of the characters related to the plot, the interaction scenes between the characters, the scenes related to the plot, and the items related to the plot. The non-main picture content refers to the visual elements presented in the video data but having no direct impact on the development of the plot. The non-main picture content can be exemplified by visual elements such as background environments or decorative items. Among them, the background environment can be exemplified by the street background where the character is located or the indoor background where the character is located. The decorative items can be exemplified by at least one of the furniture, tools, books, food, and decorative items placed in the room where the character is located.

[0028] Figure 2Schematic diagram of a video frame of video data provided by an embodiment of the present disclosure, as Figure 2 shown, the video data is exemplified by the short drama "Entrepreneurship Diary", and the video frame is the playing video picture in the 5th episode of the short drama "Entrepreneurship Diary". This video frame can be called video frame 1. The character in this video frame 1 is the heroine Xiao A in the short drama "Entrepreneurship Diary". In the picture shown in this video frame 1, Xiao A is the main picture content related to the plot of the short drama "Entrepreneurship Diary", and the other picture content except Xiao A is non-main picture content.

[0029] After identifying the main picture content and non-main picture content through the video content understanding model, the video content understanding model can be used to select a material placement area from the non-main picture content. The size information of the selected material placement area needs to meet the size requirements so that the placed material can be displayed completely and clearly in the video frame. In this embodiment, the size information of the material placement area can include length and width. Correspondingly, the size requirements include length range requirements and width range requirements. The length range requirements can be used to limit the minimum and maximum values of the length of the material placement area, and the width range requirements can be used to limit the minimum and maximum values of the width of the material placement area. In one example, the length range requirement of the size requirement can be [4 cm, 8 cm], and the width range requirement can be [4 cm, 8 cm].

[0030] Continuing with the above Figure 2 shown video frame 1 as an example, the non-main picture content is the other picture content except the heroine Xiao A. The picture content composed of items such as the computer, desk, hanging painting, and window in this video frame 1 is non-main picture content. Based on this, the video content understanding model can be used to select an area with size information meeting the size requirements from this non-main picture content as the material placement area.

[0031] Figure 3a Schematic diagram of a blank area in the non-main picture content provided by an embodiment of the present disclosure, as Figure 3a shown, it can be observed that the non-main picture content in video frame 1 includes area 1, area 2, area 3, and area 4. Area 1 is the area on the desktop, area 2 is the area on the computer screen, area 3 is the area on the window, and area 4 is the area on the wall. The above four areas are all blank areas, neither blocking items or people nor affecting the plot. Assume that among the above four areas, the size information of area 1, area 2, and area 3 meets the above size requirements. Then, area 1, area 2, and area 3 can be used as the material placement areas. Figure 3b Schematic diagram of a material placement area in the non-main picture content provided by an embodiment of the present disclosure, as Figure 3bAs shown in the figure, area 1 is determined as the material placement area 1, area 2 is determined as the material placement area 2, and area 3 is determined as the material placement area 3. The material placement area 1, the material placement area 2, and the material placement area 3 are all located in the non-main subject picture content in video frame 1.

[0032] It can be seen that through this embodiment, by using the video content understanding model, the non-main subject picture content is identified in the video frame, and the material placement area is determined in the non-main subject picture content, enabling the material to be accurately placed on the non-main subject picture content, avoiding the occlusion or interference of the material on the plot content, improving the user's viewing experience, and improving the accuracy of material placement.

[0033] After the material placement area is identified, in step S104 above, through the material placement model, the target material that matches the material placement area is determined among the materials to be placed. In one embodiment, determining the target material that matches the material placement area among the materials to be placed through the material placement model includes: Through the material placement model, an object containing the material placement area is determined in the video frame, and the object category information of the object is obtained; Through the material placement model, based on the above object category information, the target material is determined among the materials to be placed.

[0034] In this embodiment, a material placement model is pre-trained. The material placement model can be trained through AI technology, and can determine the target material that matches the material placement area among multiple materials to be placed, and place the target material in the material placement area. Based on this, first, through the material placement model, an object containing the material placement area is identified in the video frame. The object containing the material placement area is an item included in the non-main subject picture content, and the material placement area is located on this item. Then, the object category information of this object is obtained. Then, through the material placement model, according to the object category information of this object, the target material is determined among the materials to be placed. The materials to be placed can be obtained in advance. The object category information can be exemplified as categories such as table, street, mirror, computer, etc.

[0035] Continuing with the above Figure 3b as an example, on the basis of determining the material placement area 1, the material placement area 2, and the material placement area 3, it can be determined that the objects containing each material placement area in the above video frame 1 are a desk, a computer screen, and a window glass respectively. The object category information of the desk is table, the object category information of the computer screen is screen, and the object category information of the window glass is glass. Then, according to the object category information of table, screen, and glass, the target material is determined among multiple materials to be placed.

[0036] It can be seen that through this embodiment, by using the material placement model, first determine the object containing the material placement area and the object category information of the object in the video frame, and then based on the object category information, determine the target material among each material to be placed, which can make the placed target material match the material placement area, improve the overall visual coordination of the video frame, and improve the accuracy of material placement.

[0037] In one embodiment, by using the material placement model, based on the object category information, determining the target material among each material to be placed includes: By using the material placement model, among each material to be placed, select the material whose material category information matches the object category information as the first candidate material, and obtain the object style information of the object. By using the material placement model, among each first candidate material, select the material whose material style information matches the object style information as the target material.

[0038] In this embodiment, when determining the target material among each material to be placed based on the object category information by using the material placement model, first, by using the material placement model, among multiple materials to be placed, select the first candidate material whose material category information matches the object category information of the object containing the material placement area. Among them, the number of first candidate materials can be at least one. The material category information refers to the category of the product contained in the material, and the material category information can be exemplified as vehicles, food, daily necessities, beauty products, etc. The object category information can be exemplified as categories such as tables, streets, mirrors, computers, etc. The matching of the material category information and the object category information can be exemplified as the matching of vehicles and streets, food and tables, daily necessities and tables, etc.

[0039] Then, also obtain the object style information of the above object containing the material placement area. By using the material placement model, among each first candidate material, select the material whose material style information matches the object style information as the target material. Among them, the material style information refers to the style information of the product contained in the material, such as appearance and color. The object style information of the object can be the appearance information of the object such as appearance and color. The matching of the material style information and the object style information can be that the product in the material is coordinated with the object in style, the product in the material can be clearly displayed when the material is placed in the material placement area, and the style of the product in the material does not conflict with the style of the object.

[0040] Continuing with the above example, assume that the multiple materials to be placed include: images of white water cups, images of black water cups, images of lipsticks with black casings, images of lipsticks with white casings, images of lunch boxes containing fried chicken, images of fried chicken coupons, images of a certain star's concert promotional posters, images of frosted glass paper, images of New Year window grilles, images of store promotional posters, images of online game A, and images of application software B. The object category information of the desk containing the material placement area 1 determined above is table, the object category information of the computer screen containing the material placement area 2 is screen, and the object category information of the window glass containing the material placement area 3 is glass. Therefore, when determining the target materials based on the object category information, target materials that match the three categories of table, screen, and glass respectively and match the styles of the desk, computer screen, and window glass respectively can be determined from the multiple materials to be placed.

[0041] Taking the above material placement area 1 as an example, the object category information of the object containing the material placement area 1 is table. Among the above multiple materials to be placed, the materials that match the table, that is, the first candidate materials, can be: images of white water cups, images of black water cups, images of lipsticks with black casings, images of lipsticks with white casings, and images of lunch boxes containing fried chicken. Assume that the color of the desk shown in Figure 3 above is white. It can be understood that placing a white water cup or a lipstick with a white casing on a white desk may result in an unclear display effect of the white water cup or the lipstick with a white casing in the video data. However, if a lipstick with a black casing or a black water cup is placed on a white desk, it will be very obvious. Therefore, when determining the target materials, materials in which the products in the materials have an obvious color difference from the object can be selected as the target materials. For example, the images of black water cups, images of lipsticks with black casings, and images of lunch boxes containing fried chicken can be used as the target materials that match the style information of the desk.

[0042] Taking the above material placement area 2 as an example, the object category information of the object containing the material placement area 2 is screen. Among the above multiple materials to be placed, the first candidate materials that match the object category information of the screen can be images of a certain star's concert promotional posters, images of online game A, and images of application software B. Assume that the border of the computer screen shown in Figure 3 above is silver, and a pure white background image is displayed on the computer screen. Then, among the first candidate materials in this example, the images of a certain star's concert promotional posters, images of online game A, and images of application software B can all be used as the target materials that match the style information of the computer screen.

[0043] Taking the above-mentioned material placement area 3 as an example, the object category information of the object including the material placement area 3 is glass. Among the above-mentioned multiple materials to be placed, the first candidate materials matching the object category information of glass can be the images of frosted glass paper, the images of New Year window grilles, and the images of store promotional posters. Assume that the window glass shown in Figure 3 above is colorless and transparent. Among the above-mentioned first candidate materials, the style of the frosted glass paper is colorless and translucent, the color of the New Year window grilles is red, and the store promotional posters contain various eye-catching colors. Therefore, among the above-mentioned first candidate materials, the images of the New Year window grilles and the images of the store promotional posters are used as the target materials matching the style information of the window glass.

[0044] It can be seen that through this embodiment, it is possible to select, through the material placement model, the materials whose material category information matches the object category information as the first candidate materials among the various materials to be placed, and, obtain the object style information of the object, and select, through the material placement model, the materials whose material style information matches the object style information as the target materials among the various first candidate materials. Considering from two aspects of the material category and the material style, the selected target materials are more accurate, further improving the accuracy of material placement.

[0045] In one embodiment, when determining the target materials among the various materials to be placed based on the object category information through the material placement model, it may further include: Selecting, through the material placement model, the materials whose material category information matches the object category information as the first candidate materials among the various materials to be placed; Obtaining, through the material placement model, the playback scene information of the video data, and determining the playback scene of the video data according to the playback scene information; Selecting, through the material placement model, the materials whose material content matches the playback scene as the target materials among the various first candidate materials.

[0046] In this embodiment, when determining the target materials among the various materials to be placed based on the object category information through the material placement model, it is possible to first select, through the material placement model, the first candidate materials whose material category information matches the object category information of the object including the material placement area among the multiple materials to be placed, where the number of the first candidate materials can be at least one. The material category information refers to the category of the products included in the materials, and the material category information can be exemplified as vehicles, foods, daily necessities, beauty products, etc. The object category information can be exemplified as categories such as tables, streets, mirrors, computers, etc. The matching of the material category information and the object category information can be exemplified as the matching of vehicles and streets, the matching of foods and tables, the matching of daily necessities and tables, etc.

[0047] Then, the playback scene information of the video data is obtained through the material placement model, and the playback scene of the video data is determined according to the playback scene information of the video data, that is, the scene when the user plays the video data. The playback scene information can be exemplified as the playback location information, playback time information, user information of the playback user, and playback device information of the video data. The playback scene can be who plays the video data at what time and where.

[0048] Next, after determining the playback scene of the video data, through the material placement model, among the at least one first candidate material determined above, a material whose material content matches the playback scene is selected as the target material. The material content is the product to be recommended included in the material, and the material content can be exemplified as fried chicken, coffee, car, lipstick, computer, etc. The matching of the material content and the playback scene can be exemplified as fried chicken matching with late night, coffee matching with early morning, lipstick matching with shopping mall, computer matching with office building, etc.

[0049] Taking the above material placement area 1 as an example, the object category information of the object including the material placement area 1 is a table. Through the above process, the determined first candidate materials include: images of white water cups, images of black water cups, images of lipsticks with black casings, images of lipsticks with white casings, and images of lunch boxes containing fried chicken. Suppose the playback time of the video data is 11 pm and the playback location is City A, then the playback scene of the video data can be determined as the late night in City A. In this playback scene, it can be inferred that the matching degree of supper with this playback scene is relatively high. Therefore, among the above multiple first candidate materials, the image of the lunch box containing fried chicken is used as the target material.

[0050] It can be seen that through this embodiment, through the material placement model, among the materials to be placed, materials whose material category information matches the object category information are selected as the first candidate materials, and the playback scene information of the video data is obtained, the playback scene of the video data is determined according to the playback scene information, and through the material placement model, among the first candidate materials, materials whose material content matches the playback scene are selected as the target materials. Considering from two aspects of material category and material content, the selected target materials are more accurate, further improving the accuracy of material placement.

[0051] In one embodiment, determining the playback scene of the video data according to the playback scene information includes: Extracting at least one of the playback time information of the video data, the playback location information of the video data, and the playback user information corresponding to the video data from the playback scene information; Determining the playback scene of the video data according to the extracted information.

[0052] In this embodiment, when determining the playback scenario of video data based on the playback scenario information, at least one of the playback time information of the video data, the playback location information of the video data, and the playback user information corresponding to the video data may be extracted from the playback scenario information first, and the playback scenario of the video data may be determined according to the extracted information. Among them, the playback user information is the information of the user who plays the video data.

[0053] In one example, the playback time information of the short play "Entrepreneurship Diary" is 3:00 p.m. on Saturday, the playback location information is City B, and the playback user is Xiaob. According to the above playback scenario information of the short play "Entrepreneurship Diary", it can be determined that the playback scenario of the short play "Entrepreneurship Diary" is City B - Xiaob - Saturday - afternoon.

[0054] It can be seen that through this embodiment, the playback time information, playback location information, and playback user information of the video data can be extracted from the playback scenario information of the video data. According to these information, the playback scenario of the video data can be accurately located, and then the corresponding target material can be matched according to the located playback scenario, making the selected target material more accurate and further improving the accuracy of material placement.

[0055] In one embodiment, by using a material placement model to determine a target material from each material to be placed based on the object category information, it may further include: By using the material placement model, among each material to be placed, select a material whose material category information matches the object category information as the first candidate material; By using the material placement model, obtain the object style information of the object, and obtain the playback scenario information of the video data. According to the playback scenario information, determine the playback scenario of the video data; By using the material placement model, among each first candidate material, select a material whose material style information matches the object style information and whose material content matches the playback scenario as the target material.

[0056] In this embodiment, when using the material placement model to determine a target material from each material to be placed based on the object category information, first, by using the material placement model, among multiple materials to be placed, select a first candidate material whose material category information matches the object category information of the object including the material placement area. The number of the first candidate materials may be at least one. The material category information refers to the category of the product included in the material. Examples of the material category information may include vehicles, food, daily necessities, beauty products, etc. Examples of the object category information may include categories such as tables, streets, mirrors, computers, etc. The matching of the material category information and the object category information may be exemplified as the matching of vehicles and streets, food and tables, daily necessities and tables, etc.

[0057] Then, the object style information of the object and the playback scene information of the video data are obtained through the material placement model, and then the playback scene of the video data is determined according to the video playback scene information. Among them, the object style information of the object can be the appearance information of the object, and the appearance information is, for example, appearance and color. The playback scene information can be exemplified as the playback location information, playback time information, user information of the playback user, and playback device information of the video data. The playback scene can be who plays the video data at what time and where.

[0058] Next, after obtaining the object style information of the object and the playback scene of the video data, a material that matches the material style information with the object style information and the material content with the playback scene can be selected from the above-determined first candidate materials as the target material. The material style information refers to the style information of the product included in the material, such as appearance and color. The object style information of the object can be the appearance information of the object. The matching of the material style information with the object style information can be that the product in the material is coordinated with the object in terms of style, the product in the material can be clearly displayed when the material is placed in the material placement area, and the style of the product in the material does not conflict with the style of the object. The material content is the product to be recommended included in the material, and the material content can be exemplified as fried chicken, coffee, car, lipstick, computer, etc. The matching of the material content with the playback scene can be exemplified as fried chicken matching with late at night, coffee matching with early morning, lipstick matching with the mall, computer matching with the office building, etc.

[0059] Taking the above-mentioned material placement area 1 as an example, the object category information of the object containing the material placement area 1 is a table. The multiple materials to be placed include: images of products such as a white water cup, a latte, a lipstick with a black case, a lunch box containing fried chicken, a fried chicken coupon, a promotional poster of a certain star's concert, frosted glass paper, New Year window grilles, a store promotional poster, an online game A, and an application software B. The materials that match the object category information of the table, that is, the first candidate materials, can be: images of products such as a white water cup, a latte, a lipstick with a black case, a lipstick with a white case, and a lunch box containing fried chicken. Assuming that the color of the above-mentioned desk is black, among the above-mentioned first candidate materials, the images of the white water cup, the latte, the lunch box containing fried chicken, and the fried chicken coupon are significantly different in color from the black desk. When placed on the black table, the display effect of the above materials is better. Therefore, it can be determined that the images of the white water cup, the latte, the lunch box containing fried chicken, and the fried chicken coupon match the object style information. Suppose the video data is the short drama "Entrepreneurship Diary", the playback user is Xiaoc, the playback time is 8 am, and the playback city is City C. According to the determined playback time, playback location, and playback user of the short drama, the playback scene of the short drama can be determined as City C - Xiaoc - early morning. Based on Xiaoc's user information and real life, it can be inferred that coffee has a relatively high matching degree with this playback scene. Among the above-mentioned materials that are determined to match the object style information, the image of the latte is selected as the target material.

[0060] In another example, taking the above-mentioned material placement area 2 as an example, the object category information of the object including the material placement area 2 is the screen. The multiple materials to be placed include: images of a white water cup, a latte coffee, a lipstick with a black case, a food box containing fried chicken, a fried chicken coupon, a promotional poster of a certain star's concert, frosted glass paper, New Year window grilles, a store promotional poster, an online game A, an application software B, and other products. The first candidate materials that match the object category information of the screen can be: the image of the promotional poster of a certain star's concert, the image of the online game A, and the image of the application software B. If the color of the outer frame of the above computer screen is white, among the above first candidate materials, the images of the promotional poster of a certain star's concert, the image of the online game A, and the image of the application software B match the white border. Therefore, it can be determined that the images of the promotional poster of a certain star's concert, the image of the online game A, and the image of the application software B match the object style information. Suppose the video data is the short drama "Entrepreneurship Diary", the playback user is Xiaod, the playback time is 7 pm on Saturday, and the playback city is City D. According to the determined playback time and playback location of the short drama, the playback scene of the short drama can be determined as City D - Xiaod - Weekend evening. Based on Xiaod's user information and real life, it can be inferred that the online game has a high matching degree with this playback scene. Therefore, among the materials determined to match the object style information, the image of the online game A is selected as the target material.

[0061] It can be seen that through this embodiment, it is possible to select, through the material placement model, the materials whose material category information matches the object category information from each material to be placed as the first candidate materials, and obtain the style information of the object and the playback scene information of the video data. According to the playback scene information, determine the playback scene of the video data. Through the material placement model, select, from each first candidate material, the material whose material style information matches the object style information and whose material content matches the playback scene as the target material. Considering from three aspects: material category, material style, and material content, the selected target material is more accurate, further improving the accuracy of material placement.

[0062] In this embodiment, after determining the target material, the target material can also be placed in the material placement area through the material placement model, and the material component corresponding to the target material is displayed. When performing material placement, for example, through the material placement model, in the way of image editing, the target material can be edited in the corresponding material placement area. The material component corresponding to the target material can be the label corresponding to the target material. The form of the label is not limited. The label corresponding to the target material can be displayed on the target material or around the target material.

[0063] In one embodiment, placing the target material at the material placement area includes: Adjust the size and color tone of the target material according to the size of the material placement area and the color tone of the picture content at the material placement area; Place the adjusted target material at the material placement area.

[0064] In this embodiment, when placing the target material at the material placement area, the target material can be first subjected to image scaling processing according to the size of the material placement area, so that the size of the target material is the same as or smaller than the size of the material placement area. Then, according to the color tone of the picture content shown at the material placement area, the image color tone of the target material is adjusted so that the color tone of the adjusted target material is the same as or similar to the color tone of the picture content shown at the material placement area. After adjusting the size and color tone of the target material, the adjusted target material is placed at the material placement area.

[0065] In an example, assume that the length of the material placement area 1 is 6 cm and the width is 4 cm, and the target material is an image of a lunch box containing fried chicken. When placing the target material in the material placement area 1, the length of the target material can be adjusted to 6 cm and the width can be adjusted to 4 cm. Assume that the color tone of the picture content shown at the material placement area 1 is, for example, a warm color tone. Correspondingly, when adjusting the color tone of the target material, the color tone of the target material can be adjusted to a warm color tone. After adjusting the size and color tone of the target material, the adjusted target material is placed in the material placement area 1.

[0066] Figure 4a The figure is a schematic diagram of the target material provided by an embodiment of the present disclosure placed in the material placement area, as Figure 4a shown. The target material is an image of a lunch box containing fried chicken. The target material is placed in the material placement area 1 in the video frame 1. The size of the target material is the same as the size of the material placement area 1, and the color tone of the target material is the same as the color tone at the material placement area 1. A material component corresponding to the material is displayed on the image of the lunch box containing fried chicken. The material component is a label, and the words "Click to obtain recommended details" are displayed on the label.

[0067] Figure 4b The figure is a schematic diagram of the target material provided by another embodiment of the present disclosure placed in the material placement area, as Figure 4b shown. The target material is an image of the online game A. The target material is placed in the material placement area 2 in the video frame 1. The size of the target material is the same as the size of the material placement area 2, and the color tone of the target material is the same as the color tone at the material placement area 2. A material component corresponding to the material is displayed on the image of the online game A. The material component is a label, and the words "Click to obtain recommended details" are displayed on the label.

[0068] It can be seen that through this embodiment, by adjusting the size and color tone of the target material according to the size of the material placement area and the color tone of the screen content at the material placement area, and placing the adjusted target material at the material placement area, the target material can be completely and clearly placed at the material placement area, ensuring that the target material is presented in the best state within the placement area, thereby improving the accuracy of material placement.

[0069] Further, after placing the target material in the material placement area and displaying the material component corresponding to the target material, in step S106, in response to a trigger instruction for the material component, it jumps to the material recommendation page corresponding to the target material, and the material information of the target material is displayed on the material recommendation page. In this embodiment, the terminal device can detect the trigger operation of the user on the material component, generate a trigger instruction for the material component according to this trigger operation, and then, in response to this trigger instruction, jump to the material recommendation page corresponding to the target material. The material recommendation page corresponding to the target material can be the product recommendation page of the product corresponding to the target material. On this page, the products corresponding to the target material recommended for the user can be displayed. For example, if the product corresponding to the target material is fried chicken, correspondingly, the material recommendation page corresponding to the target material displays the product of fried chicken. For another example, if the product corresponding to the target material is online game A, correspondingly, the material recommendation page corresponding to the target material displays the product of online game A.

[0070] In an example, user Xiaob clicks on the label corresponding to the image of the lunch box containing fried chicken shown above on the terminal device playing the short drama "Entrepreneurship Diary". Figure 4a The terminal device jumps to the material recommendation page corresponding to the lunch box containing fried chicken. Figure 5a FIG. is a schematic diagram of the material recommendation page corresponding to the target material provided by an embodiment of the present disclosure. As Figure 5a shown, the product information and purchase component of the corresponding fried chicken are displayed on this material recommendation page. User Xiaob can learn the product information of the recommended fried chicken on this material recommendation page, and can purchase the fried chicken by clicking on the purchase component. Considering the effectiveness and timeliness of material recommendation, the store where the fried chicken shown on the material recommendation page is located can be a fried chicken store relatively close to the location where user Xiaob is, that is, the video data playback location. In addition, a component of "Click to get more recommendations" can also be displayed on this material recommendation page. User Xiaob can click on this component to obtain more recommendations for fried chicken products.

[0071] In another example, user Xiaob clicks on the material component corresponding to the image of the fried chicken coupon on the terminal device playing the short drama "Entrepreneurship Diary". After the terminal device detects this operation of user Xiaob, it jumps to the material recommendation page corresponding to the fried chicken coupon. Figure 5bSchematic diagram of a material recommendation page corresponding to a target material provided by another embodiment of the present disclosure, as Figure 5b shown. A plurality of corresponding fried chicken coupons are displayed on this material recommendation page. The plurality of fried chicken coupons are all coupons for the fried chicken products in the target material. Information such as the corresponding coupon amount, rules, and the stores to which they belong is displayed on each fried chicken coupon. In addition, corresponding receiving components are also displayed. User Xiaob can understand the detailed information of each fried chicken coupon on this recommendation page and receive each fried chicken coupon by clicking the receiving component. Considering the effectiveness and timeliness of material recommendation, the stores to which the fried chicken coupons displayed on the material recommendation page belong can be fried chicken stores that are relatively close to the location where user Xiaob is, that is, the video data playback location. A component "Go to purchase" can also be displayed on this material recommendation page. User Xiaob can click this component after receiving the fried chicken coupon to purchase the corresponding fried chicken using the received fried chicken coupon. In addition, a component "Click to get more recommendations" can also be displayed on this material recommendation page. User Xiaob can obtain more recommendations for fried chicken products by clicking this component.

[0072] In another example, user Xiaob clicks on the material component corresponding to the image of online game A shown above on the terminal device playing the short drama "Entrepreneurship Diary". After the terminal device detects this operation by user Xiaob, it jumps to the material recommendation page corresponding to this online game A. Figure 4b Figure 5c Figure 5c Schematic diagram of a material recommendation page corresponding to a target material provided by another embodiment of the present disclosure, as Figure 5c shown. Details information and installation components of the corresponding online game A are displayed on this material recommendation page. User Xiaob can understand the details information of this online game A and download and install this online game A on this recommendation page. Correspondingly, after the user finishes downloading and installing this online game A, the terminal device can also automatically jump to the game page corresponding to this online game A. In addition, a component "Click to get more recommendations" can also be displayed on this material recommendation page. User Xiaob can obtain more recommendations for online game products by clicking this component.

[0073] In the above example, when user Xiaob has downloaded this online game A on the terminal device playing the short drama "Entrepreneurship Diary", details information and opening components of the corresponding online game A can also be displayed on this material recommendation page. User Xiaob can open this online game A on this recommendation page. Similarly, a component "Click to get more recommendations" can also be displayed on this material recommendation page. User Xiaob can obtain more recommendations for online game products by clicking this component.

[0074] It can be seen that through this embodiment, the terminal device can respond to a trigger instruction for a material component, jump to the material recommendation page corresponding to the target material, and display the material information of the target material on the material recommendation page, enabling the user to quickly obtain the information of interest, reducing the time for the user to search for and browse information on different pages, and improving the efficiency of information acquisition. The appended Figure 2 appendices Figure 3a and 3b in Embodiments 4a and 4b of the present disclosure may involve video screen content generated by AI.

[0075] Figure 6 FIG. is a schematic structural diagram of an information recommendation device provided by an embodiment of the present disclosure. As Figure 6 shown, the device includes: An identification unit 601, configured to play the video data in response to a play instruction for the video data, and identify a material placement area for placing materials in the video frames of the video data through a video content understanding model; A placement unit 602, configured to determine a target material that matches the material placement area from each material to be placed through a material placement model, place the target material at the material placement area, and display a material component corresponding to the target material; A jump unit 603, configured to jump to the material recommendation page corresponding to the target material in response to a trigger instruction for the material component, and display the material information of the target material on the material recommendation page.

[0076] Optionally, the identification unit 601 is specifically configured to: identify, through the video content understanding model, the picture content related to the plot of the video data in the video frame as the main picture content, and use the picture content in the video frame other than the main picture content as the non-main picture content; select, through the video content understanding model, an area whose size information meets the size requirement from the non-main picture content as the material placement area.

[0077] Optionally, the placement unit 602 is specifically configured to: determine, through the material placement model, an object in the video frame that includes the material placement area, and obtain the object category information of the object; determine the target material from each of the materials to be placed based on the object category information through the material placement model.

[0078] Optionally, the delivery unit 602 is further specifically configured to: select, through the material delivery model, materials whose material category information matches the object category information from all the materials to be delivered as first candidate materials, and obtain the object style information of the object; select, through the material delivery model, materials whose material style information matches the object style information from all the first candidate materials as the target materials.

[0079] Optionally, the delivery unit 602 is further specifically configured to: select, through the material delivery model, materials whose material category information matches the object category information from all the materials to be delivered as first candidate materials; obtain, through the material delivery model, the playback scene information of the video data, and determine the playback scene of the video data according to the playback scene information; select, through the material delivery model, materials whose material content matches the playback scene from all the first candidate materials as the target materials.

[0080] Optionally, the delivery unit 602 is further specifically configured to: select, through the material delivery model, materials whose material category information matches the object category information from all the materials to be delivered as first candidate materials; obtain the object style information of the object, and obtain the playback scene information of the video data, and determine the playback scene of the video data according to the playback scene information; select, through the material delivery model, materials whose material style information matches the object style information and whose material content matches the playback scene from all the first candidate materials as the target materials.

[0081] Optionally, the delivery unit 602 is further specifically configured to: extract at least one of the playback time information of the video data, the playback location information of the video data, and the playback user information corresponding to the video data from the playback scene information; determine the playback scene of the video data according to the extracted information.

[0082] Optionally, the delivery unit 602 is further specifically configured to: adjust the size and color tone of the target material according to the size of the material delivery area and the color tone of the screen content at the material delivery area; deliver the adjusted target material at the material delivery area.

[0083] In this embodiment, first, in response to a play instruction for video data, the video data is played. Through the video content understanding model, in the video frames of the video data, a material placement area for placing materials is identified. Then, through the material placement model, a target material that matches the material placement area is determined from each material to be placed, the target material is placed at the material placement area, and a material component corresponding to the target material is displayed. Next, in response to a trigger instruction for the material component, a jump is made to a material recommendation page corresponding to the target material, and the material information of the target material is displayed on the material recommendation page. It can be seen that through the embodiments of the present application, it is possible to automatically identify the material placement area in the video frame and place the target material at the identified material placement area, without reserving a material placement position during the production of the video data. The video data production process is not restricted by the material placement position, improving the production efficiency of the video data. In addition, by responding to the trigger instruction for the material component, a jump can be made to the material recommendation page corresponding to the target material, and the material information of the target material is displayed on the material recommendation page, so that users can directly and quickly view the material information of the materials they are interested in, significantly improving the efficiency of users obtaining material information and enhancing the user experience.

[0084] The information recommendation device in the embodiments of the present disclosure can implement each process of the above information recommendation method embodiments and achieve the same effects and functions, which will not be repeated here.

[0085] An embodiment of the present disclosure further provides an electronic device. Figure 7 The following is a schematic structural diagram of the electronic device provided by an embodiment of the present disclosure. As Figure 7 shown, the electronic device may vary greatly due to configuration or performance differences, and may include one or more processors 701 and a memory 702. One or more application programs or data may be stored in the memory 702. Among them, the memory 702 may be a transient storage or a persistent storage. The application programs stored in the memory 702 may include one or more modules (not shown in the figure), and each module may include a series of computer-executable instructions in the electronic device. Further, the processor 701 may be configured to communicate with the memory 702 and execute a series of computer-executable instructions in the memory 702 on the electronic device. The electronic device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input or output interfaces 705, one or more keyboards 706, etc.

[0086] In a specific embodiment, the electronic device includes a processor; and a memory configured to store computer-executable instructions, and the computer-executable instructions, when executed, cause the processor to implement the following process: In response to a play instruction for video data, play the video data, and identify a material placement area for placing materials in the video frames of the video data through a video content understanding model; Through a material placement model, determine a target material that matches the material placement area from each material to be placed, place the target material at the material placement area, and display a material component corresponding to the target material; In response to a trigger instruction for the material component, jump to a material recommendation page corresponding to the target material, and display material information of the target material on the material recommendation page.

[0087] In this embodiment, first, in response to a play instruction for video data, play the video data, and identify a material placement area for placing materials in the video frames of the video data through a video content understanding model; then, through a material placement model, determine a target material that matches the material placement area from each material to be placed, place the target material at the material placement area, and display a material component corresponding to the target material; next, in response to a trigger instruction for the material component, jump to a material recommendation page corresponding to the target material, and display material information of the target material on the material recommendation page. It can be seen that through the embodiment of the present application, it is possible to automatically identify the material placement area in the video frame and place the target material at the identified material placement area, without reserving a material placement position during the production of video data, and the video data production process is not restricted by the material placement position, improving the production efficiency of video data. In addition, by responding to the trigger instruction for the material component, it is possible to jump to the material recommendation page corresponding to the target material and display the material information of the target material on the material recommendation page, so that users can directly and quickly view the material information of the materials they are interested in, significantly improving the efficiency of users obtaining material information and enhancing the user experience.

[0088] The electronic device in the embodiment of the present disclosure can implement each process of the above information recommendation method embodiment and achieve the same effects and functions, which will not be repeated here.

[0089] Another embodiment of the present disclosure further provides a computer-readable storage medium, which is used to store computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the following process: In response to a play instruction for video data, play the video data, and identify a material placement area for placing materials in the video frames of the video data through a video content understanding model; Through a material placement model, determine a target material that matches the material placement area from each material to be placed, place the target material at the material placement area, and display a material component corresponding to the target material; In response to a trigger instruction for the material component, jump to the material recommendation page corresponding to the target material, and display the material information of the target material on the material recommendation page.

[0090] In this embodiment, first, in response to a play instruction for video data, play the video data, and through a video content understanding model, identify a material placement area for placing materials in the video frames of the video data; then, through a material placement model, determine a target material that matches the material placement area among the materials to be placed, place the target material at the material placement area, and display the material component corresponding to the target material; next, in response to a trigger instruction for the material component, jump to the material recommendation page corresponding to the target material, and display the material information of the target material on the material recommendation page. It can be seen that through the embodiments of the present application, it is possible to automatically identify the material placement area in the video frame and place the target material at the identified material placement area, without reserving a material placement position during the production of the video data, and the video data production process is not restricted by the material placement position, improving the production efficiency of the video data. In addition, by responding to the trigger instruction for the material component, it is possible to jump to the material recommendation page corresponding to the target material, and display the material information of the target material on the material recommendation page, so that the user can directly and quickly view the material information of the materials of interest, significantly improving the efficiency of the user obtaining material information and enhancing the user experience.

[0091] The computer-readable storage medium in the embodiments of the present disclosure can implement each process of the above information recommendation method embodiment, and achieve the same effects and functions, which will not be repeated here.

[0092] Another embodiment of the present disclosure also provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, the following processes are implemented: In response to a play instruction for video data, play the video data, and through a video content understanding model, identify a material placement area for placing materials in the video frames of the video data; Through a material placement model, determine a target material that matches the material placement area among the materials to be placed, place the target material at the material placement area, and display the material component corresponding to the target material; In response to a trigger instruction for the material component, jump to the material recommendation page corresponding to the target material, and display the material information of the target material on the material recommendation page.

[0093] In this embodiment, first, in response to a playback instruction for video data, the video data is played, and a material placement area for placing materials is identified in the video frames of the video data through a video content understanding model; then, through a material placement model, a target material that matches the material placement area is determined from each material to be placed, the target material is placed at the material placement area, and a material component corresponding to the target material is displayed; next, in response to a trigger instruction for the material component, a jump is made to a material recommendation page corresponding to the target material, and material information of the target material is displayed on the material recommendation page. It can be seen that through the embodiment of the present application, the material placement area to be placed in the video frame can be automatically identified, and the target material can be placed at the identified material placement area without reserving a material placement position during the production of the video data. The production process of the video data is not restricted by the material placement position, improving the production efficiency of the video data. In addition, by responding to the trigger instruction for the material component, a jump can be made to the material recommendation page corresponding to the target material, and the material information of the target material is displayed on the material recommendation page, so that the user can directly and quickly view the material information of the material of interest, significantly improving the efficiency of the user obtaining the material information and enhancing the user experience.

[0094] The computer program product in the embodiments of the present disclosure can implement each process of the above information recommendation method embodiment and achieve the same effects and functions, which will not be repeated here.

[0095] In each embodiment of the present disclosure, the computer-readable storage medium includes a read-only memory (ROM for short), a random access memory (RAM for short), a magnetic disk, an optical disc, or the like.

[0096] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software improvements (improvements to method flows). However, with the development of technology, many method flow improvements today can be regarded as direct improvements to hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement to a method flow cannot be implemented using a hardware entity module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is an integrated circuit whose logical function is determined by the user programming the device. Designers can program themselves to "integrate" a digital system onto a single PLD, without having to ask a chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compilers used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a Hardware Description Language (HDL), and there is not just one type of HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones currently are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that by simply performing some logical programming on the method flow using the above-mentioned several hardware description languages and programming it into an integrated circuit, it is easy to obtain the hardware circuit that implements the logical method flow.

[0097] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to logically program the method steps to enable the controller to be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. to achieve the same function. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or the structures within the hardware component.

[0098] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0099] For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing the embodiments of the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0100] Those skilled in the art should understand that one or more embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of the present disclosure can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0101] This disclosure is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It should be understood that each flow and / or block in the flowchart and / or block diagram, and combinations of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device generate means for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks or a block.

[0102] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means that implement the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks or a block.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks or a block.

[0104] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity or device including the said element.

[0105] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

[0106] Each embodiment in the present disclosure is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for relevant details.

[0107] The above description is only for the embodiments of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, various modifications and variations can be made to the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included within the scope of the claims of the present disclosure.

Claims

1. An information recommendation method, characterized in that: include: In response to a playback instruction for video data, the video data is played, and a material placement area for placing materials is identified in a video frame of the video data by using a video content understanding model; Determine, through the material delivery model, a target material matching the material delivery area from among the materials to be delivered, deliver the target material in the material delivery area, and display the material component corresponding to the target material; In response to a trigger instruction for the material component, jump to a material recommendation page corresponding to the target material, and display material information of the target material on the material recommendation page.

2. The method according to claim 1, characterized in that The step of identifying a material placement area for placing material in a video frame of the video data by using a video content understanding model includes: Using the video content understanding model, identifying picture content related to the plot of the video data in the video frame as main picture content, and identifying picture content other than the main picture content in the video frame as non-main picture content; By using the video content understanding model, an area whose size information meets the size requirement is selected from the non-main screen content as the material placement area.

3. The method according to claim 1, characterized in that The step of determining a target material matching the material delivery area from among various materials to be delivered by using the material delivery model includes: Determine, by using the material delivery model, an object including the material delivery area in the video frame, and obtain object category information of the object; The target material is determined from among the materials to be delivered through the material delivery model based on the object category information.

4. The method according to claim 3, characterized in that The determining the target material from among the materials to be delivered through the material delivery model based on the object category information includes: By using the material delivery model, among the materials to be delivered, a material whose material category information matches the object category information is selected as a first candidate material, and object style information of the object is obtained; Through the material delivery model, among the first candidate materials, the material whose material style information matches the object style information is selected as the target material.

5. The method according to claim 3, characterized in that: The determining the target material from among the materials to be delivered through the material delivery model based on the object category information includes: By using the material delivery model, from among the materials to be delivered, a material whose material category information matches the object category information is selected as a first candidate material; Acquiring the playback scene information of the video data through the material delivery model, and determining the playback scene of the video data according to the playback scene information; Through the material delivery model, among the first candidate materials, the material whose content matches the playback scene is selected as the target material.

6. The method according to claim 3, characterized in that The determining the target material from among the materials to be delivered through the material delivery model based on the object category information includes: By using the material delivery model, from among the materials to be delivered, a material whose material category information matches the object category information is selected as a first candidate material; Obtaining object style information of the object through the material delivery model, and obtaining playback scene information of the video data, and determining the playback scene of the video data according to the playback scene information; Through the material delivery model, among the first candidate materials, the material whose material style information matches the object style information and whose material content matches the playback scene is selected as the target material.

7. The method according to claim 5 or 6, characterized in that: The determining the playback scene of the video data according to the playback scene information includes: Extracting at least one of the playback time information of the video data, the playback location information of the video data, and the playback user information corresponding to the video data from the playback scene information; The playback scene of the video data is determined according to the extracted information.

8. The method according to claim 1, characterized in that The placing of the target material in the material placing area includes: Adjusting the size and tone of the target material according to the size of the material placement area and the tone of the screen content in the material placement area; The adjusted target material is placed in the material placement area.

9. An information recommendation device, characterized in that: include: an identification unit, configured to respond to a playback instruction for video data, play the video data, and identify a material placement area for placing materials in a video frame of the video data by using a video content understanding model; A delivery unit, configured to determine, from among various materials to be delivered, a target material matching the material delivery area through a material delivery model, deliver the target material in the material delivery area, and display a material component corresponding to the target material; The jump unit is used to jump to the material recommendation page corresponding to the target material in response to the trigger instruction for the material component, and display the material information of the target material on the material recommendation page.

10. An electronic device, characterized in that: include: processor; as well as, A memory configured to store computer executable instructions, which, when executed, cause the processor to implement the method of any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the method of any one of claims 1 to 8 when executed by a processor.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images