Video recommendation method and display device

Abstract

The present invention discloses a video recommendation method and display device, belonging to the field of recommendation systems. The method comprises: upon receiving a short video recommendation request from a target user, calling a recommendation model, wherein the recommendation model is a training model obtained by migration and fusion based on the similarity between long videos and short videos using prior knowledge of long videos; controlling the recommendation model to predict and output short video recommendation information matching the target user; and displaying the short video recommendation information. The present application overcomes the cold start problem of short video recommendation and improves the performance of the recommendation system by migrating prior knowledge in the field of long videos to the field of short videos.

Description

Technical Field

[0001] The present invention relates to the technical field of recommendation systems, and in particular to a video recommendation method and a display device. Background Art

[0002] Some applications installed in the terminal are equipped with a recommendation system. The recommendation system is used to recommend objects that match the user's interests and preferences based on user behavior, user information, information about objects in the library, etc. For example, for video applications, the system can analyze the types of videos that the user may be interested in based on the user's behavior on the video (such as clicks, views, favorites, and shares, etc.), and filter and recommend video sources that match the video type from the library. This can not only meet the user's needs to watch similar or related videos, but also enhance the promotion and operation of resources within the platform.

[0003] For example, in some application scenarios, in addition to providing long video sources, the application platform also supports short video resources uploaded by users themselves, so that short video resources can be shared with other users through the application platform. Short videos are generally wonderful clips cut by users, or video content with personalized style and theme recorded by users. Therefore, short videos have gradually become another form of subject matter in the application platform to attract users to browse. The recommendation system of the application platform can also recommend short video resources that may be of interest to users based on their preferences. However, for newly registered users in the application platform or users with inactive video viewing behavior, the recommendation system cannot accurately locate their interest preferences. For short videos newly released by users, the short videos have not been watched or have been rarely watched by users, resulting in the recommendation system being unable to accurately match which users may like the short video. These are the "cold start" problems that recommendation systems often face. Summary of the invention

[0004] In order to solve the cold start problem of short video recommendation in the above-mentioned background technology, the present invention provides a video recommendation method and a display device.

[0005] A display device provided by an embodiment of the first aspect includes:

[0006] A display, for displaying recommended videos;

[0007] Controller, which performs:

[0008] When receiving a short video recommendation request from a target user, calling a recommendation model, wherein the recommendation model is a training model obtained by migrating and integrating prior knowledge of the long video based on the similarity between the long video and the short video;

[0009] Controlling the recommendation model to predict and output short video recommendation information matching the target user;

[0010] Control the display to display the short video recommendation information.

[0011] The video recommendation method provided by the second embodiment includes:

[0012] When receiving a short video recommendation request from a target user, calling a recommendation model, wherein the recommendation model is a training model obtained by migrating and integrating prior knowledge of the long video based on the similarity between the long video and the short video;

[0013] Controlling the recommendation model to predict and output short video recommendation information matching the target user;

[0014] The short video recommendation information is displayed.

[0015] In the technical solution provided in the present application, a recommendation model is pre-trained. The recommendation model utilizes the similarities between long videos and short videos, such as similar video types and themes, to transfer the prior knowledge accumulated in each long video to the short video that is most similar to it. The prior knowledge described here may include users and their behaviors associated with the long video. User behaviors include but are not limited to watching, collecting and sharing long videos. Assuming that the long video "Movie A" has been clicked and watched by user A, after transfer learning, it is believed that user A may have the same interest preference for similar short videos such as "cut clips of actor B in movie A", "highlights of movie A", and "movie reviews of movie A". The migrated model is then merged with the old prior knowledge model to obtain the final recommendation model.

[0016] Users can access the short video recommendation page specified in the application platform, or when the current video is started, the recommendation section is synchronously displayed at the specified position of the current video playback interface. The recommended related short videos are displayed to the user in the recommendation section. Therefore, when a short video recommendation request from the target user is received, the recommendation model is called for prediction processing. The recommendation model finally outputs short video recommendation information that matches the target user's interest preference, and displays the short video recommendation information on the recommendation page or recommendation section, so that users can quickly filter out the short videos they want to watch from the recommendation information. This application overcomes the cold start problem of short video recommendation and improves the performance of the recommendation system by transferring prior knowledge in the long video field to the short video field. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be accessed in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 shows a usage scenario of a display device according to some embodiments;

[0019] Figure 2 shows a hardware configuration block diagram of a control device 100 according to some embodiments;

[0020] Figure 3 shows a hardware configuration block diagram of a display device 200 according to some embodiments;

[0021] Figure 4 shows a software configuration diagram in a display device 200 according to some embodiments;

[0022] Figure 5 shows an icon control interface display diagram of an application in a display device 200 according to some embodiments;

[0023] Figure 6 A display diagram of a recommendation page is shown in FIG.

[0024] Figure 7 The example in the figure shows the video playback interface (non-full screen mode) showing the recommended section Figure 1 ;

[0025] Figure 8 The example in the figure shows the video playback interface (non-full screen mode) showing the recommended section Figure 2 ;

[0026] Fig. 9 exemplarily shows a schematic diagram of a video playback interface (full screen mode) displaying a recommendation section;

[0027] Fig.10 The example in the figure shows the video playback interface (non-full screen mode) showing the recommended section Figure 2 ;

[0028] Fig.11 The training logic diagram of the recommendation model is shown as an example in FIG;

[0029] Fig.12 A flowchart of a video recommendation method is exemplarily shown in FIG. DETAILED DESCRIPTION

[0030] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0031] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0032] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0033] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0034] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0035] Figure 1 Schematic diagram of a usage scenario of a display device according to an embodiment. Figure 1 As shown, the display device 200 also performs data communication with the server 400 , and the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0036] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and at least one of other short-range communication methods, to control the display device 200 wirelessly or wired. The user may control the display device 200 by inputting user commands through at least one of buttons on the remote controller, voice input, and control panel input.

[0037] In some embodiments, the smart device 300 may include any one of a mobile terminal, a tablet computer, a computer, a laptop computer, an AR / VR device, etc.

[0038] In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 may be controlled using an application running on the smart device.

[0039] In some embodiments, the smart device 300 and the display device may also be used to communicate data.

[0040] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0041] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0042] In some embodiments, the software steps executed by a step execution subject can be migrated to another step execution subject in data communication with it as needed for execution. Exemplarily, the software steps executed by a server can be migrated to a display device in data communication with it as needed for execution, and vice versa.

[0043] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive input operation instructions from the user, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200.

[0044] In some embodiments, the communication interface 130 is used for communicating with the outside, and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module.

[0045] In some embodiments, the user input / output interface 140 includes at least one of a microphone, a touch pad, a sensor, a button, or alternative modules.

[0046] Figure 3 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0047] In some embodiments, the display apparatus 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0048] In some embodiments, the controller includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output.

[0049] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0050] In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

[0051] In some embodiments, the tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0052] In some embodiments, the communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0053] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0054] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0055] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0056] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0057] In some embodiments, the object may be any one of the selectable objects, such as a hyperlink, an icon or other operable controls. Operations related to the selected object include: displaying a page, document, image, etc. connected to the hyperlink, or executing a program corresponding to the icon.

[0058] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (Random Access Memory, RAM), a ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0059] CPU processor. Used to execute operating system and application instructions stored in the memory, and to execute various applications, data and content according to various interactive instructions received from external input, so as to finally display and play various audio and video content. CPU processor may include multiple processors. For example, it includes a main processor and one or more sub-processors.

[0060] In some embodiments, the graphics processor is used to generate various graphic objects, such as at least one of icons, operation menus, and user input command display graphics. The graphics processor includes an operator, which performs operations by receiving various interactive instructions input by the user and displays various objects according to display attributes; and a renderer, which renders various objects obtained based on the operator, and the rendered objects are used to be displayed on the display.

[0061] In some embodiments, the video processor is used to receive an external video signal and perform at least one of the following video processing operations, such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis, according to a standard codec protocol of the input signal, to obtain a signal that can be directly displayed or played on the display device 200.

[0062] In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, and a display formatting module. The demultiplexing module is used to demultiplex the input audio and video data stream. The video decoding module is used to process the demultiplexed video signal, including decoding and scaling. The image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to the user input or itself with the scaled video image to generate an image signal for display. The frame rate conversion module is used to convert the input video frame rate. The display formatting module is used to receive the video output signal after frame rate conversion and change the signal to conform to the display format signal, such as outputting an RGB data signal.

[0063] In some embodiments, the audio processor is used to receive an external audio signal, perform decompression and decoding according to a standard codec protocol of the input signal, and perform at least one of noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in a speaker.

[0064] In some embodiments, the user may input a user command in a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0065] In some embodiments, "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, or control displayed on a display screen of an electronic device, wherein a control may include at least one of icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0066] In some embodiments, the user interface 280 is an interface that can be used to receive control input (such as a physical button on the display device body, or other interfaces).

[0067] In some embodiments, the system of the display device may include a kernel, a command parser (shell), a file system, and an application. The kernel, shell, and file system together constitute the basic operating system structure, which allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, schedulers, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. After startup, the application is compiled into machine code to form a process.

[0068] See also Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0069] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0070] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0071] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0072] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0073] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0074] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0075] In some embodiments, after the display device is started, the interface of the preset video-on-demand program can be directly entered. The interface of the video-on-demand program can be as follows: Figure 5 As shown in , at least a navigation bar 510 and a content display area located below the navigation bar 510 are included, and the content displayed in the content display area changes with the change of the selected control in the navigation bar. The program in the application layer can be integrated into the video-on-demand program and displayed through a control in the navigation bar, or further displayed after the application control in the navigation bar is selected.

[0076] In some embodiments, after the display device is started, it can directly enter the display interface of the last selected signal source, or the signal source selection interface, where the signal source can be a preset video on demand program, or at least one of an HDMI interface, a live TV interface, etc. After the user selects a different signal source, the display can display the content obtained from different signal sources.

[0077] The above embodiments introduce the hardware / software architecture and function implementation of the display device. For a terminal that is the same or similar to the display device, in an exemplary application scenario, such as Figure 6 For example, a dedicated recommendation page can be created in the application platform, for example, controls such as "recommendation" and "guess you like" can be added to the navigation bar 510 in the interface of the video on demand program. Figure 6 By clicking the first recommendation control 61 in the image, the user can enter the recommendation page 60, which displays the recommendation information 62 that meets the user's interest preferences and is predicted and output by the recommendation system after big data modeling.

[0078] In an exemplary implementation, the recommendation page can be displayed according to a preset template, which includes the number, priority, and layout of the recommended information, so that the recommendation page is displayed to the user according to the preset format, and the user can switch the recommended information 62 through operations such as turning pages and sliding.

[0079] In an exemplary implementation, a focus can be set, and the focus can move synchronously with the action of the user switching recommended information, so that the user can accurately locate the currently selected target recommended information. Optionally, the recommended information with focus generally needs to be processed in a striking manner, such as highlighting the recommended information, appropriately enlarging the recommended information, etc., so that the focus and non-focus have a more prominent contrast, which is convenient for users to distinguish.

[0080] In an exemplary implementation, Figure 6 The recommended information 62 in the example is taken as an example, and each recommended information 62 may include a preview image, video information, and a play control, etc. Among them, the preview image is a frame of image randomly selected from the video and processed as a thumbnail. The preview image can be a preset frame, or a frame randomly extracted from the video when generating the recommended information; the video information includes but is not limited to the video duration, video name, video content introduction, etc., so that the user can quickly understand the brief information of the recommended video and filter out the target video that he or she wants to watch; the play control is linked to the overall recommended information 62. For example, when the user clicks anywhere on the target recommended information, or when the user clicks the confirmation button when the focus jumps to the target recommended information, the play control can be triggered to jump to the play interface of the target video, thereby starting to play the target video.

[0081] In another exemplary application scenario, a user clicks to watch a video, and the current video can be played in full screen mode or non-full screen mode. Figure 7 and Figure 8 All videos are played in small window mode. In addition to the video screen, the video playback interface also displays a progress bar, elapsed time / video length, and is equipped with play / pause controls, speed adjustment controls, image quality adjustment controls, download controls, sharing controls, favorite controls, and window adjustment controls. The window adjustment is used to switch the window from a small window to full screen, or from full screen to a small window. Figure 7 In the example form, the current video is played on the left side of the screen, and a recommendation panel 70 is provided on the right side of the screen. The recommendation panel 70 displays various recommended information predicted and output by the recommendation system. For example, the recommendation panel 70 can display various recommended information in columns, and the user can switch the recommended information by sliding up and down. Figure 8 In another example, the recommendation section 70 can be displayed in rows below the video, and the user can switch the recommendation information by sliding left and right. This embodiment enables the user to simultaneously browse the videos recommended by the application platform while watching the current video.

[0082] In 7(b), the video is played in full screen mode, see Fig. 9 In the sub-picture (a), a second recommendation control 71 can be set in the full-screen playback interface. When the user triggers the second recommendation control 71, the UI changes to Fig. 9 The effect of the sub-picture (b) of FIG. 70 is that the recommendation panel 70 is displayed without exiting the full screen mode, and the recommendation panel 70 can be popped up at a preset position. Optionally, the recommendation panel 70 can also be hidden by a specified operation, such as the user triggering the recommendation control again, the user double-clicking the video interface, or the user not responding after the recommendation panel 70 is displayed for a preset time, etc. The display and hiding method of the recommendation panel 70 is not limited to this embodiment.

[0083] In an exemplary implementation, taking the video in non-full screen mode as an example, in the form of non-sliding switching of recommended information, the recommended section 70 can display a limited amount of recommended information. In order to facilitate users to quickly switch and browse recommended information, the UI can set switching modes such as page turning, batch switching, etc. Fig.10 In the sub-figure (a), for example, a change batch control 72 can be set in the recommendation section 70. After the recommendation system sorts its prediction results, the short video information ranked in the first N positions will form a first batch of recommendation information sets, and the first batch of recommendation information sets will be loaded and displayed in the recommendation section 70, where N is the maximum number of recommended videos allowed in a single batch by the recommendation section 70; if the user does not browse the video he wants to watch in the first batch of recommendation information sets, the change batch control 72 will be triggered, and the recommendation information will form a second batch of recommendation information sets with the short video information ranked in the N+1th to 2Nth positions, and then the recommendation section 70 will be refreshed, and the UI can be changed to the following Fig.10 As shown in the sub-figure (b), the recommendation section 70 displays the second batch of recommended information sets. Fig.10 Taking N equals 4 as an example, the first batch of recommended information sets displays the short video information from 1st to 4th place. After the user triggers "change a batch", the recommendation section is refreshed to display the short video information from 5th to 8th place.

[0084] In an exemplary implementation, after the first batch of recommended information sets are displayed, the cumulative number of times the user triggers the change batch control 72 is counted, and the count value is recorded as M, where M is greater than or equal to 1. When the recommendation system receives the Mth change batch operation instruction, it extracts the short video information sorted from the (M*N+1)th to the (M+1)*Nth positions to form the M+1th batch (including the first batch) of recommended information sets, and refreshes the recommendation section 70 to switch the recommended information set from the original Mth batch to the M+1th batch. Figure 6 The example recommendation page can also use a mode of quickly switching recommended information by changing a batch / turning pages and other similar operations, which will not be described in detail.

[0085] The above are various implementation modes of displaying recommendation information on the terminal after the recommendation system outputs the recommendation information. The UI and its operation process provided in this application are merely exemplary and shall be subject to the design and application of the actual product. The mode of displaying recommendation information is not limited to the examples in this application.

[0086] In some terminal application scenarios, for example, the application platform can provide long video sources, such as some movies, TV series, variety shows, e-sports games, etc., and can also support users to upload short video resources by themselves. For example, users can edit the highlights of long videos, or short videos shot or recorded on the terminal, and publish them with the help of the same application platform and share them with other users for viewing. For newly registered users in the application platform or users with inactive video viewing behavior, the recommendation system cannot accurately locate their interest preferences. For the short videos newly released by users, the short videos have not been or are rarely watched by users, resulting in the recommendation system being unable to accurately match which users may like the short videos. These are the "cold start" problems that the recommendation system often faces. For another example, an operator previously developed and operated a long video application platform, and later planned to develop a new short video application platform, which involves the expansion of scenarios and fields. The newly developed short video application platform will inevitably encounter cold start problems.

[0087] The recommendation algorithms currently used in recommendation systems rely more on long-term data accumulation, and the recommendation effect is insufficient in the short term or when encountering cold start problems. Although conventional transfer learning algorithms may overcome cold start, they require that the features of the training model in the new and old fields are completely consistent, which has certain limitations in cross-domain recommendations. In this regard, the present application provides a recommendation model obtained by training after migration and fusion based on the similarity between long videos and short videos using the prior knowledge of long videos. The recommendation model is applied to the recommendation system of the terminal to achieve the cross-domain recommendation of short videos using long videos, thereby solving the cold start problem of short video recommendations. The migrated training model is more adaptable to the requirements of new scenarios and new fields.

[0088] In an exemplary implementation, before the recommendation system is put into use, the recommendation model is pre-trained. Fig.11 The training logic of the example recommendation model mainly includes three aspects:

[0089] First, the first sorting model F1 is trained based on the user characteristics of each user, the behavior label of the long video, and the long video characteristics.

[0090] In an exemplary implementation, user features can be extracted from the personal information filled in by the user when registering, such as user name / user ID, location, age and other user information features required for video recommendation. Behavior records are generated when users show interest in long videos, such as users clicking to watch long videos, downloading, collecting or sharing long videos, etc., where the behavior of users actually watching long videos can better express the user's interest preferences. The behavior label is the indicative information contained in the behavior record, such as indicating whether a long video has been watched or not. In order to achieve transfer learning from the old domain to the new domain and make the transferred model better adapt to the new domain, the long video features used in calculating the first sorting model use the common features of long and short videos, rather than the difference features, such as the common features of video length, video title, video classification, etc., and the difference features, such as the director, actor and other cast and crew information of long videos, and the publisher information of short videos.

[0091] Secondly, the target long video feature with the highest similarity to each short video feature is calculated, and the behavior label corresponding to the target long video feature is transferred to the short video feature, so as to train the second sorting model F2.

[0092] In an exemplary implementation, the display device can maintain a long video collection according to the storage and out-of-stock status of the long video. Where i is used to represent the sequence number of the long video, n_L is the number of videos included in the long video set, and itemL i Used to represent the i-th long video.

[0093] In an exemplary implementation, the display device can maintain a short video collection based on the release and withdrawal of short videos. Where j is used to represent the serial number of the short video, n_S is the number of videos included in the short video set, and itemS j Used to represent the j-th short video.

[0094] In an exemplary implementation, each itemS is calculated j Each item in the long video set i , and filter out items with itemS j The target long video itemL with the highest similarity j0 , where j0 represents the sequence number of the target long video in the long video set, thereby constructing the migration association between long and short videos (itemS j ,itemL j0 ), that is, before migration, (u k ,itemL j0 ,label k,j0 ), after migration, it is (u k ,itemS j ,label k,j0 ), where (u k ,itemL j0 ,label k,j0 ) is used to represent user u k Target long video iteml j0 The generated behavior label label k,j0 , label k,j0 ∈{0,1}, when label k,j0 =0 indicates that user u k Target long video iteml not viewed j0 , when label k,j0 =1 indicates that user u k Watched target long video iteml j0 In this way, a behavioral preference migration association is established between the short video and its most similar target long video, thereby achieving recommendation prediction for short videos in unknown fields and solving the cold start problem of short video recommendation.

[0095] In a third aspect, the first sorting model F1 and the second sorting model F2 are weightedly fused to generate and store the recommendation model y.

[0096] The recommendation model y=w1*F1+w2*F2, where w1 is the first weighting coefficient, representing the weight of the first ranking model F1, and w2 is the second weighting coefficient, representing the weight of the second ranking model F2. w1 and w2 can be adjusted according to actual requirements. For example, if the ranking model after migration is more emphasized, w2 is relatively increased and w1 is relatively decreased; if the old knowledge is more emphasized, w1 is relatively increased and w2 is relatively decreased.

[0097] In some embodiments, the weighted fusion recommendation model can output a comprehensive recommendation score, which can be used to evaluate the probability and possibility of a short video being clicked and watched. For example, the prediction result of the recommendation model y is exemplarily expressed as (Score p ,itemS p ), where p represents the short video information itemS to be recommended in the prediction result p Score p Indicates short video information itemS p The recommendation score of the short video information is generally higher, and the higher the recommendation score is, the higher the priority of the short video information being recommended is. Therefore, the recommendation system can sort the prediction results according to the score and give priority to displaying the short video information with higher scores to improve the matching degree between the short video recommendation and the user's preference.

[0098] This application will be Fig.11 The training logic of the recommendation model shown in FIG. 1 is described in more detail. In some exemplary implementations, it is assumed that the user set Where n represents the number of users included in the user set. In actual user set management, each time a new user is registered on the application platform, the user information is added to the user set. Short video collection The user's behavior label data for the long video set is expressed as {(u i ,itemL j ,label i,j )}i=1,2…n; j=1,…n_L, label i,j ∈{0,1}, when label i,j =0 indicates that user u i No long videos have been watched j , when label i,j =1 indicates that user u i Watch too long videoitemL j That is, for transfer learning, long videos belong to a known old field with accumulated prior knowledge, while short videos belong to an unknown new field.

[0099] In some exemplary implementations, long videos and short videos have the same feature intersection, i.e., common features. For example, both long and short videos involve information features such as video duration, video title, video classification, and video content introduction. Long and short videos also have different features, i.e., non-intersection features. For example, long videos have information about directors, actors, and other cast members, while short videos have information about publishers and long transmission time. L =V1∪V2, short video feature set f S =V2∪V3, where V2 is the common feature of long and short videos, V1 is the difference feature contained in the long video, and V3 is the difference feature contained in the short video, that is, V2=f L ∩f S , V1=f L -f S , V3=f S -f L .

[0100] In some exemplary implementations, in order to facilitate subsequent model training and calculation, all features need to be vectorized. Optionally, for discrete features, one-hot, multi-hot and other encoding forms can be used. After encoding, each dimension can be regarded as a continuous feature. For dense features, they are normalized and then pieced together into a dense vector. The vectorization processing method is not limited to this. The feature vectors of V1, V2 and V3 are represented as:

[0101]

[0102]

[0103]

[0104] Among them, q, l and m are the dimensions of V1, V2 and V3 feature vectors, and the element value of any dimension in the above vectors is in [0,1]. After feature vectorization, the similarity between the long and short videos is calculated. Optionally, the cosine similarity of the long and short videos is calculated according to the following formula:

[0105]

[0106] In the above formula Centralize itemL for long videos i The common eigenvector of , where i=1,…n_L; ItemS is a short video collection j The common eigenvectors of , j = 1, ... n_S, for The element value of the tth dimension in , for The element value of the t-th dimension in , t = 1, ... l; sim(i, j) is item L i With itemS j The cosine similarity of .

[0107] According to the long video set and the short video set, we need to calculate n_L*n_S times cosine similarity. Since n_L and n_S can reach millions or even tens of millions in actual scenarios, the amount of calculation is large. In order to improve the calculation and training rate, in an optional solution, the LSH (Locality Sensitivity Hashing) algorithm is used to find the item S with each short video. j The target long video itemL with the highest similarity j0 . The basic idea of ​​LSH is to divide each vector into several segments (called bands) based on the signature vector obtained by Min Hashing. Each band contains multiple rows. If one or more bands of two vectors are the same, the two vectors may have a high similarity. The more the same bands are, the greater the possibility of high similarity. Therefore, LSH can perform hash bucketing on each band of the signature vector of each short video. Objects that are divided into the same bucket on any band are candidate similarity sets. In this way, it is only necessary to calculate the cosine similarity of all candidate similarity sets to find the similar long video set corresponding to each short video, and obtain the long video with the greatest similarity to the short video from the similar long video set as the target long video. LSH can use the bucketing mode to divide potentially similar objects into the same bucket with a high probability, so that each candidate similarity set will be relatively small, realizing the rapid retrieval and matching of similar data from massive data, and reducing the computational complexity.

[0108] After matching each short video itemS j The target long video itemL with the highest similarity j0 After that, the user's behavior on the long video is (u k ,itemL j0 ,label k,j0 ), label k,j0 ∈{0,1}, when label k,j0 =0 indicates that user u k Target long video iteml not viewed j0 , when label k,j0 =1 indicates that user u k Watched target long video iteml j0 , through the interest transfer between the most similar long and short videos, obtain the short video itemS j With user uk The behavioral interaction between k ,itemS j ,label k,j0 ), which is the behavior label data assigned to each user in the new field of short videos, and is also the key data required for the subsequent calculation of the second ranking model F2.

[0109] In some exemplary implementations, user features of each user are obtained according to the user set, the user features including but not limited to user name / user ID, location, age, gender, etc., and the user features are vectorized to obtain a user feature vector Where k represents the user serial number, k = 1, ... n, r is the dimension of the user feature vector, U k The element value of any dimension in the vector ∈[0,1]. Short video feature set f S =V2∪V3, the short video features are vectorized as Where j indicates the short video sequence number, j = 1, ... n_S, Vs j The behavior label data to be migrated is label k,j , label k,j ∈{0,1}, when label k,j =0 indicates user u k Not watched Vs j The corresponding short video, when label k,j =1 indicates user u k Watched Vs j The corresponding short video. k , Vs j and label k,j Combined into the second vector U2, that is Using U2 as a training sample, the second sorting model F2 is trained using algorithm models such as deep learning and neural networks.

[0110] In an optional solution, the DCN (Deep Cross Network) algorithm can be used to train the CTR (Click-Through-Rate) prediction model to obtain the second ranking model F2. The bottom layer of DCN is the Embedding and stacking layer, which includes two functions: Embed and Stack. The Embed function mainly processes discrete features (such as categorical features) and converts discrete features into dense vectors of real values. The stack function is used to process continuous features. After normalizing the continuous features, they are stacked with the vectors of the embedding layer to form the original input vector. The second vector U2 can be generated in the Embedding and stacking layer of DCN and transmitted to the upper Cross Network & Deep Network for calculation.

[0111] The middle layer of DCN includes parallel Cross Network & Deep Network. Cross Network is the core of DCN. Cross Network is used to efficiently apply explicit cross features. Since Cross Network has relatively few parameters, its expression ability is limited. In order to learn high-order nonlinear combination features, DCN introduces parallel DeepNetwork. Deep Network is a fully connected feedforward neural network, and the number of layers can be set appropriately. The specific training method of Cross Network & Deep Network can be described in the prior art, and this application will not repeat it.

[0112] The top layer of DCN is the Combination Layer, which is used to concatenate the output results of Cross Network and DeepNetwork, and then obtain logits after weighted summation, and then input it into the standard logistic regression function (such as sigmoid, softmax, etc.) to obtain the normalized prediction probability p2 (i.e. CTR) ∈ [0,1]. Based on the input of the second vector U2 and the DCN model training, the new model after migration is finally obtained as y2 = F2 The predicted probability p2 represents the probability of a user clicking to watch a short video predicted by model F2. Model F2 mainly focuses on the new short video scene after migration. The model uses its output probability as a reference for the recommendation score, which facilitates the final decision-making on the recommendation and distribution of short video information.

[0113] In some exemplary implementations, in order to make full use of the prior knowledge in the old domain, the model before migration can be integrated into the new domain, thereby providing another reference strategy for short video recommendation. L =V1∪V2, the long video features are vectorized and expressed as Where w indicates the long video sequence number, w = 1, ... n_L, The corresponding behavior label data is label k,w , label k,w ∈{0,1}, when label k,w =0 indicates user u k Not watched The corresponding long video, when label k,w =1 indicates user u k Viewed The corresponding long video. k , and label k,w Combined into vector U0, that is Taking U0 as the training sample and using the aforementioned DCN model for training, the original prediction model of the old domain before migration is obtained as

[0114] In some exemplary implementations, since the V1 feature is a differential feature included in the long video domain, and the short video domain does not include the V1 feature, in order to adapt the old domain model to the new domain, only the common feature V2 of long and short videos is used, and combined with the user's interactive behavior on the long video, the old model is retrained, that is, the first vector As a training sample, it is input into the middle layer of the DCN model for operation until the top layer of the DCN Combination Layer outputs the normalized prediction probability p1 (i.e. CTR) ∈ [0, 1]. Based on the input of the first vector U1 and the DCN model training, the model retrained in the old domain before migration is finally obtained as The predicted probability p1 represents the probability of a user clicking to watch a short video predicted by model F1. Model F1 mainly focuses on migrating the old knowledge in the field of long videos. The model uses its output probability to provide another reference for the recommendation score, thereby facilitating the final decision on the recommendation and distribution of short video information.

[0115] After obtaining model F1 and model F2, the two models are weighted and fused to generate the recommendation model y, that is, the recommendation model y = w1*F1+w2*F2, where w1 is the first weighting coefficient, representing the weight of the first sorting model F1, and w2 is the second weighting coefficient, representing the weight of the second sorting model F2. Both w1 and w2 can be adjusted according to actual requirements. For example, if the model after migration is more emphasized, w2 will be relatively increased and w1 will be relatively decreased; if the old domain knowledge is more emphasized, w1 will be relatively increased and w2 will be relatively decreased. After the recommendation model is generated, it is stored in the local memory. When the user requests a recommendation for a short video, the recommendation model can be called for prediction and the output result returned.

[0116] In an exemplary implementation, the recommendation model generated above is not static, but can be updated based on the dynamics of users and application platforms. For example, the parameter configuration in the recommendation model can be updated based on input variables, so that the recommendation model can continuously evolve, and new and old domain knowledge can be continuously accumulated and enriched, thereby improving the matching degree between recommendation information and user preferences.

[0117] In an exemplary implementation, the terminal can monitor and record user behavior dynamics, where the user behavior includes but is not limited to: user registration and login account (involving the update of user set), user behavior on long videos (may involve changes in old domain knowledge and label data), user behavior on short videos (involving new domain knowledge and changes in label data), and user response status to short video recommendation information, etc. The response status includes clicking or not clicking on the recommendation information, and the response status can reflect the accuracy of the recommendation and whether the recommendation information matches the user's interest preference to a certain extent. Optionally, by constructing the loss function Loss of the model, the deviation between the predicted value and the actual value is expressed, and by optimizing and calibrating the model, the loss is reduced, thereby improving the prediction accuracy of the recommendation model.

[0118] In an exemplary implementation, the terminal can also monitor the dynamics of the application platform, such as monitoring the storage and out-of-stock status of long videos in the application platform (involving the update of the long video set), the release and withdrawal status of short videos, etc. (involving the update of the short video set), where long videos and short videos can be different fields in the same application platform, or long videos and short videos belong to different application platforms. For example, an operator has developed application platform 1 corresponding to the long video field, and then can use the domain knowledge of platform 1 as a reference to develop application platform 2 corresponding to the short video field, and migrate and integrate the old domain knowledge into application platform 2, so as to realize short video recommendation in application platform 2.

[0119] In an exemplary implementation, the terminal updates the recommendation model according to the dynamic changes of user behavior and application platform. Optionally, the recommendation model is updated regularly according to the accumulation of dynamic changes of users and application platforms within a certain period of time (e.g., 12 hours, 24 hours).

[0120] It can be seen from the above technical solutions that the present application utilizes the similarity between long and short videos for transfer learning between new and old domains, which not only solves the cold start problem of short video recommendation, but also overcomes the limitation of conventional transfer learning that the characteristics of the new and old domains must be consistent. Similarity search has a certain degree of similarity recall characteristics, so the model after migration has a certain degree of recall rate, and the adaptive integration of the old domain model into the recommendation prediction of the new domain is conducive to improving the matching degree of short video recommendations. The recommendation model can adapt well to the new domain, and the recommendation model supports offline training. Therefore, the present application improves the performance of the recommendation system from many aspects.

[0121] In an exemplary implementation, based on the aforementioned video recommendation method and the UI content involved, as well as the training and generation scheme of the recommendation model, refer to Fig.12 The present application also provides a video recommendation method, which is specifically executed by a recommendation system running in a controller, and the method includes the following steps:

[0122] Step S1, when receiving a short video recommendation request from a target user, calling a recommendation model.

[0123] The recommendation model is a training model obtained by migrating and integrating the prior knowledge of long videos based on the similarity between long videos and short videos. The training and generation methods of the recommendation model have been described in detail in the previous article and will not be repeated here.

[0124] Step S2, controlling the recommendation model to predict and output short video recommendation information matching the target user.

[0125] Optionally, the recommendation model automatically sorts the prediction results according to a preset priority rule to obtain the short video recommendation information; the prediction result includes at least one short video information. The priority rule is, for example, sorting by prediction probability, sorting by short video popularity, sorting by short video release time, etc. The priority rule can be set according to actual requirements.

[0126] Optionally, the recommendation model automatically sorts the prediction results according to the recommendation score, for example, in descending order, that is, the recommendation scores are sorted from high to low. If the recommendation information is switched by a similar method such as page turning, batch switching, etc., then according to the preset number N of short videos that can be recommended in a single batch / single page, and the number of batch switching / page turning operations currently triggered cumulatively, N short video information of the corresponding order is extracted to form the M+1th batch / page of short video recommendation information (including the first batch), and the displayed batch / page of the recommendation page / recommendation section 70 is refreshed.

[0127] Step S3, controlling the display to display the short video recommendation information. The display method and UI of the short video recommendation information have been described in detail above and will not be repeated here.

[0128] Under the understanding of the recommendation model training mechanism of transfer learning and fusion of long videos → short videos in this application, the model algorithm adopted is not limited to DCN, and other algorithms suitable for recommendation systems can also be adopted, such as FM (Factorization Machines) model, Wide & Deep model, PNN (Product-based Neural Network) model, NFM (Neural Factorization Machine) model, AutoInt, etc. These model algorithms can all refer to the existing technology. In addition, the calculation methods of parameters such as similarity and vectorization involved in the recommendation model are not limited to the examples of this application.

[0129] This application mainly focuses on video recommendation, that is, the target type is video, the old field in transfer learning is long video, and the new field is short video. As an expansion and extension of the application scenario, the old and new fields may also involve other target types. For example, if the target type is audio, the old field in transfer learning is songs, and the new field is ringtones. Based on the recommendation mechanism of this application, according to the user's listening behavior, terminal ringtones that meet their interests and preferences can be recommended to users. The change of the target type does not affect the training and prediction mechanism of the recommendation system. The difference is only reflected in the different training sample data required by the model.

[0130] In one embodiment, the present invention further provides a computer storage medium, which may store a program. When the computer storage medium is located in a display device, the program may include program steps involved in the video recommendation method and the recommendation model training method that the controller is configured to execute. Among them, the computer storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0132] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A video recommendation method, characterized in that: include: When receiving a short video recommendation request from a target user, calling a recommendation model, wherein the recommendation model is a training model obtained by weighted fusion of the first sorting model and the second sorting model according to an adjustable first weighting coefficient and a second weighting coefficient; Controlling the recommendation model to predict and output short video recommendation information matching the target user; Display the short video recommendation information; The first ranking model is obtained by training a click-through rate prediction model using a deep cross network algorithm with a first vector as a training sample, wherein the first vector is obtained by extracting common features with short video features from long video features, user features, and behavior labels, and then combining them after vectorization processing, wherein the behavior label is used to indicate whether the user clicks to watch the long video; The second ranking model is obtained by training a click-through rate prediction model using a deep cross network algorithm using a second vector as a training sample, wherein the second vector is formed by vectorizing and combining the short video feature, the user feature, and the behavior label of the target long video feature migration, and the target long video feature is a long video feature with the highest similarity to each short video feature; The first weighting coefficient is used to represent the proportion of the recommendation model that emphasizes the prior knowledge in the old domain; the second weighting coefficient is used to represent the proportion of the recommendation model that emphasizes the knowledge in the new domain after migration.

2. The method according to claim 1, characterized in that: Before outputting the short video recommendation information matching the target user, the method further includes: The recommendation model is controlled to prioritize the prediction results to obtain the short video recommendation information; the prediction results include at least one short video information.

3. The method according to claim 1, characterized in that: After controlling the recommendation model to predict and output short video recommendation information matching the target user, the method further includes: Obtain prediction results output by the recommendation model after being sorted in descending order of scores, wherein the prediction results include at least one short video information; The short video information ranked in the top N positions is extracted to form the first batch of short video recommendation information; wherein N is a preset number, which is used to represent the maximum number of short videos allowed to be recommended in a single batch on the recommendation page.

4. The method according to claim 3, characterized in that The method further comprises: When receiving the Mth batch change operation triggered by the target user, extract the prediction results ranked from the (M*N+1)th to the (M+1)*Nth positions to form the M+1th batch of short video recommendation information; where M is the count value of the cumulative number of times the user triggers the batch change control after displaying the first batch of recommendation information sets; Switch the short video recommendation information displayed on the recommendation page from the Mth batch to the M+1th batch.

5. The method according to claim 1, characterized in that The method further comprises: Monitor user registration behavior, the entry and exit status of long videos, the release and withdrawal status of short videos, the user's response status to the short video recommendation information, and the user's behavior records for each long and short video in the library, and update the recommendation model.

6. A recommendation model training method, characterized in that: include: Extract common features between long video features and short video features; Vectorizing the common features, user features, and behavior labels, and combining them into a first vector; wherein the behavior label is used to indicate whether the user clicks to watch the long video; Using the first vector as a training sample and using a deep cross network algorithm, a click rate prediction model is trained to obtain a first ranking model; Calculate the target long video feature with the highest similarity to each short video feature; Vectorizing the short video feature, the user feature, and the behavior label of the target long video feature migration, and combining them into a second vector; Using the second vector as a training sample and using a deep cross network algorithm, a click rate prediction model is trained to obtain a second ranking model; According to the adjustable first weighting coefficient and the second weighting coefficient, the first sorting model and the second sorting model are weightedly fused to generate and store a recommendation model; wherein the first weighting coefficient is used to characterize the proportion of the recommendation model that emphasizes the prior knowledge in the old field, and the second weighting coefficient is used to characterize the proportion of the recommendation model that emphasizes the knowledge in the new field after migration.

7. The method according to claim 6, characterized in that The method further comprises: Monitor user registration behavior, the entry and exit status of long videos, the release and withdrawal status of short videos, the user's response status to short video recommendation information, and the user's behavior records for each long and short video in the library, and update the recommendation model.

Images