Content recommendation method, device, equipment, storage medium and product

Abstract

The present application discloses a content recommendation method, apparatus, device, storage medium and product, which are applied to a first recommendation system and belong to the field of artificial intelligence technology. The above method includes: obtaining an external signal corresponding to a multimedia object, the above external signal is the result obtained by predicting the recommendation information of the above multimedia object based on the content recommendation data in at least one second recommendation system, and any of the above second recommendation systems is a recommendation system different from the above first recommendation system; extracting modal information of at least one dimension of the above multimedia object to obtain modal feature information corresponding to the above multimedia object; determining the recommendation strategy of the above multimedia object based on the above external signal and the above modal feature information; and recommending the above multimedia object based on the above recommendation strategy. In the technical solution provided in the embodiment of the present application, the recommendation effect can be significantly improved by introducing external signals.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a content recommendation method, apparatus, device, storage medium, and product. Background Art

[0002] In the era of rapid development of Internet technology, in the process of content recommendation, there is a continuous increase in content entering the internal recommendation pool (storage). In particular, in order to achieve good recommendation effects on a platform, high-quality content needs to continuously improve the quality of recommendations. This requires continuous improvement in the ability to evaluate content quality. However, the recommendation quality of relevant technologies still needs to be improved, especially in some special cases, where the recommendation quality has seriously declined, such as the content cold start process that is essential for recommendation.

[0003] During the cold start phase, related technologies primarily achieve this goal by tagging incoming content, such as categories, tags, and thematic information, exploring and testing the interests of content consumers, and adjusting recommendation strategies based on their feedback and a posteriori behavioral data. However, this approach requires gradually expanding the scope of recommendations through a certain amount of traffic, which takes a long time and is not very effective. Related technologies also lack a fast and accurate cold start solution for new content consumers, resulting in poor recommendation results during the cold start phase. Summary of the Invention

[0004] The embodiments of the present application provide a content recommendation method, apparatus, device, storage medium, and product, which can improve the accuracy of content recommendation and optimize the recommendation effect, and are particularly suitable for quickly performing content recommendation in cold start scenarios.

[0005] According to one aspect of an embodiment of the present application, a content recommendation method is provided, which is applied to a first recommendation system. The method includes:

[0006] Obtaining an external signal corresponding to a multimedia object, the external signal being a result obtained from a process of predicting content distribution quality for the multimedia object, the content distribution quality prediction being performed based on content recommendation data in at least one second recommendation system, where any second recommendation system is a recommendation system different from the first recommendation system;

[0007] Extracting modal information of at least one dimension from the multimedia object to obtain modal feature information corresponding to the multimedia object;

[0008] Obtaining a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modality feature information;

[0009] The multimedia object is recommended according to the recommendation strategy.

[0010] According to one aspect of an embodiment of the present application, a content recommendation device is provided, which is applied to a first recommendation system. The device includes:

[0011] an external signal acquisition module, configured to acquire an external signal corresponding to a multimedia object, the external signal being a result obtained from a process of predicting content distribution quality for the multimedia object, the content distribution quality prediction being performed based on content recommendation data in at least one second recommendation system, any second recommendation system being a recommendation system different from the first recommendation system;

[0012] a multimodal information acquisition module, configured to extract modal information of at least one dimension from the multimedia object to obtain modal feature information corresponding to the multimedia object;

[0013] a recommendation strategy determination module, configured to obtain a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modal feature information;

[0014] A recommendation module is configured to recommend the multimedia object according to the recommendation strategy.

[0015] According to one aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-mentioned content recommendation method.

[0016] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-mentioned content recommendation method.

[0017] According to one aspect of an embodiment of the present application, a computer program product is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the above-described content recommendation method.

[0018] The technical solutions provided in the embodiments of the present application can bring the following beneficial effects:

[0019] The embodiments of the present application can effectively utilize data from other external recommendation systems, such as content distribution data, prior information about content and authors, to generate external signals for evaluating the content distribution quality of multimedia objects, thereby constructing a comprehensive content quality evaluation method at the semantic and perceptual levels, making multimedia object recommendations more scientific and reasonable, and helping the recommendation system to cold-start new incoming content and content consumption objects, avoiding the inefficiency of multiple and repeated explorations. In addition, the recommendation method can help the recommendation system better utilize recommended traffic, and the content distribution process can sort content based on the above-mentioned content distribution quality, weight and de-weight content, accelerate the distribution of high-quality content and reduce the distribution of low-quality content, and ultimately improve the content consumption experience of content consumers and enhance consumer stickiness, thereby improving product retention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 is a schematic diagram of an application program operating environment provided by an embodiment of the present application;

[0022] Figure 2 This is a flowchart of a content recommendation method provided by an embodiment of the present application;

[0023] Figure 3 An exemplary diagram of the architecture of a neural network for determining content quality distribution indicators is shown;

[0024] Figure 4 The following is an exemplary diagram of the information flow content service system architecture;

[0025] Figure 5 The following is a block diagram of a content recommendation device;

[0026] Figure 6 This is a structural block diagram of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] Before introducing the method embodiments provided in the present application, a brief introduction is first given to the relevant terms or nouns that may be involved in the method embodiments of the present application to facilitate understanding by those skilled in the art in the field of the present application.

[0028] NetVLAD (Net Vector of Locally Aggregated Descriptors) is a scene recognition algorithm that improves on VLAD (Vector of Locally Aggregated Descriptors). The VLAD algorithm uses the scale-invariant feature transform (SIFT) or similar algorithms as its basis to encode the extracted features to obtain a shorter feature string. NetVLAD uses a convolutional neural network as its basic feature extraction structure and connects it to this network to achieve end-to-end training.

[0029] FAISS (Clustering and Similarity Search Library) is an open-source library for clustering and similarity search. It provides efficient similarity search and clustering for dense vectors, supporting searches on billions of vectors. It is currently the most mature approximate nearest neighbor search library. It includes multiple algorithms for searching vector sets of any size, as well as support code for algorithm evaluation and parameter tuning.

[0030] BERT (Bidirectional Encoder Representation from Transformers) is a large-scale text pre-training model. Using a 12-layer transformer encoder, BERT significantly improves baseline performance in natural language processing tasks. Compared to word2vec (word embeddings), BERT, pre-trained on massive amounts of text, can incorporate more transferable knowledge into classification algorithms, providing more accurate text features.

[0031] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0032] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0033] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0034] Deep learning: The concept of deep learning originates from the study of artificial neural networks. A multilayer perceptron with multiple hidden layers is an example of a deep learning architecture. Deep learning discovers distributed feature representations of data by combining lower-level features to form more abstract higher-level representations of attribute categories or features.

[0035] Computer Vision Technology (CV) Computer vision is a science that studies how to make machines "see". To put it more concretely, it means using cameras and computers instead.

[0036] Machine vision involves the human eye identifying and measuring objects, and then further processing them through graphics to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0037] Key technologies in speech technology include automatic speech recognition (ASR), text-to-speech (TTS), and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech becoming one of the most promising methods of human-computer interaction.

[0038] Natural language processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to linguistics. NLP technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0039] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0040] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used flexibly and conveniently on demand. Cloud computing technology will become a key support. The backend services of technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark, which will need to be transmitted to the backend system for logical processing. Different levels of data will be processed separately. All kinds of industry data require strong system support, which can only be achieved through cloud computing.

[0041] Social networks: Social networks originated from online social networking, which began with email. The internet is essentially a network of computers. Early email solved the problem of long-distance email transmission and remains the most popular internet application to this day. It also marked the beginning of online social networking. BBS (Bulletin Board System) took it a step further, normalizing "group messaging" and "forwarding," theoretically enabling the ability to publish information and discuss topics to anyone, becoming a platform for the spontaneous generation of early internet content.

[0042] BBSs advanced online social networking, moving from simply reducing the cost of point-to-point communication to reducing the cost of face-to-face communication. Instant messaging (IM) and blogs are more like upgraded versions of these two social tools. The former improves immediacy (transmission speed) and the ability to communicate simultaneously (parallel processing); the latter begins to embody theories from sociology and psychology: information publishing nodes begin to embody a growing sense of individuality, as scattered information across time can be aggregated and become the "image" and "personality" of the information publishing node. As online social networking quietly evolved, a person's online profile became more complete, and this is when social networks emerged. Social networks encompass all forms of online services centered on human social interaction. The internet is an interactive platform for mutual exchange, communication, and participation. Social networks have expanded the internet from a platform for research, education, and business applications to a tool for human social interaction. Online social networking has even expanded its scope to mobile platforms. Leveraging the ubiquity of mobile phones and the availability of wireless networks, and utilizing various dating, instant messaging, and email software, mobile phones have become the new vehicle for social networking.

[0043] In the era of self-media, diverse voices emerge from all directions, and the voice of "mainstream media" is gradually weakening. People no longer accept being told what's right or wrong by a single "unified voice." Instead, each individual makes their own judgments based on independently acquired information. Self-media differs from information dissemination led by professional media organizations. Instead, it is an information dissemination activity led by the general public, transforming traditional "point-to-surface" communication to a peer-to-peer "point-to-point" communication concept. It also refers to a method of information dissemination that allows individuals to produce, accumulate, share, and disseminate information, combining both privacy and public access. The content structure of self-media is also unique. There's no set core; people write whatever comes to mind and share anything they find valuable. This makes viewing excellent self-media images and texts truly unique and engaging. They also maintain a strict word count, generally around 1,000 words, allowing consumers to consume content smoothly within 10 minutes, making them ideal for quick reading and consumption in the mobile age. They are often presented in a feed format for rapid refresh by consumers, and can be considered a new type of information aggregator. Subscriptions are based on friends or public figures they follow, and content is their publicly released updates. When you have a large and active group of friends, you receive constantly updated content, which is the most common form of feed flow. Time is the ultimate dimension of feed flow, as content updates are the result of continuous requests to the server. The timeline is the most primitive, intuitive, and fundamental form of feed display. This form of information flow has now become the mainstream way for content consumers to consume information and news.

[0044] Feeds: A news source, also known as source material, information flow, feed, information provider, feed, summary, source, news subscription, or web source, is a data format through which websites disseminate the latest information to content consumers. It is usually arranged in a timeline format. A timeline is the most primitive, intuitive, and basic form of a feed. A prerequisite for content consumers to subscribe to a website is that the website provides a news source.

[0045] Multimedia objects: can be considered as the carriers of "content" in the embodiments of the present application. The information flow service system recommends multimedia information entities to content consumers for reading and viewing, which may include graphic objects, picture objects, or video objects. Graphic objects are usually small vertical graphic content and / or horizontal short graphic content actively edited and published by the content consumer object corresponding to the self-media information publishing object. Videos are usually provided by PGC or UGC content producers and finally provided in the form of feeds stream.

[0046] Video: The video recommended by the information flow to the content consumer for reading, which can include small vertical videos and short horizontal videos, provided in the form of feeds stream. Short video: that is, short video, is a way of disseminating Internet content, generally a video dissemination content with a duration of less than 5 minutes that is disseminated on new Internet media; with the popularization of mobile terminals and the speeding up of the network, short, flat and fast high-traffic content has gradually gained market favor. The content in the examples of the present application uses videos or short videos as examples. It can be said without a doubt that multimedia objects in other dissemination forms can also be used as the content in the examples of the present application, so that the solutions in the examples of the present application can be used without distinction.

[0047] PGC (Professional Generated Content): An internet term referring to professionally produced content (e.g., content on video sites) and expert-produced content (e.g., content in community apps). It's also used to describe personalized content, diverse perspectives, and virtualized social connections. Also known as PPC (Professionally Produced Content).

[0048] MCN (Multi-Channel Network): is a product form of multi-channel network that combines PGC content to ensure the continuous output of content.

[0049] Server: A server program deployed on multiple groups of servers that specifically provides remote network services for terminal programs.

[0050] CNN (full name: Convolutional Neural Networks): Convolutional neural network is a type of feedforward neural network (Feedforward Neural Networks) that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. It has representational learning capabilities and can perform translation-invariant classification of input information according to its hierarchical structure.

[0051] AudioSet: Released by Google's Sound Understanding team in March 2017, it aims to provide a common large-scale evaluation task for audio event detection and a starting point for a comprehensive sound event vocabulary. AudioSet is a dataset consisting of 2 million human-labeled 10-second video audio tracks, with labels drawn from an ontology of over 600 audio event classes.

[0052] VGGish: A VGG-like model trained on a large number of AudioSet datasets generates 128-dimensional embedding features. As a feature extractor, the VGGish model converts audio input features into semantically meaningful 128-dimensional high-level feature vectors, which can be used as input for downstream models. As part of other models, VGGish can be considered a "warm start" for the lower layers of other models, which can add more layers on top of the VGGish embedding.

[0053] Social networks originated from online social networking, which began with electronic mail (e-mail). The internet is essentially a network of computers. Early e-mail solved the problem of long-distance email transmission and remains the most popular internet application to this day. It also marked the beginning of online social networking. Online forums (Bulletin Board Systems (BBSs)) took this a step further, normalizing group messaging and forwarding, theoretically enabling the ability to publish information and discuss topics to anyone. They became platforms for the spontaneous generation of early internet content. Recently, with the widespread adoption of smart devices and the rapid development of the internet, in the current context of the mobile internet, content consumers are shifting their demand for information from the era of text and images to the era of video. Taking short videos as an example, they may gradually become one of the dominant content formats on the mobile internet, replacing text and images to a certain extent and gradually gaining a dominant position in text-based media such as news and social media platforms. This content is typically displayed in the form of feeds for consumers to quickly refresh. For example, a new type of video aggregator can be set up, where the feed is based on the consumer's friends or people they follow, and the content is their publicly posted updates. When you have a large number of active friends, you can receive constantly updated content. This is the most common form of feed.

[0054] Short videos refer to frequently pushed video content, ranging from a few seconds to a few minutes, that is played on various new media platforms and is suitable for viewing on the go or during short periods of leisure time. Due to its short duration, it can be produced as a standalone film or as a series. Unlike micro-films and live broadcasts, short video production does not require the same specific expression format or team configuration as micro-films. It boasts a simple production process, a low barrier to entry, and strong participation, while also possessing greater dissemination value than live broadcasts. The emergence of short videos has enriched the forms of native advertising on new media platforms. Currently, the optimization of each link in the discovery, consumption, and production of short videos is constantly iterating, and short videos have become a key means of dissemination for content startups and social media platforms. Both producers and consumers of short video content have become a vast community.

[0055] As the barrier to entry for content production decreases, video uploads are growing exponentially. Video has become the primary entertainment choice for most internet consumers, occupying the majority of their entertainment time. Faced with such a massive video influx, how to help consumers filter out the most engaging clips from lengthy videos and improve their perceived quality is crucial. This not only impacts video distribution and accelerates the cold start of content-based applications, but also reduces the demand for discovery traffic, improves the viewing experience, and enhances consumer engagement—a crucial issue.

[0056] In the related art, during the video content recommendation process, content continuously enters the recommendation pool. In particular, for high-quality content to achieve good recommendation results on a platform, a content cold start process is required. Cold start distribution in recommendation systems is a significant issue. Cold start in related art primarily involves exploring and testing the interests of content consumers based on the tags of newly entered content, such as categories, tags, and subject information, while also adjusting recommendation strategies based on feedback from content consumers and a posteriori content consumer behavior data. This involves gradually expanding the scope of recommendations through a certain amount of traffic. Using conventional exploration and cold start methods, it takes a long time for a recommendation system to achieve an effective cold start for content consumers and content, as well as accelerated expansion of high-quality content. Furthermore, related art rarely considers the current state of multi-platform content distribution and fails to utilize the horizontal information of content distribution across multiple platforms. In other words, it fails to utilize external signals generated by other recommendation systems compared to its own recommendation system to accelerate cold start, resulting in reduced cold start and content distribution efficiency.

[0057] In fact, taking the cold start of a new content consumer as an example, the embodiment of the present application proposes that in order to better obtain the interest points of the content consumer, a weighted exploration can be performed using high-quality content that has been determined in advance and has been proven, so that the interest of the content consumer can be better obtained and the conversion can be improved. If the distributed content can be rated in advance based on the prior perception of the content consumer during this cold start process, the cold start efficiency will be effectively improved. This view is proposed taking into account the current situation that content will not be released in only one recommendation system, so the content released on different platforms will have a certain order and will be consumed by different content consumers. Taking the above into full consideration, in the embodiment of the present application, the posterior data of the distribution of the same or similar content on different platforms can be fully utilized, and then combined with the prior feature information of the content itself (such as the clarity of the content, the popularity of the characters, the author's authority certification, etc.) to introduce external signals, and use the high-quality content that the content consumer has already consumed to construct a quality model of the content consumer's perception of the distributed content to evaluate the quality of the content of its own recommendation system. According to the evaluation results, the cold start of the content can be significantly accelerated and the recommendation distribution efficiency can be improved.

[0058] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0059] Please refer to Figure 1 , which shows a schematic diagram of an application program running environment provided by an embodiment of the present application. The application program running environment may include: a terminal 10 and a server 20.

[0060] The terminal 10 includes, but is not limited to, electronic devices such as mobile phones, computers, intelligent voice interaction devices, smart home appliances, car terminals, game consoles, e-book readers, multimedia playback devices, wearable devices, etc. The terminal 10 may be installed with a client of an application.

[0061] In the embodiment of the present application, the above-mentioned application can be any application that can provide information flow content services. Typically, the application is a video application. Of course, in addition to video applications, other types of applications can also provide information flow content services. For example, news applications, social applications, interactive entertainment applications, browser applications, shopping applications, content sharing applications, virtual reality (VR) applications, augmented reality (AR) applications, etc., which are not limited in the embodiment of the present application. In addition, for different applications, the videos they push will also be different, and the corresponding functions will also be different. This can be pre-configured according to actual needs, which is not limited in the embodiment of the present application. Optionally, a client of the above-mentioned application is running in the terminal 10. In some embodiments, the above-mentioned information flow content service covers many vertical content such as variety shows, movies and TV, news, finance, sports, entertainment, games, etc., and content consumers can enjoy content services in many forms such as articles, pictures, small videos, short videos, live broadcasts, special topics, columns, etc. through the above-mentioned information flow content service.

[0062] The server 20 is used to provide background services for the client of the application in the terminal 10. For example, the server 20 can be the background server of the above-mentioned application. The server 20 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the server 20 provides background services for applications in multiple terminals 10 at the same time.

[0063] Optionally, the terminal 10 and the server 20 may communicate with each other via a network 30. The terminal 10 and the server 20 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0064] Please refer to Figure 2, which shows a flowchart of a content recommendation method provided by an embodiment of the present application, which is applied to the first recommendation system. The method can be applied to a computer device, which refers to an electronic device with data calculation and processing capabilities. For example, the execution subject of each step can be Figure 1 The server 20 in the application running environment shown. The method may include the following steps:

[0065] S101, obtaining an external signal corresponding to a multimedia object, wherein the external signal is a result obtained during a content distribution quality prediction process for the multimedia object, wherein the content distribution quality prediction is performed based on content recommendation data in at least one second recommendation system, and any of the second recommendation systems is a recommendation system other than the first recommendation system.

[0066] It should be noted that any content related to the second recommendation system in the embodiments of the present application is obtained under the premise of compliance with laws and regulations and the rules of the second recommendation system.

[0067] Optionally, the multimedia object may be a carrier of content entering the first recommendation system, such as a graphic object, a video object, a short video object, an audio object, and the like.

[0068] The external signal in the embodiment of the present application is a general term for information generated for the multimedia object based on the content recommendation data of at least one second recommendation system. In the embodiment of the present application, the second recommendation system is a recommendation system different from the first recommendation system. The main reason for introducing external signals in the embodiment of the present application is that the external signal includes information in other recommendation systems. For example, the content recommendation data may include external author information and external prior data information.

[0069] In fact, some high-quality self-media authors currently tend not to publish content on only one recommendation system. They will mainly publish content on one recommendation system and publish content on multiple recommendation systems at the same time. Therefore, the content release of different recommendation systems will have a certain order and will be consumed by different content consumers. Therefore, for a recommendation system, it can make full use of the posterior data of the distribution of the same or similar content on different recommendation systems, and then combine it with the prior feature information of the content itself (such as the clarity of the content, celebrity popularity information, author authority certification, etc.) to introduce this external signal to construct a quality standard that meets the public aesthetic standards. That is, in terms of the public aesthetic dimension, the content distribution quality of the multimedia object is determined, thereby improving the recommendation effect. In addition, in some scenarios, it can also assist and help the existing recommendation system (the first recommendation system) to identify high-quality content and accelerate the cold start process of high-quality content and content consumption objects.

[0070] In one embodiment, the external signal may include first external information. This first external information can be considered as the content distribution quality of the multimedia object in other recommendation systems predicted by the first recommendation system. This information is predicted based on prior data obtained from other recommendation systems and can be considered as a round of prediction of the multimedia object from the perspective of other recommendation systems. The descriptive capability of the content distribution quality indicator of the multimedia object in the first recommendation system is improved through multiple perspectives.

[0071] In one embodiment, obtaining an external signal corresponding to a multimedia object includes inputting the multimedia object into an external signal extractor to obtain first external information corresponding to the multimedia object, wherein the first external information is part of the external signal. The external signal extractor is trained by: determining positive sample content and negative sample content in at least one of the second recommendation systems, wherein the positive sample content meets at least one of the following conditions: a content distribution volume exceeding a first threshold, a content-targeted feedback volume exceeding a second threshold, a content provider-targeted indicator meeting a first requirement and normal content release, and a content provider having certification qualifications meeting a second requirement; and training a neural network based on the positive sample content and the negative sample content to obtain the external signal extractor. Of course, the first threshold, the second threshold, the first requirement, and the second requirement are not limited and can be designed based on actual circumstances. The content-targeted feedback volume is not limited and can be, for example, likes, comments, or reposts. The content provider-targeted indicator is not limited and can be, for example, the number of followers, number of interactions, or reputation. The certification qualifications are not limited and can be, for example, certified premium accounts. Among them, the first requirement and the second requirement can be determined according to the application scenario of the second recommendation system or the optimization direction of the first recommendation system. For example, if the second recommendation system focuses on promoting some emotional topics, the first requirement can be that the number of high-quality emotional comments published by the content provider is greater than the preset high-quality comment number threshold, and the second requirement can be that the content provider is certified as a high-quality emotional content provider. For another example, if the optimization direction of the first recommendation system is to improve the cold start effect, the first requirement can be that the content provider's fan growth rate at the beginning of content provision meets the preset speed requirement, and the second requirement can be that the content provider is certified as a fast-growing content provider, and the time required from the beginning of providing content to being certified as a high-quality content provider meets the corresponding time requirement.

[0072] In a specific implementation, high-quality, popular content from at least one recommendation system can be used as positive samples, and randomly selected content can be used as negative samples. An automated binary classification model is trained, and this binary classification model can serve as the external signal extractor, outputting the first external information. For example, government sample content can be filtered according to the following criteria:

[0073] Positive samples: (1) Video distribution with more than 100,000 views or more than 100,000 likes; (2) The author has more than 100,000 followers and the content is distributed normally; (3) The external platform's certification information for the author is: "Certified high-quality / well-known (such as history, finance, etc.) field creator" account, producing and distributing content normally;

[0074] Negative samples: Randomly select content that does not meet the positive sample criteria as negative samples.

[0075] In some embodiments, the external signal includes second external information, and the second external information includes at least one of the following: account feature information associated with the multimedia object in at least one of the second recommendation systems, behavioral feature information of content consumption objects corresponding to the first recommendation system and / or the second recommendation system; content influence information of the multimedia object, and influence information of the content provider of the multimedia object. For example, account feature information may include authentication level, number of fans, distribution volume, etc., behavioral feature information of content consumption objects may include sharing volume, likes volume, comment volume, forwarding volume, etc., content influence information of multimedia objects may include public recognition information, popularity information, and popularity information of the content and characters in the object, and content provider influence information may include author popularity information, etc.

[0076] S102. Extract modal information of at least one dimension from the multimedia object to obtain modal feature information corresponding to the multimedia object.

[0077] Specifically, the above-mentioned modal feature information includes at least one of the following:

[0078] The video modality information, audio modality information, text modality information, title modality information, and voice modality information of the multimedia object mentioned above. Taking the multimedia object as a video or short video as an example, its modality feature information can be obtained by the following method:

[0079] (1) Video mode:

[0080] The embedding vector of a multimodal video content MediaFrame represents a video using a low-dimensional vector. The distance between two embeddings represents the distance between the two videos, which can be used to calculate video similarity. Video content embeddings are based on implicit features of the content. Video content vectors have two meanings: the first meaning is representation learning, using low-dimensional, dense features and a one-dimensional array (for example, a video embedding is 128 floating-point data). The second meaning is metric learning, using a vector for similarity measurement. The distance between two vectors represents the similarity between the two objects.

[0081] In one embodiment, a video frame sequence can be input, and frames can be extracted through the TSN (Temporal Segment Networks) network. The skeleton network (Backbone) can be a RestNet network or an Xception network to extract image features. Finally, the image feature vector is obtained by using the middle layer of the network competition model of the Youtub8M-NeXtVLad network, and then the vector embeeding (video modality information) of the video is obtained by adding and averaging the image feature vectors. In the NeXtVLad large-scale video classification task, the frame-level features are aggregated into a model structure of video-level features. The basic idea is to decompose the high-dimensional features into a set of relatively low-dimensional vectors before performing NetVLAD aggregation. NeXtVLAD was originally used to aggregate spatial expressions in location recognition. The embodiment of the present application found that it is more effective and faster than conventional temporal models for aggregating visual and auditory feature tasks.

[0082] (2) Audio mode:

[0083] In one embodiment, the audio can be preprocessed to obtain a mel-spectrogram, and then features can be extracted from the mel-spectrogram using VGGish. NextVlad is then used to obtain video-level audio features. The VGGish model is pre-trained on AudioSet data. For the audio modality, a series of preprocessing steps are first required on the audio file of the video. Typically, 16kHz audio from the first 10 minutes of the video is selected. A short-time Fourier transform is performed on the audio using a 25ms Hamming window and a 10ms frame shift to obtain a spectrogram. The spectrogram is then mapped to a 64-order mel filter bank to calculate the mel spectrum. The mel spectrum features are framed in 960ms durations, with no overlap between frames. Each frame is 10ms long and contains 64 mel frequency bands. VGGish is used as a feature extractor, which has a strong and special ability to express scene-related sound events. After adding the audio modality, the viewing value and the accuracy of highlight recognition for action movies, music, and other content are significantly improved.

[0084] (3) Text mode:

[0085] Text modalities include titles and video content OCR (Optical Character Recognition) and ASR (Automatic Speech Recognition) results. A title is a subjective description of the video's content by the content provider, typically encompassing the high-level semantics of the video's intended meaning. In practice, many videos lack titles or lack sufficient information. OCR can supplement the rich textual information not provided by titles. However, OCR data also presents challenges, such as inaccurate OCR during frame switching, the need to deduplicate fixed-position OCR, the need to retain spoken content, and the need to remove news scrolling OCR. Therefore, OCR results are denoised, including filtering single-word, pure-number, and pure-letter OCR, filtering OCR with small offsets between adjacent frames and high text repetition rates, and filtering OCR with a small height at the bottom of the screen. The denoised OCR text is concatenated with the title text for the text branch's text modal input. A BERT model is trained based on the large-scale text corpus of the information stream, and the text is modeled based on the training results. The text modal also includes text information automatically recognized by ASR from the speech within the video. This method can supplement the insufficient OCR text recognition for videos without subtitles.

[0086] S103. Obtain a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modal feature information.

[0087] In some embodiments, in a cold start scenario, based on the above-mentioned external signal and the above-mentioned modal feature information, a content distribution quality index corresponding to the above-mentioned multimedia object is determined, and the above-mentioned content distribution quality index represents the probability that the above-mentioned multimedia object is high-quality content; in response to the situation that the above-mentioned content distribution quality index is higher than the first preset value, the first recommendation strategy is determined as the recommendation strategy for the above-mentioned multimedia object in the above-mentioned cold start scenario.

[0088] In some embodiments, when the distribution volume of the multimedia object reaches a second preset value and the interaction index of the multimedia object is higher than a third preset value, the recommendation strategy of the multimedia object is adjusted to the second recommendation strategy;

[0089] or,

[0090] In some embodiments, when the distribution volume of the multimedia object reaches a fourth preset value and the interaction index is higher than a fifth preset value, the recommendation strategy for the multimedia object is adjusted to a third recommendation strategy, wherein the fourth preset value is higher than the second preset value, the fifth preset value is lower than the third preset value, and the number of content consumption objects involved in the third recommendation strategy is greater than the number of content consumption objects involved in the second recommendation strategy.

[0091] Of course, there is no limitation on the specific values ​​of the first preset value, the second preset value, the third preset value, the fourth preset value and the fifth preset value, and there is no limitation on the specific contents of the first recommendation strategy, the second recommendation strategy and the third recommendation strategy.

[0092] For example, for content (multimedia objects) newly stored in the database (first recommendation system), a probability value of the content quality rate (content distribution quality index) can be output based on the above external signals and the above modal feature information. A threshold is set based on the quality concept to screen out high-quality content. For example, during the cold start period, the high-quality content is given an A rating. After the distribution volume reaches 10,000, if the interaction index is higher than the interaction average of this stage, it enters the weighted distribution stage (second recommendation strategy). After the distribution volume reaches 50,000 and the interaction is not significantly lower than the interaction average of this stage, the content enters the whole network high-quality pool (third recommendation strategy) and is pushed to more content consumers. In the cold start exploration stage of new content consumers, in order to better obtain the interest points of content consumers, it is necessary to use high-quality content that has been determined in advance and has been proven to be of high quality for weighted exploration. In this way, external information can be used to help the first recommendation system evolve, and the quality of the perceived distributed content can be predicted by using the high-quality content that has been consumed and proven, so as to evaluate the quality of the content entering its own platform (first recommendation system), accelerate the cold start of content, and improve the efficiency of recommendation distribution.

[0093] In a specific embodiment, the above-mentioned obtaining of the external signal corresponding to the multimedia object includes: inputting the multimedia object into a width information extractor to obtain the above-mentioned external signal, wherein the above-mentioned width information extractor has a flat structure; the above-mentioned extraction of modal information of at least one dimension of the above-mentioned multimedia object to obtain the modal feature information corresponding to the above-mentioned multimedia object includes: inputting the above-mentioned multimedia object into a multimodal information extractor to obtain the above-mentioned modal feature information, wherein the above-mentioned multimodal information extractor has a deep neural network. The above-mentioned obtaining of the recommendation strategy for the above-mentioned multimedia object based on the content distribution quality index determined based on the above-mentioned external signal and the above-mentioned modal feature information includes: fusing and connecting the above-mentioned external signal and the above-mentioned modal feature information to obtain the content distribution quality index corresponding to the above-mentioned multimedia object, wherein the above-mentioned content distribution quality index represents the probability that the above-mentioned multimedia object is high-quality content; and determining the corresponding recommendation strategy based on the above-mentioned content distribution quality index.

[0094] In other words, please refer to Figure 3 , which shows the architecture of the neural network for determining content quality distribution indicators in an embodiment of the present application. The neural network as a whole adopts a Deep & Wide structure. The Deep part is the multimodal information extractor mentioned above, and the Wide part is the width information extractor mentioned above. In this neural network, the following operations can undoubtedly be performed:

[0095] Multimodal feature extraction is performed on the content of multimedia objects, and author account level information is integrated with interactive behavior information of content consumers and external signals (such as the popularity and level of celebrities in the video content). This external signal can include the first external information and / or the second external information mentioned above. This allows for classification and prediction of the comprehensive distribution quality of the multimedia object content, resulting in a content distribution quality indicator that comprehensively characterizes the quality of the multimedia content. Given that the perceived quality of video content is closely related to the plot, the results of automatic OCR and ASR recognition of the video content can also be input.

[0096] exist Figure 3In the Wide part, the width information extractor can output an external signal. The width information extractor takes the output data in the multimodal input layer as input. The data output by the multimodal input layer can include audio information, title information, text recognition information and speech recognition information. The data output by the multimodal input layer can also be used as the input of the embedding layer. The embedding layer can embed and encode the data output by the multimodal input layer and transmit the result to the neural network corresponding to each modal data. For example, the three neural networks of CNN, VGG and BERT can be used to process the information of the video modality, the information of the audio modality and the information of the text modality respectively to obtain multimodal feature information. Finally, the external signal and the multimodal feature information are connected and fused to obtain the distribution content quality index, which can be used to determine the recommendation strategy.

[0097] S104. Recommend the multimedia object according to the recommendation strategy.

[0098] The embodiments of the present application can effectively utilize data from other external recommendation systems, such as content distribution data, prior information about content and authors, to generate external signals for evaluating the content distribution quality of multimedia objects, thereby constructing a comprehensive semantic and perceptual level content quality evaluation method, making more scientific and reasonable recommendations for multimedia objects, and helping the recommendation system to cold-start new incoming content and content consumption objects, avoiding the inefficiency of multiple and repeated explorations. In addition, the recommendation method can help the recommendation system better utilize recommended traffic, and the content distribution process can sort content based on the above-mentioned content distribution quality, weight and de-weight content, accelerate the distribution of high-quality content and reduce the distribution of low-quality content, and ultimately improve the content consumption experience of content consumers and enhance consumer stickiness, thereby improving product retention.

[0099] In an exemplary embodiment, the above content recommendation method can be applied to a video information stream content service system. Figure 4 As shown, it exemplarily shows a technical framework diagram of an information flow content service system. Figure 4 The various service modules and their main functions in the information flow content service system are shown as follows.

[0100] 1. Video content production and consumption

[0101] (1) PGC or UGC, MCN or PUGC content producers provide local or filmed video content through mobile terminals or back-end API (Application Programming Interface) systems. These are the main sources of distributed content;

[0102] (2) By communicating with the upstream and downstream content interface services, the upload server interface address is first obtained, and then the local file is uploaded. During the shooting process, the local video content can be matched with music, filter templates, and video beautification functions, etc.;

[0103] (3) As a consumer, the user communicates with the upstream and downstream content interface servers to obtain index information for accessing video files, and then communicates with the video storage server to download the corresponding streaming media files and play them through a local player;

[0104] (4) At the same time, the content consumption object playback behavior data during the upload and download process, such as fast forward, drag, rewind, etc., freeze, loading time, play clicks, etc., are reported to the statistics server, and a clear content quality feedback entry is provided, including the quality problem type, corresponding to the various low-quality features mentioned above, and the corresponding video file unique identifier;

[0105] (5) Consumers usually browse video data through feeds, so duplicate video files will have a great impact on the experience of content consumers.

[0106] (6) For content with high video distribution quality scores, consumers can display them in feeds with weighted weights to increase exposure probability and accelerate the cold start of content;

[0107] 2. Uplink and Downlink Content Interface Server

[0108] (1) Communicate directly with the content production end. The content submitted from the front end, usually the title, publisher, summary, cover image, release time of the video content, or the shot video, directly enters the server through the server and stores the file in the video content storage service;

[0109] (2) Writing metadata of the video content, such as video file size, cover image link, bit rate, file format, title, release time, author, etc., into the content database;

[0110] (3) Submit the uploaded files to the dispatch center server for subsequent content processing and circulation; III. Content Database

[0111] (1) The core database of video content. The metadata of all content released by producers is stored in this business database, focusing on the metadata of the video content itself, such as the video file size, cover image link, bit rate, file format, title, release time, author, and the classification of content during the manual review process (including first-level, second-level, and third-level classification and label information. For example, a video about a mobile phone may be classified as technology at the first level, mobile phone at the second level, and domestic mobile phone at the third level. The label may be the model of the mobile phone, etc.);

[0112] (2) After the upstream and downstream content interface servers store the published video files on the video content storage server, the video content storage server will perform standard transcoding operations on the content. After the transcoding is completed, the asynchronously returned meta-information mainly includes file size, bit rate, specifications, and captured cover images. These information will be stored in the content database;

[0113] (3) During the manual review process, the information in the content database will be read, and the results and status of the manual review will also be sent back to the content database;

[0114] (4) The dispatch center's content processing mainly includes machine processing and manual review processing. The core of the machine processing is to call the deduplication service. The deduplication results will be written into the content database. Completely duplicate content will not be manually processed again.

[0115] (5) Provide content metadata for the multimodal feature extraction model, such as the title of the content, the results of content OCR recognition, and the video content ASR text information video results. OCR is provided by a separate model processing the video and saved in the content data for future use;

[0116] 4. Dispatch Center

[0117] (1) Responsible for the entire scheduling process of video content flow, receiving the video content through the uplink and downlink content interface servers, and then obtaining the video metadata from the internal database;

[0118] (2) Dispatching manual review systems and machine processing systems to control the order and priority of dispatch;

[0119] (3) For video content, first communicate with the video deduplication service, which can process different bit rates, different resolutions, different sizes, partial black screens, filters, tags, and inserting some advertising content and cutting out the opening and ending credits in similar video content, effectively building the content that needs to be processed during the manual review process;

[0120] (4) Schedule video content that needs to be manually reviewed into the manual review system and filter out duplicate content;

[0121] (5) Finally, the content that has passed the manual review system is enabled through the content export distribution service, which is usually a recommendation engine or search engine or a direct display page provided to the terminal content consumers;

[0122] (6) Calling the video content distribution quality service communication to complete the prediction result of the video content distribution quality score, and saving the result in the content data for subsequent recommendation distribution;

[0123] 5. Manual review system

[0124] (1) It is necessary to read the original information of the video content itself in the content database. Usually, it is a system developed based on a web database with complex business operations. It manually performs a preliminary filtering to determine whether the video content contains some characteristics of bad content.

[0125] (2) Based on the initial review, the content is subject to a secondary review, mainly to classify and label the content or confirm the content. Since the video content itself is not fully mature through machine learning, such as deep learning, it is necessary to conduct a secondary manual review on the machine processing machine to improve the accuracy and efficiency of the video itself through human-machine collaboration;

[0126] (3) Accept the dispatch of the dispatch center service and manually review the suspected content after the video title, cover and content quality model processing

[0127] (4) During the manual review process, content related to video quality issues that fail to pass the review will be directly marked and reported to the statistical interface and analysis server as sample training data for the subsequent construction of the detection model.

[0128] 6. Video Content Storage Service

[0129] (1) It is usually a group of storage servers that are widely distributed and close to the content consumption objects. Usually there are CDN (Content Delivery Network) acceleration servers on the periphery for distributed cache acceleration. The video content uploaded by the content producer is stored through the upstream and downstream content interface servers;

[0130] (2) After obtaining the content index information, the terminal consumer can also directly access the video content storage server to download the corresponding content;

[0131] (3) In addition to being a data source for external services, it also serves as a data source for internal services, allowing the download file system to obtain raw video data for related processing. The internal and external data source pathways are usually deployed separately to avoid mutual impact;

[0132] 7. Download the file system

[0133] (1) Download and obtain the original video content from the video content storage server and control the download speed and progress. It is usually composed of a group of parallel servers with related task scheduling and distribution clusters;

[0134] (2) The downloaded file communicates with the video clip extraction and processing service to provide services for subsequent video multimodal feature extraction;

[0135] 8. Video Clip Extraction and Processing Services

[0136] (1) According to the above-mentioned algorithms and strategies, the file downloaded from the video content storage service by the download file system is subjected to primary processing of video file features, including the use of video frames and the separation of audio content in the video for subsequent video multimodal feature extraction services;

[0137] 9. Multimodal feature extraction model

[0138] (1) According to the specific model and method described above, a multimodal feature extraction model is constructed using the content title (including the ASR recognition results of the video OCR text), audio modality, and multi-dimensional information of the video clip to obtain multimodal feature information;

[0139] 10. Internal and external authors and prior data information

[0140] (1) The quality of video content distribution is not only related to the visual information, audio, and text information of the video itself, but also to many other factors, such as whether it contains celebrities and external information such as the importance and popularity of celebrities. At the same time, when training the model, manual labeling can provide celebrity level information such as first-tier, second-tier, and third-tier level information as input for the Wide part of auxiliary information; there is also the level certification information of the self-media author who published the video content;

[0141] 11. Video Distribution Quality Model and Service

[0142] According to the method described above, the Deep&Wide structure model is used to process and fuse multimodal feature information and external signals, and then the model is turned into a service;

[0143] Communicate with the dispatch center service to complete the distribution quality scoring of video content. The scoring results are used by the recommendation system online.

[0144] 12. Content sample library (internal and external)

[0145] Mainly obtain high-quality popular distribution content from other recommendation systems (second recommendation system) as positive samples, and randomly select or use content of average quality on the current platform as negative samples, that is, provide sample services;

[0146] 13. Video deduplication service

[0147] By comparing the fingerprint features of the video content itself, duplicate similar video files will retain a copy that will continue to the subsequent link, reducing unnecessary duplicate files on the link.

[0148] 14. Statistical interface and analysis server

[0149] Accept consumption flow data reports from video content consumers and feedback reports on video content quality issues. Negative feedback can be used as negative samples.

[0150] Accept reports of filtered quality issues during the manual review process, collect feedback, automatically return low-quality data, and iterate the algorithm input.

[0151] The above-mentioned information flow content service system supports the content recommendation method based on external signals provided by the embodiment of the present application. In many scenarios, high-quality content samples are screened through external signals of the distribution effect of self-media authors’ content on other platforms, and a quantitative distribution quality assessment is performed on the video content at the level of content consumption objects to meet the public’s aesthetics and ranking, and used for content recommendation and cold start of its own recommendation system, thereby improving the distribution efficiency of the recommendation system content and improving the cold start effect. By sampling the video content distributed by self-media authors who are simultaneously stationed in multiple recommendation systems, the external signals of the multimedia objects are determined from the perspective of the external recommendation system, and then the multimodal features of the video are extracted. At the same time, the consumption behavior feature information of the content consumption objects, the external information of the content (such as the popularity information of the content stars) and the posterior distribution information such as external comments, likes, number of fans and author authority can all be used as a basis to influence the content distribution quality index, and then affect the recommendation strategy. Specifically, the Deep&Wide model architecture can be used to comprehensively classify the video content from the perspective of content consumption objects’ perception, to assist the cold start and recommendation evolution of the recommendation system. The present invention can effectively utilize the content distribution signals of external platforms and the prior information of content and authors to construct a comprehensive video content quality evaluation system at the semantic level and the perception level of content consumption objects, which can help the recommendation system to cold-start new content and content consumption objects, avoiding the inefficiency of multiple and repeated explorations; it can help promote the system to better utilize recommended traffic, and the video distribution process can be based on content sorting, weighting and de-weighting content, accelerating the distribution of high-quality content and reducing the distribution of low-quality content, ultimately improving the content consumption experience of content consumption objects, enhancing consumption stickiness, and improving product retention rate.

[0152] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0153] Please refer to Figure 5 , which shows a block diagram of a content recommendation device provided by one embodiment of the present application. This device is applied to the first recommendation system and has the function of implementing the above-mentioned content recommendation method. The above-mentioned function can be implemented by hardware or by hardware executing corresponding software. The device can be a computer device or can be installed in a computer device. The device can include:

[0154] An external signal acquisition module 101 is configured to acquire an external signal corresponding to a multimedia object, wherein the external signal is a result of performing a content distribution quality prediction on the multimedia object, the content distribution quality prediction being performed based on content recommendation data from at least one second recommendation system, where any second recommendation system is a recommendation system different from the first recommendation system.

[0155] A multimodal information acquisition module 102 is configured to extract modal information of at least one dimension from the multimedia object to obtain modal feature information corresponding to the multimedia object;

[0156] A recommendation strategy determination module 103 is configured to obtain a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modality feature information;

[0157] The recommendation module 104 is configured to recommend the multimedia object according to the recommendation strategy.

[0158] In an exemplary embodiment, the above-mentioned external signal acquisition module 101 is also used to input the above-mentioned multimedia object into the external signal extractor to obtain the first external information corresponding to the above-mentioned multimedia object, and the above-mentioned first external information belongs to the above-mentioned external signal; the above-mentioned external signal extractor is trained by the following method: determining positive sample content and negative sample content in at least one of the above-mentioned second recommendation systems, and the above-mentioned positive sample content meets at least one of the following conditions: the content distribution volume is higher than the first threshold, the content-oriented feedback volume is higher than the second threshold, the content provider's directional index meets the first requirement and the content is published normally, and the content provider has the certification qualification that meets the second requirement; training a neural network based on the above-mentioned positive sample content and the above-mentioned negative sample content to obtain the above-mentioned external signal extractor.

[0159] In an exemplary embodiment, the above-mentioned external signal includes second external information, and the above-mentioned second external information includes at least one of the following: account characteristic information associated with the above-mentioned multimedia object in at least one of the above-mentioned second recommendation systems, behavioral characteristic information of content consumption objects corresponding to the above-mentioned first recommendation system and / or the above-mentioned second recommendation system; content influence information of the above-mentioned multimedia object, and influence information of the content provider of the above-mentioned multimedia object.

[0160] In an exemplary embodiment, the modal feature information includes at least one of the following:

[0161] The video modal information, audio modal information, text modal information, title modal information, and voice modal information of the above multimedia objects.

[0162] In an exemplary embodiment, the above-mentioned external signal acquisition module 101 is also used to input the above-mentioned multimedia object into a width information extractor to obtain the above-mentioned external signal, and the above-mentioned width information extractor has a flat structure; the above-mentioned multimodal information acquisition module 102 is used to input the above-mentioned multimedia object into a multimodal information extractor to obtain the above-mentioned modal feature information, and the above-mentioned multimodal information extractor has a deep neural network.

[0163] In an exemplary embodiment, the recommendation strategy determination module 103 is used to fuse the external signal and the modal feature information to obtain a content distribution quality indicator corresponding to the multimedia object, where the content distribution quality indicator represents the probability that the multimedia object is high-quality content; and determine a corresponding recommendation strategy based on the content distribution quality indicator.

[0164] In an exemplary embodiment, the above-mentioned recommendation strategy determination module 103 is used to determine the content distribution quality index corresponding to the above-mentioned multimedia object based on the above-mentioned external signal and the above-mentioned modal feature information in a cold start scenario, and the above-mentioned content distribution quality index represents the probability that the above-mentioned multimedia object is high-quality content; in response to the situation that the above-mentioned content distribution quality index is higher than the first preset value, the first recommendation strategy is determined as the recommendation strategy for the above-mentioned multimedia object in the above-mentioned cold start scenario.

[0165] In an exemplary embodiment, the recommendation strategy determination module 103 is configured to adjust the recommendation strategy of the multimedia object to the second recommendation strategy when the distribution volume of the multimedia object reaches a second preset value and the interaction index of the multimedia object is higher than a third preset value;

[0166] or,

[0167] When the distribution volume of the multimedia object reaches the fourth preset value and the interaction index is higher than the fifth preset value, the recommendation strategy for the multimedia object is adjusted to the third recommendation strategy, wherein the fourth preset value is higher than the second preset value, the fifth preset value is lower than the third preset value, and the number of content consumption objects involved in the third recommendation strategy is greater than the number of content consumption objects involved in the second recommendation strategy.

[0168] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0169] Please refer to Figure 6 , which shows a block diagram of a computer device provided by an embodiment of the present application. The computer device may be a server for executing the above-mentioned content recommendation method. Specifically:

[0170] Computer device 1600 includes a central processing unit (CPU) 1601, a system memory 1604 including a random access memory (RAM) 1602 and a read-only memory (ROM) 1603, and a system bus 1605 connecting system memory 1604 and CPU 1601. Computer device 1600 also includes a basic input / output system (I / O system) 1606 that facilitates information transfer between various components within the computer, and a mass storage device 1607 for storing an operating system 1613, application programs 1614, and other program modules 1615.

[0171] The basic input / output system 1606 includes a display 1608 for displaying information and an input device 1609, such as a mouse and keyboard, for inputting information to a content consumer. Both the display 1608 and the input device 1609 are connected to the central processing unit 1601 via an input / output controller 1610 connected to the system bus 1605. The basic input / output system 1606 may also include an input / output controller 1610 for receiving and processing input from a variety of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1610 also provides output to a display screen, printer, or other types of output devices.

[0172] The mass storage device 1607 is connected to the central processing unit 1601 via a mass storage controller (not shown) connected to the system bus 1605. The mass storage device 1607 and its associated computer-readable media provide non-volatile storage for the computer device 1600. In other words, the mass storage device 1607 may include computer-readable media (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0173] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, tape cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above-mentioned ones. The above-mentioned system memory 1604 and mass storage device 1607 can be collectively referred to as memory.

[0174] According to various embodiments of the present application, the computer device 1600 may also be connected to a remote computer on a network such as the Internet for operation. That is, the computer device 1600 may be connected to a network 1612 via a network interface unit 1611 connected to the system bus 1605, or the network interface unit 1611 may be used to connect to other types of networks or remote computer systems (not shown).

[0175] The memory further includes a computer program, which is stored in the memory and configured to be executed by one or more processors to implement the content recommendation method.

[0176] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. When the at least one instruction, the at least one program, the code set or the instruction set is executed by a processor, the content recommendation method is implemented.

[0177] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or an optical disk, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0178] In an exemplary embodiment, a computer program product or computer program is also provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described content recommendation method.

[0179] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application do not limit this.

[0180] In addition, in the specific implementation of this application, related data such as content consumption object information is involved. When the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain permission or consent from the content consumption object, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0181] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A content recommendation method, characterized in that: Applied to the first recommendation system, the method includes: Obtaining an external signal corresponding to a multimedia object, the external signal being a result obtained from a process of predicting content distribution quality for the multimedia object, the content distribution quality prediction being performed based on content recommendation data in at least one second recommendation system, where any second recommendation system is a recommendation system different from the first recommendation system; Extracting modal information of at least one dimension from the multimedia object to obtain modal feature information corresponding to the multimedia object; Obtaining a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modality feature information; The multimedia object is recommended according to the recommendation strategy.

2. The method according to claim 1, characterized in that The obtaining of the external signal corresponding to the multimedia object includes: inputting the multimedia object into an external signal extractor to obtain first external information corresponding to the multimedia object, where the first external information belongs to the external signal; The external signal extractor is trained by the following method: Determining positive sample content and negative sample content in at least one of the second recommendation systems, wherein the positive sample content meets at least one of the following conditions: a content distribution volume exceeding a first threshold, a content-targeted feedback volume exceeding a second threshold, a content provider targeting indicator meeting a first requirement and publishing content normally, and a content provider having certification qualifications meeting a second requirement; A neural network is trained according to the positive sample content and the negative sample content to obtain the external signal extractor.

3. The method according to claim 1, characterized in that The external signal includes second external information, and the second external information includes at least one of the following: Account feature information associated with the multimedia object in at least one of the second recommendation systems, Behavioral characteristic information of content consumption objects corresponding to the first recommendation system and / or the second recommendation system; The content influence information of the multimedia object, The influence information of the content provider of the multimedia object.

4. The method according to claim 1, wherein The modal feature information includes at least one of the following: The multimedia object includes video modality information, audio modality information, text modality information, title modality information, and voice modality information.

5. The method according to claim 1, wherein: The obtaining of the external signal corresponding to the multimedia object includes: inputting the multimedia object into a width information extractor to obtain the external signal, wherein the width information extractor has a flat structure; The extracting modal information of at least one dimension of the multimedia object to obtain modal feature information corresponding to the multimedia object includes: inputting the multimedia object into a multimodal information extractor to obtain the modal feature information, wherein the multimodal information extractor has a deep neural network.

6. The method according to claim 5, characterized in that The obtaining of a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modality feature information includes: fusing and connecting the external signal and the modal feature information to obtain a content distribution quality indicator corresponding to the multimedia object, wherein the content distribution quality indicator represents a probability that the multimedia object is high-quality content; Determine a corresponding recommendation strategy based on the content distribution quality indicator.

7. The method according to any one of claims 1 to 6, characterized in that The obtaining of a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modality feature information includes: In a cold start scenario, determining a content distribution quality indicator corresponding to the multimedia object based on the external signal and the modal feature information, wherein the content distribution quality indicator represents a probability that the multimedia object is high-quality content; In response to the content distribution quality indicator being higher than a first preset value, a first recommendation strategy is determined as a recommendation strategy for the multimedia object in the cold start scenario.

8. The method according to claim 7, characterized in that The step of obtaining a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modality feature information further includes: When the distribution amount of the multimedia object reaches a second preset value and the interaction index of the multimedia object is higher than a third preset value, adjusting the recommendation strategy of the multimedia object to a second recommendation strategy; or, When the distribution volume of the multimedia object reaches a fourth preset value and the interaction index is higher than a fifth preset value, the recommendation strategy of the multimedia object is adjusted to a third recommendation strategy, wherein the fourth preset value is higher than the second preset value, the fifth preset value is lower than the third preset value, and the number of content consumption objects involved in the third recommendation strategy is greater than the number of content consumption objects involved in the second recommendation strategy.

9. A content recommendation device, characterized in that: Applied to the first recommendation system, the device includes: an external signal acquisition module, configured to acquire an external signal corresponding to a multimedia object, the external signal being a result obtained from a process of predicting content distribution quality for the multimedia object, the content distribution quality prediction being performed based on content recommendation data from at least one second recommendation system, any second recommendation system being a recommendation system different from the first recommendation system; a multimodal information acquisition module, configured to extract modal information of at least one dimension from the multimedia object to obtain modal feature information corresponding to the multimedia object; a recommendation strategy determination module, configured to obtain a recommendation strategy for the multimedia object according to a content distribution quality indicator determined based on the external signal and the modal feature information; A recommendation module is configured to recommend the multimedia object according to the recommendation strategy.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the content recommendation method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the content recommendation method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the content recommendation method according to any one of claims 1 to 8.

Images