Personalized Video Push Optimization Method, Device, Storage Medium, Program Product

Abstract

An embodiment of the present disclosure discloses a personalized video push optimization method, device, storage medium, and program product. Among them, the method includes: collecting user input data, parsing the input data to obtain user preference information; after the target video application is started, tracking the dynamic changes of the currently played video content, taking screenshots of the currently played video based on the degree of video content change to obtain video screenshots; parsing the video screenshots to obtain video information; matching the preference information with the video information, and controlling the intelligent agent to interact with the target video application based on the matching result. This method can solve the problem of inaccurate push in the initial stage of user use of the target video application, guide its accurate video push, thereby reducing the time for users to browse irrelevant videos and improving user satisfaction.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular, to a method, device, storage medium, and program product for optimizing personalized video push. Background Art

[0002] Currently, various short-video tools mainly push video information according to users' personal preferences. Their push mechanisms usually rely on in-depth analysis of multi-dimensional behavioral data such as users' past browsing history, likes, and comments, and then construct accurate user interest portraits, and based on this, push videos with similar content.

[0003] However, this seemingly efficient and intelligent push mode has significant drawbacks in practical applications. Especially in the initial stage of users using short-video platforms, the platform will push a large number of videos that do not match the actual needs of users. Users have to spend a lot of time providing feedback to the platform until the platform can provide recommended content that relatively meets their personalized needs, which undoubtedly causes a great waste of users' precious time. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a method, device, storage medium, and program product for optimizing personalized video push, which can solve the problem of inaccurate push of the target video application program in the initial stage of user use, guide its accurate video push, and thus reduce the time for users to browse irrelevant videos.

[0005] In a first aspect, embodiments of the present disclosure provide a method for optimizing personalized video push, adopting the following technical solution:

[0006] Collect user input data, parse the input data, and obtain the user's preference information;

[0007] After the target video application program is started, track the dynamic changes of the currently played video content, and capture screenshots of the currently played video based on the degree of video content change to obtain video screenshots;

[0008] Parse the video screenshots to obtain video information;

[0009] Match the preference information with the video information, and based on the matching result, control the intelligent agent to interact with the target video application program.

[0010] Optionally, the collecting user input data, parsing the input data, and obtaining the user's preference information includes:

[0011] Receive the user's input data through the front-end interface, or capture the user's input data from the target video application program;

[0012] Classify and store the input data of each user in the user preference information database;

[0013] Use a preset multi-modal large model to read the input data of each user from the user preference information database and output the preference information of each user.

[0014] Optionally, tracking the dynamic changes of the currently playing video content, taking screenshots of the currently playing video based on the degree of video content change to obtain video screenshots, including:

[0015] Obtain the video stream of the currently playing video and extract video frames at preset time intervals;

[0016] Extract the scene features and the positions of human key points of the video frames;

[0017] Based on the scene features, obtain the scene change degree value of the current video content;

[0018] Based on the positions of the human key points, obtain the human action change degree value of the current video content;

[0019] When the scene change degree value is greater than the first threshold, or the human action change degree value is greater than the second threshold, take a screenshot of the currently playing video to obtain a video screenshot.

[0020] Optionally, the personalized video push optimization method further includes:

[0021] Collect the current hardware resource data of the user's intelligent terminal device according to the preset hardware resource indicators;

[0022] Obtain an adjustment coefficient based on the current hardware resource data;

[0023] Based on the adjustment coefficient, the preset first base threshold and second base threshold, obtain the first threshold and the second threshold.

[0024] Optionally, matching the preference information with the video information and controlling the intelligent agent to interact with the target video application based on the matching result, including:

[0025] Input the preference information and the video information into a preset large language model to obtain a classified or graded matching result;

[0026] Based on the matching result, control the intelligent agent to execute different simulated user operations to achieve interaction with the target video application;

[0027] The target video application optimizes the video push mechanism of the target video application based on the interaction operation.

[0028] Optionally, controlling the agent to perform different simulated user operations based on the matching result to implement interaction with the target video application, including:

[0029] Triggering the agent to generate an operation instruction based on the matching result, where the operation instruction includes a first identifier of the target video application, a second identifier of the currently playing video, and an interaction operation type;

[0030] Sending the operation instruction to the corresponding adaptation module based on the first identifier;

[0031] Using the adaptation module to convert the operation instruction into a native interaction instruction recognizable by the target video application, where the native interaction instruction includes a standard identifier converted from the second identifier and a standard interaction type converted from the interaction operation type;

[0032] Executing the native interaction instruction to perform corresponding interaction operations on the currently playing video.

[0033] Optionally, the personalized video push optimization method further includes:

[0034] When the interaction operation performed by the agent on the target video application does not include pausing the playback, continuously tracking the dynamic changes of the currently playing video content, taking a screenshot of the currently playing video based on the degree of new video content change to obtain a new video screenshot;

[0035] Updating the video information based on the accumulated video screenshots of the currently playing video.

[0036] In a second aspect, an embodiment of the present disclosure further provides a personalized video push optimization system, adopting the following technical solution:

[0037] A user input module, configured to collect user input data, parse the input data, and obtain the user's preference information;

[0038] A video screenshot module, configured to track the dynamic changes of the currently playing video content after the target video application is started, take a screenshot of the currently playing video based on the degree of video content change, and obtain a video screenshot;

[0039] A video parsing module, configured to parse the video screenshot to obtain video information;

[0040] An information matching module, configured to match the preference information with the video information, and control the agent to perform interaction operations with the target video application based on the matching result.

[0041] In a third aspect, an embodiment of the present disclosure further provides a computer device, adopting the following technical solution:

[0042] The computer device includes:

[0043] at least one processor; and,

[0044] a memory communicatively connected to the at least one processor; wherein,

[0045] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the personalized video push optimization method described in any one of the above.

[0046] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium storing computer instructions for causing a computer to execute the personalized video push optimization method described in any one of the above.

[0047] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product including a computer program / instructions, and when the computer program / instructions are executed by a processor, the steps of the method described in any one of the above are implemented.

[0048] The personalized video push optimization method provided by the embodiment of the present disclosure can more accurately understand the interests and needs of users before they use the target video application by actively collecting and analyzing input data, providing a reliable basis for subsequent optimization of video push. By tracking the dynamic changes of video content in real time to determine the screenshot timing, key moments can be accurately captured, reducing invalid screenshots, and by analyzing such video screenshots based on dynamic changes, the understanding of video content can be enhanced, and thus accurate video information can be obtained. Matching the obtained preference information with the video information obtained by analyzing the video screenshots, it can be determined whether the user is interested in the currently playing video according to the matching result, and then the agent is controlled to simulate the user's browsing of the video, and different interaction behaviors are performed on the target video application. With these specific interaction behaviors, more detailed information is fed back to the target video application, gradually improving the problem of inaccurate push in the initial stage of user use of the application, guiding it to accurately push videos, thereby reducing the time for users to browse irrelevant videos, improving the efficiency of users to obtain valuable video content, making the pushed videos more in line with user expectations, and enhancing user satisfaction.

[0049] The above description is only an overview of the technical solution of the present disclosure. In order to understand the technical means of the present disclosure more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present disclosure more obvious and understandable, the following preferred embodiments are specifically given and described in detail in conjunction with the drawings as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the accompanying drawings required for the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0051] Figure 1 It is a flowchart of the personalized video push optimization method provided by the embodiments of the present disclosure;

[0052] Figure 2 It is a flowchart of the intelligent screenshot method provided by the embodiments of the present disclosure;

[0053] Figure 3 It is a flowchart of the method for obtaining the first threshold and the second threshold provided by the embodiments of the present disclosure;

[0054] Figure 4 It is a flowchart of the method for performing an interactive operation provided by the embodiments of the present disclosure;

[0055] Figure 5 It is a principle block diagram of the personalized video push optimization system provided by the embodiments of the present disclosure;

[0056] Figure 6 It is a structural diagram of a computer device provided by the embodiments of the present disclosure. Detailed implementation manners

[0057] The following will describe the embodiments of the present disclosure in detail with reference to the accompanying drawings.

[0058] It should be clear that the following illustrates the implementation manners of the present disclosure through specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of the present disclosure, rather than all embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present disclosure without creative efforts belong to the scope of protection of the present disclosure.

[0059] Note that the following description relates to various aspects of embodiments within the scope of the appended claims. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of the aspects set forth herein can be used to implement a device and / or practice a method. Additionally, this device can be implemented and this method can be practiced using other structures and / or functionality in addition to one or more of the aspects set forth herein.

[0060] It should also be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present disclosure schematically. Only the components related to the present disclosure are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The types, quantities, and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0061] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the aspects can be practiced without these specific details.

[0062] Referring to Figure 1 , the present disclosure provides a personalized video push optimization method, including the following steps:

[0063] S1: Collect user input data, parse the input data, and obtain the user's preference information;

[0064] S2: After the target video application is started, track the dynamic changes of the currently playing video content, and capture a screenshot of the currently playing video based on the degree of change of the video content to obtain a video screenshot;

[0065] S3: Parse the video screenshot to obtain video information;

[0066] S4: Match the preference information with the video information, and control the intelligent agent to interact with the target video application based on the matching result.

[0067] At present, it is difficult for the target video application to obtain sufficient and accurate behavior data in the initial stage of user use. Due to the lack of rich data support, the constructed user interest portrait is bound to be biased and it is difficult to accurately capture the real interests and needs of users. At the same time, in order to cover various types of content as widely as possible to meet the potential needs of different users, the target video application may be overly broad in algorithm design, resulting in a large number of irrelevant videos being pushed in the initial stage of user use. In addition, in the process of data processing and analysis, the target video application may not fully consider the dynamic changes of user interests and the demand differences in different scenarios, resulting in a low matching degree between the pushed content and the actual needs of users.

[0068] The personalized video push optimization method provided by the present disclosure can more accurately understand the interests and needs of users before they use the target video application by actively collecting and analyzing input data, providing a reliable basis for subsequent optimization of video push. By tracking the dynamic changes of video content in real time to determine the screenshot timing, it is possible to accurately capture key moments, reduce invalid screenshots, and by analyzing such video screenshots based on dynamic changes, it is possible to enhance the understanding of video content and thus obtain accurate video information. Matching the obtained preference information with the video information obtained by analyzing the video screenshots, according to the matching result, it can be judged whether the user is interested in the currently playing video, and then control the intelligent agent to simulate the user browsing the video and perform different interaction behaviors on the target video application. With the help of these specific interaction behaviors, more detailed information is fed back to the target video application, gradually improving the problem of inaccurate push in the initial stage of user use, guiding it to push videos accurately, thereby reducing the time for users to browse irrelevant videos, improving the efficiency of users to obtain valuable video content, making the pushed videos more in line with user expectations, and enhancing user satisfaction.

[0069] In S1, the user can pre-select the target video application to be monitored in the front-end interface of the system and set personal interest preferences and disliked content. In addition, the user can also directly make relevant settings in the target video application. Among them, the system supports diversified data input methods, providing users with a convenient and personalized operation experience. The specific data input methods include: one is text input, where the user can directly type detailed information such as their favorite video types and content elements in the input box; the second is voice input, where the user describes their preferences through voice, and the system uses advanced speech recognition technology to convert the voice into text for processing; the third is image upload, where the user can upload pictures to express their love or dislike for specific types of videos. For example, uploading a landscape picture represents a preference for natural scenery videos; the fourth is video upload, where the user can upload video clips they like or dislike to clearly express their love or dislike for such videos.

[0070] If the user inputs data reflecting personal preferences in the target video application, obtain these preference data from the target video application by data scraping or extraction.

[0071] Based on the above, the user's input data includes at least one of text, voice, personal preference images, and personal preference video clips. Automatically collect these input data of each user, store them in a preset user preference information database, and classify and organize the data according to different users. The multimodal large model will then read the input data of each user from this database and process them. By using the user preference information database, the multimodal large model can directly read the classified and organized user data, without repeated collection and classification, saving processing time and computing resources and accelerating the model processing speed.

[0072] Among them, the multimodal large model includes a multimodal input layer, a feature fusion layer, a semantic understanding layer, and a text generation layer. The multimodal input layer is responsible for receiving different modal data from the user preference information database and preprocessing these data. The processing methods include preliminary cleaning and preprocessing of the text, removing special characters, stop words, etc.; using speech recognition technology to convert the speech signal into text and performing the same text preprocessing operations; using image feature extraction algorithms, such as convolutional neural networks (CNNs), to extract the visual features of personal preference images and convert the picture information into feature vectors; performing frame-by-frame processing on personal preference video clips, extracting features from each frame of the image, and at the same time extracting the audio information of the video and converting it into text, and comprehensively processing to obtain the feature representation of the video.

[0073] The feature fusion layer uses early fusion, late fusion, or hybrid fusion methods to fuse different modal features output by the multimodal input layer to obtain fused features. The semantic understanding layer uses pre-trained language models (such as GPT series, BERT, etc.) to further process the fused features, map them into the semantic space, mine the semantic information therein, perform operations such as classifying and clustering the features, understand information such as the video types and content elements that the user likes, and output the semantic understanding result of the user's needs and pass it to the text generation layer. The text generation layer uses natural language generation technology (such as the sequence-to-sequence (Seq2Seq) model) to convert the semantic understanding result into a comprehensive text description of the user's preferences, and this text description is the user's preference information.

[0074] During the training phase of the multimodal large model, a large amount of video-related text data is collected, including video titles, descriptions, comments, etc., as well as user evaluations and preference descriptions of the videos, to form a text sample set; a large amount of video-related voice information is recorded, such as users' oral evaluations and recommendations of the videos, and the corresponding text content is annotated to form a voice sample set; various types of pictures are collected, including video covers, screenshots, etc., and information such as the video types and content elements represented by the pictures is annotated to form an image sample set; different types of videos are collected, the content, types, tags, etc. of the videos are annotated, and at the same time, the audio information of the videos is extracted and the corresponding text is annotated to form a video clip sample set. According to a preset ratio, the collected text sample set, voice sample set, image sample set, and video clip sample set are split to generate multiple single-modal sample sets and multimodal sample sets. For example, 70% of the text sample set is split into a text modality sample set, 70% of the voice sample set is split into a voice modality sample set, 70% of the image sample set is split into an image modality sample set, 70% of the video clip sample set is split into a video modality sample set, and the remaining 30% of the text sample set, voice sample set, image sample set, and video clip sample set are combined to form a multimodal sample set.

[0075] The multimodal large model is split into exclusive submodels adapted to different modalities, and the corresponding exclusive submodels are pre-trained using single-modal sample sets. For example, the text exclusive submodel is trained using the text modality sample set to prompt it to learn the characteristics and patterns of text data; the voice exclusive submodel is trained using the voice modality sample set to enable it to master the laws of voice signals; the image exclusive submodel is trained using the image modality sample set to extract the key features of images; the video exclusive submodel is trained using the video modality sample set to enable it to understand the characteristics of video content.

[0076] The pre-trained exclusive submodels are integrated to form a fusion model, and the fusion model is jointly fine-tuned using the multimodal sample set. During the fine-tuning training process, the backpropagation algorithm is used to adjust the parameters of the fusion model. The backpropagation algorithm will backpropagate the error information from the text generation layer to the multimodal input layer according to the error between the output of the fusion model and the true label, and update each parameter in the fusion model based on this. By continuously iteratively optimizing the parameters, the fusion model can better fuse information from different modalities, accurately understand the user's needs, and finally generate a trained multimodal large model.

[0077] In S2, the system function bar has an option of "automatically obtaining control rights after the video tool is started". If the user checks this option, when the user opens the target video application, the system will automatically start monitoring the video playback content. If the user does not check this option, the system will remind the user to start the function of automatically simulating user behavior. After the user starts this function, the system will immediately start monitoring the video playback content and determine the screenshot timing according to the content of the currently played video, providing strong support for subsequent video information acquisition.

[0078] Referring to Figure 2 the schematic flowchart of the intelligent screenshot method shown, "tracking the dynamic changes of the currently played video content, taking screenshots of the currently played video based on the degree of video content change to obtain video screenshots" includes the following steps:

[0079] S21: Obtain the video stream of the currently played video and extract video frames at preset time intervals.

[0080] S22: Extract the scene features and the positions of human key points of the video frames.

[0081] S23: Based on the scene features, obtain the scene change degree value of the current video content.

[0082] S24: Based on the positions of human key points, obtain the human action change degree value of the current video content.

[0083] S25: When the scene change degree value is greater than the first threshold or the human action change degree value is greater than the second threshold, take a screenshot of the currently played video to obtain a video screenshot.

[0084] In S21, after the target video application is started, obtain the video stream through the interface provided by the operating system (such as Android or iOS) or the target video application. For the video stream, extract video frames at fixed time intervals. These video frames will be used as the basic data for subsequent analysis. Among them, the fixed time interval is the preset time, and the specific value is adjusted according to the actual situation. For example, the value can be 0.1 second or 5 seconds.

[0085] In S22 - S24, for scene change detection, the scene features include at least one of color features and texture features. Convert the video frames to a specific color space, such as the RGB or HSV color space, and then extract the color features of the video frames. Based on these color features, construct a color histogram, which can effectively reflect the overall color distribution of the video frames. Use the gray-level co-occurrence matrix (GLCM) algorithm to extract the texture features of the video frames.

[0086] In one embodiment, methods such as the histogram intersection method and the Bhattacharyya distance are used to calculate the similarity between the color histograms of adjacent video frames, which is defined as the first similarity. The Euclidean distance algorithm or the cosine similarity algorithm is used to calculate the similarity between the texture features of adjacent video frames, denoted as the second similarity. At this time, the current video content includes two adjacent video frames, and the scene change degree value can be the first similarity, the second similarity, or the weighted sum of the first similarity and the second similarity. This method is applicable to the scenario when the preset time is relatively large.

[0087] In another embodiment, for the video frames within a preset time range, the average value of the color histograms is calculated to obtain the average color histogram, and the average texture features are calculated. For example, the video frames within every 5 seconds can be averaged. The average color histograms of two adjacent video frames within the preset time range are obtained, and the first similarity between these two average color histograms is calculated. At the same time, the average texture features of two adjacent video frames within the preset time range are obtained, and the second similarity between these two average texture features is calculated. At this time, the current video content includes two adjacent video frames within the preset time range, and the value of the change degree of the human action can be the first similarity, the second similarity, or the weighted sum of the first similarity and the second similarity. This method is applicable to the scenario when the preset time is relatively small.

[0088] In the field of human pose estimation, usually, key points of the human in the video frame are detected. These key points are specific position points that describe the human pose and structure. For example, 17 key points are defined in the COCO dataset, specifically including the nose, eyes, ears, shoulders, elbows, wrists, hips, knees, and ankles, etc.

[0089] For human action change detection, in one embodiment, the positions of the key points of each human in the video frame are accurately detected. Based on the position information of the human key points in adjacent video frames, the displacement of each human key point between adjacent frames is calculated, and then the average value of the displacements of all key points in adjacent video frames is calculated. This average value is the scene change degree value of the current video content. At this time, the current video content includes two adjacent video frames, and this method is applicable to the scenario when the preset time is relatively large.

[0090] In another embodiment, the position information of the human key points in the video frames within each preset time range is extracted. Based on the position information, the average displacement of the human key points in the video frames within two adjacent preset time ranges is obtained, and this average value is the scene change degree value of the current video content. For example, for the video frames within every 5 seconds, first calculate the average position of each key point during this period, and then calculate the displacement between the average positions within two adjacent time periods. At this time, the current video content includes the video frames within two adjacent preset time ranges, and this method is applicable to the scenario when the preset time is relatively small.

[0091] In S25, compare the scene change degree value with a preset first threshold, and compare the human action change degree value with a second threshold. When any one of the scene change degree value and the human action change degree value is greater than its corresponding threshold, immediately take a screenshot of the currently playing video. Through this method, when the target video application plays a video, the timing for taking a screenshot can be accurately located to obtain a screenshot that can contain the key elements of the currently playing video.

[0092] In S3, in addition to automatically taking a screenshot of the currently playing video to obtain a video screenshot, the audio and title of the currently playing video are also extracted. Using the multi-modal large model technology, the currently playing video is comprehensively understood through the video screenshot, audio, and title, and a textual summary of the currently playing video is output, and this summary is the video information. The technical principle of extracting video information here is the same as that of obtaining preference information, and will not be elaborated here.

[0093] To avoid excessive screenshot frequency from causing too much impact on the system performance, the system will adaptively adjust the first threshold and the second threshold according to the hardware resource status of the intelligent terminal device, so as to control the screenshot frequency within a suitable range. Refer to Figure 3 the flow schematic diagram of the method for obtaining the first threshold and the second threshold shown, the method for obtaining the first threshold and the second threshold includes the following steps:

[0094] S261: Collect the current hardware resource data of the user's intelligent terminal device according to the preset hardware resource metrics;

[0095] S262: Obtain the adjustment coefficient based on the current hardware resource data;

[0096] S263: Obtain the first threshold and the second threshold based on the adjustment coefficient, the preset basic threshold, and the second basic threshold.

[0097] In S261, on a test device with a hardware configuration similar to that of the target intelligent terminal device, a series of different screenshot frequencies are set, such as 1 time per minute, 5 times per minute, 10 times per minute, etc. Run for a period of time (such as 30 minutes) at each screenshot frequency, and at the same time use the built-in system performance monitoring tool (such as the Task Manager in the Windows system, the top command in the Linux system, the Activity Monitor in the macOS system, etc.) or third-party performance monitoring software (such as HWMonitor, Master Lu, etc.) to monitor various indicators, including but not limited to CPU usage, memory usage, disk I / O read and write speed, GPU usage, etc. Compare the changes in various indicators at different screenshot frequencies, find out the indicators that change significantly as the screenshot frequency increases, and determine these indicators as the hardware resource indicators affected by the screenshot frequency.

[0098] In S262 and S263, when the target video application starts, request the user to grant necessary permissions such as access to system performance information. After the request is successful, obtain the current hardware resource data through the API provided by the device operating system according to the determined hardware resource indicators. Perform operations such as weighted summation on the current hardware resource data to obtain an adjustment coefficient, multiply the adjustment coefficient by the first base threshold to obtain the first threshold, and multiply the adjustment coefficient by the second base threshold to obtain the second threshold.

[0099] To further reduce the impact of screenshot operations on system performance, the system adaptively adjusts the resolution of the intercepted video screenshots according to the hardware resource status of the intelligent terminal device, divides different intervals according to the adjustment coefficient, and each interval corresponds to a fixed screenshot resolution.

[0100] In S4, the matching result of the preference information and the video information is output to the Agent control module, and the agent automatically interacts with the target video application according to the matching result. Among them, the matching result is specifically divided into a classification result or a grading result. In the first case, the binary classification method is used to distinguish the matching result into two categories: success and failure. When the matching is successful, the agent will automatically perform a series of positive feedback operations, specifically including liking, commenting, favoriting, sharing, and continuing to watch the currently playing video; when the matching fails, the agent will automatically perform negative feedback operations, specifically including negatively marking the currently playing video (for example, selecting "dislike" after long pressing the screen) and terminating the current video playback. This binary classification method has a simple and efficient logical architecture, and only needs to judge the compliance of the matching result according to the preset standard, so as to achieve fast processing of low computational cost and large-scale data. By clearly setting the threshold, this method can accurately screen out the video content that fits the user's preferences. Subsequently, the agent feeds back the user's preferences to the target video application through interactive operations, enabling it to accurately grasp the user's preferences in a short time, and optimize its own mechanism accordingly, effectively reducing the probability of ineffective recommendations and significantly improving the recommendation efficiency and accuracy.

[0101] In the second case, according to the matching degree between the preference information and the video information, the matching results are divided into multiple levels, for example, divided into 5 levels, namely "especially like", "like", "average", "dislike" and "very dislike" in sequence. The agent performs different simulated user operations according to the levels to achieve interaction with the target video application. For example, when the level is "especially like", the agent likes the currently playing video with a high probability, posts positive comments, collects the video, searches for relevant keywords, and may trigger the sharing operation; when the level is "like", the agent likes, comments on, and collects the currently playing video with a certain probability; when the level is "average", the agent views the comment area of the currently playing video with a low probability; when the level is "dislike", the agent makes a negative mark on the currently playing video; when the level is "very dislike", the agent makes a negative mark on the currently playing video and immediately terminates the current video playback. For example, when the agent simulates a user watching a food video, it will monitor the video content in real time and generate video information (such as "A food blogger named XX is enjoying steak, the environment is luxurious, the background music is elegant, and the picture color scheme is gorgeous"), and predict the matching degree according to the user's preference information (such as "The user likes food, photography, music, and prefers high-end restaurants with an elegant environment"). The prediction result shows that the user's preference for this video is "like", and the agent controls the background to perform operations such as liking, commenting on, and collecting the video with a certain probability. After the currently playing video is played, the next video will be automatically played. This multi-level division method not only evaluates whether the user likes the video, but further subdivides the user's preference degree, realizing the refined capture of the user's preferences. The agent feeds back this refined preference information to the target video application through interactive operations, helping it gradually optimize the video push mechanism. During the optimization process, the target video application can consider more dimensions of content, making the video push mechanism more flexible, and thus meeting the personalized needs of different users in diverse scenarios.

[0102] Among them, after the currently playing video is played, the following two methods can be selected to achieve automatic playback of the next video. One is to trigger the "automatic continuous playback" function of the target video application by the user's active opening or system call, and play the next video in the video list according to the preset logic of the target video application; the other is to use the agent to simulate user instructions (such as simulating the sliding event of the touch screen or sending the corresponding mouse wheel event) to trigger the sliding operation of the video playback interface, so as to load and play new video content.

[0103] In a specific implementation, the method for obtaining the matching result is as follows: Input the preference information and video information into a pre-set large language model (such as ChatGPT). Through the large language model, perform semantic comparison and analysis on the preference information and video information, and output the matching result. Before the large language model is put into actual application, judgment cases for binary classification (success and failure) or five-level rating (especially like, like, average, dislike, very dislike) need to be embedded in its prompt, so that the model can accurately understand the specific classification criteria of the matching degree. This matching method using the large language model can deeply understand the semantics of the preference information and video information, understand the context, and process complex comparison logics, thus providing more accurate results.

[0104] In another specific implementation, the method for obtaining the matching result is as follows: Extract multiple first keywords from the user preference information and determine the sentiment tendency of each first keyword. Assign weights to each first keyword according to the sentiment tendency. For example, assign positive weights (such as +1, +2, etc.) to the keywords with positive sentiment tendency according to the degree of preference, and assign negative weights (such as -1, -2, etc.) to the keywords with negative sentiment tendency according to the degree of dislike. Extract multiple second keywords from the video information. Through the cosine similarity algorithm or Jaccard similarity algorithm, obtain the similarity between each second keyword and the first keyword, and use the weight of the first keyword as the weighting coefficient of the similarity. Based on the weighting coefficient, perform weighted summation on the similarity to obtain the matching score, and determine the matching result based on the matching score. This keyword matching method is simple to implement and has a fast calculation speed, which is suitable for the rapid screening of large-scale data.

[0105] Referring to Figure 4 the flow schematic diagram of the method for performing interactive operations shown, "Based on the matching result, control the intelligent agent to perform different simulated user operations to achieve interaction with the target video application" includes the following steps:

[0106] S41: Trigger the intelligent agent to generate an operation instruction based on the matching result. The operation instruction includes the first identifier of the target video application, the second identifier of the currently playing video, and the type of interactive operation;

[0107] S42: Send the operation instruction to the corresponding adaptation module based on the first identifier;

[0108] S43: Use the adaptation module to convert the operation instruction into a native interactive instruction recognizable by the target video application. The native interactive instruction includes the standard identifier converted from the second identifier and the standard interactive type converted from the type of interactive operation;

[0109] S44: Execute the native interactive instruction to perform corresponding interactive operations on the currently playing video.

[0110] In S41, the operation instruction is a formatted instruction that contains the necessary parameters indicating the operation. Among them, the first identifier can be the name, package name, or unique number of the application, which is used to clearly identify the specific application targeted by the operation. The second identifier includes the title, ID, or URL of the video, etc., which is used to accurately point to the currently playing video. Different numbers or symbols can be used to represent the types of interaction operations. For example, "1" represents the like operation, "2" represents the share operation, "3" represents the favorite operation, "4" represents the pause operation, etc.

[0111] In S42, for each video application that reaches a cooperation, a corresponding adaptation module is pre-constructed. Since each video application has unique interfaces and protocols, the main function of the adaptation module is to convert the general operation instruction into a format that the corresponding application can recognize. After the agent generates the operation instruction, it will send the instruction to the corresponding adaptation module according to the first identifier in the instruction.

[0112] In S43, the adaptation module parses the operation instruction according to the interface specifications and protocols of the target video application, maps and converts the second identifier and the interaction operation type in it into the standard identifier and standard interaction type that the target video application can directly execute, thereby generating a native interaction instruction.

[0113] In S44, the adaptation module sends the converted native interaction instruction to the target video application for execution. After receiving the native interaction instruction, the application performs the corresponding interaction operations on the currently playing video according to the requirements of the instruction, such as liking, sharing, and favoriting.

[0114] There are differences in the interfaces and protocols of different video applications. This method can make the general operation instructions generated by the agent adapt to various applications through the instruction conversion of the adaptation module, realizing the effective interaction between the agent and different video applications, and enhancing the compatibility and generality of the system. For each video application, only one adaptation module needs to be constructed to handle the instruction conversion, rather than developing different interaction logics separately for the agent for each application, reducing the development workload and cost. Moreover, the adaptation module converts the general operation instruction into a native interaction instruction that the application can directly execute, avoiding the application from performing additional parsing and processing on the instruction, enabling the interaction behavior to be executed more quickly and accurately, and improving the interaction efficiency of the system. When the interface or protocol of the video application changes, only the corresponding adaptation module needs to be modified and adjusted, without affecting the normal operation of the agent and other applications, facilitating the maintenance and subsequent expansion of the system.

[0115] Furthermore, the matching result between the preference information and the video information determines whether the agent continues to watch the currently playing video. If the interaction operation performed by the agent on the target video application includes pausing the playback, the dynamic change tracking of the currently playing video content will end. At the same time, the agent controls the target video application to play the next video, and then performs dynamic change tracking on the next video. If the interaction operation performed by the agent on the target video application does not include pausing the playback, when the interaction operation performed by the agent on the target video application does not include pausing the playback, continue to track the dynamic changes of the currently playing video content, take a screenshot of the currently playing video based on the degree of change of the new video content to obtain a new video screenshot, and then based on the accumulated video screenshots of the currently playing video, new and more accurate video information can be obtained to update the video information, and the new video information is continued to be matched with the preference information until the currently playing video is paused or played to completion. Through this method, the agent can simulate the user's viewing habits, pause the playback in a timely manner when not interested in the video content, and watch the interested video content completely, so as to better adapt to the interaction requirements of the target video application and make the interaction behavior with the target video application more close to the operation habits of real users.

[0116] The target video application comprehensively collects various types of interaction behavior data according to the interaction behavior, such as playing, pausing, fast-forwarding, rewinding, liking, commenting, favoriting, sharing, search keywords, etc. At the same time, it records the relevant information of the video, including the title, tags, category, duration, release time, etc. Based on the interaction behavior data, it helps the target video application to build and optimize the user interest portrait and improve the push effect of the target video application.

[0117] In summary, the personalized video push optimization method of the present disclosure can accurately extract the user's preference information and the video information of the currently playing video by means of multi-modal data processing technology, and provides two matching mechanisms of binary classification and rank division for users to choose to meet the different needs of users. Based on the matching result between the preference information and the video information, the agent can automatically perform relevant operations to replace the user's manual intervention, realize the automatic optimization of the video push mechanism, and without the user manually operating the intelligent terminal device (including various common devices such as mobile phones, tablets, and computers), the video content can be automatically updated, thus liberating the user's hands and effectively saving the user's time.

[0118] Refer to Figure 5 , the present disclosure provides a personalized video push optimization system, including:

[0119] A user input module 101, configured to collect user input data, parse the input data, and obtain the user's preference information;

[0120] The video screenshot module 102 is used to track the dynamic changes of the currently playing video content after the target video application is launched, take screenshots of the currently playing video based on the degree of video content change, and obtain video screenshots.

[0121] The video parsing module 103 is used to parse the video screenshots to obtain video information.

[0122] The information matching module 104 is used to match the preference information with the video information, and control the intelligent agent to interact with the target video application based on the matching result.

[0123] The various change methods and specific examples in the above-provided personalized video push optimization method are equally applicable to the personalized video push optimization system provided by the present disclosure. Through the foregoing detailed description of the personalized video push optimization method, those skilled in the art can clearly know the implementation method of the personalized video push optimization system. For the sake of simplicity of the specification, it will not be elaborated here.

[0124] The computer device according to an embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0125] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to execute the desired functions. In an embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device executes all or part of the steps of the personalized video push optimization method according to the various embodiments of the present disclosure described above.

[0126] Those skilled in the art should understand that in order to solve the technical problem of how to obtain good user experience effects, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included in the protection scope of the present disclosure.

[0127] As Figure 6 FIG. is a schematic structural diagram of a computer device provided by an embodiment of the present disclosure. It shows a schematic structural diagram of a computer device suitable for implementing the computer device in the embodiments of the present disclosure. Figure 6 The shown computer device is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present disclosure.

[0128] As shown Figure 6 in the figure, a computer device may include a processor (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0129] Generally, the following devices may be connected to the I / O interface: an input device including, for example, a sensor or a visual information acquisition device, etc.; an output device including, for example, a display screen, etc.; a storage device including, for example, a magnetic tape, a hard disk, etc.; and a communication device. The communication device may allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or wiredly to exchange data. Although Figure 6 a computer device with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0130] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by the processor, all or part of the steps of the personalized video push optimization method of the embodiments of the present disclosure are executed.

[0131] For a detailed description of this embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, and details are not repeated here.

[0132] A computer-readable storage medium according to an embodiment of the present disclosure has non-temporary computer-readable instructions stored thereon. When the non-temporary computer-readable instructions are run by a processor, all or part of the steps of the personalized video push optimization method of the foregoing embodiments of the present disclosure are executed.

[0133] The above-mentioned computer-readable storage medium includes but is not limited to: optical storage media (such as: CD-ROM and DVD), magneto-optical storage media (such as: MO), magnetic storage media (such as: magnetic tape or removable hard disk), media with built-in rewritable non-volatile memory (such as: memory card) and media with built-in ROM (such as: ROM cartridge).

[0134] For a detailed description of this embodiment, reference may be made to the corresponding descriptions in the foregoing embodiments, which will not be elaborated herein.

[0135] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations. It cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-mentioned specific details of the disclosure are only for illustrative and easy-to-understand purposes and are not limitations. The above details do not limit the present disclosure to necessarily adopt the above specific details to implement.

[0136] In the present disclosure, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of the devices, apparatuses, equipment, and systems involved in the present disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended terms meaning "including but not limited to" and can be used interchangeably with them. The words "or" and "and" used herein refer to the word "and / or" and can be used interchangeably with it, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably with it.

[0137] In addition, as used herein, the "or" used in the listing of items starting with "at least one" indicates a separate listing, so that for example, the listing of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). In addition, the term "exemplary" does not mean that the described examples are preferred or better than other examples.

[0138] It should also be noted that in the systems and methods of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present disclosure.

[0139] Various changes, substitutions, and alterations to the technology described herein can be made without departing from the teachings defined by the appended claims. Additionally, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and acts described above. Current or later-developed processes, machines, manufactures, compositions of events, means, methods, or acts that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Accordingly, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or acts within their scope.

[0140] The foregoing description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0141] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some of their variations, modifications, alterations, additions, and subcombinations.

Claims

1. A personalized video push optimization method, characterized in that Including: Collect user input data, parse the input data, and obtain the user's preference information; After the target video application is launched, track the dynamic changes of the currently playing video content, take a screenshot of the currently playing video based on the degree of video content change, and obtain a video screenshot; Parse the video screenshot to obtain video information; Match the preference information with the video information, and based on the matching result, control the agent to interact with the target video application; Among them, the matching the preference information with the video information and controlling the agent to interact with the target video application based on the matching result includes: Input the preference information and the video information into a preset large language model to obtain a classified or graded matching result; Based on the matching result, control the agent to execute different simulated user operations to achieve interaction with the target video application; The target video application optimizes the video push mechanism of the target video application based on the interaction operation; Among them, the controlling the agent to execute different simulated user operations based on the matching result to achieve interaction with the target video application includes: Based on the matching result, trigger the agent to generate an operation instruction, and the operation instruction includes the first identifier of the target video application, the second identifier of the currently playing video, and the interaction operation type; Send the operation instruction to the corresponding adaptation module based on the first identifier; Use the adaptation module to convert the operation instruction into a native interaction instruction recognizable by the target video application, and the native interaction instruction includes a standard identifier converted from the second identifier and a standard interaction type converted from the interaction operation type; Execute the native interaction instruction to perform corresponding interaction operations on the currently playing video.

2. The personalized video push optimization method according to claim 1, wherein The collecting user input data, parsing the input data, and obtaining the user's preference information includes: Receive the user's input data through the front-end interface or capture the user's input data from the target video application; Classify and store the input data of each user in the user preference information database; Use a preset multi-modal large model to read the input data of each user from the user preference information database and output the preference information of each user.

3. The personalized video push optimization method according to claim 1, characterized in that The tracking the dynamic changes of the currently playing video content, taking a screenshot of the currently playing video based on the degree of video content change, and obtaining a video screenshot includes: Obtain the video stream of the currently playing video and extract video frames at preset time intervals; Extract the scene features and the positions of human key points of the video frame; Based on the scene features, obtain the scene change degree value of the current video content; Based on the positions of the human key points, obtain the human action change degree value of the current video content; When the scene change degree value is greater than the first threshold or the human action change degree value is greater than the second threshold, take a screenshot of the currently playing video to obtain a video screenshot.

4. The personalized video push optimization method according to claim 3, wherein It also includes: Collect the current hardware resource data of the user's intelligent terminal device according to the preset hardware resource indicators; Obtain an adjustment coefficient based on the current hardware resource data; Obtain the first threshold and the second threshold based on the adjustment coefficient, a preset first base threshold, and a second base threshold.

5. The personalized video push optimization method according to claim 3, wherein It further includes: When the interaction operation performed by the intelligent agent on the target video application does not include pausing the playback, continue to track the dynamic changes of the currently played video content, and capture a screenshot of the currently played video based on the degree of change of the new video content to obtain a new video screenshot. Update the video information based on the video screenshots accumulated for the currently played video.

6. A computer device, characterized in that, The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the personalized video push optimization method according to any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to execute the personalized video push optimization method according to any one of claims 1-5.

8. A computer program product, comprising computer instructions, characterized in that, When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1-5 are implemented.

Images