Video optimization method, device, equipment and storage medium

Abstract

The present application discloses a video optimization method, device, equipment and storage medium, which relates to the technical field of big data processing and data visualization. The method includes: screening the initial video data of the target video obtained from at least one data source to obtain the target video data; determining the video optimization data of the target video according to the target video data; wherein the target video data includes the target video content, network information and user feedback information; generating a video optimization strategy for the target video according to the video optimization data; optimizing the target video according to the video optimization strategy. The above technical solution realizes dynamic optimization of the video by integrating data from video content, network status and user feedback, and adaptively adjusting the video optimization strategy according to the integrated data, and helps to improve user satisfaction.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, in particular to the field of big data processing and data visualization technology, and specifically to a video optimization method, device, equipment and storage medium. Background Art

[0002] With the rapid development of Internet technology, video streaming and playback has become a daily network behavior. High definition, no buffering, and fast loading have become basic expectations of users. In order to meet these expectations, a variety of video optimization technologies have emerged.

[0003] A common traditional method is to dynamically adjust the bit rate based on network bandwidth. This method monitors the network bandwidth in real time and dynamically adjusts the video bit rate according to bandwidth changes to achieve smooth playback. However, this method only relies on network conditions and ignores video content and user feedback. It may result in video quality that cannot meet user needs when the network condition is good but the video content is complex or the user has high requirements for clarity. Summary of the invention

[0004] The present application provides a video optimization method, apparatus, device and storage medium to achieve dynamic optimization of videos and improve user satisfaction.

[0005] According to one aspect of the present application, a video optimization method is provided, the method comprising:

[0006] Screening initial video data of a target video obtained from at least one data source to obtain target video data;

[0007] Determine the video optimization data of the target video according to the target video data; wherein the target video data includes target video content, network information and user feedback information;

[0008] Generating a video optimization strategy for the target video according to the video optimization data;

[0009] The target video is optimized according to the video optimization strategy.

[0010] According to another aspect of the present application, a video optimization device is provided, the device comprising:

[0011] A data screening module, used for screening the initial video data of the target video obtained from at least one data source to obtain the target video data;

[0012] A data integration module, used to determine the video optimization data of the target video according to the target video data; wherein the target video data includes target video content, network information and user feedback information;

[0013] A strategy generation module, used to generate a video optimization strategy for the target video according to the video optimization data;

[0014] The video optimization module is used to optimize the target video according to the video optimization strategy.

[0015] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0016] one or more processors;

[0017] A memory for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement any one of the video optimization methods provided in the embodiments of the present application.

[0019] According to another aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, any one of the video optimization methods provided in the embodiments of the present application is implemented.

[0020] The present application obtains target video data by screening the initial video data of the target video obtained from at least one data source; determines the video optimization data of the target video according to the target video data; wherein the target video data includes the target video content, network information and user feedback information; generates a video optimization strategy for the target video according to the video optimization data; optimizes the target video according to the video optimization strategy. The above technical solution realizes dynamic optimization of the video by integrating data from video content, network status and user feedback, and adaptively adjusts the video optimization strategy according to the integrated data, and helps to improve user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a video optimization method provided according to Embodiment 1 of the present application;

[0022] Figure 2 is a flow chart of a video optimization method provided according to Embodiment 2 of the present application;

[0023] Figure 3 is a structural schematic diagram of a video optimization device provided according to Embodiment 3 of the present application;

[0024] Figure 4It is a structural diagram of an electronic device that implements the video optimization method of an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In addition, it should be noted that in the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of relevant data such as initial video data and target video data are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0028] Embodiment 1

[0029] Figure 1 This is a flow chart of a video optimization method provided according to the first embodiment of the present application. This embodiment is applicable to the case of optimizing the transmission and playback of videos, and can be executed by a video optimization device. The video optimization device can be implemented in the form of hardware and / or software. The video optimization device can be configured in a computer device, such as a server. Figure 1 As shown, the method includes:

[0030] S110, screening initial video data of a target video acquired from at least one data source to obtain target video data;

[0031] The data source refers to the source of video data, which may include at least one of the video content, network status, and user feedback of the target video. The initial video data refers to the relevant data related to the target video directly obtained from the data source, which may include at least one of the content currently played by the target video, the current network status of the target video, and user opinions. The target video data refers to the initial video data that meets the preset data conditions, which are artificially set according to actual conditions or experience values.

[0032] Exemplarily, at least one data source is selected to obtain initial video data of a target video; the initial video data of the target video is collected from the selected data source, and the collected initial video data is screened; and the target video data is extracted from the screened initial video data.

[0033] Optionally, data sampling is performed on initial video data of the target video obtained from at least one data source to obtain intermediate video data; the intermediate video data is screened, and the intermediate video data that meets data quality conditions is used as the target video data.

[0034] Data sampling refers to selecting a portion of sample data from the overall data to represent the entire data set. Intermediate video data refers to video data that can be used for analysis and processing, such as frame rate, resolution, color mode, network delay, network packet loss rate, user operation instructions, etc. Data quality conditions are set manually based on actual conditions or experience values, and can include data quality requirements, such as clarity, stability, and noise-free.

[0035] It should be noted that the data sampling method is artificially set according to actual conditions or empirical values, and may include at least one of random sampling, uniform sampling and systematic sampling.

[0036] S120, determining video optimization data of the target video according to the target video data; wherein the target video data includes target video content, network information and user feedback information;

[0037] The video optimization data refers to data used to determine the direction and strategy of video optimization. The target video content refers to the various attribute information constituting the target video, which may include at least one of the resolution and frame rate of the video. The network information refers to the network environment and transmission information related to the target video, which may include at least one of the network delay and network packet loss rate of the video. The user feedback information refers to the user's evaluation and feedback on the target video, which may include at least one of the user's viewing experience, playback problems, clarity requirements, opinions and suggestions, etc.

[0038] Exemplarily, the target video data is analyzed to obtain analysis results of the target video data; based on the analysis results, the optimization direction of the target video is determined; based on the optimization direction, the optimization parameters of the target video are determined; the analysis results, optimization direction, and optimization parameters are integrated to obtain video optimization data of the target video.

[0039] The analysis result refers to the valuable information in the target video data, which may include at least one of the video clarity, bit rate, frame rate and playback speed. The optimization direction is used to characterize the optimization method of the video, which may include at least one of improving the video quality and improving the video playback speed. The optimization parameters refer to a set of parameters adjusted during the video optimization process, which are used to improve the video quality, enhance the playback effect or meet specific requirements, and may include at least one of the video resolution, compression ratio and frame rate.

[0040] S130, generating a video optimization strategy for a target video according to the video optimization data;

[0041] Among them, video optimization strategy refers to a series of methods and solutions adopted to achieve specific video optimization goals.

[0042] Exemplarily, based on the video optimization data, a video optimization strategy for a target video may be generated by analyzing the video optimization data to determine the optimization goals and optimization priorities of the target video; based on the optimization goals and optimization priorities, selecting appropriate optimization means, such as video transcoding, compression, editing, adding special effects, etc., and formulating optimization plans corresponding to the optimization means; integrating the optimization goals, optimization priorities, optimization means and optimization plans to generate a video optimization strategy for the target video.

[0043] Optionally, based on a Strategic Meta-Learning (SML) algorithm, the video optimization data is analyzed to generate a video optimization strategy for the target video.

[0044] Among them, the SML algorithm refers to a deep learning method that aims to make quick decisions by learning and adjusting strategies. It can quickly adapt and optimize strategies when facing different tasks by leveraging previously learned knowledge and experience.

[0045] Exemplarily, a utility function is defined based on target video data; based on the gradient ascent method of the SML algorithm, the utility function is maximized to obtain a target weight vector; based on the target weight vector, a video optimization strategy for the target video is generated.

[0046] Among them, the utility function is a function that maps features to utility values, and is usually used to describe the contribution of a decision or behavior to a goal. The gradient ascent algorithm is an optimization algorithm used to maximize the value of a function. The target weight vector is a set of parameters used to calculate the utility function, which controls the contribution of each feature to the final utility.

[0047] Furthermore, the utility function can be defined by the following formula:

[0048] U(P,V,N,C)=w1P+w2f(V)+w3g(N)+w4h(C);

[0049] Among them, U refers to the utility function. P refers to the video optimization strategy. V refers to the video content in the video optimization data. N refers to the network information in the video optimization data. C refers to the user feedback information in the video optimization data. w1 refers to the weight of the video optimization strategy, which is used to characterize the influence of the video optimization strategy on the utility function. w2 refers to the weight of the video content in the video optimization data, which is used to characterize the influence of the video content in the video optimization data on the utility function. w3 refers to the weight of the network information in the video optimization data, which is used to characterize the influence of the network information in the video optimization data on the utility function. w4 refers to the weight of the user feedback information in the video optimization data, which is used to characterize the influence of the user feedback information in the video optimization data on the utility function. f refers to the processing function of the video content in the video optimization data. g refers to the processing function of the network information in the video optimization data. h refers to the processing function of the user feedback information in the video optimization data.

[0050] It should be noted that the processing function for video optimization data is based on the SML algorithm and is manually set according to actual conditions or experience values.

[0051] S140: Optimize the target video according to the video optimization strategy.

[0052] Among them, video optimization strategy refers to improving the quality, performance and user experience of video through a series of strategies and technologies, which may include at least one of compression encoding, adaptive bitrate adjustment, video stream optimization and responsive design.

[0053] Exemplarily, after the video optimization strategy is determined, the video optimization strategy is converted into actual output in real time, and the transmission and playback of the video are controlled according to the video optimization strategy to achieve optimization of the target video.

[0054] Optionally, the video optimization strategy is converted into a video control instruction; and the target video is optimized according to the video control instruction.

[0055] The video control instructions refer to instructions for controlling video behavior during video playback and transmission, and may include at least one of controlling the fast forward of the video, adjusting the clarity of the video, and jumping to a specified time period of the video.

[0056] Exemplarily, the video optimization strategy is converted into a control instruction that can directly control the transmission and playback of the target video; according to the video control instruction, the transmission and playback of the target video are controlled to optimize the target video.

[0057] It should be noted that the conversion method of the video optimization strategy is set according to the actual content of the video optimization strategy, and the embodiments of the present application do not specifically limit this.

[0058] Optionally, after optimizing the target video according to the video optimization strategy, target feedback information of the user on the target video is obtained; and the target video data is updated according to the target feedback information.

[0059] The target feedback information refers to the user's evaluation and feedback on the optimized target video, which may include at least one of the user's viewing experience, playback problems, clarity requirements, and opinions and suggestions.

[0060] Exemplarily, after the video optimization strategy is converted into actual output in real time and the target video is optimized, target feedback information of the user on the target video is collected in real time; based on the target feedback information, the initial video data of the target video obtained from at least one data source is re-screened to obtain the target video data, so as to update the target video data and generate a new video optimization strategy.

[0061] Understandably, real-time output and feedback loops are key to ensuring that the video dynamic optimization method can continue and gradually improve. The system not only generates output based on the determined strategies, but also collects user feedback to continuously adjust and optimize these strategies. This continuous learning and optimization process ensures that users always get the best video experience.

[0062] The embodiment of the present application obtains target video data by screening the initial video data of the target video obtained from at least one data source; determines the video optimization data of the target video according to the target video data; wherein the target video data includes the target video content, network information and user feedback information; generates a video optimization strategy for the target video according to the video optimization data; and optimizes the target video according to the video optimization strategy. The above technical solution realizes dynamic optimization of the video by integrating data from video content, network status and user feedback, and adaptively adjusts the video optimization strategy according to the integrated data, and helps to improve user satisfaction.

[0063] Embodiment 2

[0064] Figure 2 It is a flow chart of a video optimization method provided according to Example 2 of the present application. Based on the technical solutions of the above-mentioned embodiments, this embodiment refines "determining the video optimization data of the target video according to the target video data; wherein the target video data includes the target video content, network information and user feedback information" into "based on the video optimization model, parsing the target video content to obtain the target video optimization content; predicting the network status of the target video according to the network information to obtain the network prediction data of the target video; identifying the user feedback information to obtain the user demand information; merging the target video optimization content, network prediction data and user demand information to obtain the video optimization data of the target video". It should be noted that for the parts not described in detail in the embodiments of the present application, please refer to the relevant statements of other embodiments. Figure 2 As shown, the method includes:

[0065] S210: Screening initial video data of a target video acquired from at least one data source to obtain target video data.

[0066] S220: Analyze the target video content based on the video optimization model to obtain the target video optimized content.

[0067] The video optimization model refers to a model that analyzes and processes video content based on artificial intelligence, machine learning and other technologies. Its goal is to provide targeted optimization suggestions through in-depth understanding and analysis of video content to improve video quality, reduce resource consumption, or improve user experience. The target video optimization content refers to the optimization suggestions or processing results generated by the video optimization model for the target video, which may include at least one of video encoding parameter optimization, key frame selection and editing suggestions, and object recognition and motion analysis results.

[0068] Optionally, based on the feature extraction network in the video optimization model, features of the target video content are extracted to determine the target key frames of the target video; wherein the feature extraction network is a convolutional neural network; based on the content optimization network in the video optimization model, the video content is analyzed and optimized according to the target key frames to obtain the optimized content of the target video; wherein the content optimization network is a generative adversarial network.

[0069] Among them, the feature extraction network refers to a neural network that extracts features from input data. The target key frame refers to a specific frame with important information or representative content in a video sequence. The convolutional neural network is a deep learning neural network that is mainly used for processing and analyzing two-dimensional data such as images and videos. It consists of an input layer, a convolution layer, an activation function layer, a pooling layer, a fully connected layer, and an output layer. The content optimization network refers to a neural network that generates high-quality video content data. Target video optimization content refers to obtaining higher quality video content that better meets specific requirements by processing and optimizing the target video content video. Generative adversarial networks are a type of deep learning model composed of generators and discriminators. They compete with each other to improve their respective performance through adversarial training to generate realistic samples.

[0070] Furthermore, the training process of the video optimization model is as follows: the initial optimization model is iteratively trained for n rounds using sample video content to obtain a video optimization model: for the first round, based on the feature extraction network in the initial optimization model, feature extraction is performed on the sample video content to obtain sample key frames corresponding to the first round; based on the content optimization network in the initial optimization model, according to the sample key frames corresponding to the first round, the sample video content is analyzed and optimized to obtain sample video optimized content corresponding to the first round; for the i round, based on the feature extraction network in the initial optimization model, feature extraction is performed on the sample video optimized content corresponding to the i-1 round to obtain sample key frames corresponding to the i round; based on the content optimization network in the initial optimization model, according to the sample key frames corresponding to the i round, the sample optimized video content corresponding to the i-1 round is analyzed and optimized to obtain sample video optimized content corresponding to the i round; wherein, i is a natural number greater than 1 and less than n, and n is a natural number greater than 2.

[0071] The sample video content refers to the video content used for model training. The sample key frame refers to the specific frame with important information or representative content extracted from the video sequence by the feature extraction network. The video optimized content refers to the high-quality video content data generated by the content optimization network.

[0072] Exemplarily, the training process of the feature extraction network is as follows: prepare data sets and labels, and divide the training data sets into training sets, validation sets, and test sets; define the structure and hierarchy of the feature extraction network, and build a feature extraction network model; initialize the parameters of the feature extraction network, and input the training data into the network, and calculate layer by layer from the input layer through the forward propagation process until the final prediction result is obtained; during the forward propagation process, the convolution layer will perform convolution operations on the input data and use activation functions to process, the pooling layer will downsample the convolution output, and the fully connected layer will flatten the downsampled feature map and calculate; by comparing the network output and the true label, the loss function is calculated to measure the difference between the network prediction result and the true result; through the chain rule, the gradient is propagated layer by layer from the output layer to the input layer, and the parameters in the network are updated; according to the gradient information, the parameters of the network are updated using the optimization algorithm; during the training process, the validation set is used to evaluate and adjust the model; after the training is completed, the test set is used to perform a final evaluation and test on the model to evaluate the performance of the model on unseen data; after the training is completed, the feature extraction network is obtained.

[0073] Exemplarily, the training process of the content optimization network is as follows: prepare a training data set, which contains a set of real samples; the content optimization network is composed of a generator and a discriminator; wherein the generator is responsible for generating synthetic samples similar to real samples, and the discriminator is responsible for judging whether a given sample is a real sample or a sample generated by the generator; initialize the parameters of the generator and the discriminator; train the generator and the discriminator, and during the training process, first fix the generator parameters and optimize the discriminator by minimizing the loss function of the discriminator; then fix the discriminator parameters and optimize the generator by maximizing the loss function of the generator, and these two steps are performed alternately to continuously update the parameters of the generator and the discriminator; during the training process, some random noise can be given as input, samples can be generated using the generator, and the quality and realism of the generated samples can be evaluated by the discriminator; after the training is finally completed, a feature extraction network is obtained.

[0074] It can be understood that the present application combines convolutional neural networks and generative adversarial networks, uses the sample key frames extracted by the convolutional neural network as the training data set of the generative adversarial network, and uses the sample video optimization content generated by the generative adversarial network as the training data set of the convolutional neural network; the feature extraction capability of the convolutional neural network and the generation capability of the generative adversarial network are closely integrated to work together in the analysis and optimization of video content, making the processing of video content more accurate and efficient.

[0075] S230: Predict the network status of the target video according to the network information to obtain network prediction data of the target video.

[0076] The network condition refers to the quality and performance of the network connection of the target video. The network prediction data refers to the possible changes in the network condition of the target video in the future.

[0077] Exemplarily, based on a Neural Time Translation (NTT) model, the network status of the target video may be predicted according to network information to obtain network prediction data of the target video.

[0078] Among them, the NTT model is a time series forecasting method based on machine learning, which uses neural networks to learn the patterns of time series data and use these patterns to predict future values.

[0079] Optionally, the network prediction data may be determined by the following formula:

[0080] NTT t )=W*N t-1 +b;

[0081] Where NTT is the time series prediction model. W refers to the weight of the network status at the current moment. t Refers to network information. t-1 refers to the network status at the previous moment. B refers to the bias term, which is used to adjust the overall fit of the model to make it better fit the data.

[0082] It is understandable that by predicting network conditions through the NTT model, possible network fluctuations and instabilities can be predicted in advance, thereby providing users with a smoother and higher-quality video experience.

[0083] S240: Identify user feedback information to obtain user demand information.

[0084] The user demand information refers to the user's demand for target video improvement, which may include at least one of the following: the user wants to reduce the video quality, the user wants to reduce the video sound, and the user wants to improve the video quality.

[0085] Exemplarily, user feedback information may be identified based on a contextual transformer analysis (CTA) method to obtain user demand information.

[0086] Among them, the CTA method is a method for explaining the prediction results of deep learning models. It aims to help us understand how the model makes predictions and discover which words have a greater impact on the model's prediction results.

[0087] Optionally, based on natural language processing technology, context recognition is performed on user feedback information to obtain user context information; user context information is classified and identified to obtain user demand information.

[0088] Natural language processing technology is a method and technology for understanding, processing and generating human language, which may include CTA methods. User context information refers to information describing the user's current context, which may include at least one of the environment, mood and expectation of the user watching the video.

[0089] Furthermore, the user context information can be classified and identified to obtain the user demand information through the following classification model calculation formula:

[0090] Category(C)=argmax i *(Softmax(W c *C(U)+b c ));

[0091] Category(C) is the returned context category with the highest probability. i refers to the index i of the maximum value. Softmax is a function used to normalize the elements in a vector, ensuring that their sum is 1, and convert it into a probability distribution. c Refers to the weight matrix in the classification model. C refers to the input feature vector or feature matrix. U refers to the user context information. c Refers to the bias vector in the classification model.

[0092] It can be understood that the contextual information is first extracted from the user feedback information through the CTA method, and then the contextual information is classified to determine the user's actual needs and expectations, ensuring that the video optimization process always revolves around the user's real experience, thereby providing users with a more personalized and satisfactory video viewing experience.

[0093] S250: Merge the target video optimization content, network prediction data, and user demand information to obtain video optimization data of the target video.

[0094] Specifically, the target video optimization content, network prediction data, and user demand information obtained through different models are formatted in a unified manner to obtain video optimization data for the target video.

[0095] It should be noted that this merger does not mean integrating the target video optimization content, network prediction data, and user demand information into one data set, but unifying their formats to ensure that they can be used as a unified type of input for the subsequent generation of video optimization strategies.

[0096] S260: Generate a video optimization strategy for the target video according to the video optimization data.

[0097] Specifically, the target video optimization content, network prediction data and user demand information in the video optimization content are respectively used as inputs of the SML algorithm to generate a video optimization strategy for the target video.

[0098] S270: Optimize the target video according to the video optimization strategy.

[0099] The embodiment of the present application obtains target video data by screening the initial video data of the target video obtained from at least one data source; analyzes the target video content based on the video optimization model to obtain the target video optimization content; predicts the network status of the target video based on the network information to obtain the network prediction data of the target video; identifies the user feedback information to obtain the user demand information; merges the target video optimization content, the network prediction data and the user demand information to obtain the video optimization data of the target video; generates the video optimization strategy of the target video based on the video optimization data; optimizes the target video based on the video optimization strategy. The above technical scheme realizes the dynamic optimization of the video by integrating the data from the video content, the network status and the user feedback, and adaptively adjusts the video optimization strategy based on the integrated data, and helps to improve user satisfaction.

[0100] Embodiment 3

[0101] Figure 3 It is a structural diagram of a video optimization device provided according to Example 3 of the present application, which can be applicable to the situation of optimizing the transmission and playback of video streams. The video optimization device can be implemented in the form of hardware and / or software. The video optimization device can be configured in a computer device, such as a server.

[0102] like Figure 3 As shown, the device comprises:

[0103] The data screening module 310 is used to screen the initial video data of the target video obtained from at least one data source to obtain the target video data;

[0104] The data integration module 320 is used to determine the video optimization data of the target video according to the target video data; wherein the target video data includes the target video content, network information and user feedback information;

[0105] A strategy generation module 330, for generating a video optimization strategy for a target video according to the video optimization data;

[0106] The video optimization module 340 is used to optimize the target video according to the video optimization strategy.

[0107] The embodiment of the present application obtains target video data by screening the initial video data of the target video obtained from at least one data source; determines the video optimization data of the target video according to the target video data; wherein the target video data includes the target video content, network information and user feedback information; generates a video optimization strategy for the target video according to the video optimization data; and optimizes the target video according to the video optimization strategy. The above technical solution realizes dynamic optimization of the video by integrating data from video content, network status and user feedback, and adaptively adjusts the video optimization strategy according to the integrated data, and helps to improve user satisfaction.

[0108] Optionally, the data integration module 320 includes:

[0109] A content analysis unit, used to analyze the target video content based on the video optimization model to obtain the optimized content of the target video;

[0110] A network prediction unit, used to predict the network status of the target video according to the network information, and obtain network prediction data of the target video;

[0111] An information identification unit, used to identify user feedback information and obtain user demand information;

[0112] The data integration unit is used to merge the target video optimization content, network prediction data and user demand information to obtain the video optimization data of the target video.

[0113] Optional content parsing unit, specifically used for:

[0114] Based on the feature extraction network in the video optimization model, feature extraction is performed on the target video content to determine the target key frame of the target video; wherein the feature extraction network is a convolutional neural network;

[0115] Based on the content optimization network in the video optimization model, the video content is analyzed and optimized according to the target key frames to obtain the target video optimized content; wherein the content optimization network is a generative adversarial network.

[0116] Optionally, the training process of the video optimization model is as follows:

[0117] The sample video content is used to perform n rounds of iterative training on the initial optimization model to obtain the video optimization model:

[0118] For the first round, based on the feature extraction network in the initial optimization model, feature extraction is performed on the sample video content to obtain the sample key frames corresponding to the first round;

[0119] Based on the content optimization network in the initial optimization model, the sample video content is analyzed and optimized according to the sample key frames corresponding to the first round, and the optimized sample video content corresponding to the first round is obtained;

[0120] For round i, feature extraction is performed on the sample video optimization content corresponding to round i-1 based on the feature extraction network in the initial optimization model to obtain the sample key frame corresponding to round i;

[0121] Based on the content optimization network in the initial optimization model, the sample optimized video content corresponding to the i-1th round is analyzed and optimized according to the sample key frames corresponding to the i-th round, so as to obtain the sample video optimized content corresponding to the i-th round; wherein i is a natural number greater than 1 and less than n, and n is a natural number greater than 2.

[0122] Optionally, the information identification unit is specifically used for:

[0123] Based on natural language processing technology, the user feedback information is identified in the context to obtain the user context information;

[0124] Classify and identify user context information to obtain user demand information.

[0125] Optionally, the video optimization module 340 is specifically used for:

[0126] Convert video optimization strategies into video control instructions;

[0127] According to the video control instructions, the target video is optimized.

[0128] Optionally, the device further includes: a data updating module, specifically configured to:

[0129] Obtain target feedback information from users on target videos;

[0130] According to the target feedback information, the target video data is updated.

[0131] The video optimization device provided in the embodiments of the present application can execute the video optimization method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing each video optimization method.

[0132] Embodiment 4

[0133] Figure 44 is a schematic diagram of the structure of an electronic device 410 that implements the video optimization method of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0134] like Figure 4 As shown, the electronic device 410 includes at least one processor 411, and a memory connected to the at least one processor 411 in communication, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. In RAM413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, ROM412 and RAM413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0135] Multiple components in the electronic device 410 are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device 410 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0136] The processor 411 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 411 performs the various methods and processes described above, such as a video optimization method.

[0137] In some embodiments, the video optimization method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the video optimization method described above may be performed. Alternatively, in other embodiments, the processor 411 may be configured as a video optimization method in any other appropriate manner (e.g., by means of firmware).

[0138] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] The computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable video optimization device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0140] In the context of the present application, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device or equipment. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0143] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0144] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of this application can be achieved, and this document is not limited here.

[0145] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A video optimization method, characterized in that: include: Screening initial video data of a target video obtained from at least one data source to obtain target video data; Determine the video optimization data of the target video according to the target video data; wherein the target video data includes target video content, network information and user feedback information; Generating a video optimization strategy for the target video according to the video optimization data; Optimizing the target video according to the video optimization strategy; Wherein, determining the video optimization data of the target video according to the target video data includes: Based on the video optimization model, the target video content is parsed to obtain the target video optimization content; wherein the target video optimization content refers to the optimization suggestions or processing results generated by the video optimization model for the target video; the target video optimization content includes video encoding parameter optimization, key frame selection and editing suggestions, and object recognition and motion analysis results; Predicting the network status of the target video according to the network information to obtain network prediction data of the target video; Identifying the user feedback information to obtain user demand information; Merging the target video optimization content, the network prediction data and the user demand information to obtain video optimization data of the target video; Accordingly, generating a video optimization strategy for the target video according to the video optimization data includes: Respectively taking the target video optimization content, network prediction data and user demand information in the video optimization data as inputs of the SML algorithm to generate a video optimization strategy for the target video; The method of parsing the target video content based on the video optimization model to obtain the target video optimized content includes: Based on the feature extraction network in the video optimization model, feature extraction is performed on the target video content to determine the target key frame of the target video; wherein the feature extraction network is a convolutional neural network; Based on the content optimization network in the video optimization model, the video content is analyzed and optimized according to the target key frame to obtain the target video optimized content; wherein the content optimization network is a generative adversarial network.

2. The method according to claim 1, characterized in that The training process of the video optimization model is as follows: The sample video content is used to perform n rounds of iterative training on the initial optimization model to obtain the video optimization model: For the first round, based on the feature extraction network in the initial optimization model, feature extraction is performed on the sample video content to obtain sample key frames corresponding to the first round; Based on the content optimization network in the initial optimization model, the sample video content is analyzed and optimized according to the sample key frames corresponding to the first round to obtain the sample video optimized content corresponding to the first round; For round i, extract features of the sample video optimization content corresponding to round i-1 based on the feature extraction network in the initial optimization model to obtain sample key frames corresponding to round i; Based on the content optimization network in the initial optimization model, the sample optimized video content corresponding to the i-1th round is analyzed and optimized according to the sample key frames corresponding to the i-th round, so as to obtain the sample video optimized content corresponding to the i-th round; wherein i is a natural number greater than 1 and less than n, and n is a natural number greater than 2.

3. The method according to claim 1, characterized in that The identifying the user feedback information to obtain the user demand information includes: Based on natural language processing technology, context recognition is performed on the user feedback information to obtain user context information; The user context information is classified and identified to obtain user demand information.

4. The method according to claim 1, characterized in that Optimizing the target video according to the video optimization strategy includes: Converting the video optimization strategy into video control instructions; The target video is optimized according to the video control instruction.

5. The method according to claim 1, characterized in that: After optimizing the target video according to the video optimization strategy, the method further includes: Obtaining target feedback information from the user on the target video; The target video data is updated according to the target feedback information.

6. A video optimization device, characterized in that: include: A data screening module, used for screening the initial video data of the target video obtained from at least one data source to obtain the target video data; A data integration module, used to determine the video optimization data of the target video according to the target video data; wherein the target video data includes target video content, network information and user feedback information; A strategy generation module, used to generate a video optimization strategy for the target video according to the video optimization data; A video optimization module, used for optimizing the target video according to the video optimization strategy; Wherein, the data integration module includes: A content analysis unit, configured to analyze the target video content based on the video optimization model to obtain the target video optimization content; wherein the target video optimization content refers to the optimization suggestions or processing results generated by the video optimization model for the target video; the target video optimization content includes video encoding parameter optimization, key frame selection and editing suggestions, and object recognition and motion analysis results; A network prediction unit, used to predict the network status of the target video according to the network information, and obtain network prediction data of the target video; An information identification unit, used to identify user feedback information and obtain user demand information; A data integration unit, used to merge the target video optimization content, network prediction data and user demand information to obtain video optimization data of the target video; Accordingly, the strategy generation module is specifically used for: Respectively taking the target video optimization content, network prediction data and user demand information in the video optimization data as inputs of the SML algorithm to generate a video optimization strategy for the target video; Wherein, the content parsing unit is specifically used for: Based on the feature extraction network in the video optimization model, feature extraction is performed on the target video content to determine the target key frame of the target video; wherein the feature extraction network is a convolutional neural network; Based on the content optimization network in the video optimization model, the video content is analyzed and optimized according to the target key frames to obtain the target video optimized content; wherein the content optimization network is a generative adversarial network.

7. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the video optimization method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the video optimization method as described in any one of claims 1 to 5 is implemented.

Images