A method, apparatus, and medium for optimizing user video data.

Abstract

This invention relates to the field of video image compression technology, and in particular to a method, device, and medium for optimizing a program based on user video data. The method includes: extracting one image from each second of video segments in a target video to be transmitted to obtain several key images; extracting features from the key images to obtain several target image features and several target image labels; inputting the image features and image labels into a machine learning model to obtain an importance score corresponding to each key image; segmenting the target video based on the importance score corresponding to each key image to obtain a preset number of target video slices; determining the corresponding compression algorithm based on the video data information of the target video slices; and completing the optimization of the compression program. This invention can reasonably segment a target video into several video slices and optimize the video slice compression program, thereby ensuring the transmission efficiency and transmission quality of the target video.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video image compression, in particular to a method, device and medium for optimizing program based on user video data. BACKGROUND

[0002] When a user searches and watches a video, the process from sending the video to a playing platform to the final playing involves multiple technical links, including video uploading, video slicing, generating video stream, sending to a cache channel, decoding and playing, etc. During the playing process, video freezing often occurs. This situation may be caused by network fluctuation or too large video stream data. Since the network fluctuation is affected by the user's network bandwidth and cannot be fixed, in the case of not considering network fluctuation, in order to improve the freezing reason, the transmission process of the video needs to be optimized, that is, in the case of fixed bandwidth, the segmentation and compression program of the video needs to be optimized to improve the video playing freezing phenomenon. SUMMARY

[0003] In view of the above technical problems, the present application provides a method, device and medium for optimizing program based on user video data, which can reasonably segment a target video into several video slices and optimize the video slice compression program, so as to ensure the transmission efficiency and transmission quality of the target video and improve the continuity of video playing.

[0004] According to a first aspect of the present application, a method for optimizing program based on user video data is provided, comprising the following steps:

[0005] S1, obtaining video data information of a target video to be sent; the video data information of the target video includes a frame rate and a key frame interval corresponding to the target video.

[0006] S2, extracting an image from each second video segment in the target video to obtain several key images.

[0007] S3, for any key image, performing feature extraction on the key image to obtain several target image features and several target image labels, and inputting the several target image features and the several target image labels into a pre-trained machine learning model to obtain an importance score corresponding to the key image.

[0008] S4, segmenting the target video based on the importance score corresponding to each key image to obtain a preset number of target video slices.

[0009] S5, based on the frame rate, keyframe interval, and data volume corresponding to each target video slice, determine the compression algorithm corresponding to each target video slice from the given compression algorithm library, and compress the corresponding target video slice size into a video data packet no larger than the preset slice data size based on the determined compression algorithm, so as to complete the optimization of the compression program; wherein, the frame rate and keyframe interval corresponding to the target video slice are the frame rate and keyframe interval corresponding to the target video.

[0010] According to a second aspect of the present invention, a non-transitory computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the above-described method of optimizing a program based on user video data.

[0011] According to a third aspect of the present invention, an electronic device is provided, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0012] The present invention has at least the following beneficial effects:

[0013] This invention provides a method for optimizing a program based on user video data. First, one image is extracted from each second of the target video to be transmitted, resulting in several key images. Feature extraction is performed on these key images to obtain several target image features and labels. These features and labels are then input into a machine learning model to obtain an importance score for each key image. Next, the target video is segmented based on the importance score of each key image, resulting in a preset number of target video slices. When segmenting the target video using the importance score as a benchmark, high-scoring video slices are further segmented, while low-scoring video slices are merged. This ensures that the number of video slices remains at the preset number, while guaranteeing that the compressed slice data size meets requirements, thereby ensuring transmission efficiency and video quality. Finally, a corresponding compression algorithm is determined based on the video data information of the target video slices, completing the optimization of the compression program. By screening compression algorithms, algorithms with higher compression efficiency and quality, while ensuring the compressed data size meets requirements, are selected. This optimization of the compression program improves transmission efficiency and ensures the continuity of video playback. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart of a method for optimizing a program based on user video data provided in an embodiment of the present invention;

[0016] Figure 2 A flowchart of a method for obtaining importance scores provided in an embodiment of the present invention;

[0017] Figure 3 A flowchart for obtaining a preset number of slices provided in an embodiment of the present invention;

[0018] Figure 4 A flowchart for obtaining a target video slice provided in an embodiment of the present invention;

[0019] Figure 5 A flowchart of a compression algorithm for determining a target video slice provided in an embodiment of the present invention;

[0020] Figure 6 Another flowchart of the method for optimizing a program based on user video data provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This invention provides a method for optimizing programs based on user video data, such as... Figure 1 As shown, the method includes the following steps:

[0023] S1, Obtain the video data information of the target video to be sent; the video data information of the target video includes the frame rate and keyframe interval corresponding to the target video.

[0024] As mentioned above, since the amount of data in keyframes is much larger than that in forward prediction frames and bidirectional prediction frames, using keyframes as a dimension in subsequent filtering and compression algorithms can reduce the complexity of the filtering algorithm and ensure the accuracy of the filtering.

[0025] S2, extract one image from each video segment per second in the target video to obtain several key images; this can be understood as: randomly extracting one image from each video segment per second as a key image.

[0026] Furthermore, step S2 includes the following steps:

[0027] S201, for any second video segment of the target video, if the number of images in the video segment is odd, extract the middle image of the video segment as the key image.

[0028] S202, when the number of images in a video clip is even, extract any one of the two middle images of the video clip as the key image.

[0029] As described above, this image filtering method can ensure the same interval between adjacent images, allowing for the filtering of more and more uniform information, making the content covered by all images more comprehensive, and ensuring the reliability of subsequent feature extraction.

[0030] S3. For any key image, perform feature extraction on the key image to obtain several target image features and several target image labels. Then, input the several target image features and several target image labels into a pre-trained machine learning model to obtain the importance score corresponding to the key image.

[0031] Specifically, such as Figure 2 As shown, step S3 includes the following steps:

[0032] S301, an image recognition model is used to extract features from each key image that conform to any preset image features as target image features. Those skilled in the art can set corresponding preset image features according to different application scenarios; for example, preset image features may include semantic features, character features, and image meaning features.

[0033] S302, based on the preset feature-label mapping relationship, obtain several target image labels corresponding to each key image according to several target image features corresponding to each key image; it can be understood that the preset image features and preset image labels in the feature-label mapping relationship have a many-to-many relationship.

[0034] Specifically, target image labels include object-level labels and scene-level labels; those skilled in the art can set object-level labels and scene-level labels according to actual needs, and no limitation is made here.

[0035] S303: For any key image, convert each target image label corresponding to the key image into a numerical vector, and concatenate several numerical vectors with the vectors corresponding to several target image features corresponding to the key image to obtain the feature label vector corresponding to the key image.

[0036] S304, input the feature label vector corresponding to the key image into the trained machine learning model to obtain the importance score of the key image; wherein, the machine learning model is trained based on several preset image features and several preset image labels. In the specific training process, features and labels with high weights are determined, and the weights of important features and important labels can be manually assigned according to actual needs.

[0037] As described above, by selectively extracting several target image features from key images and scoring the importance of key images based on the extracted image features and corresponding image labels, a reliable key image importance score can be obtained. This allows for increased cutting frequency for highly important parts during subsequent slicing, thereby reducing the data size of the slices. This ensures that the compressed video meets the preset slice data size while maintaining the quality of the compressed video, even when lossless or near-lossless compression is used.

[0038] S4. The target video is segmented based on the importance score corresponding to each key image to obtain a preset number of target video slices.

[0039] Furthermore, such as Figure 3 As shown, the preset number of slices is obtained through the following steps:

[0040] P1. Obtain a historical video sample set according to several preset slice data size levels; where each preset slice data size level corresponds to several historical video samples, and the data size of each historical video sample is not equal. For example, the preset slice data size levels can be 2kb, 4kb, 6kb, etc.; it can be understood that when the preset slice data size level is 2kb, the corresponding slice data size is 2kb.

[0041] P2, for any preset slice data size, divides each historical video sample corresponding to the preset slice data size according to the preset slice data size, resulting in several video slice samples corresponding to each historical video sample. For example, if the video is cut according to a slice data size of 2kb, when the video is 1000kb in total, 500 video slice samples can be obtained.

[0042] P3, conduct video transmission experiments on several video slice samples corresponding to each historical video sample to obtain the packet loss rate, transmission time and restoration time corresponding to each historical video sample; it can be understood that the restoration time refers to the time to decompress the transmission data packet corresponding to the historical video sample and obtain the historical video sample.

[0043] P4. Calculate the evaluation index value for each historical video sample based on the packet loss rate, transmission time, and restoration time.

[0044] Among them, the evaluation index value F corresponding to any historical video sample meets the following conditions:

[0045] F = ε + (t - tmin) / (tmax - tmin), where ε is the packet loss rate for any historical video sample, t is the sum of the transmission time and restoration time for any historical video sample, tmax is the maximum value among the sums of the transmission time and restoration time for each historical video sample, and tmin is the minimum value among the sums of the transmission time and restoration time for each historical video sample.

[0046] P5 determines the number of video slice samples corresponding to the historical video samples with the minimum evaluation index value as the preset number of slices.

[0047] The above-mentioned data uses a large number of historical video samples to obtain the packet loss rate, transmission time, and restoration time corresponding to different numbers and sizes of video slice samples. The evaluation index values ​​of historical video samples are considered from these three dimensions, reflecting the transmission quality and efficiency of the video samples. The number and size of slices determined by the evaluation index values ​​are used as the benchmark for video segmentation, which can improve the transmission efficiency and quality of the segmented slices, making the segmentation of the target video more reliable.

[0048] In a specific embodiment, such as Figure 4 As shown, step S4 includes the following steps:

[0049] S401, based on the duration of the target video, the target video is divided into an average number of segments to obtain a preset number of first video segments; wherein any two first video segments have the same duration.

[0050] S402, calculate the slice score corresponding to each first video slice based on the importance score corresponding to each key image. The slice score corresponding to the first video slice can be the average score of several key images contained within the first video slice itself.

[0051] S403, when the slice score corresponding to any first video slice is greater than the score threshold, the first video slice itself is divided into several second video slices by average.

[0052] S404, traverse several second video slices, and calculate the slice score corresponding to each adjacent two second video slices after merging based on the importance score corresponding to each key image, so as to merge the two second video slices with the smallest corresponding slice score to obtain several third video slices.

[0053] S405, when the number of third video slices is greater than the preset number of slices, the third video slice is used as the second video slice and the process returns to step S404 until the number of third video slices is no greater than the preset number of slices, and the obtained third video slice is used as the target video slice.

[0054] As described above, the target video is segmented based on the importance score corresponding to each key image. When the slice score corresponding to the first video slice is high, in order to ensure the video compression quality, the compression method should tend to be lossless compression. At this time, the data volume of the compressed data packet is large, which will reduce the data packet transmission efficiency and cause a certain packet loss rate during transmission. Therefore, the method of further cutting the video slice with high score and merging the video slice with low score is adopted so that the number of video slices remains at the preset number, thereby ensuring transmission efficiency and video quality.

[0055] S5, based on the frame rate, keyframe interval, and data volume corresponding to each target video slice, determine the compression algorithm corresponding to each target video slice from the given compression algorithm library, and compress the corresponding target video slice size into a video data packet no larger than the preset slice data size based on the determined compression algorithm, so as to complete the optimization of the compression program; wherein, the frame rate and keyframe interval corresponding to the target video slice are the frame rate and keyframe interval corresponding to the target video.

[0056] Further, the preset slice data size is obtained through the following steps:

[0057] The preset slice data size is determined by the preset slice data size level corresponding to the historical video samples with the minimum evaluation index value.

[0058] In a specific embodiment, such as Figure 5 As shown, the compression algorithm corresponding to each target video slice is determined through the following steps:

[0059] S501, for any target video slice, based on the frame rate, keyframe interval, and data volume corresponding to the target video slice, obtain the compression duration and compressed data packet size for each compression algorithm in the given compression algorithm library corresponding to the target video slice. In this embodiment, the prediction result can be obtained based on the relationship between a pre-trained prediction model or a fitted compression algorithm and the frame rate, keyframe interval, and data volume of the target video slice. Since those skilled in the art are familiar with the training and fitting processes, and this does not involve innovative content, it will not be described in detail here.

[0060] S502, select compression algorithms whose corresponding compressed data packet size is not greater than the preset slice data size as pre-compression algorithms.

[0061] S503, determine the pre-compression algorithm with the shortest corresponding compression time among several pre-compression algorithms as the compression algorithm corresponding to the target video slice.

[0062] The above process involves first screening the compression algorithms to ensure that the compressed data volume meets the requirements, and then performing a second screening to ensure that the compression efficiency meets the requirements. Through these two screenings, compression algorithms with higher compression efficiency and the ability to ensure that the compressed data volume meets the requirements can be selected, thereby improving transmission efficiency and ensuring the continuity of video viewing.

[0063] Specifically, the following steps are included after step S5:

[0064] S6, the compressed data packets and the recorded log information are sent to the video receiving platform together, so that the video receiving platform can use the corresponding decompression algorithm to restore each video data packet and combine them to obtain the target video according to the compression algorithm corresponding to each target video slice recorded in the log information.

[0065] In a parallel embodiment, such as Figure 6 As shown, the following steps are included after step S1:

[0066] K1 divides the target video into a preset number of target video slices on an equal basis; this can be understood as: each target video slice corresponds to the same amount of data.

[0067] K2 determines the compression algorithm corresponding to each target video slice from a given compression algorithm library based on the frame rate, keyframe interval, and data volume of each target video slice, and compresses the corresponding target video slice size into a data packet no larger than the preset slice data size based on the determined compression algorithm. In this embodiment, the method of determining the compression algorithm is the same as step S5, and will not be repeated here.

[0068] The above provides another implementation method, which directly divides the target video into equal segments, and then compresses each segment of the target video to a size no larger than a preset segment data size, so that the number and data size of the target video segments meet the aforementioned threshold, thereby ensuring the transmission efficiency and transmission quality of the target video segments.

[0069] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the method of the optimized program based on user video data provided in the above embodiments.

[0070] Embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0071] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended technical solutions.

Claims

1. A method of optimizing a program based on user video data, the method comprising: The method comprises the following steps: S1, obtaining video data information of a target video to be sent; the video data information of the target video comprises a frame rate and a key frame interval corresponding to the target video; S2, extracting an image from each second video segment in the target video to obtain a plurality of key images; S3, for any key image, performing feature extraction on the key image to obtain a plurality of target image features and a plurality of target image labels, and inputting the plurality of target image features and the plurality of target image labels into a pre-trained machine learning model to obtain an importance score corresponding to the key image; S4, segmenting the target video based on the importance score corresponding to each key image to obtain a preset number of target video slices; the step S4 comprises the following steps: S401, performing average segmentation on the target video based on the time length of the target video to obtain a preset number of first video slices; wherein, the time length of any two first video slices is equal; S402, calculating a slice score corresponding to each first video slice according to the importance score corresponding to each key image; S403, when the slice score corresponding to any first video slice is greater than a score threshold, performing average segmentation on the first video slice itself again to obtain a plurality of second video slices; S404, traversing the plurality of second video slices, and calculating a slice score corresponding to each adjacent two second video slices after merging according to the importance score corresponding to each key image, so as to merge the two second video slices corresponding to the smallest slice score to obtain a plurality of third video slices; S405, when the number of third video slices is greater than the preset number of slices, taking the third video slice as a second video slice and returning to execute step S404 until the number of third video slices obtained is not greater than the preset number of slices, and taking the third video slice obtained as a target video slice; S5, determining a compression algorithm corresponding to each target video slice from a given compression algorithm library according to the frame rate, the key frame interval and the data volume corresponding to each target video slice, and compressing the size of the corresponding target video slice to a video data packet not greater than a preset slice data size based on the determined compression algorithm, so as to complete the optimization of the compression program; wherein, the frame rate and the key frame interval corresponding to the target video slice are the frame rate and the key frame interval corresponding to the target video.

2. The method of claim 1, wherein, The step S3 comprises the following steps: S301, extracting a feature conforming to any preset image feature from each key image as a target image feature by using an image recognition model; S302, obtaining a plurality of target image labels corresponding to each key image based on a preset feature-label mapping relationship according to the plurality of target image features corresponding to each key image; S303, for any key image, converting each target image label corresponding to the key image into a numerical vector, and splicing the plurality of numerical vectors with vectors corresponding to the plurality of target image features corresponding to the key image to obtain a feature-label vector corresponding to the key image; S304, input the feature label vector corresponding to the key image into the trained machine learning model to obtain the importance score corresponding to the key image; wherein the machine learning model is trained according to a plurality of preset image features and a plurality of preset image labels.

3. The method of claim 1, wherein, The preset slice number is obtained through the following steps: P1, according to a plurality of preset slice data amount gears, a historical video sample set is obtained; wherein each preset slice data amount gear corresponds to a plurality of historical video samples, and the data amount of each corresponding historical video sample is not equal; P2, for any preset slice data amount gear, each historical video sample corresponding to the preset slice data amount gear is segmented according to the preset slice data amount gear to obtain a plurality of video slice samples corresponding to each historical video sample; P3, video transmission tests are performed on the plurality of video slice samples corresponding to each historical video sample to obtain the packet loss rate, transmission time and restoration time corresponding to each historical video sample; P4, according to the packet loss rate, transmission time and restoration time corresponding to each historical video sample, the evaluation index value corresponding to each historical video sample is calculated; Wherein the evaluation index value F corresponding to any historical video sample satisfies the following condition: F = ε + (t - t min ) / (t max -t min ), wherein ε is a packet loss rate corresponding to any historical video sample, t is a sum of transmission time and restoration time corresponding to any historical video sample, t max is a maximum value of the sum of transmission time and restoration time corresponding to each historical video sample, and t min is a minimum value of the sum of transmission time and restoration time corresponding to each historical video sample. P5, the number of video slice samples corresponding to the historical video sample with the minimum evaluation index value is determined as the preset slice number.

4. The method of claim 3, wherein, The preset slice data size is obtained through the following steps: The preset slice data amount gear corresponding to the historical video sample with the minimum evaluation index value is determined as the preset slice data size.

5. The method of claim 1, wherein, In the S5 step, the compression algorithm corresponding to each target video slice is determined through the following steps: S501, for any target video slice, according to the frame rate, key frame interval and data amount corresponding to the target video slice, the compression time and the data packet size after compression of each compression algorithm in the given compression algorithm library corresponding to the target video slice are obtained; S502, the compression algorithm corresponding to the data packet size after compression not greater than the preset slice data size is screened out as a pre-compression algorithm; S503, the pre-compression algorithm corresponding to the shortest compression time in the plurality of pre-compression algorithms is determined as the compression algorithm corresponding to the target video slice.

6. The method of claim 1, wherein, After the S1 step, the following steps are further included: K1, the target video is evenly segmented into a plurality of target video slices with the preset slice number; K2, according to the frame rate, key frame interval and data amount corresponding to each target video slice, the compression algorithm corresponding to each target video slice is determined from the given compression algorithm library, and the size of the corresponding target video slice is compressed to a data packet not greater than the preset slice data size based on the determined compression algorithm.

7. The method of claim 1, wherein, After the S5 step, the following steps are further included: S6, the compressed plurality of data packets and the recorded log information are sent to the video receiving platform together, so that the video receiving platform uses the corresponding decompression algorithm to restore and combine each video data packet according to the compression algorithm corresponding to each target video slice recorded in the log information to obtain the target video. 8.A non-transitory computer readable storage medium having stored therein at least one instruction or at least one piece of program, characterized in that, The at least one instruction or the at least one program is loaded and executed by the processor to implement the method of the optimization program based on user video data as claimed in any one of claims 1-7.

9. An electronic device, comprising: A non-transitory computer readable storage medium including a processor and the non-transitory computer readable storage medium of claim 8.

Images