Video classification method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a video classification method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring video data of a to-be-classified video; decomposing the video data to obtain at least one piece of video component data; for each piece of video component data, performing feature extraction on the video component data to obtain feature data corresponding to the video component data; and determining category information of the to-be-classified videos according to the feature data corresponding to the video component data. By adopting the method, detail information in the video data can be comprehensively captured, and rich and effective features are provided for video classification, so that an accurate and reliable video classification result can be provided under the condition that the video data lacks annotation information.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a video classification method, apparatus, computer equipment, storage medium, and program product. Background Technology

[0002] With the development of digital media technology and the widespread adoption of the internet, video production and viewing have become increasingly convenient, leading to an explosive growth in video data volume. Against this backdrop, how to efficiently classify and process this massive amount of video data has become a crucial issue.

[0003] Current methods for video classification mainly include deep learning-based methods and methods that combine video tag information. These methods require video annotation. However, a large number of videos in reality lack annotation information, severely impacting accurate video classification.

[0004] Therefore, how to accurately classify videos when there is a lack of annotation information is a problem worth paying attention to. Summary of the Invention

[0005] Therefore, it is necessary to provide a video classification method, apparatus, computer equipment, storage medium, and program product to address the aforementioned technical problems, enabling accurate video classification even when video annotation information is lacking.

[0006] Firstly, this application provides a video classification method, including:

[0007] Obtain video data for the videos to be categorized;

[0008] The video data is decomposed to obtain at least one video component data;

[0009] For each video component data, feature extraction is performed on the video component data to obtain the feature data corresponding to the video component data;

[0010] Based on the feature data corresponding to each video component, the category information of the video to be classified is determined.

[0011] In one embodiment, the video data is decomposed to obtain at least one video component data, including:

[0012] In each iteration, the local mean data of the local extreme points in the current video data surplus under this iteration, as well as the envelope estimation data corresponding to the current video data surplus, are obtained; among them, local extreme points include local maxima and local minima; the pixel value of a local maxima is greater than the pixel value of its neighboring pixel; the pixel value of a local minima is less than the pixel value of its neighboring pixel;

[0013] Based on the current video data surplus, local mean data, and envelope estimation data, determine the pure frequency modulation data for this iteration process;

[0014] Based on the pure frequency modulation data and envelope estimation data, the video component data for this iteration process is determined;

[0015] Based on the current video data surplus and video component data in this iteration, determine the current video data surplus in the next iteration, and return to the steps of obtaining the local mean data and envelope estimation data corresponding to the current video data surplus, until the preset cutoff condition is reached; where the current video data surplus in the first iteration is video data.

[0016] In one embodiment, the video data includes the pixel values ​​of each pixel in a plurality of consecutive frame images;

[0017] Accordingly, the local mean data of local extreme points in the remaining video data during this iteration are obtained, including:

[0018] For each pixel, if its pixel value is greater than the pixel values ​​of its two adjacent pixels, the pixel is determined to be a local maximum; if its pixel value is less than the pixel values ​​of its two adjacent pixels, the pixel is determined to be a local minimum. Specifically, if the pixel is a row pixel, the two adjacent pixels are the two pixels in the same frame that are adjacent to the pixel in the row direction; if the pixel is a column pixel, the two adjacent pixels are the two pixels in the same frame that are adjacent to the pixel in the column direction; and if the pixel is a pixel at a target image location, the two adjacent pixels are the pixels in the two frames adjacent to the pixel's frame that are at the target image location.

[0019] Based on the pixel values ​​of each local extreme point, determine the local mean data of the local extreme points in the current video data margin.

[0020] In one embodiment, determining the local mean data of the local extreme points in the current video data margin based on the pixel values ​​of each local extreme point includes:

[0021] For each preset sliding window, the local mean value corresponding to the preset sliding window is determined based on the pixel values ​​and values ​​of each local extreme point within the preset sliding window.

[0022] Based on the local mean corresponding to each preset sliding window, determine the local mean data of the local extreme points in the current video data margin.

[0023] In one embodiment, the video data includes the pixel values ​​of each pixel in a plurality of consecutive frame images;

[0024] Accordingly, obtain the envelope estimation data corresponding to the current video data surplus, including:

[0025] For each pixel, based on its pixel value, determine the first pixel difference between the pixel and its neighboring pixels in the frame image; and,

[0026] Determine the second pixel difference between other pixels located at the target image position in two adjacent frame images of the same pixel;

[0027] Based on the differences between the first and second pixels, determine the envelope estimation data corresponding to each pixel.

[0028] Based on the envelope estimation data corresponding to each pixel, determine the envelope estimation data corresponding to the current video data margin.

[0029] In one embodiment, determining the envelope estimation data corresponding to a pixel based on the first pixel difference and the second pixel difference includes:

[0030] Determine the average value of the first pixel difference based on the first pixel difference;

[0031] Determine the average value of the second pixel difference based on the second pixel difference;

[0032] The envelope estimation data corresponding to the pixel is determined based on the average value of the first pixel difference and the average value of the second pixel difference.

[0033] In one embodiment, the pure frequency modulation data for the current iteration is determined based on the current video data margin, local mean data, and envelope estimation data, including:

[0034] Based on the current remaining video data and local mean data, determine the intermediate data;

[0035] Based on intermediate data and envelope estimation data, pure frequency modulation data are determined.

[0036] In one embodiment, determining the current video data reserve for the next iteration based on the current video data reserve and video component data in the current iteration includes:

[0037] The difference between the current video data margin and the video component data is determined as the current video data margin for the next iteration.

[0038] In one embodiment, the category information of the video to be classified is determined based on the feature data corresponding to each video component data, including:

[0039] For each video component data, the target component category corresponding to the video component data is determined based on the feature data corresponding to each video component data.

[0040] Based on the target component category corresponding to each video component data, determine the category information of the video to be classified.

[0041] In one embodiment, determining the target component category corresponding to each video component data based on the feature data corresponding to each video component data includes:

[0042] Determine the distance between the feature data corresponding to the video component data and the feature data corresponding to other video component data.

[0043] Select a preset number of target video component data from the other video component data in ascending order of distance;

[0044] Based on the component category corresponding to each target video component data, determine the first quantity of target video component data under each component category;

[0045] The component category with the highest number of corresponding components is determined as the target component category of the video component data.

[0046] In one embodiment, the category information of the video to be classified is determined based on the target component category corresponding to each video component data, including:

[0047] Determine the second quantity of video component data corresponding to each target component category;

[0048] The category of the second most numerous target component is determined as the category information of the video to be classified.

[0049] Secondly, this application also provides a video classification device, comprising:

[0050] The acquisition module is used to acquire video data of the videos to be classified.

[0051] The decomposition module is used to decompose video data to obtain at least one video component data.

[0052] The extraction module is used to extract features from each video component data to obtain the feature data corresponding to the video component data.

[0053] The determination module is used to determine the category information of the video to be classified based on the feature data corresponding to each video component data.

[0054] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0055] Obtain video data for the videos to be categorized;

[0056] The video data is decomposed to obtain at least one video component data;

[0057] For each video component data, feature extraction is performed on the video component data to obtain the feature data corresponding to the video component data;

[0058] Based on the feature data corresponding to each video component, the category information of the video to be classified is determined.

[0059] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0060] Obtain video data for the videos to be categorized;

[0061] The video data is decomposed to obtain at least one video component data;

[0062] For each video component data, feature extraction is performed on the video component data to obtain the feature data corresponding to the video component data;

[0063] Based on the feature data corresponding to each video component, the category information of the video to be classified is determined.

[0064] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0065] Obtain video data for the videos to be categorized;

[0066] The video data is decomposed to obtain at least one video component data;

[0067] For each video component data, feature extraction is performed on the video component data to obtain the feature data corresponding to the video component data;

[0068] Based on the feature data corresponding to each video component, the category information of the video to be classified is determined.

[0069] The aforementioned video classification method, apparatus, computer equipment, storage medium, and program product can acquire video data of a video to be classified; decompose the video data to obtain at least one video component data; extract features from each video component data to obtain corresponding feature data; and determine the category information of the video to be classified based on the feature data corresponding to each video component data. In the above process, by decomposing the video data to obtain at least one video component data and extracting feature data from at least one video component data, detailed information in the video data can be comprehensively captured, providing rich and effective features for video classification. Thus, even when the video data lacks annotation information, accurate and reliable video classification results can be provided. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a diagram illustrating the application environment of a video classification method in one embodiment;

[0072] Figure 2 This is a flowchart illustrating a video classification method in one embodiment;

[0073] Figure 3 This is a flowchart illustrating the steps of decomposing video data in one embodiment;

[0074] Figure 4 This is a flowchart illustrating the steps for determining the category information of videos to be classified in one embodiment.

[0075] Figure 5 This is a flowchart illustrating a video classification method in another embodiment;

[0076] Figure 6 This is a structural block diagram of a video classification device in one embodiment;

[0077] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0079] The video classification method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0080] In one exemplary embodiment, such as Figure 2 As shown, a video classification method is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0081] S210, Obtain video data of the video to be classified.

[0082] The videos to be classified can include short videos with a relatively short duration and long videos with a relatively long duration. This application does not specifically limit the type of videos to be classified.

[0083] In this context, video data can be understood as the content data of the video to be classified. Specifically, video data can include the pixel values ​​of each pixel in multiple consecutive frames of the video to be classified.

[0084] In some embodiments, the acquired video data can be converted into a three-dimensional array, where the two spatial dimensions correspond to the width and height of the image, respectively, and the temporal dimension corresponds to the video frame sequence. Assuming the video has a frame rate of fps, a duration of t seconds, and a resolution of width × height, the converted three-dimensional array would have a size of width × height × (fps × t), representing the image data arranged frame-by-frame in the temporal dimension. The video data is then decomposed.

[0085] S220, decompose the video data to obtain at least one video component data.

[0086] Among them, video component data can be understood as component data containing various components in the video to be classified, obtained by adaptively decomposing the video data according to the features of the video data at different spatiotemporal scales, such as component data containing the background, component data containing foreground objects, etc.

[0087] In some embodiments, video data can be viewed as a non-stationary, non-linear signal, and the video data can be decomposed to obtain at least one product function (PF) component, i.e., video component data.

[0088] In some embodiments, to improve the efficiency of video classification, the video data of each video to be classified can be assigned to different executors, and each executor can decompose its own video data to achieve distributed processing.

[0089] In some embodiments, to improve video classification efficiency, operators developed in Rust can be used to perform the operation of decomposing video data.

[0090] S230: For each video component data, feature extraction is performed on the video component data to obtain the feature data corresponding to the video component data.

[0091] In some embodiments, video component data can be viewed as signal components obtained by decomposing video data.

[0092] In some embodiments, the feature data may include feature data under at least one of the following preset feature dimensions: energy, peak frequency, average frequency, mean, standard deviation, skewness, and kurtosis.

[0093] Among them, energy can be understood as signal strength; peak frequency can be understood as the maximum frequency in the frequency spectrum; average frequency can be understood as the weighted average of the amplitudes in the frequency spectrum; mean can be understood as the average value of the signal in the time domain or frequency domain; standard deviation can be understood as the degree of dispersion of the signal relative to the mean; skewness can be understood as the degree of skewness of the signal on both sides of the mean; and kurtosis can be understood as the sharpness of the signal.

[0094] S240, determine the category information of the video to be classified based on the feature data corresponding to each video component data.

[0095] In some embodiments, the feature data corresponding to each video component data can be standardized to obtain a standardized feature vector corresponding to each video component data. Based on the standardized feature vector corresponding to each video component data, the category information of the video to be classified can be determined.

[0096] In the above video classification method, video data of the video to be classified can be obtained; the video data can be decomposed to obtain at least one video component data; for each video component data, feature extraction can be performed to obtain the corresponding feature data; based on the feature data corresponding to each video component data, the category information of the video to be classified can be determined. In the above process, by decomposing the video data to obtain at least one video component data and extracting feature data from at least one video component data, the detailed information in the video data can be comprehensively captured, providing rich and effective features for video classification. Thus, even when the video data lacks annotation information, accurate and reliable video classification results can be provided.

[0097] Based on the above embodiments, this application also provides another optional embodiment in which the video data decomposition steps are refined.

[0098] See Figure 3 The steps for decomposing video data as shown include:

[0099] S310, in each iteration, obtain the local mean data of the local extreme points in the current video data surplus under the current iteration, and the envelope estimation data corresponding to the current video data surplus; wherein, the local extreme points include local maxima and local minima; the pixel value of the local maxima is greater than the pixel value of the neighboring pixel; the pixel value of the local minima is less than the pixel value of the neighboring pixel.

[0100] In this iteration, the remaining video data can be understood as the remaining video data from the previous iteration minus the remaining video component data from the previous iteration. In the first iteration, the remaining video data consists of the video data of the video to be classified.

[0101] Local maxima can be understood as pixels whose pixel values ​​are greater than those of their neighboring pixels.

[0102] Local minima can be understood as pixels whose pixel values ​​are less than those of their neighboring pixels.

[0103] Local mean data can be understood as mean data determined based on the pixel values ​​of local extreme points.

[0104] The envelope estimation data can be understood as data determined based on the pixel difference between each pixel and its neighboring pixels.

[0105] In some embodiments, obtaining the local mean data of local extreme points in the current video data surplus during the current iteration process may include: for each pixel, if the pixel value of the pixel is greater than the pixel values ​​of the two adjacent pixels, determining the pixel as a local maximum point; if the pixel value of the pixel is less than the pixel values ​​of the two adjacent pixels, determining the pixel as a local minimum point; wherein, when the pixel is a row pixel, the two adjacent pixels are the two pixels in the frame image where the row pixel is located that are adjacent to the pixel in the row direction; when the pixel is a column pixel, the two adjacent pixels are the two pixels in the frame image where the row pixel is located that are adjacent to the pixel in the column direction; when the pixel is a pixel at the target image position, the two adjacent pixels are the pixels at the target image position on the two frames adjacent to the frame image where the pixel is located; and determining the local mean data of local extreme points in the current video data surplus based on the pixel values ​​of each local extreme point.

[0106] Here, row pixels can be understood as pixels arranged in the row direction.

[0107] Column pixels can be understood as pixels arranged in the column direction.

[0108] In some embodiments, for each frame of an image, each row of pixels can be traversed from left to right. If the pixel value of any row of pixels is greater than the pixel values ​​of the two adjacent pixels in the row direction, then that row of pixels is determined as a local maximum point.

[0109] In some embodiments, for each frame of an image, each row of pixels can be traversed from left to right. If the pixel value of any row of pixels is less than the pixel values ​​of the two pixels adjacent to that row in the row direction, that row of pixels is determined as a local minimum point.

[0110] In some embodiments, for each frame of an image, each column of pixels can be traversed from top to bottom. If the pixel value of any column of pixels is greater than the pixel values ​​of the two pixels adjacent to that column of pixels in the column direction, that column of pixels is determined as a local maximum point.

[0111] In some embodiments, for each frame of an image, each column of pixels can be traversed from top to bottom. If the pixel value of any column of pixels is less than the pixel values ​​of the two pixels adjacent to that column of pixels in the column direction, that column of pixels is determined as a local minimum point.

[0112] In some embodiments, for each target image location, the pixels located at the target image location in each frame image can be traversed in the time order of the frame images. If the pixel value of any pixel is greater than the pixel value of the pixel at the target image location in two adjacent frame images of the frame image where the pixel is located, the pixel is determined as a local maximum point.

[0113] In some embodiments, for each target image location, the pixels located at the target image location in each frame image can be traversed in the time order of the frame images. If the pixel value of any pixel is less than the pixel value of the pixel at the target image location in two adjacent frame images of the frame image where the pixel is located, the pixel is determined as a local minimum point.

[0114] The above embodiments determine the local extreme points in the current video data surplus from both spatial and temporal dimensions, providing a basis for accurately obtaining local mean data in the future.

[0115] In some embodiments, for each preset sliding window, the local mean value corresponding to the preset sliding window is determined based on the pixel values ​​and values ​​of each local extreme point within the preset sliding window; and the local mean value data of the local extreme points in the current video data margin is determined based on the local mean value corresponding to each preset sliding window.

[0116] For example, the preset size of the sliding window can be 5. 5 5 means 5 pixels in each direction (row and column) in the spatial dimension, and 5 frames in the temporal dimension.

[0117] For each preset sliding window, the pixel values ​​of each local extreme point within the preset sliding window can be weighted and summed to obtain the local mean value corresponding to that preset sliding window.

[0118] In some embodiments, the pixel value of each local extreme point within a preset sliding window can be set to... The corresponding weight is The formula for calculating the local mean M is: The weights can be determined based on the actual situation; for example, the closer a local extreme point is to the center of a preset sliding window, the greater its weight.

[0119] By calculating the local mean corresponding to each preset sliding window, the local mean data of local extreme points in the current video data margin can be determined. In some embodiments, the local mean function of local extreme points in the current video data margin can be determined based on the local mean corresponding to each preset sliding window.

[0120] The above embodiments determine local extreme points in the current video data margin and, based on the pixel values ​​of each local extreme point, determine local mean data, providing a basis for decomposition in subsequent iterations.

[0121] In some embodiments, obtaining envelope estimation data corresponding to the current video data surplus includes: for each pixel, determining a first pixel difference between other pixels adjacent to the pixel in the frame image where the pixel is located, based on the pixel value of each pixel; and determining a second pixel difference between other pixels located at the target image position in two adjacent frame images of the pixel; determining envelope estimation data corresponding to the pixel based on the first pixel difference and the second pixel difference; and determining envelope estimation data corresponding to the current video data surplus based on the envelope estimation data corresponding to the pixel.

[0122] The first pixel difference can be understood as the absolute value of the pixel difference between any two adjacent pixels in the same frame of the image.

[0123] The second pixel difference can be understood as the absolute value of the pixel difference between any two adjacent frames at the same image location.

[0124] In some embodiments, a first pixel difference average value is determined based on a first pixel difference; a second pixel difference average value is determined based on a second pixel difference; and envelope estimation data corresponding to a pixel is determined based on the sum of the first pixel difference average value and the second pixel difference average value.

[0125] The above embodiments determine the envelope estimation data corresponding to the current video data margin based on the difference of each first pixel in each frame image and the difference of each second pixel at each image position, providing a basis for decomposition in subsequent iterations.

[0126] S320: Based on the current video data margin, local mean data, and envelope estimation data, determine the pure frequency modulation data for this iteration process.

[0127] In some embodiments, intermediate data can be determined based on the current video data margin and local mean data; pure frequency modulation data can be determined based on the intermediate data and envelope estimation data.

[0128] In some embodiments, the difference between the current video data margin and the local mean data can be used as intermediate data, and then the ratio of the intermediate data to the envelope estimation data can be used as pure frequency modulation data.

[0129] S330: Based on the pure frequency modulation data and envelope estimation data, determine the video component data for this iteration.

[0130] In some embodiments, the product of pure frequency modulation data and envelope estimation data can be used as the video component data in this iteration process.

[0131] S340, based on the current video data surplus and video component data in this iteration, determine the current video data surplus in the next iteration, and return the steps of obtaining the local mean data and envelope estimation data corresponding to the current video data surplus, until the preset cutoff condition is reached; wherein, the current video data surplus in the first iteration is video data.

[0132] In some embodiments, the difference between the current video data margin and the video component data can be determined as the current video data margin for the next iteration.

[0133] The preset cutoff condition can be reaching a preset number of iterations.

[0134] The above embodiments accurately locate local extrema in the remaining video data based on the Local Mean Decomposition (LMD) algorithm, and decompose the video data into at least one video component data by calculating the local mean data and envelope estimation data, thus providing a foundation for subsequent feature data extraction based on the video component data and comprehensive capture of video detail information.

[0135] Based on the above embodiments, this application also provides another optional embodiment, in which the step of determining the category information of the video to be classified is refined.

[0136] See Figure 4 The steps for determining the category information of the video to be classified, as shown, include:

[0137] S410: For each video component data, determine the target component category corresponding to the video component data based on the feature data corresponding to each video component data.

[0138] In some embodiments, for each video component data, the distance between the feature data corresponding to the video component data and the feature data corresponding to other video component data can be determined; a preset number of target video component data are selected from the other video component data in ascending order of distance; a first number of target video component data under each component category is determined according to the component category corresponding to each target video component data; and the component category with the largest corresponding first number is determined as the target component category of the video component data.

[0139] In some embodiments, for each video component data, the Euclidean distance between the feature data corresponding to that video component data and the feature data corresponding to other video component data can be determined, as follows:

[0140]

[0141] in, Euclidean distance; For any video component data, the feature data is used. The feature data corresponds to the other video component data; n is the number of feature dimensions in the feature data.

[0142] In some embodiments, K target video component data can be selected from the other video component data in ascending order of distance. Here, K is a preset value and is a positive integer greater than or equal to 1.

[0143] In some embodiments, the component categories in each target video component data can be counted, and a first quantity of target video component data under each component category can be determined. The component category with the largest corresponding first quantity can be taken as the target component category of the video component data.

[0144] In the above embodiments, based on Euclidean distance, the target video component data that is most similar to the feature data of the current video component data can be determined, thereby providing a basis for the accurate classification of the current video component data; furthermore, taking the component category with the most target video component data as the target component category of the current video component data can make the classification result of the current video component data more accurate.

[0145] S420: Determine the category information of the video to be classified based on the target component category corresponding to each video component data.

[0146] In some embodiments, a second quantity of video component data corresponding to each target component category can be determined; the target component category with the largest corresponding second quantity is determined as the category information of the video to be classified.

[0147] In the above embodiments, by determining the category of the target component with the most video component data as the category information of the video to be classified, the classification result of the current video component data can be made more accurate.

[0148] In the above embodiments, by classifying each video component data and then determining the category information of the video to be classified according to the target component category corresponding to each video component data, since each video component data has been decomposed by LMD, more comprehensive detailed information within the video data can be obtained, thereby effectively improving the accuracy of video classification.

[0149] Based on the above embodiments, this application also provides another optional embodiment in which the video classification method is described in detail.

[0150] See Figure 5 The video classification methods shown include:

[0151] S501, Obtain video data of the video to be classified.

[0152] S502, in each iteration, for each pixel, if the pixel value of the pixel is greater than the pixel values ​​of the two adjacent pixels, the pixel is determined to be a local maximum point; if the pixel value of the pixel is less than the pixel values ​​of the two adjacent pixels, the pixel is determined to be a local minimum point.

[0153] Specifically, when a pixel is a row pixel, two adjacent pixels are two pixels in the frame image where the row pixel is located that are adjacent to the pixel in the row direction; when a pixel is a column pixel, two adjacent pixels are two pixels in the frame image where the row pixel is located that are adjacent to the pixel in the column direction; when a pixel is a pixel at the target image location, two adjacent pixels are pixels at the target image location on two frames adjacent to the pixel's frame image.

[0154] S503, for each preset sliding window, determine the local mean value corresponding to the preset sliding window based on the pixel values ​​and values ​​of each local extreme point within the preset sliding window.

[0155] S504, based on the local mean corresponding to each preset sliding window, determine the local mean data of the local extreme points in the current video data surplus during this iteration process.

[0156] S505, for each pixel, determine the first pixel difference between the pixel and other adjacent pixels in the frame image where the pixel is located, based on the pixel value of each pixel.

[0157] S506, for each pixel, based on the pixel value of each pixel, determine the second pixel difference between the other pixels located at the target image position in the two adjacent frame images of the frame image where the pixel is located and the pixel.

[0158] S507, based on the differences between each first pixel and each second pixel, determine the envelope estimation data corresponding to the current video data margin in this iteration process.

[0159] In specific implementation, the average value of the first pixel differences is determined based on the differences between each first pixel; the average value of the second pixel differences is determined based on the differences between each second pixel; and the envelope estimation data corresponding to the pixel is determined based on the sum of the average values ​​of the first and second pixel differences. Based on the envelope estimation data corresponding to the pixel, the envelope estimation data corresponding to the current video data margin is determined.

[0160] S508 determines intermediate data based on the current video data margin and local mean data.

[0161] S509 determines pure frequency modulation data based on intermediate data and envelope estimation data.

[0162] S510: Based on the pure frequency modulation data and envelope estimation data, determine the video component data for this iteration.

[0163] S511, determine the difference between the current video data margin and the video component data as the current video data margin in the next iteration process, and repeat S502 until the preset cutoff condition is reached.

[0164] In the first iteration, the remaining amount of video data is the video data itself.

[0165] S512, for each video component data, determine the distance between the feature data corresponding to the video component data and the feature data corresponding to other video component data.

[0166] S513, select a preset number of target video component data from the other video component data in ascending order of distance.

[0167] S514, determine the first quantity of target video component data under each component category based on the component category corresponding to each target video component data.

[0168] S515, the component category with the largest number of corresponding components is determined as the target component category of the video component data.

[0169] S516, determine the second quantity of video component data corresponding to each target component category.

[0170] S517, the category of the second most numerous target component is determined as the category information of the video to be classified.

[0171] In some embodiments, the above video classification method can be implemented using the Spark big data video classification model.

[0172] In some embodiments, a cluster consisting of multiple servers can be pre-built. The servers are equipped with high-performance Central Processing Units (CPUs), large-capacity memory, sufficient storage devices, and high-speed network interfaces. A suitable operating system is selected to build the big data cluster, including a distributed storage system and a distributed scheduling system. A suitable distributed database is chosen to store video metadata, and corresponding database configuration and metadata management settings are performed. Based on this, the aforementioned Spark big data video classification model is deployed.

[0173] In some embodiments, large-scale video data can be collected from multiple video platforms. For example, the video duration can be between 5 and 100 seconds, covering various fields and types, including but not limited to entertainment, education, news, sports, and lifestyle, and containing videos of different resolutions and formats. The video files are stored in a distributed system and partitioned into directories according to certain rules. The acquired video data is used to train the aforementioned Spark big data video classification model. The trained Spark big data video classification model is then used to implement the aforementioned video classification method.

[0174] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0175] Based on the same inventive concept, this application also provides a video classification apparatus for implementing the video classification method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more video classification apparatus embodiments provided below can be found in the limitations of the video classification method described above, and will not be repeated here.

[0176] In one exemplary embodiment, such as Figure 6 As shown, a video classification device is provided, including: an acquisition module 610, a decomposition module 620, an extraction module 630, and a determination module 640, wherein:

[0177] The acquisition module 610 is used to acquire video data of the video to be classified;

[0178] The decomposition module 620 is used to decompose video data to obtain at least one video component data;

[0179] The extraction module 630 is used to extract features from the video component data for each video component data to obtain the feature data corresponding to the video component data.

[0180] The determination module 640 is used to determine the category information of the video to be classified based on the feature data corresponding to each video component data.

[0181] In one embodiment, the decomposition module 620 is specifically used for:

[0182] In each iteration, the local mean data of local extrema in the current video data surplus and the envelope estimation data corresponding to the current video data surplus are obtained. Local extrema include local maxima and local minima. The pixel value of a local maxima is greater than the pixel value of its neighboring pixel; the pixel value of a local minima is less than the pixel value of its neighboring pixel. Based on the current video data surplus, local mean data, and envelope estimation data, the pure frequency modulated data for the current iteration is determined. Based on the pure frequency modulated data and envelope estimation data, the video component data for the current iteration is determined. Based on the current video data surplus and video component data for the current iteration, the current video data surplus for the next iteration is determined, and the steps to obtain the local mean data and envelope estimation data corresponding to the current video data surplus are returned, until a preset cutoff condition is reached. The current video data surplus in the first iteration is the video data itself.

[0183] In one embodiment, the video data includes the pixel values ​​of each pixel in a plurality of consecutive frame images;

[0184] Correspondingly, the decomposition module 620 is specifically used for: for each pixel, determining the pixel as a local maximum point when the pixel value of the pixel is greater than the pixel values ​​of the two adjacent pixels; determining the pixel as a local minimum point when the pixel value of the pixel is less than the pixel values ​​of the two adjacent pixels; wherein, when the pixel is a row pixel, the two adjacent pixels are the two pixels in the frame image where the row pixel is located that are adjacent to the pixel in the row direction; when the pixel is a column pixel, the two adjacent pixels are the two pixels in the frame image where the row pixel is located that are adjacent to the pixel in the column direction; when the pixel is a pixel at the target image position, the two adjacent pixels are the pixels at the target image position on the two frames image adjacent to the frame image where the pixel is located; and determining the local mean data of the local extreme points in the current video data surplus based on the pixel values ​​of each local extreme point.

[0185] In one embodiment, the decomposition module 620 is specifically used to: for each preset sliding window, determine the local mean value corresponding to the preset sliding window based on the pixel values ​​and values ​​of each local extreme point within the preset sliding window; and determine the local mean value data of the local extreme points in the current video data margin based on the local mean values ​​corresponding to each preset sliding window.

[0186] In one embodiment, the video data includes the pixel values ​​of each pixel in a plurality of consecutive frame images;

[0187] Accordingly, the decomposition module 620 is specifically used for: for each pixel, determining the first pixel difference between other pixels adjacent to the pixel in the frame image where the pixel is located, based on the pixel value of each pixel; and determining the second pixel difference between other pixels located at the target image position in two adjacent frame images; determining the envelope estimation data corresponding to the pixel based on the first pixel difference and the second pixel difference; and determining the envelope estimation data corresponding to the current video data margin based on the envelope estimation data corresponding to the pixel.

[0188] In one embodiment, the decomposition module 620 is specifically used to: determine the average value of the first pixel differences based on the differences between each first pixel; determine the average value of the second pixel differences based on the differences between each second pixel; and determine the envelope estimation data corresponding to the pixel based on the average value of the first pixel differences and the average value of the second pixel differences.

[0189] In one embodiment, the decomposition module 620 is specifically used to: determine intermediate data based on the current video data margin and local mean data; and determine pure frequency modulation data based on the intermediate data and envelope estimation data.

[0190] In one embodiment, the decomposition module 620 is specifically used to: determine the difference between the current video data surplus and the video component data as the current video data surplus in the next iteration process.

[0191] In one embodiment, the determining module 640 is specifically used to: for each video component data, determine the target component category corresponding to the video component data based on the feature data corresponding to each video component data; and determine the category information of the video to be classified based on the target component category corresponding to each video component data.

[0192] In one embodiment, the determining module 640 is specifically used to: determine the distance between the feature data corresponding to the video component data and the feature data corresponding to other video component data; select a preset number of target video component data from the other video component data in ascending order of distance; determine the first number of target video component data under each component category according to the component category corresponding to each target video component data; and determine the component category with the largest corresponding first number as the target component category of the video component data.

[0193] In one embodiment, the determining module 640 is specifically used to: determine the second quantity of video component data corresponding to each target component category; and determine the target component category with the largest corresponding second quantity as the category information of the video to be classified.

[0194] Each module in the aforementioned video classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0195] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores video data, video component data, feature data, and category information. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a video classification method.

[0196] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0197] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the video classification method provided in any of the above embodiments.

[0198] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the video classification method provided in any of the above embodiments.

[0199] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the video classification method provided in any of the above embodiments.

[0200] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0201] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0202] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A video classification method, characterized in that, The method includes: Obtain video data for the videos to be categorized; The video data is decomposed to obtain at least one video component data; For each video component data, feature extraction is performed on the video component data to obtain the feature data corresponding to the video component data; The category information of the video to be classified is determined based on the feature data corresponding to each video component data.

2. The method according to claim 1, characterized in that, The video data is decomposed to obtain at least one video component data, including: In each iteration, the local mean data of local extreme points in the current video data surplus under this iteration, and the envelope estimation data corresponding to the current video data surplus are obtained; wherein, the local extreme points include local maxima and local minima; the pixel value of the local maxima is greater than the pixel value of the neighboring pixel; the pixel value of the local minima is less than the pixel value of the neighboring pixel; Based on the current video data margin, the local mean data, and the envelope estimation data, determine the pure frequency modulation data for this iteration process; Based on the pure frequency modulation data and the envelope estimation data, determine the video component data for this iteration process; Based on the current video data surplus and the video component data in this iteration, determine the current video data surplus in the next iteration, and return to the steps of obtaining the local mean data and envelope estimation data corresponding to the current video data surplus, until the preset cutoff condition is reached; wherein, the current video data surplus in the first iteration is the video data.

3. The method according to claim 2, characterized in that, The video data includes the pixel values ​​of each pixel in multiple consecutive frame images; Accordingly, the local mean data of local extreme points in the remaining video data during this iteration are obtained, including: For each pixel, if the pixel value is greater than the pixel values ​​of its two adjacent pixels, the pixel is determined to be a local maximum; if the pixel value is less than the pixel values ​​of its two adjacent pixels, the pixel is determined to be a local minimum. Specifically, if the pixel is a row pixel, the two adjacent pixels are the two pixels in the frame image containing the row pixel that are adjacent to the pixel in the row direction; if the pixel is a column pixel, the two adjacent pixels are the two pixels in the frame image containing the row pixel that are adjacent to the pixel in the column direction; if the pixel is a pixel at a target image location, the two adjacent pixels are the pixels in the two frames adjacent to the frame image containing the pixel that are at the target image location. Based on the pixel values ​​of each local extreme point, determine the local mean data of the local extreme points in the current video data margin.

4. The method according to claim 3, characterized in that, The step of determining the local mean data of the local extreme points in the current video data margin based on the pixel values ​​of each of the local extreme points includes: For each preset sliding window, the local mean value corresponding to the preset sliding window is determined based on the pixel values ​​and values ​​of each local extreme point within the preset sliding window. Based on the local mean corresponding to each preset sliding window, determine the local mean data of the local extreme points in the current video data margin.

5. The method according to claim 2, characterized in that, The video data includes the pixel values ​​of each pixel in multiple consecutive frame images; Accordingly, obtaining the envelope estimation data corresponding to the current video data surplus includes: For each pixel, based on its pixel value, determine the first pixel difference between the pixel and other adjacent pixels in the frame image containing the pixel; and, Determine the second pixel difference between the pixel and other pixels located at the target image position in two adjacent frame images; Based on the first pixel difference and the second pixel difference, the envelope estimation data corresponding to the pixel is determined; Based on the envelope estimation data corresponding to the pixel, determine the envelope estimation data corresponding to the current video data margin.

6. The method according to claim 5, characterized in that, The step of determining the envelope estimation data corresponding to the pixel based on the differences between the first pixels and the differences between the second pixels includes: Determine the average value of the first pixel differences based on the differences between each first pixel; Determine the average value of the second pixel difference based on the differences between each second pixel; The envelope estimation data corresponding to the pixel is determined based on the average value of the first pixel difference and the average value of the second pixel difference.

7. The method according to claim 2, characterized in that, The step of determining the pure frequency modulation data for this iteration process based on the current video data surplus, the local mean data, and the envelope estimation data includes: Based on the current remaining video data and the local mean data, determine the intermediate data; The pure frequency modulation data is determined based on the intermediate data and the envelope estimation data.

8. The method according to claim 2, characterized in that, Based on the current video data reserve and the video component data in this iteration, determine the current video data reserve for the next iteration, including: The difference between the current video data margin and the video component data is determined as the current video data margin in the next iteration.

9. The method according to claim 1, characterized in that, The step of determining the category information of the video to be classified based on the feature data corresponding to each of the video component data includes: For each video component data, the target component category corresponding to the video component data is determined based on the feature data corresponding to each video component data. The category information of the video to be classified is determined based on the target component category corresponding to each video component data.

10. The method according to claim 9, characterized in that, Based on the feature data corresponding to each of the video component data, the target component category corresponding to the video component data is determined, including: Determine the distance between the feature data corresponding to the video component data and the feature data corresponding to each other video component data; According to the order of increasing distance, a preset number of target video component data are selected from the other video component data. Based on the component category corresponding to each target video component data, determine the first quantity of target video component data under each component category; The component category with the largest number of corresponding components is determined as the target component category of the video component data.

11. The method according to claim 9 or 10, characterized in that, The step of determining the category information of the video to be classified based on the target component category corresponding to each of the video component data includes: Determine the second quantity of video component data corresponding to each target component category; The category of the target component with the largest number of corresponding second components is determined as the category information of the video to be classified.

12. A video classification device, characterized in that, The device includes: The acquisition module is used to acquire video data of the videos to be classified. The decomposition module is used to decompose the video data to obtain at least one video component data; The extraction module is used to extract features from each video component data to obtain the feature data corresponding to the video component data. The determination module is used to determine the category information of the video to be classified based on the feature data corresponding to each of the video component data.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images