A scene classification method and device based on feature points and image transformation frequency

Abstract

The application provides a scene classification method and device based on feature points and image transformation frequency, comprising: acquiring a video to be classified and converting it into a continuous frame image sequence, extracting key frames from the continuous frame image sequence based on a histogram variation method; extracting a first preset number of FAST feature points from each key frame based on a preset grayscale value threshold; calculating the average value and distribution variance of the response values of the first preset number of FAST feature points; comparing the average value and distribution variance of the response values with preset average value thresholds and distribution variance thresholds of response values under natural scenes and screen scenes according to a preset classification rule to classify the key frames; when the key frames cannot be classified by comparing the thresholds or the number of FAST feature points is insufficient, acquiring the second preset number of frames of images after the corresponding key frames, and classifying the key frames by comparing the image transformation frequency with a preset frequency threshold. The method provided by the application does not need to train a classification model, and has high classification speed and precision.

Description

Technical Field

[0001] This invention relates to the field of scene classification technology, and in particular to a scene classification method and apparatus based on feature points and image transformation frequency. Background Technology

[0002] Scene classification technology refers to classifying input images or videos into different scene categories, such as natural scenes and screen scenes. Natural scenes typically refer to real outdoor or indoor environments, while screen scenes typically refer to interfaces on devices such as computer screens and mobile phone screens.

[0003] Feature points are widely used in scene classification tasks. In the field of image processing, feature points refer to points in an image that possess specific characteristics, such as edges, corners, and textures. The location and description of these feature points can be used to represent the local features of an image. Patent publication number "CN114373145A," entitled "A Scene Classification Method for Surveillance Videos Based on Keyframe Acquisition using the ORB Algorithm," describes a method of extracting ORB feature points and matching them with a pre-trained model to obtain the corresponding labels for video classification. Patent publication number "CN104680173A," entitled "A Scene Classification Method for Remote Sensing Images," describes a method of pre-classifying images based on the distribution of locally invariant feature points, dividing them into two categories: uniformly distributed and non-uniformly distributed feature points. Different models are then used to train and classify the two types of images. However, existing scene classification schemes require pre-training of classification models or classifiers, which is cumbersome and results in slow actual classification speeds.

[0004] The patent published under the title "CN107666610A," entitled "Desktop Video Content Analysis Method," describes a method that uses image transformation frequency as a standard for video classification. It detects the block hash values ​​of video frames within a viewing window, compares them with the first frame, and classifies the video based on the number of differing frames. While this method reflects the image transformation speed, its classification accuracy is poor. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a scene classification method and apparatus based on feature points and image transformation frequency, so as to eliminate or improve one or more defects existing in the prior art, and solve the problems of existing classification techniques requiring pre-training of classification models or classifiers, cumbersome training process, slow classification speed and poor classification accuracy.

[0006] On one hand, the present invention provides a scene classification method based on feature points and image transformation frequency, characterized in that the method includes the following steps:

[0007] The video to be classified is obtained, the video is converted into a continuous frame image sequence, and keyframes are extracted from the continuous frame image sequence based on the histogram change method.

[0008] Based on a preset grayscale threshold, a first preset number of FAST feature points are extracted from each keyframe;

[0009] Calculate the average response value and distribution variance of the first preset number of FAST feature points;

[0010] Obtain a preset threshold for the average value of a first response value and a threshold for the first distribution variance under a natural scene, and a threshold for the average value of a second response value and a threshold for the second distribution variance under a screen scene; if the average response value is less than the threshold for the first average response value and the distribution variance is less than the threshold for the first distribution variance, then the corresponding keyframe is classified as a natural scene; if the average response value is greater than the threshold for the second average response value and the distribution variance is greater than the threshold for the second distribution variance, then the corresponding keyframe is classified as a screen scene.

[0011] When the average response value and distribution variance of the FAST feature points do not meet the above classification rules, or when the number of FAST feature points is less than the first preset number, a second preset number of frame images are obtained after the corresponding key frame. The image transformation frequency is calculated by determining whether two adjacent frames in the second preset number of frame images have changed. If the image transformation frequency is greater than a preset frequency threshold, the corresponding key frame is classified as a natural scene. If the image transformation frequency is less than the frequency threshold, the corresponding key frame is classified as a screen scene.

[0012] In some embodiments of the present invention, extracting keyframes from the continuous frame image sequence based on the histogram change method further includes:

[0013] The continuous frame image sequence is converted into a grayscale image, and a grayscale histogram is calculated;

[0014] Calculate the grayscale histogram difference between adjacent frames. If the difference is greater than a preset difference threshold, then the current frame is taken as the key frame.

[0015] In some embodiments of the present invention, extracting a first preset number of FAST feature points from each keyframe based on a preset grayscale value threshold further includes:

[0016] In one loop, FAST feature points are extracted from each keyframe based on a preset grayscale threshold; the extracted FAST feature points are sorted according to their response values; and the preset grayscale threshold is modified to the grayscale value of the FAST feature point with the lowest response value.

[0017] Repeat the above steps until the first preset number of FAST feature points with high response values ​​are obtained.

[0018] In some embodiments of the present invention, the extracted FAST feature points are sorted according to their response values, wherein the method for calculating the response values ​​of the FAST feature points includes the following steps:

[0019] Obtain a preset grayscale threshold for extracting the FAST feature points, and calculate the median value between the preset grayscale threshold and the maximum grayscale value as the starting median value;

[0020] In one loop, if the initial intermediate value is less than the gray value of the FAST feature point, then the midpoint between the initial intermediate value and the maximum gray value is calculated and used as the new initial intermediate value; if the initial intermediate value is greater than the gray value of the FAST feature point, then the midpoint between the preset gray value threshold and the initial intermediate value is calculated and used as the new initial intermediate value.

[0021] Repeat the above steps until the boundary gray value is obtained, and use the boundary gray value as the response value of the FAST feature point.

[0022] In some embodiments of the present invention, the method for calculating the distribution variance of the first preset number of FAST feature points includes the following steps:

[0023] The corresponding keyframes are divided into multiple regions, and the number of FAST feature points in each region is counted.

[0024] Calculate the variance of the number of FAST feature points in all regions to obtain the distribution variance.

[0025] In some embodiments of the present invention, after scene classification is achieved based on the image transformation frequency, the average threshold and the first distribution variance threshold of the first response value under natural scenes, or the average threshold and the second distribution variance threshold of the second response value under screen scenes, are adjusted according to the scene type.

[0026] In some embodiments of the present invention, it further includes:

[0027] When a corresponding keyframe is determined to be a natural scene, if the average response value is greater than the first average response value threshold, then the first average response value threshold is increased, calculated as follows:

[0028]

[0029] Wherein, Tne represents the threshold value of the first response value average; e represents the average value of the response value;

[0030] If the average response value is less than the first average response value threshold, and the distribution variance is greater than the first distribution variance threshold, then the first distribution variance threshold is increased, calculated as follows:

[0031]

[0032] Wherein, Tnd represents the first distribution variance threshold; d represents the distribution variance;

[0033] When a corresponding keyframe is determined to be a screen scene, if the average response value is less than the first average response value threshold, then the first average response value threshold is reduced, calculated as follows:

[0034]

[0035] Where Tse represents the threshold value of the second response value; e represents the average value of the response value;

[0036] If the average response value is greater than the first average response value threshold, and the distribution variance is less than the first distribution variance threshold, then the first distribution variance threshold is reduced, calculated as follows:

[0037]

[0038] Where Tsd represents the second distribution variance threshold; d represents the distribution variance.

[0039] In some embodiments of the present invention, the calculation of the image transformation frequency by determining whether two adjacent frames in the second preset number of frames have changed further includes:

[0040] Obtain two adjacent frames from the second preset number of frames;

[0041] Divide two adjacent frames into multiple regions and calculate the grayscale histogram for each region.

[0042] Compare the grayscale histograms of two adjacent frames and record the number of regions that have changed; if more than half of the regions have changed, mark the current frame as changed.

[0043] The image transformation frequency refers to the number of image frames that undergo transformation in the second preset number of image frames.

[0044] On the other hand, the present invention provides a scene classification device based on feature points and image transformation frequency, including a processor and a memory, characterized in that the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the scene classification method based on feature points and image transformation frequency as described above.

[0045] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the program is executed by a processor, it implements the steps of the scene classification method based on feature points and image transformation frequency as described in any of the above.

[0046] The beneficial effects of the present invention are at least as follows:

[0047] This invention provides a scene classification method and apparatus based on feature points and image transformation frequency, comprising: acquiring a video to be classified and converting it into a sequence of continuous frame images; extracting keyframes from the continuous frame image sequence based on the histogram change method; extracting a first preset number of FAST feature points from each keyframe based on a preset grayscale threshold; calculating the average response value and distribution variance of the first preset number of FAST feature points; classifying the keyframes by comparing the average response value and distribution variance with preset thresholds for the average response value and distribution variance under natural scenes and screen scenes according to preset classification rules; when classification cannot be performed by comparing thresholds, or when the number of FAST feature points is insufficient, the transformation frequency of the image is used to supplement the classification by acquiring a second preset number of frame images after the corresponding keyframe and comparing the image transformation frequency with a preset frequency threshold. The scene classification method provided by this invention does not require training a classification model, avoiding a cumbersome training process, and achieves fast classification speed and high accuracy.

[0048] Furthermore, when classification cannot be performed by comparing thresholds and scene classification is required by using image transformation frequency, the average threshold and distribution variance threshold of the response value under the corresponding scene can be dynamically adjusted according to the final classified scene type to improve classification accuracy.

[0049] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0050] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0051] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0052] Figure 1 This is a schematic diagram of the steps of a scene classification method based on feature points and image transformation frequency in one embodiment of the present invention.

[0053] Figure 2 This is a flowchart of a scene classification method based on feature points and image transformation frequency in one embodiment of the present invention.

[0054] Figure 3 This is a flowchart of a method for calculating the response value of a feature point in one embodiment of the present invention.

[0055] Figure 4 This is a flowchart illustrating the dynamic adjustment of scene thresholds based on image transformation frequency judgment results in one embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0057] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0058] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0059] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0060] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0061] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0062] To address the problems of existing classification techniques, such as the need for pre-training classification models or classifiers, cumbersome training processes, slow classification speeds, and poor classification accuracy, this invention provides a scene classification method based on feature points and image transformation frequencies, such as... Figure 1 and Figure 2 As shown, the method includes the following steps S101 to S105:

[0063] Step S101: Obtain the video to be classified, convert the video into a continuous frame image sequence, and extract keyframes from the continuous frame image sequence based on the histogram change method.

[0064] Step S102: Based on a preset grayscale threshold, extract a first preset number of FAST feature points from each keyframe.

[0065] Step S103: Calculate the average value and distribution variance of the response values ​​of the first preset number of FAST feature points.

[0066] Step S104: Obtain the preset threshold values ​​of the first average response value and the first distribution variance threshold for natural scenes, and the threshold values ​​of the second average response value and the second distribution variance threshold for screen scenes; if the average response value is less than the threshold value of the first average response value and the distribution variance is less than the threshold value of the first distribution variance, then the corresponding keyframe is classified as a natural scene; if the average response value is greater than the threshold value of the second average response value and the distribution variance is greater than the threshold value of the second distribution variance, then the corresponding keyframe is classified as a screen scene.

[0067] Step S105: When the average response value and distribution variance of the FAST feature points do not meet the above classification rules, or the number of FAST feature points is less than the first preset number, the second preset number of frame images after the corresponding key frame are obtained. The image transformation frequency is calculated by judging whether the two adjacent frames in the second preset number of frame images have changed. If the image transformation frequency is greater than the preset frequency threshold, the corresponding key frame is classified as a natural scene. If the image transformation frequency is less than the frequency threshold, the corresponding key frame is classified as a screen scene.

[0068] In step S101, the video that needs to be classified into scenes is acquired, and the video is converted into a sequence of consecutive frame images using video editing software.

[0069] In some embodiments, the VideoCapture function in the open-source computer vision library OpenCV can be used to read video and convert it into a sequence of consecutive frame images.

[0070] After acquiring a sequence of consecutive image frames, keyframes are extracted from the sequence based on the histogram variation method. A histogram, also known as a grayscale histogram, is a statistical chart used to represent the distribution of pixels at each grayscale level in an image. A grayscale histogram typically consists of 256 bars, representing all grayscale levels from 0 to 255. The horizontal axis represents the grayscale level, and the vertical axis represents the number of pixels at that grayscale level in the image. The grayscale histogram provides a visual understanding of the grayscale distribution of an image.

[0071] In some embodiments, the method for extracting keyframes includes the following steps S1011 to S1012:

[0072] Step S1011: Convert the continuous frame image sequence into grayscale images and calculate the grayscale histogram of each grayscale image.

[0073] Step S1012: Calculate the difference between the grayscale histograms of adjacent frames. If the difference is greater than the preset difference threshold, then the current frame is taken as the key frame.

[0074] In some embodiments, histogram similarity measurement methods, such as cross-entropy and correlation coefficient, can be used to calculate the gray-level histogram difference between adjacent frames.

[0075] In some embodiments, the calcHist function in the open-source computer vision library OpenCV can be used to calculate the grayscale histogram for each grayscale image.

[0076] In some embodiments, in step S101, the video is converted into a sequence of consecutive frame images without extracting keyframes. That is, in steps S102 to S105, feature point extraction and scene classification are performed on each frame of the consecutive frame image sequence.

[0077] In step S102, a grayscale threshold is preset, and N high-response-value FAST feature points are extracted from each keyframe using the grayscale threshold, where N represents a first preset number. The FAST feature point extraction algorithm is an algorithm for extracting image corner points. It can quickly extract feature points that can represent image information. In this algorithm, a FAST corner point (feature point) is defined as follows: if a pixel differs significantly from a sufficient number of its surrounding pixels, then that pixel may be a corner point, and the degree of difference between these corner points and their surrounding pixels is the intensity of the corner point.

[0078] Step S102 can be further divided into the following steps S1021 to S1023:

[0079] Step S1021: Obtain the grayscale image of the keyframe in step S101. In some embodiments, a smoothing filter can be used to reduce noise in the grayscale image to reduce the false detection rate.

[0080] Step S1022: Use the FAST feature point extraction algorithm to detect feature points, extract feature points in key frames according to the preset grayscale threshold, and calculate the response value of the feature points.

[0081] Step S1023: Sort the extracted feature points according to the response value from high to low, and select the top N feature points with high response values ​​as the FAST feature points required by this invention.

[0082] In some embodiments, it may not be possible to extract N FAST feature points at once based on a preset grayscale threshold. Therefore, the grayscale threshold needs to be continuously adjusted to meet the required number of FAST feature points. Specifically:

[0083] In one iteration, the FAST feature point extraction algorithm is used to extract FAST feature points from each keyframe based on a preset grayscale threshold (the number of extracted FAST feature points is less than N at this point); the extracted FAST feature points are sorted according to their response values; the preset grayscale threshold is then modified to the grayscale value of the FAST feature point with the lowest response value. This process is repeated until N FAST feature points with high response values ​​are obtained.

[0084] In some embodiments, such as Figure 3 As shown, the method for calculating the response value of each FAST feature point in step S1022 may further include the following steps S10221 to S10223:

[0085] Step S10221: Obtain the preset gray value threshold for extracting the corresponding FAST feature points, and calculate the median value between the preset gray value threshold and the maximum gray value as the starting median value.

[0086] Step S10222: In one loop, if the initial intermediate value is less than the gray value of the FAST feature point, calculate the median value between the initial intermediate value and the maximum gray value, and use it as the new initial intermediate value; if the initial intermediate value is greater than the gray value of the FAST feature point, calculate the median value between the preset gray value threshold and the initial intermediate value, and use it as the new initial intermediate value.

[0087] Step S10223: Repeat the above steps until the boundary gray value is obtained, and use the boundary gray value as the response value of the FAST feature point.

[0088] In step S10221, the obtained preset grayscale threshold is further explained: For example, if the corresponding FAST feature point is not extracted in the first feature point extraction, after using the feature point extraction method described above, the grayscale threshold is adjusted to 95, and the FAST feature point is extracted. Then, the preset grayscale threshold obtained in step S10221 is 95. The maximum grayscale value refers to the largest grayscale value among the pixel values ​​of the image. In an 8-bit grayscale image, the maximum grayscale value is 255. Therefore, the initial intermediate value at this time is 175.

[0089] In step S10222, the initial intermediate value is compared with the grayscale value of the FAST feature point. That is, it tests whether the FAST feature point will still be extracted and identified as a feature point at the initial intermediate grayscale value. Specifically, if the initial intermediate value is less than the grayscale value of the FAST feature point, that is, the FAST feature point will still be identified as a feature point, then the midpoint between the initial intermediate value and the maximum grayscale value is taken as the new initial intermediate value. For example, if the initial intermediate value is 175 and the maximum grayscale value is 255, then the new initial intermediate value is 215. If the initial intermediate value is greater than the grayscale value of the FAST feature point, that is, the FAST feature point is no longer a feature point, then the midpoint between the preset grayscale value threshold and the initial intermediate value is taken as the new initial intermediate value. For example, if the preset grayscale value threshold is 95 and the initial intermediate value is 175, then the new initial intermediate value is 135.

[0090] In step S10223, step S10222 is repeated, new starting intermediate values ​​are continuously obtained, and the starting intermediate values ​​are used to test whether the FAST feature point will still be extracted and identified as a feature point. The range of gray values ​​is continuously narrowed, and finally the boundary gray value is obtained. This boundary gray value is used as the response value of the feature point.

[0091] exist Figure 3 In the initialization part T, it represents the preset grayscale value threshold obtained in step S10221, and bmax is the maximum grayscale value of 255. Then, step S10222 is repeated to continuously update the values ​​of bmax and bmin, narrowing the range between bmax and bmin. Finally, bmax = bmin, and the boundary grayscale value is obtained.

[0092] In step S103, the average response value and distribution variance of the N FAST feature points on the corresponding keyframe are calculated.

[0093] Obtain the response value of each FAST feature point obtained in step S102, and divide the sum of the response values ​​of each FAST feature point by the first preset number N to obtain the average response value.

[0094] The method for calculating the variance of the distribution includes the following steps S1031~S1032:

[0095] Step S1031: Divide the corresponding keyframe into multiple regions and count the number of FAST feature points in each region. In some embodiments, a grid partitioning method can be used to divide the corresponding keyframe into multiple regions.

[0096] Step S1032: Calculate the variance of the number of FAST feature points in all regions to obtain the distribution variance. Specifically, first calculate the sum of the number of FAST feature points in all regions, divide it by the number of regions to obtain the average; then calculate the sum of the squares of the number of FAST feature points in all regions, divide it by the number of regions, and subtract the square of the average to obtain the final variance.

[0097] In step S104, preset threshold values ​​for the average response value and the first distribution variance are obtained for natural scenes, and threshold values ​​for the average response value and the second distribution variance are obtained for screen scenes. The average response value and distribution variance of N FAST feature points on the corresponding keyframe are compared with the threshold values ​​for the two scenes to determine the scene type of the keyframe. Specifically, the classification rule is as follows:

[0098] If the average response value is less than the first average response value threshold and the distribution variance is less than the first distribution variance threshold, then the corresponding keyframe is classified as a natural scene; if the average response value is greater than the second average response value threshold and the distribution variance is greater than the second distribution variance threshold, then the corresponding keyframe is classified as a screen scene.

[0099] In step S105, when the average response value and distribution variance of the FAST feature points on the corresponding keyframe do not conform to the classification rules in step S104, i.e., the scene type cannot be determined by comparing thresholds, or when the first preset number N FAST feature points cannot be extracted in step S102, the image transformation frequency can be used to compensate. Here, "unable to determine the scene type by comparing thresholds" refers to situations such as: the average response value is less than the first average response value threshold, and the distribution variance is greater than the first distribution variance threshold; or the average response value is greater than the first average response value threshold, and the distribution variance is less than the first distribution variance threshold, etc.

[0100] Specifically, a second preset number of frame images are extracted after the corresponding keyframes for which the scene type cannot be determined or sufficient FAST feature points cannot be extracted. For example, the second preset number is denoted as M.

[0101] In some embodiments, the method for determining whether an image has undergone a transformation includes the following steps S1051 to S1053:

[0102] Step S1051: Obtain two adjacent frames from the M-frame image.

[0103] Step S1052: Divide two adjacent frames into multiple regions and calculate the grayscale histogram of each region.

[0104] Step S1053: Compare the grayscale histograms of adjacent two pins and record the number of regions that have changed. If more than half of the regions have changed, mark the current frame image as changed.

[0105] After determining whether any transformation has occurred in each of the M frames, the image transformation frequency is obtained. If the image transformation frequency is greater than a preset frequency threshold, the corresponding keyframe is classified as a natural scene; if the image transformation frequency is less than the frequency threshold, the corresponding keyframe is classified as a screen scene. For example, M is 10, meaning 10 frames are acquired after the corresponding keyframe, and each of the 10 frames is determined to have undergone a transformation. If 7 of these frames are marked as transformed, the image transformation frequency is 7. The frequency threshold is usually set to half of the second preset number, i.e., 5. Therefore, the image transformation frequency of 7 is greater than the frequency threshold of 5, and in this case, the corresponding keyframe is classified as a natural scene.

[0106] In some embodiments, such as Figure 4 As shown, after scene classification is achieved based on image transformation frequency in step S105, the average threshold and first distribution variance threshold of the first response value in natural scenes, or the average threshold and second distribution variance threshold of the second response value in screen scenes, can be adjusted according to the scene type.

[0107] Specifically, when a corresponding keyframe is determined to be a natural scene, if the average response value is greater than the first average response value threshold, the first average response value threshold is increased, as shown in formula (1):

[0108]

[0109] Where Tne represents the threshold value of the first response value average; e represents the average response value.

[0110] If the average response value is less than the first average response value threshold and the variance of the distribution is greater than the first variance of the distribution threshold, then the first variance of the distribution threshold is increased, as shown in formula (2):

[0111]

[0112] Where Tnd represents the first distribution variance threshold; d represents the distribution variance.

[0113] When the corresponding keyframe is determined to be a screen scene, if the average response value is less than the first average response value threshold, the first average response value threshold is reduced, and the calculation formula is shown in formula (3):

[0114]

[0115] Where Tse represents the threshold value of the second response value; e represents the average value of the response value;

[0116] If the average response value is greater than the first average response value threshold and the variance of the distribution is less than the first variance of the distribution threshold, then the first variance of the distribution threshold is reduced, as shown in formula (4):

[0117]

[0118] Where Tsd represents the second distribution variance threshold; d represents the distribution variance.

[0119] Based on the above steps and formulas, the thresholds for the two scenarios are dynamically adjusted to improve classification accuracy.

[0120] As shown in Table 1, based on a large amount of experimental data, it can be determined that the scene classification method based on feature points and image transformation frequency provided by this invention achieves a classification accuracy of over 90%.

[0121] Table 1

[0122]

[0123] The present invention also provides a scene classification device based on feature points and image transformation frequency, including a processor and a memory, characterized in that the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the scene classification method based on feature points and image transformation frequency.

[0124] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a scene classification method based on feature points and image transformation frequency.

[0125] Corresponding to the above method, the present invention also provides an apparatus comprising a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus performs the steps of the method as described above.

[0126] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0127] In summary, this invention provides a scene classification method and apparatus based on feature points and image transformation frequency, comprising: acquiring a video to be classified and converting it into a sequence of continuous frame images; extracting keyframes from the continuous frame image sequence based on the histogram change method; extracting a first preset number of FAST feature points from each keyframe based on a preset grayscale value threshold; calculating the average response value and distribution variance of the first preset number of FAST feature points; classifying the keyframes by comparing the average response value and distribution variance with preset thresholds for the average response value and distribution variance under natural scenes and screen scenes according to preset classification rules; when classification cannot be performed by comparing thresholds, or when the number of FAST feature points is insufficient, the transformation frequency of the image is used to supplement the classification by acquiring a second preset number of frame images after the corresponding keyframe and comparing the image transformation frequency with a preset frequency threshold. The scene classification method provided by this invention does not require training a classification model, avoiding a cumbersome training process, and offers fast classification speed and high accuracy.

[0128] Furthermore, when classification cannot be performed by comparing thresholds and scene classification is required by using image transformation frequency, the average threshold and distribution variance threshold of the response value under the corresponding scene can be dynamically adjusted according to the final classified scene type to improve classification accuracy.

[0129] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0130] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0131] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0132] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A scene classification method based on feature points and image transform frequency, characterized in that, The method comprises the following steps: Obtaining a video to be classified, converting the video into a sequence of continuous frame images, and extracting key frames from the sequence of continuous frame images based on a histogram change amount method; Extracting a first preset number of FAST feature points from each key frame based on a preset grayscale value threshold; Calculating the average value and distribution variance of the response values of the first preset number of FAST feature points; wherein the calculation of the response values comprises: obtaining a preset grayscale value threshold of the FAST feature point, and calculating the intermediate value between the preset grayscale value threshold and the maximum grayscale value as a starting intermediate value; in one cycle, if the starting intermediate value is less than the grayscale value of the FAST feature point, the intermediate value between the starting intermediate value and the maximum grayscale value is calculated and used as a new starting intermediate value; if the starting intermediate value is greater than the grayscale value of the FAST feature point, the intermediate value between the preset grayscale value threshold and the starting intermediate value is calculated and used as a new starting intermediate value; the above steps are repeated until a boundary grayscale value is obtained, and the boundary grayscale value is used as the response value of the FAST feature point; the calculation of the distribution variance comprises: dividing the corresponding key frame into multiple regions, counting the number of FAST feature points in each region, and calculating the variance of the number of FAST feature points in all regions to obtain the distribution variance; Obtaining a first response value average threshold and a first distribution variance threshold in a preset natural scene, and a second response value average threshold and a second distribution variance threshold in a screen scene; if the average value of the response values is less than the first response value average threshold and the distribution variance is less than the first distribution variance threshold, the corresponding key frame is classified as a natural scene, and if the average value of the response values is greater than the second response value average threshold and the distribution variance is greater than the second distribution variance threshold, the corresponding key frame is classified as a screen scene; When the average value of the response values of the FAST feature points and the distribution variance do not meet the above classification rules, or the number of FAST feature points is less than the first preset number, a second preset number of frames of images after the corresponding key frame is obtained, the image transformation frequency is calculated by judging whether the adjacent two frames of images in the second preset number of frames of images are transformed, if the image transformation frequency is greater than a preset frequency threshold, the corresponding key frame is classified as a natural scene, and if the image transformation frequency is less than the frequency threshold, the corresponding key frame is classified as a screen scene; wherein the step of judging whether the adjacent two frames of images are transformed comprises: obtaining the adjacent two frames of images in the second preset number of frames of images; dividing the adjacent two frames of images into multiple regions, and counting the grayscale histogram of each region, comparing the region grayscale histograms of the adjacent two frames, recording the number of regions that are transformed, and marking the current frame of image as transformed if more than half of the regions are transformed; wherein the image transformation frequency represents the number of image frames that are transformed in the second preset number of frames of images.

2. The method of claim 1, wherein, The key frames are extracted from the sequence of continuous frame images based on the histogram change amount method, and further comprise: convert the continuous frame image sequence into gray scale images and calculate a gray scale histogram; calculate the gray scale histogram difference between adjacent frames, and if the difference is greater than a preset difference threshold, the current frame is taken as the key frame. 3.The method of claim 1, wherein, extract a first preset number of FAST feature points from each key frame based on a preset gray scale value threshold, and the method further comprises: extract FAST feature points from each key frame based on a preset gray scale value threshold in one loop; sort the extracted FAST feature points according to response values; modify the preset gray scale value threshold to the gray scale value of the FAST feature point with the lowest response value; repeat the above steps until the first preset number of FAST feature points with high response values are obtained. 4.The method of claim 1, wherein, After the scene classification based on the image transformation frequency, adjust the first response value average threshold and the first distribution variance threshold for natural scenes, or the second response value average threshold and the second distribution variance threshold for screen scenes according to the scene type.

5. The method of claim 4, wherein, the method further comprises: when the corresponding key frame is determined to be a natural scene, if the response value average is greater than the first response value average threshold, increase the first response value average threshold, and the calculation formula is: ； wherein represents the first response value average threshold value; represents the response value average value; if the response value average is less than the first response value average threshold and the distribution variance is greater than the first distribution variance threshold, increase the first distribution variance threshold, and the calculation formula is: ； wherein, represents the first distribution variance threshold; represents the distribution variance; when the corresponding key frame is determined to be a screen scene, if the response value average is less than the first response value average threshold, decrease the first response value average threshold, and the calculation formula is: ； wherein represents the second response value average value threshold; represents the response value average value; if the response value average is greater than the first response value average threshold and the distribution variance is less than the first distribution variance threshold, decrease the first distribution variance threshold, and the calculation formula is: ； wherein, represents the second distribution variance threshold; represents the distribution variance.

6. A scene classification apparatus based on feature points and image transform frequency, comprising a processor and a memory, characterized in that, the memory stores computer instructions, and the processor is configured to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the device implements the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that the program is executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

Images