Image classification method and device, electronic device, and storage medium

Abstract

An image classification method and device, an electronic device, and a storage medium, the image classification method comprising: sequentially taking each pixel point in an input image as a current pixel point, for each preset direction of a plurality of preset directions, determining a spatial domain feature value and a frequency domain feature value of the current pixel point based on pixel values of the current pixel point and adjacent pixel points of the current pixel point in the preset direction; screening the spatial domain feature values and the frequency domain feature values of each pixel point in the input image to obtain a spatial domain feature and a frequency domain feature of the input image; performing feature fusion processing on the spatial domain feature and the frequency domain feature to obtain a fusion feature; inputting the fusion feature into a neural network model for classification processing to obtain a classification result, the classification result being used to represent that the input image is a native image or a fake image. The embodiment of the present application improves the effect of image classification.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image classification method and apparatus, an electronic device, and a storage medium. Background Technology

[0002] With the rapid development of the digital television industry, 4K (3840×2160 pixels) ultra-high-definition technology, as the next stage of high-definition technology, has become a hot topic. 4K video significantly improves image sharpness and detail, providing viewers with a more realistic and immersive visual experience. Currently, over 60% of televisions in the Chinese market support 4K ultra-high-definition technology, and many online video operators have also launched ultra-high-definition content or programs. The smartphone industry is also turning its attention to 4K technology, with an increasing number of smartphones supporting the recording of 4K images and videos.

[0003] 4K video production often involves multiple stages, including video capture, production, post-processing, and transmission. To transmit 4K content over networks with limited bandwidth, many source 4K videos undergo downsampling and upsampling. These "fake 4K" videos, upsampled to high-resolution content using super-resolution technology, typically suffer from image quality degradation. This means that the advertised 4K content often falls short of the expected 4K quality, resulting in a poor viewing experience for viewers. Therefore, accurately classifying and identifying native 4K and fake 4K content is crucial for implementing targeted image optimization techniques to improve the user's viewing experience.

[0004] However, the effectiveness of image classification still needs to be improved. Summary of the Invention

[0005] The problem addressed by the embodiments of the present invention is to provide an image classification method and apparatus, electronic device and storage medium to improve the image classification effect.

[0006] To address the aforementioned problems, this invention provides an image classification method, comprising: acquiring an input image; sequentially using each pixel in the input image as a current pixel; for each of a plurality of preset directions, determining the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel values ​​of the current pixel and its neighboring pixels in that preset direction; filtering the spatial domain feature value and frequency domain feature value of each pixel in the input image to obtain the spatial domain feature and the frequency domain feature of the input image; performing feature fusion processing on the spatial domain feature and the frequency domain feature to obtain fused features; inputting the fused features into a neural network model for classification processing to obtain a classification result, wherein the classification result is used to characterize whether the input image is a native image or a fake image.

[0007] Accordingly, this embodiment of the invention also provides an image classification device, comprising: an image acquisition module for acquiring an input image; a feature extraction module for sequentially taking each pixel in the input image as a current pixel, and for each of a plurality of preset directions, determining the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel value of the current pixel and its neighboring pixels in that preset direction; a feature filtering module for filtering the spatial domain feature value and frequency domain feature value of each pixel in the input image to obtain the spatial domain feature and the frequency domain feature of the input image; a feature fusion module for performing feature fusion processing on the spatial domain feature and the frequency domain feature to obtain fused features; and a classification module for inputting the fused features into a neural network model for classification processing to obtain a classification result, wherein the classification result is used to characterize whether the input image is a native image or a fake image.

[0008] Accordingly, embodiments of the present invention also provide an electronic device, including at least one memory and at least one processor, wherein the memory stores one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the image classification method of the present invention.

[0009] This invention also provides a storage medium storing one or more computer instructions for implementing the image classification method of this invention.

[0010] Compared with the prior art, the technical solution of the embodiments of the present invention has the following advantages: The image classification method provided in this invention sequentially takes each pixel in the input image as the current pixel. For each of a plurality of preset directions, based on the pixel values ​​of the current pixel and its neighboring pixels in that preset direction, it determines the spatial domain feature value and frequency domain feature value of the current pixel. This allows the image features to simultaneously reflect the structural and spectral distribution information of the image, improving the richness and comprehensiveness of the image features. Furthermore, since the spatial and frequency domain feature values ​​are obtained based on the pixel values ​​of each pixel and its neighboring pixels in each preset direction, they can reflect the correlation and differences between pixels in the input image, which is beneficial for improving the sensitivity of image features to local details and texture changes in the image, thus enabling accurate feature representation of scenes with changing scenes. Moreover, this invention also filters the spatial and frequency domain feature values ​​of each pixel in the input image, thereby reducing redundant image feature information, and thus reducing the computational load of subsequent feature fusion and classification processing, and improving the running speed. Correspondingly, the spatial and frequency domain feature values ​​obtained after filtering are... Feature fusion processing of rate domain features yields fused features that organically combine image structural features with spectral distribution features, enhancing the representational and discriminative capabilities of the fused features and providing a more reliable feature basis for subsequent classification. When the fused features are input into a neural network model for classification, the model's inherent learning and nonlinear modeling capabilities allow it to automatically learn the complex relationships inherent in the fused features. This eliminates the need for manually designed classification rules and avoids excessively demanding requirements on the completeness of the input feature data, thereby improving image classification accuracy and enhancing the model's generalization ability. It also reduces the risk of performance degradation due to missing or outlier input data. In summary, the image classification method provided in this invention, by acquiring rich and comprehensive image features, filtering feature values, performing effective feature fusion, and utilizing a neural network model for automatic learning and classification, can accurately distinguish between native and false images. It ensures classification accuracy while maintaining the model's generalization ability, reduces computational load, increases running speed, and ultimately improves the overall image classification effect. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating an embodiment of the image classification method of the present invention; Figure 2 This is a schematic diagram of an embodiment of a preset window W and the current pixel P in an input image; Figure 3 This is a flowchart illustrating an embodiment of step S6; Figure 4 This is a flowchart illustrating an embodiment of step S61; Figure 5This is a flowchart illustrating an embodiment of step S7; Figure 6 This is a schematic diagram of the structure of an embodiment of the neural network model of the present invention; Figure 7 This is a functional block diagram of an embodiment of the image classification device of the present invention; Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device of the present invention. Detailed Implementation

[0012] As can be seen from the background technology, the current image classification performance still needs to be improved.

[0013] Specifically, full-reference image quality assessment algorithms, such as PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity), evaluate the quality of the image to be evaluated by comparing it with its corresponding reference image and calculating the difference between the two. However, in many practical applications, reference image information is unavailable, resulting in poor performance in distinguishing between native and fake images, and limited applicability.

[0014] Unlike full-reference image quality assessment algorithms, no-reference image methods can directly evaluate the quality of distorted images without relying on a reference image, making them more versatile. However, existing no-reference image classification methods are mainly designed for low-resolution images, targeting synthetic distortion types such as JPEG compression and Gaussian noise. They struggle to effectively classify high-resolution real and fake images. This is because, compared to classifying synthetic distortion types, real and fake image classification faces two unique technical challenges: first, upsampling increases the spatial resolution of images, significantly increasing the computational complexity of classification algorithms; second, upsampling fake images leads to the loss of high-frequency features, resulting in degradation or distortion of texture, details, and image structure, further increasing the difficulty of classifying original and fake images.

[0015] Besides the image quality assessment-based schemes mentioned above, manual feature classification schemes based on traditional algorithms extract features from images based on certain patterns they exhibit, and then classify the images accordingly. However, manual feature classification based on traditional algorithms suffers from drawbacks such as poor feature applicability and low classification accuracy; furthermore, traditional algorithms struggle to accurately classify images where the scene is changing.

[0016] Image classification schemes based on machine learning learn from input image data to build a classification model, thereby achieving image classification. However, machine learning-based methods have high requirements for input data; missing data or a few outliers will lead to a decrease in the model's classification performance. Furthermore, machine learning-based classification models are also prone to insufficient generalization ability.

[0017] Furthermore, deep learning-based image classification schemes perform end-to-end feature learning and classification of images by constructing neural network models. However, deep learning-based network models have a large number of parameters and require significant computation, making it difficult to meet real-time requirements on devices without GPUs (Graphics Processing Units) or NPUs (Neural Network Processing Units).

[0018] Furthermore, some existing image classification methods obtain an overall image quality assessment by predicting a small number of local image patches. Such methods fail to effectively filter out key regions that reflect the overall image quality, resulting in low classification accuracy.

[0019] Therefore, there is an urgent need for an image classification method to improve the overall effect of image classification.

[0020] To address the aforementioned technical problems, this invention provides an image classification method. Each pixel in the input image is sequentially taken as the current pixel. For each of a plurality of preset directions, based on the pixel values ​​of the current pixel and its neighboring pixels in that preset direction, spatial domain feature values ​​and frequency domain feature values ​​are determined. This allows image features to simultaneously reflect both structural and spectral distribution information, improving the richness and comprehensiveness of image features. Furthermore, since the spatial and frequency domain feature values ​​are obtained based on the pixel values ​​of each pixel and its neighboring pixels in each preset direction, they reflect the correlation and differences between pixels in the input image. This enhances the sensitivity of image features to local details and texture changes, enabling accurate feature representation of scenes with changing conditions. Moreover, this invention also addresses the spatial domain features of each pixel in the input image. The spatial and frequency domain features are filtered to reduce redundant image feature information, thereby reducing the computational load and improving the running speed of subsequent feature fusion and classification processing. Correspondingly, the spatial and frequency domain features obtained after filtering are fused. The resulting fused features can organically combine image structural features with spectral distribution features, improving the representation and discrimination capabilities of the fused features and providing a more reliable feature basis for subsequent classification processing. When the fused features are input into the neural network model for classification processing, the neural network model itself has learning and nonlinear modeling capabilities, which can automatically learn the complex relationships contained in the fused features. It does not rely on manually designed classification rules, nor does it place excessive demands on the completeness of the input feature data. This can improve the accuracy of image classification, enhance the generalization ability of the model, and reduce the risk of model performance degradation due to missing or outlier input data.

[0021] In summary, the image classification method provided by the embodiments of the present invention can accurately identify original images and fake images by acquiring rich and comprehensive image features, filtering feature values, performing effective feature fusion, and using a neural network model for automatic learning and classification. It ensures classification accuracy while taking into account the generalization ability of the model, and also reduces the amount of computation and improves the running speed, thereby improving the overall effect of image classification.

[0022] It should be noted that the image classification method provided in this embodiment of the invention is executed by an electronic device with image processing capabilities. In specific embodiments, the electronic device may be a terminal device such as a smartphone, tablet computer, smart TV, or high-definition display, or it may be a server or cloud processing platform, or it may be an edge-cloud system composed of terminal devices and servers. This embodiment of the invention does not limit the specific type of the executing entity.

[0023] In some embodiments, the executing entity is a terminal device. Taking a smart TV or smartphone as an example, after receiving an input image to be classified, the device extracts image features, performs feature fusion, and uses a neural network for classification processing, outputting a classification result that characterizes whether the input image is a native image or a fake image, so that subsequent processing (such as targeted image optimization processing) can be performed based on the classification result. This is suitable for scenarios with high real-time requirements.

[0024] In some alternative embodiments, the terminal device is an edge device. For example, it can be a low-power edge display device, such as a smart TV or a high-definition display. More specifically, the low-power edge display device can be an edge display device that does not include a graphics processing unit (GPU) or a neural network processing unit (NPU).

[0025] In other embodiments, the execution entity is a server. The terminal device uploads the received input image to be classified to the server, which then performs image classification processing and sends the classification result back to the terminal device. This approach is suitable for scenarios where terminal computing power is limited but high classification accuracy is required.

[0026] In some other embodiments, the execution entity adopts an edge-cloud collaborative approach. The terminal device is responsible for acquiring the input image and extracting image features, while the server is responsible for feature fusion and neural network classification processing. The classification results are returned to the terminal device for subsequent judgment or storage, thus balancing real-time performance and classification effectiveness.

[0027] In this embodiment, the execution subject and application scenario described above are only examples for illustration, and the execution subject and application scenario in this embodiment are not limited to these.

[0028] refer to Figure 1 The diagram illustrates a flowchart of an embodiment of the image classification method of the present invention. As an example, the image classification method includes the following basic steps: Step S1: Obtain the input image; Step S2: Sequentially take each pixel in the input image as the current pixel. For each of the multiple preset directions, determine the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel value of the current pixel and its neighboring pixels in the preset direction. Step S3: Filter the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image to obtain the spatial domain features and frequency domain features of the input image; Step S4: Perform feature fusion processing on the spatial domain features and frequency domain features to obtain fused features; Step S5: Input the fused features into a neural network model for classification processing to obtain a classification result. The classification result is used to characterize whether the input image is a native image or a fake image.

[0029] To make the above-mentioned objects, features and advantages of the embodiments of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] refer to Figure 1 Step S1: Obtain the input image.

[0031] The input image is used for subsequent feature extraction and image classification.

[0032] An input image is acquired so that each pixel in the input image is used as the current pixel in turn. For each of a plurality of preset directions, the spatial domain feature value and frequency domain feature value of the current pixel are determined based on the pixel value of the current pixel and its neighboring pixels in the preset direction.

[0033] In this embodiment, the input image is either a native image or a fake image. A native image refers to an image that has not undergone lossy processing such as downsampling and upsampling and has the original image quality level corresponding to the corresponding resolution. A fake image refers to an image that has been downsampled and then upsampled (such as through super-resolution technology) to restore it to the target resolution, or has undergone image enhancement, contrast adjustment, color adjustment, or other processing, resulting in image quality degradation or distortion compared to the native image.

[0034] It should be noted that this embodiment uses a fake image obtained through downsampling and then upsampling as an example for detailed explanation. In other embodiments, the fake image may also be an image that has been processed in other ways, resulting in a degradation or distortion in image quality compared to the original image. This embodiment of the invention does not limit the specific causes of fake images.

[0035] As an example of an application scenario, the input image is a 4K image. More specifically, the 4K image is either a native 4K image or a pseudo-4K image.

[0036] The so-called fake 4K image usually refers to a 4K resolution image obtained by downsampling and then upsampling through super-resolution technology. It may also have undergone image enhancement, contrast or color adjustment operations. The fake 4K image usually has image quality degradation problems such as loss of high-frequency details, texture and structure degradation, and is fundamentally different from the native 4K image.

[0037] It should be noted that the input image is not limited to a 4K image. In other embodiments, the input image can also be an image of other resolutions. The image classification method can also be used to distinguish between native images at other resolutions and fake images whose image quality has been degraded after processing. The specific resolution of the input image is not limited in the embodiments of the present invention.

[0038] Continue to refer to Figure 1 Step S2: Sequentially take each pixel in the input image as the current pixel. For each of the multiple preset directions, determine the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel value of the current pixel and its neighboring pixels in the preset direction.

[0039] The spatial domain feature value and frequency domain feature value of the current pixel are determined so that the spatial domain feature value and frequency domain feature value of each pixel in the input image can be filtered to obtain the spatial domain feature and frequency domain feature of the input image.

[0040] In this embodiment, for each of the multiple preset directions, based on the pixel values ​​of the current pixel and its neighboring pixels in that preset direction, the spatial domain feature value and the frequency domain feature value of the current pixel are determined. The spatial domain feature value is a feature value obtained by processing the pixel values ​​of the input image and its neighboring pixels using a spatial domain filter, and can reflect the structural feature information of the image. The frequency domain feature value is a feature value obtained by processing the pixel values ​​of the input image and its neighboring pixels using a frequency domain filter, and can reflect the spectral distribution feature information of the image.

[0041] Therefore, the image features extracted in this embodiment can simultaneously reflect the structural feature information and spectral distribution feature information of the image, thereby improving the richness and comprehensiveness of the image features. Furthermore, since the spatial domain feature values ​​and frequency domain feature values ​​are obtained based on the pixel values ​​of each pixel and its neighboring pixels in each preset direction, they can reflect the correlation and differences between pixels in the input image, which is beneficial to improving the sensitivity of the image features to local details and texture changes in the image, thus enabling accurate feature representation of scenes with changing scenes.

[0042] Correspondingly, the spatial domain features and frequency domain features obtained after screening are then subjected to feature fusion processing. The resulting fused features can organically combine image structural features with spectral distribution features, thereby improving the representation and discrimination capabilities of the fused features. This provides a more reliable feature basis for subsequent classification processing and helps to improve the accuracy of subsequent classification processing.

[0043] In this embodiment, the input image is a native 4K image or a fake 4K image. The spatial domain feature value and frequency domain feature value of each pixel can reflect the feature information of the essential difference between the native image and the fake image.

[0044] Specifically, fake images often suffer from high-frequency detail loss, texture and structure degradation during processing; these degradation phenomena are often reflected in the changes in pixel values ​​of pixels within local areas of the image. Therefore, by determining the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel values ​​of the current pixel and its neighboring pixels in the preset direction, the aforementioned local degradation phenomena can be captured more sensitively, thus accurately representing the differences between fake and real images.

[0045] As an example of an application scenario, the input image is a 4K image. More specifically, the 4K image is either a native 4K image or a pseudo-4K image.

[0046] Therefore, in this embodiment, each pixel in the input 4K image is taken as the current pixel. For each of the multiple preset directions, the spatial domain feature value and frequency domain feature value of the current pixel are determined based on the pixel value of the current pixel and its neighboring pixels in the preset direction. This can fully capture the unique image quality degradation features of fake 4K images compared to native 4K images, such as loss of high-frequency details, texture and structure degradation, thereby making the improvement effect on the accuracy of subsequent classification processing more significant.

[0047] Reference Figure 2 This diagram illustrates an embodiment of a preset window W and a current pixel P in an input image. In this embodiment, determining the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel values ​​of the current pixel and its neighboring pixels in the preset direction includes: within the preset window centered on the current pixel, for each preset direction, determining the spatial domain feature value of the current pixel in the preset direction according to a pre-configured spatial domain filter and the pixel values ​​of the current pixel and its neighboring pixels in the preset direction; and determining the frequency domain feature value of the current pixel in the preset direction according to a pre-configured frequency domain filter and the pixel values ​​of the current pixel and its neighboring pixels in the preset direction.

[0048] Reference Figure 2 As an example, the plurality of preset directions include: the horizontal direction (e.g., Figure 2 (as shown in the hor direction), vertical direction (as shown in the hor direction) Figure 2 (as shown in the middle ver direction), the first diagonal direction (as shown in the middle ver direction) Figure 2 (as shown in the middle diag1 direction) and the second diagonal direction (as shown in the second diagonal direction) Figure 2 At least two of the directions shown in diag2.

[0049] In the field of image processing technology, structural and texture changes in an image typically exhibit different characteristics in different directions. By introducing directionality into the process of determining the spatial domain feature value and frequency domain feature value of the current pixel, the extracted feature values ​​can more comprehensively cover the changes in the image in various directions, reflecting the pixel value change relationship between the current pixel and its neighboring pixels in different directions. This is beneficial for more fully capturing the directional differences between the original image and the fake image in terms of detail texture, structural integrity, etc., thereby improving the feature discrimination ability.

[0050] As an example, the multiple preset directions include: horizontal direction, vertical direction, first diagonal direction and second diagonal direction, so as to cover pixel change information in four directions: horizontal, vertical and two diagonals, making the extracted image features more comprehensive and complete.

[0051] In other embodiments, the plurality of preset directions may also be a combination of any two of the four directions mentioned above. For example, it may include a horizontal direction, a vertical direction, and a first diagonal direction; or it may include a horizontal direction, a first diagonal direction, and a second diagonal direction, etc., and will not be listed in detail here.

[0052] It should be noted that this embodiment uses at least two of the following directions as examples: horizontal direction, vertical direction, first diagonal direction, and second diagonal direction. In other embodiments, the plurality of preset directions may include more subdivided directions (such as dividing more directions with smaller angular intervals). This embodiment of the invention does not limit the specific number and angle of the preset directions.

[0053] A pre-configured spatial domain filter is used to obtain the spatial domain feature values ​​of the current pixel in each preset direction; a pre-configured frequency domain filter is used to obtain the frequency domain feature values ​​of the current pixel in each preset direction.

[0054] Specifically, spatial domain filters are used to extract feature values ​​that reflect image structure information, while frequency domain filters are used to extract feature values ​​that reflect image spectral distribution information.

[0055] In this embodiment, the spatial domain filter includes: a first spatial domain filter and a second spatial domain filter, wherein the weighting coefficients of the first spatial domain filter and the second spatial domain filter are different; and / or, the frequency domain filter includes a first frequency domain filter, a second frequency domain filter and a third frequency domain filter, wherein the frequency bands of the first frequency domain filter, the second frequency domain filter and the third frequency domain filter are different.

[0056] In the field of image processing, the generation of fake images typically involves interpolation based on pixel values ​​in the spatial domain, and the number of inserted pixels may be one or more. In this embodiment, two spatial domain filters are used, with different weight coefficients for the first and second filters. This allows the first and second filters to correspond to different numbers of inserted pixel values ​​in the fake image, thereby improving the representational ability of the acquired spatial domain feature values ​​and enhancing the ability of subsequent spatial domain features to represent the spatial structural feature differences between the fake and original images.

[0057] In this embodiment, the spatial domain filter includes a first spatial domain filter and a second spatial domain filter as an example. In other embodiments, the number of spatial domain filters can be other values, such as three, four, etc., and the weighting coefficients of the multiple spatial domain filters are all different.

[0058] In this embodiment, the frequency domain filter includes a first frequency domain filter, a second frequency domain filter, and a third frequency domain filter. The first frequency domain filter, the second frequency domain filter, and the third frequency domain filter have different frequency bands, thereby enabling the capture of feature information and spectral distribution characteristics in the preset direction from multiple frequency bands, thereby improving the ability of subsequent frequency domain features to characterize the differences in spectral distribution characteristics between the original image and the fake image.

[0059] In this embodiment, the frequency domain filter is described as including a first frequency domain filter, a second frequency domain filter, and a third frequency domain filter. In other embodiments, the number of frequency domain filters can be other values, such as two, four, etc., and the frequency bands of the multiple frequency domain filters are all different.

[0060] Specifically, in this embodiment, within a preset window centered on the current pixel, for each preset direction, the spatial domain feature value of the current pixel in the preset direction is determined based on a pre-configured spatial domain filter and the pixel values ​​of the current pixel and its neighboring pixels in the preset direction. Furthermore, determining the frequency domain feature value of the current pixel in the preset direction based on a pre-configured frequency domain filter and the pixel values ​​of the current pixel and its neighboring pixels in the preset direction includes: within the preset window centered on the current pixel, for each preset direction, the pre-configured spatial domain filter and the pre-configured frequency domain filter are convolved with the pixel values ​​within the current preset window to obtain the spatial domain feature value and frequency domain feature value of the current pixel in the preset direction.

[0061] In this process, by performing convolution operations within a preset window centered on the current pixel, the resulting spatial domain feature values ​​and frequency domain feature values ​​not only reflect the value of the current pixel itself, but also incorporate the pixel information of the surrounding areas in each preset direction, thereby more accurately reflecting the spatial structure and spectral characteristics of the local region of the image.

[0062] In this embodiment, the spatial domain filter includes a first spatial domain filter and a second spatial domain filter. Correspondingly, the number of spatial domain feature values ​​of the current pixel in each preset direction is also two, corresponding to the first spatial domain filter and the second spatial domain filter, respectively.

[0063] In this embodiment, the frequency domain filter includes a first frequency domain filter, a second frequency domain filter, and a third frequency domain filter. Correspondingly, the number of frequency domain feature values ​​of the current pixel in each preset direction is also three, corresponding to the first frequency domain filter, the second frequency domain filter, and the third frequency domain filter, respectively.

[0064] Continue to refer to Figure 1 In this embodiment, before filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image, the image classification method further includes: performing step S6: based on each pixel in the input image and the pixel values ​​around the pixel in the neighborhood of the preset window, performing local importance estimation processing on each pixel to obtain a local importance reference threshold for each pixel.

[0065] Based on the pixel values ​​of each pixel in the input image and the pixel values ​​of the surrounding pixels within a preset window neighborhood, local importance estimation processing is performed on each pixel to obtain a reference threshold for evaluating the importance of each pixel in a local region based on the pixel values ​​of each pixel and its surrounding pixels. This allows for subsequent filtering of the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image based on the local importance reference threshold.

[0066] The local importance reference threshold is used as a reference threshold to evaluate the importance of each pixel in a local region. Subsequently, the spatial domain feature value and frequency domain feature value of each pixel in each preset direction are compared with the local importance reference threshold to filter the spatial domain feature value and the frequency domain feature value.

[0067] Figure 3A flowchart illustrating an embodiment of step S6 is shown. As an example, step S6, which involves estimating the local importance of each pixel based on the pixel values ​​in the input image and the pixel values ​​surrounding the pixel within a preset window neighborhood, includes: step S61, performing numerical statistical processing on the pixel information within the preset window neighborhood along the preset direction to obtain a local importance reference threshold for the current pixel in each preset direction; and step S62, traversing the input image by sliding the preset window neighborhood to obtain a local importance reference threshold for each pixel in the input image in each preset direction.

[0068] Within a preset window neighborhood along a preset direction, pixel information is numerically statistically processed to obtain a value for each pixel in each preset direction that can be used to evaluate the importance of the pixel in the local region, i.e., a local importance reference threshold, which provides a basis for subsequent screening of spatial domain feature values ​​and frequency domain feature values.

[0069] As an example, the neighborhood of the preset window along the preset direction is a one-dimensional window along the preset direction. A local importance reference threshold for each pixel in each preset direction is obtained to match the spatial domain feature values ​​and frequency domain feature values ​​in each subsequent preset direction, and serves as a reference standard for filtering the spatial domain feature values ​​and frequency domain feature values ​​in each preset direction.

[0070] In this embodiment, the local importance reference threshold of the current pixel in each preset direction can be determined based on the difference between the maximum and minimum values ​​of pixels in the preset window neighborhood of the current pixel in that preset direction.

[0071] The difference between the maximum and minimum values ​​of pixels within the preset window reflects the fluctuation range of pixel values ​​in that local area: a larger difference indicates more drastic fluctuations in pixel values ​​within that local area, and richer structural or texture information; a smaller difference indicates smoother pixel values ​​within that local area, more information redundancy, and relatively lower importance of the corresponding pixel's feature values. Therefore, by statistically analyzing the maximum and minimum values ​​of pixels within the preset window, a value used to evaluate the importance of a pixel in a local area can be determined, i.e., a local importance reference threshold.

[0072] As an example, Figure 4 A flowchart illustrating one embodiment of step S61 is shown. Figure 4As shown, in a specific embodiment, the step of performing numerical statistical processing on pixel information within a preset window neighborhood along the preset direction to obtain the local importance reference threshold S61 of the current pixel in each preset direction includes: step S611: calculating the difference between the maximum and minimum values ​​of pixels within the preset window neighborhood along each preset direction; step S612: comparing the product of the difference and a preset gain coefficient with a preset threshold range to obtain the local importance reference threshold of the current pixel in each preset direction.

[0073] The difference between the maximum and minimum values ​​of pixels within the preset window reflects the fluctuation range of pixel values ​​in that local area: a larger difference indicates more drastic fluctuations in pixel values ​​within that local area, and richer structural or texture information; a smaller difference indicates smoother pixel values ​​within that local area, more information redundancy, and relatively lower importance of the corresponding pixel's feature values. Therefore, by statistically analyzing the maximum and minimum values ​​of pixels within the preset window, a basis can be provided for subsequently obtaining the local importance reference thresholds of the current pixel in each preset direction.

[0074] In this embodiment, the product of the difference and the preset gain coefficient is compared with the preset threshold range, so that the local importance reference threshold can be adjusted by flexibly setting the gain coefficient based on actual needs.

[0075] Specifically, in this embodiment, comparing the product of the difference and the preset gain coefficient with a preset threshold range to obtain the local importance reference threshold of the current pixel in each preset direction includes: when the product is within the preset threshold range, the product is used as the local importance reference threshold; otherwise, the lower limit of the preset threshold range is used as the local importance reference threshold.

[0076] It should be noted that the above example uses the preset window neighborhood along the preset direction as a one-dimensional window along the preset direction, and each pixel has a corresponding local importance reference threshold in each preset direction as an example for explanation.

[0077] In other embodiments, the preset window neighborhood can also be a two-dimensional window centered on the current pixel. Each pixel has a local importance reference threshold, and the subsequent filtering of the feature values ​​of each pixel in each preset direction is based on this local importance reference threshold.

[0078] Accordingly, in this embodiment, the local importance estimation processing of each pixel based on the pixel values ​​of each pixel in the input image and the pixel values ​​around the pixel in the preset window neighborhood includes: performing numerical statistical processing on the pixel information in the preset window neighborhood centered on the current pixel to obtain the local importance reference threshold of the current pixel; and traversing the input image by sliding the preset window neighborhood to obtain the local importance reference threshold of each pixel in the input image.

[0079] Accordingly, in this embodiment, the pixel information within the preset window neighborhood centered on the current pixel is subjected to numerical statistical processing to obtain the local importance reference threshold of the current pixel, including: calculating the difference between the maximum and minimum values ​​of the pixels within the preset window neighborhood centered on the current pixel; and comparing the product of the difference and a preset gain coefficient with a preset threshold range to obtain the local importance reference threshold of the current pixel.

[0080] Accordingly, in this embodiment, each pixel corresponds to a local importance reference threshold. In the subsequent step of filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image based on the local importance reference threshold, the feature values ​​of each pixel in each preset direction are filtered, all based on the same local importance reference threshold.

[0081] For detailed steps on obtaining the local importance reference threshold in this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0082] It should also be noted that, in this embodiment, the order of steps S2 and S6 is not limited. Step S6 can be performed before, after, or simultaneously with step S2.

[0083] Continue to refer to Figure 1 In this embodiment, step S3 is performed: the spatial domain feature value and frequency domain feature value of each pixel in the input image are filtered to obtain the spatial domain feature and frequency domain feature of the input image.

[0084] The spatial domain features and frequency domain features are used to characterize the spatial structure features and spectral distribution features of the input image, respectively; the spatial domain features and frequency domain features are also used for subsequent feature fusion processing.

[0085] In the field of image processing technology, there is often a strong correlation between adjacent pixels or between different frequencies in an image. Therefore, the spatial and frequency domain feature values ​​of each pixel often contain redundant feature information. This redundant feature information not only increases the computational load of subsequent feature fusion and classification processes but may also affect the classification results of neural network models. Therefore, this embodiment simplifies image features by filtering the spatial and frequency domain feature values, thereby reducing the computational load of subsequent feature fusion and classification processes and improving their running speed.

[0086] In this embodiment, filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image includes: filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image based on the local importance reference threshold to obtain the spatial domain features and frequency domain features.

[0087] Based on the local importance reference threshold, the spatial domain feature values ​​and the frequency domain feature values ​​of each pixel are filtered to remove redundant features with low importance and retain features with high importance, thereby obtaining the filtered spatial domain features and frequency domain features.

[0088] Specifically, the spatial domain feature value and the frequency domain feature value are compared with a local importance reference threshold; when the spatial domain feature value and the frequency domain feature value are greater than or equal to the local importance reference threshold, the feature value is retained; when the spatial domain feature value and the frequency domain feature value are less than the local importance reference threshold, the feature value is discarded, thereby obtaining the filtered spatial domain feature and frequency domain feature.

[0089] By comparing the spatial domain feature values ​​and the frequency domain feature values ​​with a local importance reference threshold, the spatial domain feature values ​​and frequency domain feature values ​​can be screened with a unified and objective judgment standard. The judgment logic is simple, the amount of computation is small, and it is easy to implement in scenarios with limited computing resources.

[0090] Specifically, in this embodiment, the neighborhood of the preset window along the preset direction is a one-dimensional window along the preset direction; each pixel has a local importance reference threshold in each preset direction; based on the local importance reference threshold, filtering the spatial domain feature value and frequency domain feature value of each pixel in the input image includes: in each preset direction, comparing the spatial domain feature value and frequency domain feature value of each pixel with the local importance reference threshold to filter the spatial domain feature value and the frequency domain feature value.

[0091] In this embodiment, the pixel values ​​surrounding the pixel can characterize the distribution of pixel values ​​within the local area where the current pixel is located; the preset direction can characterize the directional attributes corresponding to the spatial domain feature and the frequency domain feature, respectively. In this embodiment, based on the local importance reference threshold, the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image are filtered in each preset direction. This can identify and eliminate redundant feature information caused by the correlation between adjacent pixels or between different frequencies in the image. The filtering process can reflect the differences in the importance of features in different directions and can also be adaptively adjusted according to the actual changes in the local content of the image. Compared with feature filtering methods that do not distinguish directions or do not consider local pixel changes, this embodiment is beneficial for accurately retaining features that are truly valuable for subsequent classification processing, thereby improving the accuracy of subsequent classification processing.

[0092] In other embodiments, the preset window neighborhood can also be a two-dimensional window centered on the current pixel, with each pixel corresponding to a local importance reference threshold. Accordingly, in this embodiment, filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image based on the local importance reference threshold includes: comparing the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in each preset direction with the local importance reference threshold to filter the spatial domain feature values ​​and frequency domain feature values. That is, the filtering of the feature values ​​of each pixel in each preset direction is based on the same local importance reference threshold.

[0093] Continue to refer to Figure 1 Step S4: Perform feature fusion processing on the spatial domain features and frequency domain features to obtain fused features.

[0094] The spatial domain features and the frequency domain features are subjected to feature fusion processing to organically combine the two types of feature information to obtain fused features, which are then input into a neural network model for classification processing.

[0095] In this embodiment, feature fusion combines feature information from different sources, enhancing the model's understanding of feature data and improving its accuracy and robustness. Since the image features extracted in this embodiment include spatial and frequency domain features, the fused features obtained by performing feature fusion processing on these features organically combine image structural features with spectral distribution features. Compared to subsequent classification methods that rely solely on single-domain features, this embodiment improves the representational and discriminative capabilities of the fused features, providing more reliable feature input for subsequent classification and thus enhancing the accuracy of the classification process.

[0096] In this embodiment, the spatial domain features and the frequency domain features are obtained by filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image. By performing feature fusion processing on the filtered spatial domain features and frequency domain features, on the one hand, since the filtering step has eliminated redundant feature information caused by the correlation between adjacent pixels or different frequencies in the image, the amount of feature data involved in the feature fusion processing is reduced, thereby reducing the computational complexity of the feature fusion processing and correspondingly improving the running speed of the feature fusion processing; on the other hand, since the filtering step retains image features with higher importance, that is, more distinguishing value for classification processing, the fused features obtained after feature fusion processing maintain high representational and distinguishing abilities while reducing the dilution or interference of redundant feature information on the quality of the fused features, which is beneficial to improving the efficiency and accuracy of subsequent classification processing by the neural network model.

[0097] As one embodiment, the spatial domain features and the frequency domain features are subjected to feature fusion processing to obtain fused features, including: splicing the spatial domain features and the frequency domain features in the channel dimension to obtain the fused features.

[0098] By concatenating the spatial domain features and the frequency domain features along the channel dimension, the two types of features can be integrated into a unified feature expression without changing the independent numerical meaning of the spatial domain features and the frequency domain features. This allows the fused features to simultaneously contain the structural information and spectral distribution information of the image. The method is simple, computationally inexpensive, and easy to implement in scenarios with limited computing resources.

[0099] In this embodiment, both the spatial domain features and the frequency domain features belong to lower-level features in image processing; therefore, channel-dimensional stitching, an early fusion method, is adopted. In other embodiments, other fusion methods can also be used to perform feature fusion processing on the spatial domain features and the frequency domain features. This embodiment of the invention does not limit the specific method of feature fusion processing.

[0100] It should be noted that continued reference is necessary. Figure 1 In this embodiment, after performing feature fusion processing on the spatial domain features and the frequency domain features to obtain fused features, and before inputting the fused features into the neural network model for classification processing, the image classification method further includes: performing step S7: normalizing the fused features to obtain normalized fused features.

[0101] In this embodiment, the fused features are normalized to obtain normalized fused features, which can map different feature data to the same data range. In the subsequent process of inputting the fused features into the neural network model for classification, the probability of the neural network model's training feature weight imbalance caused by inconsistent input data magnitude (i.e., data range) can be reduced.

[0102] Specifically, Figure 5 A flowchart illustrating an embodiment of step S7 is shown. In this embodiment, step S7, which normalizes the fused features, includes: performing statistical processing on the numerical distribution of features corresponding to each channel of the fused features to obtain a statistical feature distribution of the fused features; and executing step S8: based on the statistical feature distribution, normalizing the fused features to obtain normalized fused features.

[0103] The numerical distribution of the corresponding features for each channel of the fusion feature is statistically processed to obtain a statistical feature distribution that reflects the overall characteristics of each channel of the fusion feature. This facilitates the normalization of the fusion feature based on the statistical feature distribution, and the normalized fusion feature is used as the input of the subsequent neural network model.

[0104] In this embodiment, the statistical feature distribution includes a maximum value and a minimum value, thereby obtaining the numerical range of the fused feature; correspondingly, in the process of normalizing the fused feature based on the statistical feature distribution, different feature data can be mapped to the same data range.

[0105] In this embodiment, the statistical distribution of the numerical distribution of each channel corresponding to the fused feature is statistically processed to obtain the statistical feature distribution corresponding to the fused feature; and the fused feature is normalized based on the statistical feature distribution as an example. In other embodiments, when the data range of the fused feature is known, the statistical processing step can be omitted.

[0106] refer to Figure 1 Step S5: Input the fused features into the neural network model for classification processing to obtain a classification result. The classification result is used to characterize whether the input image is a native image or a fake image.

[0107] The fused features are input into a neural network model for classification processing to obtain a classification result that can characterize whether the input image is a native image or a fake image, thereby achieving accurate identification of native and fake images.

[0108] The image classification method provided in this invention sequentially takes each pixel in the input image as the current pixel. For each of a plurality of preset directions, based on the pixel values ​​of the current pixel and its neighboring pixels in that preset direction, the spatial domain feature value and frequency domain feature value of the current pixel are determined. This allows the image features to simultaneously reflect the structural and spectral distribution information of the image. Compared with related technical solutions based on single-domain features (such as extracting only spatial domain features), or based on the overall statistical regularity of the image for manual feature extraction, or by predicting a small number of local image patches to obtain the overall image quality, this invention improves the richness and comprehensiveness of image features. Furthermore, since the spatial and frequency domain feature values ​​are obtained based on the pixel values ​​of each pixel and its neighboring pixels in each preset direction, they can reflect the correlation and differences between pixels in the input image. This is beneficial for improving the sensitivity of image features to local details and texture changes in the image, thereby enabling accurate feature representation of scenes with changing scenes.

[0109] Furthermore, this embodiment of the invention also filters the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image, thereby reducing redundant image feature information, which in turn reduces the computational load of subsequent feature fusion processing and classification processing and improves the running speed. Correspondingly, feature fusion processing is performed on the spatial domain features and frequency domain features obtained after filtering. The resulting fused features can organically combine image structural feature information and spectral distribution feature information, improving the representation and discrimination capabilities of the fused features and providing a more reliable feature basis for subsequent classification processing. In the process of inputting the fused features into the neural network model for classification processing, since the neural network model itself has learning ability and nonlinear modeling ability, it can automatically learn the complex relationships contained in the fused features, without relying on manually designed classification rules, and without placing excessive requirements on the completeness of the input feature data. This can improve the accuracy of image classification, enhance the generalization ability of the model, and reduce the risk of model performance degradation due to missing or outlier input data.

[0110] In summary, the image classification method provided by the embodiments of the present invention can accurately identify original images and fake images by acquiring rich and comprehensive image features, filtering feature values, performing effective feature fusion, and using a neural network model for automatic learning and classification. It ensures classification accuracy while taking into account the generalization ability of the model, and also reduces the amount of computation and improves the running speed, thereby improving the overall effect of image classification.

[0111] In this embodiment, inputting the normalized fused features into the neural network model for classification processing helps reduce the probability of imbalance in training feature weights caused by inconsistent magnitude (i.e., data range) of input data in the neural network model.

[0112] Reference Figure 6 The diagram illustrates a structural schematic of an embodiment of the neural network model of the present invention. In this embodiment, the neural network model includes convolutional layers and fully connected layers.

[0113] The convolutional layer extracts and abstracts local correlation information from the fused features, while the fully connected layer synthesizes the features output by the convolutional layer and completes the final classification judgment. Compared to deep learning network models with more complex structures and a larger number of parameters, the neural network model composed of convolutional and fully connected layers has a simple structure and fewer parameters, effectively reducing the computational complexity and dependence on hardware computing power while ensuring classification performance.

[0114] In this embodiment, the training method of the neural network model includes: constructing a training dataset, which includes native images and fake images, and classification labels, wherein the classification labels are used to indicate whether the input image is a native image or a fake image; extracting image features from the native images and fake images in the training dataset, wherein the image features include spatial domain features and frequency domain features in multiple preset directions, including: taking each pixel in the input image as the current pixel in turn, and for each preset direction, determining the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel value of the current pixel and its neighboring pixels in that preset direction; filtering the spatial domain feature value and frequency domain feature value of each pixel in the input image to obtain the spatial domain feature and the frequency domain feature of the input image; performing feature fusion processing on the spatial domain feature and the frequency domain feature to obtain fused features; and training the neural network model using the fused features as training input and the classification labels as training output.

[0115] In this embodiment, the neural network model takes the fused features as input, trains and optimizes the neural network model by traversing the training dataset, and obtains a trained neural network model, which is used to classify the fused features and output the classification result.

[0116] In this embodiment, by repeatedly training and optimizing the neural network model through the training dataset, the neural network model can gradually learn the correspondence between the fusion features and the two categories of classification results: original images and fake images. Thus, after training is completed, it can accurately output the corresponding classification results based on the input fusion features.

[0117] As an example of an application scenario, the input image is a 4K image, and the classification result is used to characterize whether the input image is a native 4K image or a fake 4K image.

[0118] In this embodiment, accurately identifying native 4K images and fake 4K images is beneficial for taking targeted image optimization measures in the future, thereby significantly improving the user's viewing experience.

[0119] It should be noted that the classification result in this embodiment is not limited to characterizing the input image as a native 4K image or a pseudo 4K image. In other embodiments, the classification result can also be used to characterize input images at other resolutions as native images or pseudo images of the corresponding resolution. This embodiment of the invention does not limit the specific content characterized by the classification result.

[0120] It should also be noted that the image classification method provided in this embodiment of the invention has the advantages of high recognition accuracy, strong robustness, and high real-time performance. It can accurately identify original images and fake images, and has low computational load and fast inference speed. It can run stably in edge devices that do not contain graphics processing units (GPUs) or neural network processing units (NPUs).

[0121] Specifically, the obtained image feature information is relatively rich and comprehensive, which can fully reflect the differences between the original image and the fake image in terms of structure and spectral distribution, which is conducive to improving the recognition accuracy of classification processing. At the same time, since the obtained image features cover multiple dimensions of information such as multiple preset directions, spatial domain and frequency domain, compared with the method of classification based on a single feature or a single dimension of information, the image features obtained by the embodiments of the present invention have stronger information redundancy and complementarity. Even if some features are disturbed or have local errors, the remaining features can still provide effective support for classification processing, which is conducive to improving the robustness of neural network models in classification processing.

[0122] Furthermore, the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image are filtered to remove redundant feature information, simplifying the image features and reducing the amount of feature data involved in subsequent processing. In addition, the neural network model used for classification processing only includes convolutional layers and fully connected layers, with a relatively simple network structure and a small number of parameters. The combined effect of the above factors makes the overall computational load of the embodiment of the present invention small and the computational speed fast, thereby meeting the real-time requirements.

[0123] Therefore, because the embodiments of the present invention have the characteristics of low computational load and fast inference speed, the image classification method provided by the embodiments of the present invention can run stably and meet real-time requirements even in edge devices that do not contain GPUs or NPUs and whose computing power and power consumption are relatively limited, which is beneficial to reducing the hardware cost of related products.

[0124] Accordingly, the present invention also provides an image classification device. Figure 7This is a functional block diagram of an embodiment of the image classification device of the present invention. The image classification device of this embodiment will be described in detail below with reference to the accompanying drawings.

[0125] refer to Figure 7 In this embodiment, the image classification device 10 includes: an image acquisition module 101, used to acquire an input image; a feature extraction module 102, used to sequentially take each pixel in the input image as the current pixel, and for each of a plurality of preset directions, determine the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel value of the current pixel and its neighboring pixels in the preset direction; a feature filtering module 103, used to filter the spatial domain feature value and frequency domain feature value of each pixel in the input image to obtain the spatial domain feature and the frequency domain feature of the input image; a feature fusion module 104, used to perform feature fusion processing on the spatial domain feature and the frequency domain feature to obtain fused features; and a classification module 105, used to input the fused features into a neural network model for classification processing to obtain a classification result, wherein the classification result is used to characterize whether the input image is a native image or a fake image.

[0126] In the image classification device 10, the feature extraction module 102 sequentially takes each pixel in the input image as the current pixel. For each of the multiple preset directions, based on the pixel values ​​of the current pixel and its neighboring pixels in that preset direction, it determines the spatial domain feature value and frequency domain feature value of the current pixel. This allows the image features to simultaneously reflect the structural feature information and spectral distribution feature information of the image, improving the richness and comprehensiveness of the image features. Furthermore, since the spatial domain feature value and frequency domain feature value are obtained based on the pixel values ​​of each pixel and its neighboring pixels in each preset direction, they can reflect the correlation and differences between pixels in the input image. This is beneficial for improving the sensitivity of image features to local details and texture changes in the image, thereby enabling accurate feature representation of scenes with changing scenes.

[0127] Furthermore, the feature filtering module 103 filters the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image, thereby reducing redundant image feature information and reducing the computational load and improving the running speed of subsequent feature fusion and classification processing. Correspondingly, the feature fusion module 104 performs feature fusion processing on the spatial domain features and frequency domain features obtained after filtering. The resulting fused features can organically combine image structural feature information with spectral distribution feature information, improving the representation and discrimination capabilities of the fused features and providing a more reliable feature basis for the classification module 105 to perform classification processing. In the process of inputting the fused features into the neural network model for classification processing, since the neural network model itself has learning ability and nonlinear modeling ability, it can automatically learn the complex relationships contained in the fused features. It does not need to rely on manually designed classification rules, nor does it place excessive demands on the completeness of the input feature data. This can improve the accuracy of image classification and enhance the generalization ability of the model, reducing the risk of model performance degradation due to missing or outlier input data.

[0128] In summary, the image classification device 10 can accurately distinguish between native and fake images, ensuring classification accuracy while taking into account the generalization ability of the model, reducing computational load and increasing running speed, thereby improving the overall effect of image classification.

[0129] In this embodiment, the image classification device 10 further includes an importance estimation module 106, which is used to perform local importance estimation processing on each pixel based on each pixel in the input image and the pixel values ​​around the pixel in the neighborhood of a preset window, to obtain a local importance reference threshold for each pixel.

[0130] The importance estimation module 106 performs local importance estimation processing on each pixel in the input image and the pixel values ​​around the pixel in the neighborhood of the preset window, so as to obtain a reference threshold for evaluating the importance of each pixel in the local region based on the pixel values ​​of each pixel and the surrounding pixels, so that the feature filtering module 103 can filter the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image based on the local importance reference threshold.

[0131] The image classification device 10 in this embodiment of the invention can be used to execute the image classification method of the foregoing embodiments, or other functional modules can be used to execute the image classification method of the foregoing embodiments. For a detailed description of the image classification method in the foregoing embodiments, please refer to the description here, which will not be repeated here.

[0132] To address the aforementioned problem, embodiments of the present invention also provide an electronic device that can implement the image classification method provided by the embodiments of the present invention by loading a program in the form of the above-described image classification method.

[0133] An optional hardware structure for the electronic device provided in this embodiment of the invention can be as follows: Figure 8 As shown, it includes: at least one processor 01, at least one communication interface 02, at least one memory 03, and at least one communication bus 04.

[0134] In this embodiment of the invention, the number of processor 01, communication interface 02, memory 03 and communication bus 04 is at least one, and processor 01, communication interface 02 and memory 03 communicate with each other through communication bus 04.

[0135] Optionally, the communication interface 02 can be an interface for a communication module used for network communication, such as the interface of a GSM module.

[0136] Optionally, processor 01 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0137] Optionally, memory 03 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0138] The memory 03 stores one or more computer instructions, which are executed by the processor 01 to implement the image classification method provided in this embodiment of the invention.

[0139] In this embodiment, the electronic device is a device capable of image processing, such as a smartphone, tablet computer, smart TV, set-top box, network video recorder, medical imaging workstation, or server, etc. This embodiment of the invention does not limit the specific type of the electronic device.

[0140] It should be noted that the aforementioned terminal device may also include other devices (not shown) that may not be essential to understanding the content disclosed in the embodiments of the present invention; given that these other devices may not be essential for understanding the content disclosed in the embodiments of the present invention, the embodiments of the present invention will not describe them one by one.

[0141] Accordingly, embodiments of the present invention also provide a storage medium storing one or more computer instructions, which are used to implement the image classification method described in the embodiments of the present invention.

[0142] The storage medium is a computer-readable storage medium, which can be a read-only memory (ROM), random access memory (RAM), USB flash drive, portable hard drive, magnetic disk or optical disk, or any other medium that can store program code.

[0143] The embodiments of the present invention described above are combinations of elements and features of the present invention. Unless otherwise stated, the elements or features described are optional. Individual elements or features may be practiced without combination with other elements or features. Furthermore, embodiments of the present invention may be constructed by combining some elements and / or features. The order of operations described in the embodiments of the present invention may be rearranged. Some constructions of any embodiment may be included in another embodiment and may be replaced by corresponding constructions of another embodiment. It will be apparent to those skilled in the art that claims in the appended claims that are not expressly referenced to each other may be combined to form embodiments of the present invention, or may be included as new claims in amendments made after the filing of this application.

[0144] Embodiments of the present invention can be implemented by various means, such as hardware, firmware, software, or combinations thereof. In a hardware configuration, the method according to an exemplary embodiment of the present invention can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0145] In firmware or software configuration, embodiments of the present invention can be implemented in the form of modules, processes, functions, etc. Software code can be stored in a memory unit and executed by a processor. The memory unit is located inside or outside the processor and can send data to and receive data from the processor via various known means.

[0146] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An image classification method, characterized in that, include: Obtain the input image; Each pixel in the input image is taken as the current pixel in turn. For each of the multiple preset directions, the spatial domain feature value and frequency domain feature value of the current pixel are determined based on the pixel value of the current pixel and its neighboring pixels in the preset direction. The spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image are filtered to obtain the spatial domain feature and the frequency domain feature of the input image; The spatial domain features and frequency domain features are fused to obtain fused features; The fused features are input into a neural network model for classification processing to obtain a classification result, which is used to characterize whether the input image is a native image or a fake image.

2. The image classification method as described in claim 1, characterized in that, Based on the current pixel and the pixel values ​​of its neighboring pixels in the preset direction, determining the spatial domain feature value and frequency domain feature value of the current pixel includes: Within a preset window centered on the current pixel, for each preset direction, the spatial domain feature value of the current pixel in the preset direction is determined based on a pre-configured spatial domain filter and the pixel values ​​of the current pixel and its neighboring pixels in the preset direction; and the frequency domain feature value of the current pixel in the preset direction is determined based on a pre-configured frequency domain filter and the pixel values ​​of the current pixel and its neighboring pixels in the preset direction. The spatial domain filter includes a first spatial domain filter and a second spatial domain filter, wherein the weighting coefficients of the first spatial domain filter and the second spatial domain filter are different. And / or, The frequency domain filter includes a first frequency domain filter, a second frequency domain filter, and a third frequency domain filter, wherein the first frequency domain filter, the second frequency domain filter, and the third frequency domain filter have different frequency bands.

3. The image classification method as described in claim 1, characterized in that, Before filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image, the image classification method further includes: performing local importance estimation processing on each pixel based on the pixel values ​​of each pixel in the input image and the pixel values ​​around the pixel in the neighborhood of a preset window, to obtain a local importance reference threshold for each pixel; Filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image includes: filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image based on the local importance reference threshold to obtain the spatial domain feature and frequency domain feature.

4. The image classification method as described in claim 3, characterized in that, The step of performing local importance estimation processing on each pixel based on each pixel in the input image and the pixel values ​​around the pixel in the neighborhood of the preset window includes: performing numerical statistical processing on the pixel information in the neighborhood of the preset window along the preset direction to obtain the local importance reference threshold of the current pixel in each preset direction; and traversing the input image by sliding the neighborhood of the preset window to obtain the local importance reference threshold of each pixel in the input image in each preset direction. Based on the local importance reference threshold, filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image includes: comparing the spatial domain feature values ​​and frequency domain feature values ​​of each pixel with the local importance reference threshold in each preset direction to filter the spatial domain feature values ​​and frequency domain feature values.

5. The image classification method as described in claim 4, characterized in that, Numerical statistical processing is performed on the pixel information within the neighborhood of a preset window along the preset direction to obtain the local importance reference threshold of the current pixel in each preset direction, including: The difference between the maximum and minimum values ​​of pixels within the neighborhood of the preset window along each preset direction is calculated. The product of the difference and the preset gain coefficient is compared with a preset threshold range to obtain the local importance reference threshold of the current pixel in each preset direction.

6. The image classification method as described in claim 3, characterized in that, The step of performing local importance estimation processing on each pixel based on each pixel in the input image and the pixel values ​​around the pixel in the preset window neighborhood includes: performing numerical statistical processing on the pixel information in the preset window neighborhood centered on the current pixel to obtain the local importance reference threshold of the current pixel; and traversing the input image by sliding the preset window neighborhood to obtain the local importance reference threshold of each pixel in the input image. Based on the local importance reference threshold, filtering the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image includes: comparing the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in each preset direction with the local importance reference threshold to filter the spatial domain feature values ​​and frequency domain feature values.

7. The image classification method as described in claim 6, characterized in that, Numerical statistical processing is performed on the pixel information within the preset window neighborhood centered on the current pixel to obtain the local importance reference threshold of the current pixel, including: The difference between the maximum and minimum values ​​of pixels within the preset window neighborhood centered on the current pixel is calculated. The product of the difference and the preset gain coefficient is compared with a preset threshold range to obtain the local importance reference threshold of the current pixel.

8. The image classification method as described in claim 1, characterized in that, The image classification method further includes: performing feature fusion processing on the spatial domain features and the frequency domain features to obtain fused features, and before inputting the fused features into a neural network model for classification processing, the fused features are normalized to obtain normalized fused features. In the step of inputting the fused features into the neural network model for classification processing, the normalized fused features are input into the neural network model for classification processing.

9. The image classification method as described in claim 1, characterized in that, The training methods for the neural network model include: Construct a training dataset, which includes native images and fake images, as well as classification labels, which are used to identify whether the input image is a native image or a fake image; Extracting image features from the original and fake images in the training dataset, the image features include spatial domain features and frequency domain features in multiple preset directions, including: taking each pixel in the input image as the current pixel in turn; for each preset direction, determining the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel value of the current pixel and its neighboring pixels in that preset direction; filtering the spatial domain feature value and frequency domain feature value of each pixel in the input image to obtain the spatial domain feature and frequency domain feature of the input image; The spatial domain features and frequency domain features are fused to obtain fused features; The neural network model is trained using the fused features as training input and the classification labels as training output.

10. The image classification method as described in claim 1, characterized in that, The input image is either a native 4K image or a pseudo 4K image; the classification result is used to characterize whether the input image is a native 4K image or a pseudo 4K image.

11. An image classification device, characterized in that, include: The image acquisition module is used to acquire the input image; The feature extraction module is used to sequentially take each pixel in the input image as the current pixel, and for each of a plurality of preset directions, determine the spatial domain feature value and frequency domain feature value of the current pixel based on the pixel value of the current pixel and its neighboring pixels in the preset direction. The feature filtering module is used to filter the spatial domain feature values ​​and frequency domain feature values ​​of each pixel in the input image to obtain the spatial domain features and frequency domain features of the input image. The feature fusion module is used to perform feature fusion processing on the spatial domain features and frequency domain features to obtain fused features; The classification module is used to input the fused features into a neural network model for classification processing to obtain a classification result, which is used to characterize whether the input image is a native image or a fake image.

12. An electronic device, characterized in that, It includes at least one memory and at least one processor, the memory storing one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the image classification method as described in any one of claims 1-10.

13. A storage medium, characterized in that, The storage medium stores one or more computer instructions, which are used to implement the image classification method as described in any one of claims 1-10.

Images