Image classification method and device, storage medium and electronic device

Abstract

This application discloses an image classification method, apparatus, storage medium, and electronic device, relating to the field of financial technology. The method includes: acquiring a target image to be classified; performing image segmentation on the target image using a first segmentation module in a target network model to obtain a first segmented image; and classifying the target image based on the first segmented image using a classification module in the target network model to obtain a classification result for the target image, wherein the classification result of the target image indicates whether the target image has been tampered with. This application solves the problem in related technologies where image segmentation is used to determine whether an image has been tampered with, resulting in relatively low accuracy in determining whether an image has been tampered with.

Description

Technical Field

[0001] This application relates to the field of financial technology, and more specifically, to an image classification method and apparatus, storage medium and electronic device. Background Technology

[0002] With the development of deep learning, image recognition based on deep learning is being applied more and more widely in the financial industry. Financial institutions need to use intelligent models to identify whether images in customer-uploaded review materials have been tampered with, thereby improving the efficiency of their staff in reviewing materials. In existing technologies, traditional image recognition methods based on manual features can only identify images tampered with using the same method. While highly targeted, they have poor generalization ability. In financial institutions' review materials, the same image may be processed by multiple tampering methods (e.g., splicing and removal), so image recognition methods based on manual features are insufficient to meet the accuracy requirements of financial institutions. Furthermore, current deep neural network algorithms for image tampering mainly involve complex modeling of the core neural network structure, resulting in models with poor generalization ability. However, in the business scenarios of financial institutions, most image tampering detection scenarios involve text content modification, without involving many other objects. Therefore, using existing models to identify whether images in review materials have been tampered with through scene segmentation results in low accuracy.

[0003] There is currently no effective solution to the problem that the accuracy of determining whether an image has been tampered with is relatively low when using image segmentation in related technologies. Summary of the Invention

[0004] The main objective of this application is to provide an image classification method, apparatus, storage medium, and electronic device to solve the problem that the accuracy of determining whether an image has been tampered with is relatively low in related technologies, which rely on image segmentation.

[0005] To achieve the above objectives, according to one aspect of this application, an image classification method is provided. The method includes: acquiring a target image to be classified; performing image segmentation on the target image using a first segmentation module in a target network model to obtain a first segmented image; and classifying the target image based on the first segmented image using a classification module in the target network model to obtain a classification result for the target image, wherein the classification result of the target image indicates whether the target image has been tampered with.

[0006] Further, the classification module in the target network model classifies the target image based on the first segmented image to obtain the classification result of the target image, including: cropping the first segmented image and the target image to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image; extracting features from the multiple first image blocks based on the multiple second image blocks using the feature extraction module in the target network model to obtain multiple target feature images; segmenting the multiple first image blocks based on the multiple second image blocks using the second segmentation module in the target network model to obtain segmented image blocks corresponding to each first image block, and obtaining a second segmented image based on the segmented image blocks, wherein the second segmented image includes the location information of abnormal regions, and the pixel value of the abnormal region is a preset pixel value; classifying the second segmented image based on the multiple target feature images using the classification module in the target network model to determine the classification result of the second segmented image, wherein the classification result of the second segmented image indicates whether the abnormal region in the second segmented image is a tampered region; and obtaining the classification result of the target image based on the classification result of the second segmented image.

[0007] Further, the classification module in the target network model classifies the second segmented image based on the multiple target feature images, and the classification result of the second segmented image is determined by: calculating the probability value corresponding to the second segmented image based on the multiple target feature images by the classification module, wherein the probability value represents the probability that the abnormal region in the second segmented image is a tampered region; if the probability value is greater than or equal to a preset value, the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is a tampered region; if the probability value is less than the preset value, the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is not a tampered region.

[0008] Furthermore, the feature extraction module in the target network model extracts features from the multiple first image blocks based on the multiple second image blocks to obtain multiple target feature images, including: determining the second image block corresponding to each first image block; for each first image block, the feature extraction module extracts features from the first image block based on the second image block corresponding to the first image block to obtain the target feature map corresponding to the first image block.

[0009] Furthermore, after obtaining the classification result of the target image based on the classification result of the second segmented image, the method further includes: if the classification result of the target image indicates that the target image has been tampered with, then obtaining the location information of the tampered region based on the second segmented image; correcting the second segmented image based on the location information using a preset algorithm to obtain a corrected second segmented image; and obtaining the location result of the tampered region in the target image based on the corrected second segmented image.

[0010] Furthermore, before acquiring the target image to be classified, the method further includes: acquiring an initial training sample set, wherein the initial training sample set includes at least multiple initial sample images, the true classification result of each sample image, and the true location information of the abnormal region in each sample image; processing the initial training sample set to obtain a target training sample set; and training the initial network model based on the target training sample set to obtain the target network model.

[0011] Further, processing the initial training sample set to obtain the target training sample set includes: preprocessing the initial sample images in the initial training sample set to obtain multiple first sample images, wherein the preprocessing includes at least one of the following operations: image rotation, image scaling, image deformation, and mirror transformation; padding the multiple first sample images to obtain multiple second sample images, wherein the multiple second sample images have the same size; standardizing the multiple second sample images to obtain multiple target sample images; and obtaining the target training sample set based on the multiple target sample images.

[0012] Further, training the initial network model based on the target training sample set to obtain the target network model includes: processing the multiple target sample images using the initial network model to obtain the predicted classification results of the multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images; determining a target loss function based on the predicted classification results of the multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images; and training the initial network model based on the target loss function to obtain the target network model.

[0013] Further, determining the target loss function based on the predicted classification results of the plurality of target sample images and the predicted location information of abnormal regions in the plurality of target sample images includes: determining a first loss function based on the predicted classification results of the plurality of target sample images and the actual classification results of the plurality of target sample images in the target training sample set; determining a second loss function based on the actual location information of abnormal regions in the plurality of target sample images and the predicted location information of abnormal regions in the plurality of target sample images; and determining the target loss function based on the first loss function and the second loss function.

[0014] To achieve the above objectives, according to another aspect of this application, an image classification apparatus is provided. The apparatus includes: a first acquisition unit for acquiring a target image to be classified; a segmentation unit for segmenting the target image using a first segmentation module in a target network model to obtain a first segmented image; and a classification unit for classifying the target image based on the first segmented image using a classification module in the target network model to obtain a classification result for the target image, wherein the classification result of the target image indicates whether the target image has been tampered with.

[0015] Further, the classification unit includes: a cropping module, used to crop the first segmented image and the target image to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image; an extraction module, used to extract features from the multiple first image blocks based on the multiple second image blocks using the feature extraction module in the target network model to obtain multiple target feature images; a segmentation module, used to segment the multiple first image blocks based on the multiple second image blocks using the second segmentation module in the target network model to obtain segmented image blocks corresponding to each first image block, and to obtain a second segmented image based on the segmented image blocks, wherein the second segmented image includes location information of abnormal regions, and the pixel value of the abnormal region is a preset pixel value; a classification module, used to classify the second segmented image based on the multiple target feature images using the classification module in the target network model to determine the classification result of the second segmented image, wherein the classification result of the second segmented image characterizes whether the abnormal region in the second segmented image is a tampered region; and a first determination module, used to obtain the classification result of the target image based on the classification result of the second segmented image.

[0016] Further, the classification module includes: a calculation submodule, used to calculate a probability value corresponding to the second segmented image based on the plurality of target feature images, wherein the probability value represents the probability that the abnormal region in the second segmented image is a tampered region; a first determination submodule, used to determine that the classification result of the second segmented image is that the abnormal region in the second segmented image is a tampered region if the probability value is greater than or equal to a preset value; and a second determination submodule, used to determine that the classification result of the second segmented image is that the abnormal region in the second segmented image is not a tampered region if the probability value is less than the preset value.

[0017] Furthermore, the extraction module includes: a third determining submodule, used to determine a second image block corresponding to each first image block; and an extraction submodule, used to extract features from each first image block by means of the feature extraction module based on the second image block corresponding to the first image block, to obtain a target feature map corresponding to the first image block.

[0018] Furthermore, the device further includes: a second acquisition unit, configured to acquire the location information of the tampered region based on the second segmented image if the classification result of the target image indicates that the target image has been tampered with; a correction unit, configured to correct the second segmented image based on the location information using a preset algorithm to obtain a corrected second segmented image; and a positioning unit, configured to obtain the positioning result of the tampered region in the target image based on the corrected second segmented image.

[0019] Furthermore, the device further includes: a third acquisition unit, configured to acquire an initial training sample set, wherein the initial training sample set includes at least multiple initial sample images, the true classification result of each sample image, and the true location information of abnormal regions in each sample image; a processing unit, configured to process the initial training sample set to obtain a target training sample set; and a training unit, configured to train the initial network model based on the target training sample set to obtain the target network model.

[0020] Further, the processing unit includes: a preprocessing module for preprocessing the initial sample images in the initial training sample set to obtain multiple first sample images, wherein the preprocessing includes at least one of the following operations: image rotation, image scaling, image deformation, and mirror transformation; a padding module for padding the multiple first sample images to obtain multiple second sample images, wherein the multiple second sample images have the same size; a standardization module for standardizing the multiple second sample images to obtain multiple target sample images; and a second determination module for obtaining the target training sample set based on the multiple target sample images.

[0021] Further, the training unit includes: a processing module, used to process the plurality of target sample images through the initial network model to obtain the predicted classification results of the plurality of target sample images and the predicted location information of abnormal regions in the plurality of target sample images; a third determining module, used to determine a target loss function based on the predicted classification results of the plurality of target sample images and the predicted location information of abnormal regions in the plurality of target sample images; and a training module, used to train the initial network model based on the target loss function to obtain the target network model.

[0022] Furthermore, the third determining module includes: a fourth determining submodule, used to determine a first loss function based on the predicted classification results of the plurality of target sample images and the actual classification results of the plurality of target sample images in the target training sample set; a fifth determining submodule, used to determine a second loss function based on the actual location information of the abnormal regions in the plurality of target sample images and the predicted location information of the abnormal regions in the plurality of target sample images; and a sixth determining submodule, used to determine the target loss function based on the first loss function and the second loss function.

[0023] To achieve the above objectives, according to one aspect of this application, a computer-readable storage medium is provided, the storage medium storing a program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the image classification method described in any one of the above claims.

[0024] To achieve the above objectives, according to another aspect of this application, an electronic device is also provided, the electronic device including one or more processors and a memory, the memory being used to store the image classification method implemented by the one or more processors according to any one of the above.

[0025] This application employs the following steps: acquiring a target image to be classified; segmenting the target image using a first segmentation module in the target network model to obtain a first segmented image; and classifying the target image using a classification module in the target network model based on the first segmented image to obtain a classification result. The classification result indicates whether the target image has been tampered with, thus solving the problem in related technologies where image segmentation alone leads to low accuracy in determining whether an image has been tampered with. In this application, the target image is first segmented using a first segmentation module in the target network model, and then classified using a classification module based on the first segmented image. By adding a classification module to the image segmentation, the accuracy of target image classification is improved, thereby increasing the accuracy of determining whether an image has been tampered with. Attached Figure Description

[0026] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0027] Figure 1 This is a flowchart of an image classification method provided according to an embodiment of this application. Figure 1 ;

[0028] Figure 2 This is a flowchart of an image classification method provided according to an embodiment of this application. Figure 2 ;

[0029] Figure 3 This is a schematic diagram of an image classification device according to an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0031] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0035] Binary image: A binary image is an image in which each pixel has only two possible values ​​or grayscale levels. The grayscale value of any pixel in the image is 0 or 255, representing black and white respectively.

[0036] Connected component analysis: Connected component analysis is the most important method for binary image analysis. By marking the white pixels in the binary image, the connected regions of each individual pixel form a marked block, thereby obtaining the geometric parameters such as the outline, centroid, and circumscribed rectangle of these blocks.

[0037] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0038] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of an image classification method provided according to an embodiment of this application. Figure 1 ,like Figure 1 As shown, the method includes the following steps:

[0039] Step S101: Obtain the target image to be classified.

[0040] For example, acquiring a target image to be identified as to whether it has been tampered with, such as an image of a business license.

[0041] Step S102: The target image is segmented by the first segmentation module in the target network model to obtain the first segmented image.

[0042] For example, the target image is segmented by the first segmentation module in the target network model to obtain a low-resolution segmented image (i.e., the first segmented image mentioned above). It should be noted that the first segmentation module mentioned above can be a U-net neural network or a SegNet network.

[0043] Step S103: The target image is classified according to the first segmented image by the classification module in the target network model to obtain the classification result of the target image. The classification result of the target image indicates whether the target image has been tampered with.

[0044] For example, based on a low-resolution segmented image, the classification module in the target network model identifies and classifies whether the target image has been tampered with, thus determining whether the target image has been tampered with. It should be noted that the classification module in this application can be a fully convolutional neural network or a recurrent neural network.

[0045] It should be noted that, in order to improve the image segmentation accuracy and efficiency of the first segmentation module, the target image can be downsampled before segmentation to reduce its size. After the first segmentation module completes image segmentation, the segmented image is upsampled to ensure that the resulting low-resolution segmented image is the same size as the target image.

[0046] In summary, in this application, the target image is first segmented by the first segmentation module in the target network model, and then the target image is classified by the classification module in the target network model based on the first segmented image. By adding a classification module on the basis of image segmentation, the classification accuracy of the target image is improved, thereby achieving the effect of improving the accuracy of judging whether the image has been tampered with.

[0047] Optionally, in the image classification method provided in this application embodiment, classifying the target image based on the first segmented image by the classification module in the target network model to obtain the classification result of the target image includes: cropping the first segmented image and the target image to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image; extracting features from the multiple first image blocks based on the multiple second image blocks by the feature extraction module in the target network model to obtain multiple target feature images; segmenting the multiple first image blocks based on the multiple second image blocks by the second segmentation module in the target network model to obtain segmented image blocks corresponding to each first image block, and obtaining a second segmented image based on the segmented image blocks, wherein the second segmented image includes the location information of abnormal regions, and the pixel value of the abnormal region is a preset pixel value; classifying the second segmented image based on the multiple target feature images by the classification module in the target network model to determine the classification result of the second segmented image, wherein the classification result of the second segmented image characterizes whether the abnormal region in the second segmented image is a tampered region; and obtaining the classification result of the target image based on the classification result of the second segmented image.

[0048] For example, the obtained first segmented image and target image are first cropped to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image. Then, the first image blocks and second image blocks are input into the feature extraction module of the target network model to extract features from the multiple first image blocks and obtain the aforementioned multiple target feature images. It should be noted that the aforementioned target feature images can be feature maps with minimum resolution.

[0049] Simultaneously, the second segmentation module of the target network model performs image segmentation on multiple first image blocks to obtain segmented image blocks corresponding to each first image block. These segmented image blocks are then merged to obtain the aforementioned second segmented image. The classification module in the target network model then identifies multiple target feature images. Based on the identification results of the classification module, it is determined whether the abnormal region in the second segmented image is a tampered region. Finally, based on the classification results of the second segmented image, the classification result of the target image is determined.

[0050] If the classification result of the second segmented image is that the abnormal region in the second segmented image is a tampered region, then the classification result of the corresponding target image is that the target image is a tampered image. The above classification module can be a fully convolutional neural network, and the second segmentation module and the feature extraction module can be included in the same U-net neural network.

[0051] It should be noted that after the second segmentation module segments the first image block, the resulting segmented image block is a binary image. It should also be noted that, generally, white areas in a binary image are considered anomalous areas, meaning the white portions in the segmented image block are anomalous areas, potentially areas that have been tampered with. The classification module's identification results determine whether the white areas in the segmented image are tampered with.

[0052] By simultaneously classifying and segmenting multiple first image patches using the target network model, the system can both classify whether an image has been tampered with and locate the tampered region, thereby improving the efficiency of image classification and ensuring the accuracy of determining whether an image has been tampered with.

[0053] Optionally, in the image classification method provided in this application embodiment, classifying the second segmented image based on multiple target feature images by the classification module in the target network model, and determining the classification result of the second segmented image includes: calculating the probability value corresponding to the second segmented image based on multiple target feature images by the classification module, wherein the probability value represents the probability that the abnormal region in the second segmented image is a tampered region; if the probability value is greater than or equal to a preset value, the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is a tampered region; if the probability value is less than the preset value, the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is not a tampered region.

[0054] For example, the classification module identifies the target feature image and outputs the probability value that the abnormal region in the second segmented image is a tampered region among multiple target feature images. If the probability value (e.g., 0.8) is greater than or equal to a preset value (e.g., 0.5), then the abnormal region in the second segmented image is determined to be a tampered region. If the predicted probability value (e.g., 0.4) is less than the preset value (e.g., 0.5), then the abnormal region in the second segmented image is determined not to be a tampered region.

[0055] By using the predicted probability values ​​output by the classification module, it can be determined whether the abnormal region in the second segmented image is a tampered region. This allows for a quick and accurate determination of the classification result of the second segmented image, without the need for extensive computational analysis. This improves the efficiency of classifying the second segmented image, and consequently, the efficiency of classifying the target image.

[0056] Optionally, in the image classification method provided in this application embodiment, the feature extraction module in the target network model performs feature extraction on multiple first image blocks based on multiple second image blocks to obtain multiple target feature images, including: determining the second image block corresponding to each first image block; for each first image block, the feature extraction module performs feature extraction on the first image block based on the second image block corresponding to the first image block to obtain the target feature map corresponding to the first image block.

[0057] For example, the correspondence between multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image is determined, with each first image block corresponding to one second image block, and the two being the same region of the target image. Then, each first image block and its corresponding second image block are input together into the feature extraction module. The feature extraction module learns the feature information of the second image block, and then performs feature extraction on the first image block to obtain the target feature image corresponding to the first image block.

[0058] By inputting the first image block and the corresponding second image block into the feature extraction model, the feature extraction module can learn the feature information of the second image block while extracting features from the first image block, thereby improving the accuracy of feature extraction from the first image block.

[0059] Optionally, in the image classification method provided in this application embodiment, after obtaining the classification result of the target image based on the classification result of the second segmented image, the method further includes: if the classification result of the target image indicates that the target image has been tampered with, then obtaining the location information of the tampered area based on the second segmented image; correcting the second segmented image based on the location information using a preset algorithm to obtain a corrected second segmented image; and obtaining the location result of the tampered area in the target image based on the corrected second segmented image.

[0060] For example, if the classification result of the target image indicates that the target image has been tampered with, then the abnormal region in the second segmented image is also the tampered region. Based on the second segmented image, the location information of the tampered region is obtained. Then, the connected component analysis of the second segmented image is performed by an algorithm, that is, the tampered region in the second segmented image is connected to obtain the boundary of the tampered region. Then, the boundary of the tampered region in the second segmented image is corrected and identified to obtain the corrected second segmented image. Finally, based on the location of the tampered region in the corrected second segmented image, the location result of the tampered region in the target image is determined. In other words, the region in the target image that corresponds to the tampered region in the second segmented image is the tampered region.

[0061] It should be noted that if the classification result of the target image indicates that the target image has not been tampered with, then there is no need to perform connected component analysis, and the classification result can be directly output through the target network model.

[0062] Based on the classification results of the target image, it is determined whether to perform connected component analysis on the second segmented image. This avoids unnecessary analysis and calculations on images that are not tampered with according to the classification results. Through connected component analysis, the location of the tampered region in the second segmented image is more accurate. Then, based on the second segmented image, the specific location of the tampered region in the target image is determined, which improves the accuracy of locating the tampered region in the target image.

[0063] Optionally, in the image classification method provided in the embodiments of this application, before obtaining the target image to be classified, the method further includes: obtaining an initial training sample set, wherein the initial training sample set includes at least multiple initial sample images, the true classification result of each sample image, and the true location information of the abnormal region in each sample image; processing the initial training sample set to obtain a target training sample set; and training the initial network model based on the target training sample set to obtain a target network model.

[0064] For example, first obtain an initial training sample set, then process the initial sample images in the initial training sample set to obtain a target training sample set including multiple target training samples, and then train the initial network model based on the target training sample set to obtain the target network model. The aforementioned initial training sample set includes multiple initial sample images, which may include tampered images and untampered images. The tampered sample images have corresponding label information in the tampered areas.

[0065] By training the initial network model with the target training sample set, the target network model can better classify, segment, and locate images, thereby improving the prediction accuracy of the target network model and ensuring the accuracy of the target network model in classifying images.

[0066] Optionally, in the image classification method provided in this application embodiment, processing the initial training sample set to obtain the target training sample set includes: preprocessing the initial sample images in the initial training sample set to obtain multiple first sample images, wherein the preprocessing includes at least one of the following operations: image rotation, image scaling, image deformation, and mirror transformation; padding the multiple first sample images to obtain multiple second sample images, wherein the multiple second sample images have the same size; standardizing the multiple second sample images to obtain multiple target sample images; and obtaining the target training sample set based on the multiple target sample images.

[0067] For example, the initial sample images in the initial training sample set are first preprocessed to obtain multiple first sample images. For instance, random image rotation, random image scaling, random image elastic deformation, gamma correction, or mirror transformation can be applied to the initial sample images. It should be noted that during the preprocessing of the sample images, random cropping is not performed to avoid confusing the initial network model with cropping caused by image manipulation during preprocessing.

[0068] Then, padding is applied to multiple first sample images to make them all the same size (e.g., 256*256 pixels). The images are then normalized using the z-score normalization method to obtain multiple target sample images. Based on these multiple target sample images, a target training sample set is obtained. The formula for the z-score normalization method is shown below:

[0069]

[0070] Where z is the target sample image, x is the second sample image, μ is the pixel mean image of multiple second sample images, and σ is the pixel variance image of multiple second sample images.

[0071] First, the initial sample images in the initial sample set are preprocessed, which effectively expands the number of sample images. Then, the sample images are standardized, which reduces the training difficulty of the initial network model and enables the network model to converge faster, thus ensuring the efficiency of model training.

[0072] Optionally, in the image classification method provided in this application embodiment, training an initial network model based on a target training sample set to obtain a target network model includes: processing multiple target sample images through the initial network model to obtain predicted classification results of multiple target sample images and predicted location information of abnormal regions in multiple target sample images; determining a target loss function based on the predicted classification results of multiple target sample images and the predicted location information of abnormal regions in multiple target sample images; and training the initial network model based on the target loss function to obtain the target network model.

[0073] For example, multiple target sample images are first classified using an initial network model to obtain the predicted classification results of the target sample images and the predicted location information of abnormal regions in the target sample images. Then, the target loss function is determined based on the output of the initial network model. The initial network model is then trained based on the target loss function. By minimizing the target loss function value, the initial network model is made to converge, and finally, a target network model with the expected prediction effect is obtained.

[0074] It should be noted that during model training, the learning rate of the model also needs to be adjusted according to the loss function value to ensure the training effect of the model.

[0075] The initial network model is used to predict the target sample image to obtain the prediction result. Then, the target loss function is determined based on the prediction result. Finally, the initial network model is trained using the target loss function to obtain the target network model. By training the initial network model, the accuracy of the target network model in image classification is improved.

[0076] Optionally, in the image classification method provided in this application embodiment, determining the target loss function based on the predicted classification results of multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images includes: determining a first loss function based on the predicted classification results of multiple target sample images and the actual classification results of multiple target sample images in the target training sample set; determining a second loss function based on the actual location information of abnormal regions in the multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images; and determining the target loss function based on the first loss function and the second loss function.

[0077] For example, based on the predicted classification results of multiple target sample images output by the initial network model and the true classification results of multiple training sample images, the first loss function L for the classification task is determined. c The first loss function mentioned above can be the binary classification cross-entropy loss function. Then, based on the predicted location information of abnormal regions in multiple sample images output by the initial network model and the actual location information of abnormal regions in multiple sample images, the second loss function L for the segmentation task is determined. segThe second loss function mentioned above can be a weighted cross-entropy loss function, L seg The specific formula can be seen as follows:

[0078]

[0079] Where M = 2, c is the pixel category (including black and white), and y c For the pixel category c in the real location information, p c To predict the label of pixel category c in the location information, w c The weighting parameters are as follows:

[0080]

[0081] Where N represents the total number of pixels in the sample image, N c This indicates the number of pixels of type c.

[0082] The target loss function is determined based on the first loss function and the second loss function, as shown below:

[0083] L total =L seg +L c (4)

[0084] By designing a target loss function, the initial network model is trained simultaneously for image classification and anomaly region localization, ensuring the accuracy of the target network model in image classification and anomaly region localization, and thus ensuring the accuracy of determining whether an image has been tampered with.

[0085] In an alternative embodiment, the following can be employed: Figure 2 The flowchart shown implements image classification. The specific process is as follows: In the first network stage, the original image is downsampled and scaled down to obtain a low-resolution original image. The low-resolution original image is then input into the first U-net neural network module. The first U-net neural network module performs image segmentation on the low-resolution original image to obtain a low-resolution segmented image. Subsequently, the low-resolution segmented image is upsampled to restore the size of the low-resolution segmented image to the same as the original image. The original image is copied to the low-resolution segmented image side through jumper links. Then, the low-resolution segmented image and the original image are cropped together to obtain image blocks of each segmented image and their corresponding image blocks of the original image.

[0086] In the second network stage, each segmented image patch and its corresponding original image patch are fed into the second U-net neural network module. Since the U-net neural network has a symmetrical structure, with the first half encoding the image and extracting features, and the second half decoding the image and segmenting it, the central feature map in the U-net neural network is the lowest-resolution feature map after feature extraction. This lowest-resolution feature map is input into a fully convolutional neural network (WCNN), which performs the recognition task, predicting whether the image has been tampered with. Simultaneously, the second U-net neural network module also outputs the original-resolution segmented image. The original-resolution segmented image is a binary image, where the white areas represent abnormal regions, i.e., regions that may have been tampered with.

[0087] Based on the prediction results of the fully convolutional neural network, it is determined whether the white regions in the segmented image at the original resolution are tampered regions. If they are determined to be tampered regions, connected component analysis is performed on the segmented image at the original resolution, that is, the white regions are connected. The boundaries of the white regions in the segmented image at the original resolution are corrected to achieve accurate localization of the tampered regions.

[0088] The image classification method provided in this application involves acquiring a target image to be classified; performing image segmentation on the target image using a first segmentation module in a target network model to obtain a first segmented image; and then classifying the target image based on the first segmented image using a classification module in the target network model to obtain a classification result. The classification result indicates whether the target image has been tampered with. This solves the problem in related technologies where image segmentation is used to determine whether an image has been tampered with, resulting in low accuracy. In this application, the target image is first segmented using a first segmentation module in the target network model, and then classified based on the first segmented image by a classification module in the target network model. By adding a classification module to the image segmentation, the accuracy of target image classification is improved, thereby increasing the accuracy of determining whether an image has been tampered with.

[0089] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0090] This application also provides an image classification apparatus. It should be noted that the image classification apparatus of this application can be used to execute the image classification method provided in this application. The image classification apparatus provided in this application will be described below.

[0091] Figure 3 This is a schematic diagram of an image classification apparatus according to an embodiment of this application. Figure 3 As shown, the device includes: a first acquisition unit 301, a segmentation unit 302, and a classification unit 303.

[0092] The first acquisition unit 301 is used to acquire the target image to be classified;

[0093] The segmentation unit 302 is used to segment the target image through the first segmentation module in the target network model to obtain a first segmented image;

[0094] The classification unit 303 is used to classify the target image based on the first segmented image through the classification module in the target network model to obtain the classification result of the target image, wherein the classification result of the target image indicates whether the target image has been tampered with.

[0095] The image classification apparatus provided in this application embodiment acquires a target image to be classified through a first acquisition unit 301; a segmentation unit 302 performs image segmentation on the target image through a first segmentation module in a target network model to obtain a first segmented image; and a classification unit 303 classifies the target image based on the first segmented image through a classification module in the target network model to obtain a classification result of the target image. The classification result of the target image indicates whether the target image has been tampered with, thus solving the problem in related technologies where image segmentation is used to determine whether an image has been tampered with, resulting in a relatively low accuracy rate in this assessment. In this application, the target image is first segmented by the first segmentation module in the target network model, and then classified by the classification module in the target network model based on the first segmented image. By adding a classification module on top of image segmentation, the accuracy of target image classification is improved, thereby increasing the accuracy of determining whether an image has been tampered with.

[0096] Optionally, in the image classification apparatus provided in this application embodiment, the classification unit 303 includes: a cropping module, used to crop a first segmented image and a target image to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image; an extraction module, used to extract features from multiple first image blocks based on multiple second image blocks using a feature extraction module in the target network model to obtain multiple target feature images; a segmentation module, used to segment multiple first image blocks based on multiple second image blocks using a second segmentation module in the target network model to obtain segmented image blocks corresponding to each first image block, and to obtain a second segmented image based on the segmented image blocks, wherein the second segmented image includes location information of abnormal regions, and the pixel value of the abnormal region is a preset pixel value; a classification module, used to classify the second segmented image based on multiple target feature images using a classification module in the target network model to determine the classification result of the second segmented image, wherein the classification result of the second segmented image characterizes whether the abnormal region in the second segmented image is a tampered region; and a first determination module, used to obtain the classification result of the target image based on the classification result of the second segmented image.

[0097] Optionally, in the image classification apparatus provided in this application embodiment, the classification module includes: a calculation submodule, used to calculate the probability value corresponding to the second segmented image based on multiple target feature images through the classification module, wherein the probability value represents the probability that the abnormal region in the second segmented image is a tampered region; a first determination submodule, used to determine that the classification result of the second segmented image is that the abnormal region in the second segmented image is a tampered region if the probability value is greater than or equal to a preset value; and a second determination submodule, used to determine that the classification result of the second segmented image is that the abnormal region in the second segmented image is not a tampered region if the probability value is less than the preset value.

[0098] Optionally, in the image classification apparatus provided in this application embodiment, the extraction module includes: a third determining submodule, used to determine a second image block corresponding to each first image block; and an extraction submodule, used to extract features from each first image block by a feature extraction module based on the second image block corresponding to the first image block, to obtain a target feature map corresponding to the first image block.

[0099] Optionally, in the image classification apparatus provided in this application embodiment, the apparatus further includes: a second acquisition unit, configured to acquire the location information of the tampered region based on the second segmentation image if the classification result of the target image indicates that the target image has been tampered with; a correction unit, configured to correct the second segmentation image based on the location information using a preset algorithm to obtain a corrected second segmentation image; and a positioning unit, configured to obtain the positioning result of the tampered region in the target image based on the corrected second segmentation image.

[0100] Optionally, in the image classification apparatus provided in the embodiments of this application, the apparatus further includes: a third acquisition unit, configured to acquire an initial training sample set, wherein the initial training sample set includes at least a plurality of initial sample images, the true classification result of each sample image, and the true location information of abnormal regions in each sample image; a processing unit, configured to process the initial training sample set to obtain a target training sample set; and a training unit, configured to train the initial network model based on the target training sample set to obtain a target network model.

[0101] Optionally, in the image classification apparatus provided in this application embodiment, the processing unit includes: a preprocessing module, used to preprocess the initial sample images in the initial training sample set to obtain a plurality of first sample images, wherein the preprocessing includes at least one of the following operations: image rotation, image scaling, image deformation, and mirror transformation; a filling module, used to fill the plurality of first sample images to obtain a plurality of second sample images, wherein the plurality of second sample images have the same size; a standardization module, used to standardize the plurality of second sample images to obtain a plurality of target sample images; and a second determination module, used to obtain a target training sample set based on the plurality of target sample images.

[0102] Optionally, in the image classification apparatus provided in this application embodiment, the training unit includes: a processing module, used to process multiple target sample images through an initial network model to obtain predicted classification results of multiple target sample images and predicted location information of abnormal regions in multiple target sample images; a third determining module, used to determine a target loss function based on the predicted classification results of multiple target sample images and the predicted location information of abnormal regions in multiple target sample images; and a training module, used to train the initial network model based on the target loss function to obtain a target network model.

[0103] Optionally, in the image classification apparatus provided in this application embodiment, the third determining module includes: a fourth determining submodule, used to determine a first loss function based on the predicted classification results of multiple target sample images and the actual classification results of multiple target sample images in the target training sample set; a fifth determining submodule, used to determine a second loss function based on the actual location information of abnormal regions in multiple target sample images and the predicted location information of abnormal regions in multiple target sample images; and a sixth determining submodule, used to determine a target loss function based on the first loss function and the second loss function.

[0104] The image classification device includes a processor and a memory. The first acquisition unit 301, the segmentation unit 302 and the classification unit 303 mentioned above are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.

[0105] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of low accuracy in determining image tampering through image segmentation.

[0106] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0107] This invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements an image classification method.

[0108] This invention provides a processor for running a program, wherein the program executes an image classification method during runtime.

[0109] like Figure 4 As shown, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring a target image to be classified; performing image segmentation on the target image through a first segmentation module in the target network model to obtain a first segmented image; and classifying the target image based on the first segmented image through a classification module in the target network model to obtain a classification result of the target image, wherein the classification result of the target image indicates whether the target image has been tampered with.

[0110] Optionally, in the image classification method provided in this application embodiment, classifying the target image based on the first segmented image by the classification module in the target network model to obtain the classification result of the target image includes: cropping the first segmented image and the target image to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image; extracting features from the multiple first image blocks based on the multiple second image blocks by the feature extraction module in the target network model to obtain multiple target feature images; segmenting the multiple first image blocks based on the multiple second image blocks by the second segmentation module in the target network model to obtain segmented image blocks corresponding to each first image block, and obtaining a second segmented image based on the segmented image blocks, wherein the second segmented image includes the location information of abnormal regions, and the pixel value of the abnormal region is a preset pixel value; classifying the second segmented image based on the multiple target feature images by the classification module in the target network model to determine the classification result of the second segmented image, wherein the classification result of the second segmented image characterizes whether the abnormal region in the second segmented image is a tampered region; and obtaining the classification result of the target image based on the classification result of the second segmented image.

[0111] Optionally, in the image classification method provided in this application embodiment, classifying the second segmented image based on multiple target feature images by the classification module in the target network model, and determining the classification result of the second segmented image includes: calculating the probability value corresponding to the second segmented image based on multiple target feature images by the classification module, wherein the probability value represents the probability that the abnormal region in the second segmented image is a tampered region; if the probability value is greater than or equal to a preset value, the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is a tampered region; if the probability value is less than the preset value, the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is not a tampered region.

[0112] Optionally, in the image classification method provided in this application embodiment, the feature extraction module in the target network model performs feature extraction on multiple first image blocks based on multiple second image blocks to obtain multiple target feature images, including: determining the second image block corresponding to each first image block; for each first image block, the feature extraction module performs feature extraction on the first image block based on the second image block corresponding to the first image block to obtain the target feature map corresponding to the first image block.

[0113] Optionally, in the image classification method provided in this application embodiment, after obtaining the classification result of the target image based on the classification result of the second segmented image, the method further includes: if the classification result of the target image indicates that the target image has been tampered with, then obtaining the location information of the tampered area based on the second segmented image; correcting the second segmented image based on the location information using a preset algorithm to obtain a corrected second segmented image; and obtaining the location result of the tampered area in the target image based on the corrected second segmented image.

[0114] Optionally, in the image classification method provided in the embodiments of this application, before obtaining the target image to be classified, the method further includes: obtaining an initial training sample set, wherein the initial training sample set includes at least multiple initial sample images, the true classification result of each sample image, and the true location information of the abnormal region in each sample image; processing the initial training sample set to obtain a target training sample set; and training the initial network model based on the target training sample set to obtain a target network model.

[0115] Optionally, in the image classification method provided in this application embodiment, processing the initial training sample set to obtain the target training sample set includes: preprocessing the initial sample images in the initial training sample set to obtain multiple first sample images, wherein the preprocessing includes at least one of the following operations: image rotation, image scaling, image deformation, and mirror transformation; padding the multiple first sample images to obtain multiple second sample images, wherein the multiple second sample images have the same size; standardizing the multiple second sample images to obtain multiple target sample images; and obtaining the target training sample set based on the multiple target sample images.

[0116] Optionally, in the image classification method provided in this application embodiment, training an initial network model based on a target training sample set to obtain a target network model includes: processing multiple target sample images through the initial network model to obtain predicted classification results of multiple target sample images and predicted location information of abnormal regions in multiple target sample images; determining a target loss function based on the predicted classification results of multiple target sample images and the predicted location information of abnormal regions in multiple target sample images; and training the initial network model based on the target loss function to obtain the target network model.

[0117] Optionally, in the image classification method provided in this application embodiment, determining the target loss function based on the predicted classification results of multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images includes: determining a first loss function based on the predicted classification results of multiple target sample images and the actual classification results of multiple target sample images in the target training sample set; determining a second loss function based on the actual location information of abnormal regions in the multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images; and determining the target loss function based on the first loss function and the second loss function. The device mentioned herein can be a server, PC, PAD, mobile phone, etc.

[0118] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: acquiring a target image to be classified; performing image segmentation on the target image through a first segmentation module in a target network model to obtain a first segmented image; and classifying the target image based on the first segmented image through a classification module in the target network model to obtain a classification result of the target image, wherein the classification result of the target image indicates whether the target image has been tampered with.

[0119] Optionally, in the image classification method provided in this application embodiment, classifying the target image based on the first segmented image by the classification module in the target network model to obtain the classification result of the target image includes: cropping the first segmented image and the target image to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image; extracting features from the multiple first image blocks based on the multiple second image blocks by the feature extraction module in the target network model to obtain multiple target feature images; segmenting the multiple first image blocks based on the multiple second image blocks by the second segmentation module in the target network model to obtain segmented image blocks corresponding to each first image block, and obtaining a second segmented image based on the segmented image blocks, wherein the second segmented image includes the location information of abnormal regions, and the pixel value of the abnormal region is a preset pixel value; classifying the second segmented image based on the multiple target feature images by the classification module in the target network model to determine the classification result of the second segmented image, wherein the classification result of the second segmented image characterizes whether the abnormal region in the second segmented image is a tampered region; and obtaining the classification result of the target image based on the classification result of the second segmented image.

[0120] Optionally, in the image classification method provided in this application embodiment, classifying the second segmented image based on multiple target feature images by the classification module in the target network model, and determining the classification result of the second segmented image includes: calculating the probability value corresponding to the second segmented image based on multiple target feature images by the classification module, wherein the probability value represents the probability that the abnormal region in the second segmented image is a tampered region; if the probability value is greater than or equal to a preset value, the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is a tampered region; if the probability value is less than the preset value, the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is not a tampered region.

[0121] Optionally, in the image classification method provided in this application embodiment, the feature extraction module in the target network model performs feature extraction on multiple first image blocks based on multiple second image blocks to obtain multiple target feature images, including: determining the second image block corresponding to each first image block; for each first image block, the feature extraction module performs feature extraction on the first image block based on the second image block corresponding to the first image block to obtain the target feature map corresponding to the first image block.

[0122] Optionally, in the image classification method provided in this application embodiment, after obtaining the classification result of the target image based on the classification result of the second segmented image, the method further includes: if the classification result of the target image indicates that the target image has been tampered with, then obtaining the location information of the tampered area based on the second segmented image; correcting the second segmented image based on the location information using a preset algorithm to obtain a corrected second segmented image; and obtaining the location result of the tampered area in the target image based on the corrected second segmented image.

[0123] Optionally, in the image classification method provided in the embodiments of this application, before obtaining the target image to be classified, the method further includes: obtaining an initial training sample set, wherein the initial training sample set includes at least multiple initial sample images, the true classification result of each sample image, and the true location information of the abnormal region in each sample image; processing the initial training sample set to obtain a target training sample set; and training the initial network model based on the target training sample set to obtain a target network model.

[0124] Optionally, in the image classification method provided in this application embodiment, processing the initial training sample set to obtain the target training sample set includes: preprocessing the initial sample images in the initial training sample set to obtain multiple first sample images, wherein the preprocessing includes at least one of the following operations: image rotation, image scaling, image deformation, and mirror transformation; padding the multiple first sample images to obtain multiple second sample images, wherein the multiple second sample images have the same size; standardizing the multiple second sample images to obtain multiple target sample images; and obtaining the target training sample set based on the multiple target sample images.

[0125] Optionally, in the image classification method provided in this application embodiment, training an initial network model based on a target training sample set to obtain a target network model includes: processing multiple target sample images through the initial network model to obtain predicted classification results of multiple target sample images and predicted location information of abnormal regions in multiple target sample images; determining a target loss function based on the predicted classification results of multiple target sample images and the predicted location information of abnormal regions in multiple target sample images; and training the initial network model based on the target loss function to obtain the target network model.

[0126] Optionally, in the image classification method provided in this application embodiment, determining the target loss function based on the predicted classification results of multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images includes: determining a first loss function based on the predicted classification results of multiple target sample images and the actual classification results of multiple target sample images in the target training sample set; determining a second loss function based on the actual location information of abnormal regions in the multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images; and determining the target loss function based on the first loss function and the second loss function.

[0127] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0132] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0133] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0134] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0135] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image classification method, characterized in that, include: Obtain the target image to be classified; The target image is segmented using the first segmentation module in the target network model to obtain a first segmented image; The target image is classified by the classification module in the target network model based on the first segmented image to obtain the classification result of the target image, wherein the classification result of the target image indicates whether the target image has been tampered with; The classification module in the target network model classifies the target image based on the first segmented image to obtain the classification result of the target image, including: The first segmented image and the target image are cropped to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image; The feature extraction module in the target network model extracts features from the multiple first image blocks based on the multiple second image blocks to obtain multiple target feature images; The second segmentation module in the target network model performs image segmentation on the multiple first image blocks based on the multiple second image blocks to obtain a segmented image block corresponding to each first image block, and obtains a second segmented image based on the segmented image block. The second segmented image includes the location information of abnormal regions, and the pixel value of the abnormal region is a preset pixel value. The classification module in the target network model classifies the second segmented image based on the multiple target feature images to determine the classification result of the second segmented image. The classification result of the second segmented image indicates whether the abnormal region in the second segmented image is a tampered region. Based on the classification result of the second segmented image, the classification result of the target image is obtained.

2. The method according to claim 1, characterized in that, The classification module in the target network model classifies the second segmented image based on the multiple target feature images, and the classification result of the second segmented image includes: The classification module calculates the probability value corresponding to the second segmented image based on the multiple target feature images, wherein the probability value represents the probability that the abnormal region in the second segmented image is a tampered region; If the probability value is greater than or equal to the preset value, then the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is a tampered region; If the probability value is less than the preset value, then the classification result of the second segmented image is determined to be that the abnormal region in the second segmented image is not a tampered region.

3. The method according to claim 1, characterized in that, The feature extraction module in the target network model extracts features from the multiple first image patches based on the multiple second image patches to obtain multiple target feature images, including: Determine the second image block corresponding to each first image block; For each first image block, the feature extraction module performs feature extraction on the first image block based on the second image block corresponding to the first image block to obtain the target feature map corresponding to the first image block.

4. The method according to claim 1, characterized in that, After obtaining the classification result of the target image based on the classification result of the second segmented image, the method further includes: If the classification result of the target image indicates that the target image has been tampered with, then the location information of the tampered region is obtained based on the second segmented image; The second segmented image is corrected based on the location information using a preset algorithm to obtain a corrected second segmented image; Based on the corrected second segmented image, the location result of the tampered region in the target image is obtained.

5. The method according to claim 1, characterized in that, Before acquiring the target image to be classified, the method further includes: Obtain an initial training sample set, wherein the initial training sample set includes at least multiple initial sample images, the true classification result of each sample image, and the true location information of abnormal regions in each sample image; The initial training sample set is processed to obtain the target training sample set; The initial network model is trained based on the target training sample set to obtain the target network model.

6. The method according to claim 5, characterized in that, The initial training sample set is processed to obtain the target training sample set, which includes: The initial sample images in the initial training sample set are preprocessed to obtain multiple first sample images, wherein the preprocessing includes at least one of the following operations: image rotation, image scaling, image deformation, and mirror transformation; The plurality of first sample images are filled to obtain a plurality of second sample images, wherein the plurality of second sample images are of the same size; The multiple second sample images are standardized to obtain multiple target sample images; The target training sample set is obtained based on the multiple target sample images.

7. The method according to claim 6, characterized in that, The initial network model is trained based on the target training sample set to obtain the target network model, which includes: The initial network model is used to process the multiple target sample images to obtain the predicted classification results of the multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images. Based on the predicted classification results of the multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images, the target loss function is determined; The initial network model is trained based on the target loss function to obtain the target network model.

8. The method according to claim 7, characterized in that, Based on the predicted classification results of the multiple target sample images and the predicted location information of abnormal regions in the multiple target sample images, the target loss function is determined as follows: Based on the predicted classification results of the multiple target sample images and the actual classification results of the multiple target sample images in the target training sample set, a first loss function is determined; A second loss function is determined based on the actual location information of the abnormal regions in the multiple target sample images and the predicted location information of the abnormal regions in the multiple target sample images; The target loss function is determined based on the first loss function and the second loss function.

9. An image classification device, characterized in that, include: The first acquisition unit is used to acquire the target image to be classified. A segmentation unit is used to segment the target image using the first segmentation module in the target network model to obtain a first segmented image. A classification unit is used to classify the target image based on the first segmented image using the classification module in the target network model, and obtain a classification result of the target image, wherein the classification result of the target image indicates whether the target image has been tampered with; The classification unit includes: a cropping module for cropping the first segmented image and the target image to obtain multiple first image blocks corresponding to the first segmented image and multiple second image blocks corresponding to the target image; an extraction module for extracting features from the multiple first image blocks based on the multiple second image blocks using the feature extraction module in the target network model to obtain multiple target feature images; a segmentation module for segmenting the multiple first image blocks based on the multiple second image blocks using the second segmentation module in the target network model to obtain segmented image blocks corresponding to each first image block, and obtaining a second segmented image based on the segmented image blocks, wherein the second segmented image includes the location information of abnormal regions, and the pixel values ​​of the abnormal regions are preset pixel values; a classification module for classifying the second segmented image based on the multiple target feature images using the classification module in the target network model to determine the classification result of the second segmented image, wherein the classification result of the second segmented image indicates whether the abnormal regions in the second segmented image are tampered regions; and a first determination module for obtaining the classification result of the target image based on the classification result of the second segmented image.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, the storage medium controls the device to perform the image classification method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the image classification method according to any one of claims 1 to 8.

Images