Method and device for evaluating image classification model

Abstract

The invention discloses an evaluation method and device of an image classification model. The method comprises the following steps: acquiring image data, and dividing the image data into a training set, an evaluation data set and a test set; based on the test set, the performance of the image classification model in image similarity detection is evaluated by adopting a quantitative analysis method and utilizing classification performance indexes, and the classification performance indexes comprise at least one of the following indexes: the accuracy rate, the precision rate, the recall rate and the harmonic average value. The technical problem that the detection result of the image classification model is inaccurate is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an evaluation method and device for an image classification model. Background Art

[0002] With the development of computer vision technology, image classification models have gained wide attention in many application fields. In order to ensure the effectiveness and reliability of these models in practical applications, it is particularly important to evaluate their performance.

[0003] Current image classification models are usually built based on predefined rules or machine learning algorithms, and these models often perform satisfactorily during the training phase. However, in practical applications, the accuracy of their evaluation methods is often insufficient. Specifically, when the model processes a large amount of complex real data, it is prone to misjudgment, resulting in inaccurate image classification.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for evaluating an image classification model, so as to at least solve the technical problem of inaccurate detection results of the image classification model.

[0006] According to one aspect of an embodiment of the present invention, there is provided a method for evaluating an image classification model, comprising: acquiring image data, and dividing the image data into a training set, an evaluation data set, and a test set; based on the test set, using a quantitative analysis method, and utilizing classification performance indicators to evaluate the performance of the image classification model in image similarity detection, wherein the classification performance indicators include at least one of the following: accuracy, precision, recall, and harmonic mean.

[0007] According to another aspect of an embodiment of the present invention, there is also provided an evaluation device for an image classification model, comprising: an acquisition module, configured to acquire image data, and divide the image data into a training set, an evaluation data set, and a test set; an evaluation module, configured to evaluate the image classification model's detection performance for image similarity based on the test set by using a quantitative analysis method, using accuracy, precision, recall, and harmonic mean, so as to evaluate the image classification model.

[0008] In an embodiment of the present invention, image data is obtained and divided into a training set, an evaluation data set and a test set; based on the test set, a quantitative analysis method is used to evaluate the performance of the image classification model in image similarity detection using classification performance indicators, wherein the classification performance indicators include at least one of the following: accuracy, precision, recall, and harmonic mean. Through the above solution, the technical problem of inaccurate detection results of the image classification model is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0010] Figure 1 is a flow chart of a method for evaluating an image classification model according to an embodiment of the present invention;

[0011] Figure 2 is a flow chart of another method for evaluating an image classification model according to an embodiment of the present invention;

[0012] Figure 3 is a flowchart of a method for training an image classification model according to an embodiment of the present invention;

[0013] Figure 4 is a flow chart of a model evaluation method according to an embodiment of the present invention;

[0014] Figure 5 is a structural schematic diagram of an evaluation device for an image classification model according to an embodiment of the present invention;

[0015] Figure 6 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] The present application embodiment provides a method for evaluating an image classification model. Figure 1 As shown, the evaluation method of the image classification model includes the following steps:

[0019] Step S102, acquiring image data, and dividing the image data into a training set, an evaluation data set, and a test set;

[0020] First, image data needs to be collected to ensure that it covers a variety of scenes and objects to improve the generalization ability of the model. The acquired dataset should be preprocessed, including denoising, normalization, and image resizing, to meet the input requirements of the subsequent model. After preprocessing, these image data are divided into three subsets: training set, evaluation dataset, and test set. Generally, the dataset can be divided in proportion, such as 70% of the data is used for training, 15% for evaluation, and 15% for testing, which can effectively balance the needs of training and evaluation.

[0021] After that, the data set is further checked and analyzed to ensure that the sample distribution in each subset is consistent to avoid bias in the model during training and testing. At the same time, ensure that the number of each type of sample is sufficient to meet the needs of model learning, especially when facing unbalanced data, if necessary, techniques such as oversampling or undersampling can be used for processing.

[0022] Step S104, based on the test set, a quantitative analysis method is used to evaluate the performance of the image classification model in image similarity detection using classification performance indicators, wherein the classification performance indicators include at least one of the following: accuracy, precision, recall, and harmonic mean.

[0023] Accuracy refers to the ratio of the number of samples correctly predicted by the model to the total number of samples, which is used to measure the overall classification effect. Precision refers to the ratio of truly positive samples among all samples predicted as positive samples, which reflects the accuracy of the model in predicting the positive class. Recall refers to the ratio of correctly predicted positive samples among all true positive samples, which measures the model's ability to capture positive samples. The harmonic mean (F1 Score) is the harmonic mean of precision and recall, which provides a comprehensive evaluation indicator, especially suitable for situations where positive and negative samples are unbalanced. A high F1 value indicates that the model performs well in both accuracy and capture ability.

[0024] In some embodiments, based on a binary confusion matrix, the predicted positive examples TP and the predicted negative examples FN are determined when the test data in the test set are true positive examples, as well as the predicted positive examples FP and the predicted negative examples TN are determined when the test data in the test set are true negative examples; based on at least part of the predicted positive examples TP and the predicted negative examples FN in the case of the true positive examples, and the predicted positive examples FP and the predicted negative examples TN in the case of the true negative examples, the accuracy, the precision, and the recall are determined, and the harmonic mean is determined based on the accuracy and the recall; the detection performance of the image classification model for image similarity is evaluated based on the accuracy, the precision, the recall, and the harmonic mean.

[0025] This embodiment can accurately evaluate the performance of the image classification model in image similarity detection through analysis based on the binary confusion matrix. First, by determining the predicted positive examples (TP) and predicted negative examples (FN) of the true positive examples, as well as the predicted positive examples (FP) and predicted negative examples (TN) of the true negative examples, the classification effect of the model can be fully understood. Based on these data, the calculated accuracy, precision, and recall rates provide a detailed evaluation of the performance of the model, while the harmonic mean effectively integrates the information of precision and recall, reflecting the comprehensive ability of the model in dealing with the imbalance of positive and negative examples. This embodiment can not only reveal the performance of the model in different sample situations, but also provide a clear direction for subsequent optimization, thereby improving the accuracy and reliability of image similarity detection.

[0026] In some embodiments, with fixed variables, the basic network, optimization strategy and data enhancement of the image classification model are adjusted respectively; for the adjusted basic network, the optimization strategy and the data enhancement, the accuracy, the precision, the recall and the harmonic mean are calculated respectively to evaluate the impact of the basic network, the optimization strategy and the data enhancement on the performance of the image classification model. This embodiment can accurately measure how changes in the basic network architecture, optimization algorithm and data enhancement technology affect the accuracy, precision, recall and harmonic mean. Through the above method, not only can the contribution of each component in improving model performance be revealed, but also empirical evidence can be provided for future model optimization, thereby effectively enhancing the performance and robustness of image classification models in practical applications.

[0027] In some embodiments, under the same test set, color distribution analysis is used to extract the color attributes of the images in the test set, and a texture analysis method is used to extract the texture attributes of the images. The similarity of the images is calculated according to the extracted color attributes and texture attributes, and the accuracy, precision, recall and harmonic mean of the color distribution analysis and the texture analysis method are calculated based on the calculated similarity; the accuracy, precision, recall and harmonic mean of the color distribution analysis and the texture analysis method are compared with the accuracy, precision, recall and harmonic mean of the image classification model to evaluate the image classification model. This embodiment can comprehensively evaluate the performance of the image classification model by using color distribution analysis and texture analysis under the same test set. Through the above method, not only the effectiveness of color distribution analysis and texture analysis is revealed, but also an important benchmark is provided for evaluating the image classification model. Through the systematic comparison of these indicators, the most influential feature extraction method can be identified, which provides valuable guidance for subsequent optimization and improvement, thereby improving the effect and reliability of overall image similarity detection.

[0028] Figure 2 is another evaluation method of an image classification model according to an embodiment of the present invention, such as Figure 2 As shown, the method includes:

[0029] Step S202, constructing an image classification model.

[0030] This application builds a convolutional neural network (CNN) model based on a Siamese neural network to improve the performance and accuracy of image classification. A Siamese neural network is two sub-networks that share weights. These two networks process different input images in parallel and can effectively capture and compare the similarities and differences between images. This structure is particularly suitable for tasks that require contrastive learning, such as similarity detection and image classification.

[0031] In the feature extraction module, the convolutional neural network gradually extracts the hierarchical features of the image through convolutional layers and pooling layers. The convolutional layer can learn low-level to high-level features, thereby deeply analyzing the image. In order to enhance the model's attention to important features, this application specifically introduces the Convolutional Block Attention Module (CBAM). CBAM consists of two parts: channel attention and spatial attention.

[0032] The channel attention module first evaluates the contribution of each channel, generates a channel descriptor through global average pooling and global maximum pooling, and then generates a channel attention map through a simple feed-forward network. In this way, the model can emphasize those channels that are more important in the feature extraction process, thereby improving the effectiveness of feature representation.

[0033] Subsequently, the spatial attention module weights the spatial dimensions of the feature map. By processing the average and maximum values ​​of the channel dimension, a spatial attention map is generated, which enables the model to focus on important areas in the image. Finally, CBAM enhances the model's sensitivity to key features by weighted combination of feature maps.

[0034] In addition, in order to improve the generalization ability of the model, this application adopts a transfer learning strategy to transfer the weights of the pre-trained model to the new task. In this way, this application can make full use of the knowledge gained from training on large-scale data sets, reduce the number of samples required for training, and improve the learning efficiency and classification accuracy of the model.

[0035] Step S204: training an image classification model.

[0036] like Figure 3 As shown, the method for training an image classification model includes the following steps:

[0037] Step S2042, data preprocessing.

[0038] 1) Generate a Gaussian kernel and perform normalization.

[0039] Assume that the image has been loaded as a matrix I, where I(i,j) represents the pixel value of the image at position (i,j). First, select the size k of the Gaussian kernel (such as a positive odd number such as 3x3, 5x5, 7x7, etc.) and the standard deviation σ. A larger Gaussian kernel will increase the blurring effect, but may lose more details; the standard deviation σ is a measure of the dispersion of the data set. It represents the average deviation between the data points and the mean, reflecting the variability of the data. The larger the standard deviation, the more dispersed the data points are; the smaller the standard deviation, the more concentrated the data points are. The larger the standard deviation σ, the more significant the blurring effect. Next, generate the Gaussian kernel matrix K according to the Gaussian function. The Gaussian kernel matrix is ​​a kernel function used in machine learning, especially in algorithms such as support vector machines (SVM) and Gaussian processes, which is used to map input data to a high-dimensional feature space to better handle nonlinear problems.

[0040]

[0041] in, σ is the standard deviation, which determines the width of the Gaussian distribution, (x, y) is the coordinate of the pixel in the picture relative to the center of the kernel, and k is the size of the Gaussian kernel. The present application uses a Gaussian kernel to process images, which can effectively smooth and reduce noise, reduce details to improve image quality, and the blurring effect is more significant when a larger kernel size and standard deviation are selected. This embodiment not only helps to emphasize the overall structure, but also extracts important features and improves the robustness of subsequent image processing (such as edge detection). In addition, the application of Gaussian kernels can simplify data, improve computational efficiency, and make machine learning algorithms more effective in dealing with complex problems.

[0042] For a k×k Gaussian kernel, whose center is (k / / 2, k / / 2), the Gaussian kernel is normalized to ensure that its weighted sum is 1:

[0043]

[0044] Wherein, m represents the offset of the currently processed pixel position in the horizontal direction, and n represents the offset of the currently processed pixel position in the vertical direction.

[0045] This application can significantly improve the visual effect by applying Gaussian blur when processing images. Assume that the image has been loaded as a matrix I, where each element I (i, j) represents the pixel value at the corresponding position. First, it is necessary to select an appropriate Gaussian kernel size k (such as odd numbers such as 3x3, 5x5, 7x7, etc.) and standard deviation σ. Although a larger Gaussian kernel can enhance the blurring effect, it may lead to the loss of more details, while a larger standard deviation further enhances the intensity of the blur. Next, a Gaussian kernel matrix K is generated based on the Gaussian function, and the weighted value of each pixel is calculated based on its standard deviation and center coordinates. By normalizing the k×k Gaussian kernel to ensure that the weighted sum is 1, blurring is effectively achieved. In this way, not only the smoothness of the image is improved, but also the noise can be effectively reduced, making subsequent image analysis and processing more accurate and stable.

[0046] 2) Perform denoising.

[0047] Perform denoising. Multiscale Gaussian blur smoothes the image by applying Gaussian blur at different scales. It can effectively remove noise and details while retaining important structural information. Multiple Gaussian kernels are applied to the image at different standard deviations to generate images with different blur levels, which can then be combined to achieve a smoother visual effect. Multiscale Gaussian blur is used for feature extraction, edge detection, and image segmentation, etc. In this application, multiscale Gaussian blur captures the features of the image at different scales by applying multiple Gaussian kernels with different standard deviations σ, thereby enhancing the denoising effect. The improved denoising processing formula is:

[0048]

[0049] Among them, I(i,j) is the pixel value of the original image at position (i,j), m,n are the offsets of the horizontal and vertical coordinates relative to the center of the Gaussian kernel, tx is the horizontal gradient of the image at each pixel position, ty is the vertical gradient of the image at each pixel position, β is the gradient weight, and N is the noise intensity parameter, which is used to control the sensitivity of the pixel value difference of the image during the denoising process. It is obtained based on the noise intensity η, the pixel value of the image at position (i,j), and the offset value of the pixel value of the image at position (i,j).

[0050] in,

[0051] This embodiment performs denoising through multi-scale Gaussian blurring, which can effectively enhance image quality. Multiple Gaussian kernels with different standard deviations σ are used to capture the features of the image at different scales, thereby improving the denoising effect. The improved denoising processing formula of this embodiment combines the pixel values ​​of the original image I (i, j) and the offsets m and n relative to the center of the Gaussian kernel, while considering the horizontal and vertical gradients (tx and ty) of each pixel position. By introducing the gradient weight β and the noise intensity parameter N, the sensitivity to the difference in pixel values ​​during the denoising process can be accurately controlled, so that the impact of noise can be reduced during denoising, and the image details can be better preserved. The processing method of this embodiment significantly improves the clarity and visual quality of the image, making subsequent image analysis more accurate and reliable.

[0052] Step S2044, model training.

[0053] This application integrates ResNet50 combined with the attention mechanism with the twin network to form a dual-branch network structure. This structure inputs two images at the same time, generates positive or negative examples, and performs feature extraction and mapping. The extracted feature vector is converted to a one-dimensional form, its L1 distance is calculated, feature comparison and fusion are achieved through the fully connected layer, and finally the matching similarity is output through the Sigmoid function. After that, back propagation is performed through the loss function to optimize the parameters of the feature extraction layer and the fully connected layer to improve the accuracy of the model prediction. The output is compressed to the [0,1] interval after Sigmoid activation, indicating the predicted probability of the positive example. A cross entropy operation is then performed with the true label (y=1 for similarity, y=0 for dissimilarity) to evaluate the difference between the output and the true label.

[0054] This application adopts binary cross entropy loss, calculates the loss by the actual and predicted similarity of the current training data, accumulates the losses of all samples and takes the average to obtain the final loss value. Specifically, this application determines the binary cross entropy loss of the current training data based on the actual similarity and predicted similarity of the current training data in the current training set; accumulates all binary cross entropy losses in the training set and averages all accumulated binary cross entropy losses to obtain the final loss.

[0055] For example, the binary cross entropy loss of the current training data can be determined based on the actual similarity and predicted similarity of the current training data in the current training set, and the corresponding weights, and all binary cross entropy losses in the training set are accumulated and averaged to obtain an average loss; the regularization term Lreg of the image classification model is calculated based on the network parameters and regularization parameter λ of the image classification model; and the final loss is calculated based on the average loss and regularization term of the binary cross entropy loss.

[0056] First, calculate the binary cross entropy loss:

[0057] BCE=-[α·y i ·log(σ(z i ))+β·(1-y i )·log(1-σ(z i ))]

[0058] Binary cross entropy loss is a loss function for binary classification problems, which is used to measure the difference between the probability distribution predicted by the model and the actual label. In the above formula, α and β are weight parameters for positive and negative examples, respectively, which are used to adjust the impact of positive and negative examples on the overall loss to deal with the problem of class imbalance. And α+β=1 keeps the scale of the total loss consistent, ensuring the stability of training and the interpretability of the loss value. Then, the regularization term is calculated.

[0059] The regularization term is used to prevent the model from overfitting by adding an additional term to the loss function to limit the complexity of the model. In the above formula, λ is the regularization parameter. The regularization parameter is a hyperparameter that controls the degree of influence of the regularization term on the loss function. During the model training process, the regularization parameter determines the strength of the penalty term, which in turn affects the complexity and generalization ability of the model. θj is a model parameter that represents the weights and biases between neurons. In each layer, the weight matrix and bias vector are represented as θ and are continuously updated during the training process. Finally, the total loss function after combining the binary cross entropy loss and the regularization term is:

[0060]

[0061] N represents the total number of samples, and yi is the true label of the i-th image pair, which takes the value of 0 or 1, representing negative and positive examples respectively. The predicted output of the model is represented by zi, and σ(zi) is the result after applying the Sigmoid function, which converts the model output into a probability value ranging from 0 to 1. In order to optimize the training process, this application adopts an improved loss function, which reduces the excessive impact of a single sample on the overall training process by adjusting the weight of each sample. In this way, the model can learn more stably when facing outliers or noisy data, thereby improving the robustness and generalization ability of the model. In this way, the contribution of each sample in the training process is balanced to ensure that the overall loss can more accurately reflect the performance of the model.

[0062] In some other embodiments, another improved cross entropy loss function may be used:

[0063]

[0064] Among them, Loss cross-entropy represents the cross entropy loss. ikis the label of the model, indicating the true probability that sample i belongs to category j. ik is the predicted probability output by the model, indicating the model's confidence in predicting that sample i belongs to category k. K is the total number of categories, indicating the number of different categories that the model needs to predict. S is the temperature parameter, which is used to adjust the smoothness of the output probability distribution. β ij It is the model's predicted probability that sample i belongs to category j, reflecting the model's prediction confidence for each category on the sample.

[0065] After that, the gradient is calculated. When calculating the gradient, in addition to using the back-propagation algorithm to calculate the gradient of the loss, a gradient clipping strategy can also be introduced to prevent gradient explosion during training. By setting the threshold ∈, the gradient calculation process can be adjusted:

[0066]

[0067] in, It represents the gradient of the weighted loss function with respect to the model parameter θ, guiding the direction and magnitude of parameter update. is the norm of the gradient, which is used to detect the gradient size to achieve gradient clipping. ∈ is the threshold of gradient clipping. When the norm of the gradient exceeds the threshold, adjustment is made to prevent gradient explosion.

[0068] Finally, the network parameters are updated by calculating the gradient and the clipped value. For example, an optimization algorithm such as stochastic gradient descent (SGD), Adam, or RMSprop can be used to gradually optimize the model parameters by adjusting the learning rate and gradient information, thereby improving the performance of the model on the training set and validation set, and ultimately achieving higher prediction accuracy. The updated parameters will provide a new starting point for the next round of training, forming a cycle of continuous iterative optimization.

[0069] Step S204: model evaluation.

[0070] like Figure 4 As shown, the model evaluation method includes the following steps:

[0071] Step S2042, collecting image data.

[0072] This application uses a dataset specially customized for image similarity matching, which is constructed from video materials collected from multiple video websites and converted into frame images.

[0073] Since there may be temporal changes and perspective diversity during the shooting process, this application introduces the automatic optimization strategy of RandAugment. This strategy achieves data enhancement through parameter adjustment, mainly involving two key parameters: the number of operations (N) and the transformation amplitude (M). RandAugment randomly selects predefined transformation operations and applies them with a uniform transformation amplitude to achieve combined enhancement. These transformation operations include random occlusion, rotation, scaling, flipping, and adjustment of brightness and contrast. The purpose of these enhancement techniques is to simulate the shooting conditions of the same asset at different times and perspectives, thereby helping the model match complex images during the training and testing phases, improving its robustness and reducing the risk of overfitting.

[0074] Step S2044, setting evaluation criteria.

[0075] In the process of setting the evaluation criteria, the self-constructed image dataset must first be processed uniformly, and the size of all original images must be adjusted to 105×105 JPG format. Next, the dataset is divided into training set, evaluation set, and test set in a ratio of 7:2:1. In the training set, the focus will be on optimizing the weight parameters of the model; the evaluation set is used to fine-tune the hyperparameters of the model; and finally, the test set will be used to verify the performance of the model. In order to evaluate the effect of image similarity detection, a quantitative analysis method is adopted, and the main indicators include accuracy, precision, recall, and harmonic mean. These indicators will be evaluated by constructing similar image pairs and dissimilar image pairs. Specifically, if the similarity between the input matching target image and the same image successfully matched in the test set is greater than 0.85, it is judged to be the same asset; otherwise, it is regarded as a different asset.

[0076] Step S2046, evaluating the model.

[0077] The first step involves running the trained model on the test set to obtain similarity match predictions for each pair of images. The model scores the similarity of each pair of images in the test set and classifies the results into “similar images” and “dissimilar images” based on a preset threshold (0.85).

[0078] Next, we will build a confusion matrix to fully evaluate the performance of the model. The confusion matrix consists of four key parts: True Positives (TP), which refers to the number of positive examples that are actually positive and predicted as positive; False Positives (FP), which refers to the number of negative examples that are actually predicted as positive; True Negatives (TN), which refers to the number of negative examples that are actually predicted as negative; False Negatives (FN), which refers to the number of positive examples that are actually predicted as negative. From these values, several important metrics can be calculated.

[0079] The first is accuracy, which measures the correctness of the model's overall predictions. The calculation formula is the sum of TP and TN divided by the number of all samples. Next is precision, which reflects the accuracy of the model in all predicted positive examples. The calculation method is the ratio of TP to FP. Recall measures the model's ability to identify all positive examples. The calculation formula is the ratio of TP to TP plus FN. In addition, the harmonic mean (F1 Score) can also be calculated. It combines precision and recall to provide a comprehensive performance evaluation. The formula is the harmonic mean of precision and recall. Finally, specificity can also be introduced to evaluate the model's ability to correctly identify negative examples. The calculation method is the ratio of TN to TN plus FP.

[0080] After the evaluation is completed, these indicators are counted and analyzed to determine the performance of the model in different scenarios. At the same time, by visualizing the confusion matrix and various indicators, the model performance is made more intuitive. In-depth analysis of false positives and false negatives helps to identify the causes of model misjudgment, thereby providing a clear direction for subsequent optimization. Through these steps, the model evaluation process will be more systematic and clear, providing effective support for further optimization and practical application of the model.

[0081] The present application also provides an evaluation device for an image classification model, such as Figure 5 As shown, the acquisition module 52 is configured to acquire image data and divide the image data into a training set, an evaluation data set and a test set; the evaluation module 54 is configured to evaluate the performance of the image classification model in image similarity detection based on the test set by using a quantitative analysis method and a classification performance indicator, wherein the classification performance indicator includes at least one of the following: accuracy, precision, recall, and harmonic mean.

[0082] It should be noted that the image classification model evaluation device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the image classification model evaluation device provided in the above embodiment and the image classification model evaluation method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0083] Figure 6 FIG. 1 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. It should be noted that: Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0084] like Figure 6 As shown, the electronic device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0085] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0086] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for evaluating an image classification model, characterized in that: include: Acquire image data, and divide the image data into a training set, an evaluation data set, and a test set; Based on the test set, a quantitative analysis method is adopted to use classification performance indicators to evaluate the performance of the image classification model in image similarity detection, wherein the classification performance indicators include at least one of the following: accuracy, precision, recall, and harmonic mean.

2. The method according to claim 1, characterized in that The quantitative analysis method is used to evaluate the performance of the image classification model in image similarity detection using classification performance indicators, including: Based on the binary confusion matrix, determine the predicted positive example TP and the predicted negative example FN when the test data in the test set is a true positive example, and the predicted positive example FP and the predicted negative example TN when the test data in the test set is a true negative example; Determine the accuracy, the precision, and the recall based on at least part of the predicted positive examples TP and the predicted negative examples FN in the case of the true positive examples, and the predicted positive examples FP and the predicted negative examples TN in the case of the true negative examples, and determine the harmonic mean based on the accuracy and the recall; The performance of the image classification model in image similarity detection is evaluated based on the accuracy, the precision, the recall, and the harmonic mean.

3. The method according to claim 1, characterized in that The performance of the image classification model in image similarity detection is evaluated using classification performance indicators, including: Under the condition of fixed variables, adjusting the basic network, optimization strategy and data enhancement of the image classification model respectively; The accuracy, the precision, the recall, and the harmonic mean are calculated for the adjusted base network, the optimization strategy, and the data enhancement, respectively, to evaluate the impact of the base network, the optimization strategy, and the data enhancement on the performance of the image classification model.

4. The method according to claim 1, characterized in that The performance of the image classification model in image similarity detection is evaluated using classification performance indicators, including: under the same test set, Extracting color attributes of the images in the test set using color distribution analysis, extracting texture attributes of the images using a texture analysis method, calculating image similarities based on the extracted color attributes and texture attributes, and calculating the accuracy, precision, recall and harmonic mean of the color distribution analysis and the texture analysis method based on the calculated similarities; The accuracy, precision, recall and harmonic mean of the color distribution analysis and the texture analysis method are compared with the accuracy, precision, recall and harmonic mean of the image classification model to evaluate the image classification model.

5. The method according to claim 1, characterized in that After dividing the image data into a training set, an evaluation data set and a test set, the method further includes: preprocessing the training set, wherein the preprocessing includes applying data enhancement technology to generate diversified training samples by adjusting the number of operations and the magnitude of transformations.

6. The method according to claim 5, characterized in that The data enhancement includes at least one of the following: Changing the brightness of the image so that the image classification model can adapt to images under different lighting conditions; By enhancing or weakening the contrast of the image, the image classification model is enabled to identify detailed features in the image; Rotating the image at random angles to simulate objects at different viewing angles; By changing the size of the image, the image classification model can learn the features of objects at different scales; By flipping the image horizontally or vertically, the image classification model can adapt to changes in image orientation; By simulating partial occlusion in the image, the image classification model learns to make judgments under incomplete information.

7. An evaluation device for an image classification model, characterized in that: include: An acquisition module is configured to acquire image data and divide the image data into a training set, an evaluation data set, and a test set; The evaluation module is configured to evaluate the performance of the image classification model in image similarity detection based on the test set by using a quantitative analysis method and a classification performance indicator, wherein the classification performance indicator includes at least one of the following: accuracy, precision, recall, and harmonic mean.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A computer device, characterized in that: include: Memory and processor, The memory stores a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is run, the processor is enabled to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images