Image classification methods, devices, electronic devices, and readable storage media
By training a deep learning network using the teacher-student network learning method and optimizing the model through reinforcement loop training, the problem of low image classification accuracy of deep learning networks is solved, and high-accuracy classification of target objects in target images is achieved.
Patent Information
- Application Number
- CN202110910053.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-09
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-08-09
AI Technical Summary
The accuracy of existing deep learning networks in image classification needs to be improved.
The first classification model is obtained by training the second classification model using its output, and the weights of the second classification model are updated using the output of the first classification model. The training process is optimized by using a teacher-student network learning method for reinforcement loop training.
It significantly improves the classification accuracy of target objects in target images and enhances the accuracy of image classification.
Smart Images

Figure CN115705691B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and more specifically to an image classification method, apparatus, electronic device, and readable storage medium. Background Technology
[0002] Image classification is an image processing method that distinguishes different categories of objects based on the different features they reflect in image information. It uses computers to perform quantitative analysis of images, classifying each pixel or region in an image into one of several categories, thus replacing human visual interpretation.
[0003] Currently, deep learning networks are mostly used for image classification, but the accuracy of deep learning networks in image classification needs to be improved. Summary of the Invention
[0004] Based on the above research, embodiments of the present invention provide an image classification method, apparatus, electronic device, and readable storage medium, which improve the accuracy of image classification.
[0005] The embodiments of the present invention can be implemented through the following aspects:
[0006] In a first aspect, embodiments of the present invention provide an image classification method, the method comprising:
[0007] Obtain the target image to be classified;
[0008] The target image is input into a preset first classification model, and the target image is processed by the first classification model to obtain the classification result of the target object in the target image; wherein, the first classification model is trained by using the output of a second classification model, and the second classification model updates its weights by using the output of the first classification model.
[0009] In an optional implementation, before inputting the target image into a preset classification model, the method further includes:
[0010] Acquire sample data, which includes a training sample set and a validation sample set;
[0011] Based on the classification results of the initial second classification model on the training sample set, the initial first classification model is trained to obtain the intermediate first classification model;
[0012] The first evaluation index of the initial second classification model is calculated based on the classification results of the validation sample set by the intermediate first classification model, and the weights of the initial second classification model are updated to obtain the intermediate second classification model.
[0013] Based on the classification results of the intermediate second classification model on the training sample set, the intermediate first classification model is trained, and based on the classification results of the trained intermediate first classification model on the validation sample set, the second evaluation index of the intermediate second classification model is calculated.
[0014] Based on the first evaluation index and the second evaluation index, the training endpoint is determined, and the first classification model and the second classification model are obtained.
[0015] In an optional implementation, the step of updating the weights of the initial second classification model based on the classification results of the validation sample set by the intermediate first classification model to obtain the intermediate second classification model includes:
[0016] The validation sample set is input into the initial second classification model to obtain the classification result of the initial second classification model on the validation sample set;
[0017] Calculate the loss between the initial second classification model's classification result on the validation sample set and the intermediate first classification model's classification result on the validation sample set; wherein, the intermediate first classification model's classification result on the validation sample set is the label value of the validation sample set;
[0018] Backpropagation of the loss yields the first gradient of the initial second classification model.
[0019] The weights of the initial second classification model are updated based on the first gradient to obtain an intermediate second classification model.
[0020] In an optional implementation, the step of updating the weights of the initial second classification model based on the first gradient to obtain the intermediate second classification model includes:
[0021] Differentiating the first gradient yields the second gradient;
[0022] A perturbation within a defined neighborhood is applied to the second gradient to obtain the neighborhood gradient;
[0023] The weights of the initial second classification model are updated based on the first gradient and the neighborhood gradient to obtain an intermediate second classification model.
[0024] In an optional implementation, the step of calculating the first evaluation index of the initial second classification model based on the classification results of the validation sample set by the intermediate first classification model includes:
[0025] The validation sample set is input into the initial second classification model to obtain the classification result of the initial second classification model on the validation sample set;
[0026] The accuracy of the initial second classification model is obtained by comparing the classification results of the initial second classification model on the validation sample set with the classification results of the intermediate first classification model on the validation sample set.
[0027] Based on the accuracy of the initial second classification model, the first evaluation index of the initial second classification model is obtained.
[0028] In an optional implementation, the step of training the initial first classification model based on the classification results of the initial second classification model on the training sample set to obtain the intermediate first classification model includes:
[0029] The training sample set is input into the initial first classification model to obtain the classification result of the initial first classification model on the training sample set;
[0030] Calculate the loss between the classification result of the initial first classification model on the training sample set and the classification result of the initial second classification model on the training sample set; wherein, the classification result of the initial second classification model on the training sample set is the label value of the training sample set;
[0031] Determine whether the loss meets the first preset condition;
[0032] If the condition is not met, the loss is backpropagated, the weights of the initial first classification model are adjusted, and this process is repeated until the loss meets the first preset condition, thus obtaining the intermediate second classification model.
[0033] In an optional implementation, the step of calculating the second evaluation index of the intermediate second classification model based on the classification results of the intermediate first classification model on the validation sample set after training includes:
[0034] The validation sample set is input into the intermediate second classification model to obtain the classification result of the intermediate second classification model on the validation sample set;
[0035] The accuracy of the intermediate second classification model is obtained by comparing the classification results of the intermediate second classification model on the validation sample set with the classification results of the trained intermediate first classification model on the validation sample set.
[0036] Based on the accuracy of the intermediate second classification model, a second evaluation index for the intermediate second classification model is obtained.
[0037] In an optional implementation, the step of determining the training endpoint based on the first evaluation index and the second evaluation index to obtain the first classification model and the second classification model includes:
[0038] Calculate the difference between the first evaluation index and the second evaluation index;
[0039] Determine whether the difference satisfies the second preset condition;
[0040] If the conditions are met, the first classification model and the second classification model are obtained;
[0041] If the condition is not met, the weights of the intermediate second classification model are updated based on the classification results of the intermediate first classification model on the validation sample set after training. The intermediate first classification model is then trained again based on the classification results of the intermediate second classification model on the training sample set after weight update. This process is iterated until the difference between the first evaluation index and the second evaluation index meets the second preset condition, thus obtaining the first classification model and the second classification model.
[0042] In an optional implementation, before training the initial first classification model based on the classification results of the initial second classification model on the training sample set, the method further includes:
[0043] The training sample set is input into a preset original classification model to obtain the classification result of the original classification model on the training sample set;
[0044] Calculate the loss between the classification result of the original classification model on the training sample set and the original label values of the training sample set;
[0045] Determine whether the loss meets the third preset condition;
[0046] If the condition is not met, the loss is backpropagated, the weights of the original classification model are adjusted, and this process is repeated until the loss meets the third preset condition to obtain the initial second classification model.
[0047] Secondly, embodiments of the present invention provide an image classification device, comprising:
[0048] The image acquisition module is used to acquire the target image to be classified.
[0049] The image classification module is used to input the target image into a preset first classification model, and to perform feature processing on the target image through the first classification model to obtain the classification result of the target object in the target image; wherein, the first classification model is obtained by training with the output of a second classification model, and the second classification model is obtained by training with the output of the first classification model.
[0050] Thirdly, embodiments of the present invention provide an electronic device, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image classification method described in any of the foregoing embodiments.
[0051] Fourthly, embodiments of the present invention provide a readable storage medium storing a computer program, which, when executed by a processor, implements the image classification method described in any of the foregoing embodiments.
[0052] The image classification method, apparatus, electronic device, and readable storage medium provided in this invention obtain a first classification model by training it with the output of a second classification model, and update the weights of the second classification model with the output of the first classification model. In this way, the first classification model is obtained through reinforcement loop training. After acquiring the target image to be classified, the target image is input into the first classification model for feature processing. When classifying the target objects in the target image, the classification accuracy of the target objects in the target image can be greatly improved. Attached Figure Description
[0053] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0054] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.
[0055] Figure 2 This is a schematic flowchart of an image classification method provided in an embodiment of the present invention.
[0056] Figure 3 This is another schematic diagram of the image classification method provided in an embodiment of the present invention.
[0057] Figure 4 This is a block diagram of an image classification device provided in an embodiment of the present invention.
[0058] Icons: 100 - Electronic device; 10 - Image classification device; 11 - Image acquisition module; 12 - Image classification module; 20 - Memory; 30 - Processor; 40 - Communication unit. Detailed Implementation
[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0060] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0061] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0062] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0063] As described in the background section, deep learning networks are currently widely used for image classification. However, the accuracy of deep learning networks in image classification needs improvement. Therefore, to improve the accuracy of image classification, optimization is usually performed during the training of deep learning networks. Optimization of deep learning networks is mostly achieved through the following methods: 1. Modifying the network by adding more structures; 2. Using different data processing methods to optimize samples; 3. Optimizing the loss function by using different loss functions to stretch intra-class and inter-class distances; 4. Using different training methods to improve performance. Of the above optimization methods, the first one modifies the network structure, thus affecting the final model size and execution performance. The second, third, and fourth methods do not modify the network structure and improve the network's expressive power and performance.
[0064] Based on the above research, this embodiment provides an image classification method, apparatus, electronic device, and readable storage medium. A first classification model is obtained by training with the output of a second classification model, and the weights of the second classification model are updated with the output of the first classification model. In this way, the training process of the model is optimized by strengthening the iterative training method. After acquiring the target image to be classified, the target image is input into the first classification model for feature processing. When classifying the target objects in the target image, the classification accuracy of the target objects in the target image can be greatly improved.
[0065] Please see Figure 1 , Figure 1 This is a structural block diagram of an electronic device 100 provided in this embodiment. The electronic device 100 includes an image classification device 10, a memory 20, a processor 30, and a communication unit 40. The memory 20 stores machine-readable instructions that can be executed by the processor 30. When the electronic device 100 is running, the processor 30 communicates with the memory 20 through a bus, and the processor 30 executes the machine-readable instructions and performs the image classification method.
[0066] The memory 20, processor 30, and communication unit 40 are electrically connected directly or indirectly to each other to achieve signal transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The image classification device 10 includes at least one software function module that can be stored in the memory 20 in the form of software or firmware. The processor 30 is used to execute the executable module (e.g., the software function module or computer program included in the image classification device 10) stored in the memory 20.
[0067] The memory 20 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.
[0068] In some embodiments, processor 30 is used to perform one or more functions described in this embodiment. In some embodiments, processor 30 may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (S)). By way of example only, processor 30 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction-set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computing (RISC) computer, or a microprocessor, or any combination thereof.
[0069] In this embodiment, the memory 20 is used to store the program, and the processor 30 is used to execute the program after receiving the execution instruction. The process definition method disclosed in any implementation of this embodiment can be applied to the processor 30, or implemented by the processor 30.
[0070] The communication unit 40 is used to establish a communication connection between the electronic device 100 and other devices via a network, and to send and receive data via the network.
[0071] In some implementations, the network can be any type of wired or wireless network, or a combination thereof. By way of example only, the network may include wired networks, wireless networks, fiber optic networks, telecommunications networks, intranets, the Internet, local area networks (LANs), wide area networks (WANs), wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), Bluetooth networks, ZigBee networks, or near field communication (NFC) networks, or any combination thereof.
[0072] In this embodiment, the electronic device 100 may be, but is not limited to, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or other electronic devices. This embodiment does not impose any restrictions on the specific type of electronic device.
[0073] Understandably, Figure 1 The structure shown is for illustrative purposes only. The electronic device 100 may also have... Figure 1 Showing more or fewer components, or having with Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.
[0074] based on Figure 1 The implementation architecture of this embodiment provides an image classification method, which is based on... Figure 1 The electronic device shown performs the following detailed explanation of the steps of the image classification method provided in this embodiment. Please refer to the reference below. Figure 2 The image classification method provided in this embodiment includes steps S101 to S102.
[0075] Step S101: Obtain the target image to be classified.
[0076] Optionally, the target image can be obtained by taking a picture of the target object to be monitored using a camera device. After the camera device takes the picture of the target image, it can send the target image to an electronic device.
[0077] In this embodiment, the target image can be obtained from surveillance video captured by a camera device. Optionally, the target image can be obtained by directly selecting multiple consecutive frames from the surveillance video, or by dividing the surveillance video into multiple consecutive sub-videos of equal length, and then selecting an equal number of frames from each sub-video to form the target image.
[0078] To achieve real-time image classification, in this embodiment, two queues of N frames (assuming the target image includes N frames) can be maintained: an S queue and a Q queue. S is the working queue, and Q is the buffer queue. The frames in the buffer queue are new frames from the surveillance video. Whenever the Q queue is full of N frames, N / 2 frames are uniformly sampled from the S queue and the Q queue to form a new S queue. Then, the target image is obtained from the S queue.
[0079] After obtaining the target image, step S102 can be performed to classify the target image.
[0080] Step S102: Input the target image into the preset first classification model, perform feature processing on the target image through the first classification model, and obtain the classification result of the target object in the target image.
[0081] In this embodiment, the first classification model can be a model for action recognition, such as the ECO model (an action classification network based on 3D convolution), and the second classification model can be a model with a larger network structure and better performance, such as the TANet model (a 3D point cloud object detection framework).
[0082] In this embodiment, a teacher-student network learning method is used to train a first classification model and a second classification model. The second classification model can be used as a teacher model, and the first classification model can be used as a student model. The first classification model is trained by using the output of the second classification model, and the second classification model updates its weights by using the output of the first classification model. In this way, the learning process is repeated to steadily improve the ability of the first classification model.
[0083] After obtaining the first classification model, the target image can be input into the first classification model, which can then perform feature processing on the target image to obtain the classification result of the target object in the target image.
[0084] Using the first classification model as the ECO model, the target image comprises multiple frames for illustration. After inputting the target image into the first classification model, a conventional 2D convolutional network is used for feature processing of each frame to obtain a feature map corresponding to each frame. After obtaining the feature maps for each frame, the feature maps are stacked to obtain a stacked feature map. Then, a 3D convolutional network is used to process the stacked feature map, outputting a one-dimensional vector corresponding to the number of categories, which is the classification result of the target image.
[0085] In practical applications, the action classification network takes 10 frames of image data as input, performs 3D convolution, and outputs a 5-class classification result. Due to data complexity, the final classification accuracy can only reach an optimal 81%, which cannot be improved. Therefore, this embodiment utilizes a teacher-student network learning method for training to improve the classification accuracy of the first classification model. Please refer to [reference needed]. Figure 3 The training process for the first classification model and the second classification model can be as follows:
[0086] Step S201: Obtain sample data, which includes training sample set and validation sample set.
[0087] Step S202: Based on the classification results of the initial second classification model on the training sample set, train the initial first classification model to obtain the intermediate first classification model.
[0088] Step S203: Calculate the first evaluation index of the initial second classification model based on the classification results of the validation sample set by the intermediate first classification model, and update the weights of the initial second classification model to obtain the intermediate second classification model.
[0089] Step S204: Based on the classification results of the intermediate second classification model on the training sample set, train the intermediate first classification model, and calculate the second evaluation index of the intermediate second classification model based on the classification results of the trained intermediate first classification model on the validation sample set.
[0090] Step S205: Determine the training endpoint based on the first evaluation index and the second evaluation index to obtain the first classification model and the second classification model.
[0091] After obtaining the sample data, it can be divided into a training sample set and a validation sample set. Optionally, the training sample set and the validation sample set can be divided according to a set ratio, for example, the ratio of the training sample set to the validation sample set can be 80%:20%, or any other set ratio. This embodiment does not impose specific limitations and can be set according to actual needs.
[0092] In this embodiment, the initial second classification model is a trained model with superior performance. To obtain the initial second classification model, before training the initial first classification model based on the classification results of the initial second classification model on the training sample set, the image classification method provided in this embodiment further includes:
[0093] The training sample set is input into the preset original classification model to obtain the classification result of the original classification model on the training sample set.
[0094] Calculate the loss of the original classification model on the training sample set compared to the original label values of the training sample set.
[0095] Determine whether the loss meets the third preset condition.
[0096] If the condition is not met, the loss is backpropagated, the weights of the original classification model are adjusted, and this process is repeated until the loss meets the third preset condition, thus obtaining the initial second classification model.
[0097] After the sample data is divided, the original classification model can be trained using the training sample set to obtain the initial second classification model.
[0098] In this embodiment, the original classification model is a classification model with a large network structure. By training the original classification model, a second classification model with better performance can be obtained.
[0099] In this embodiment, the training sample set contains labeled original label values. After inputting the training sample set into a preset original classification model and obtaining the classification result of the original classification model on the training sample set, the loss between the classification result of the original classification model on the training sample set and the original label values of the training sample set can be calculated. Then, it is determined whether the loss meets a third preset condition. If the third preset condition is met, the structure of the current original classification model is saved to obtain an initial second classification model. If the third preset condition is not met, the weights of the original classification model are adjusted in reverse using the loss, and the loss between the classification result of the original classification model on the training sample set and the original label values of the training sample set is calculated again after the weight adjustment. It is then determined whether the loss obtained from the recalculation meets the third preset condition. If it is met, the structure of the original classification model after the weight adjustment is saved to obtain an initial second classification model. If it is not met, the recalculated loss is backpropagated, and the weights of the original classification model after the weight adjustment are adjusted again. This process is iterated until the loss meets the third preset condition to obtain an initial second classification model.
[0100] Optionally, in this embodiment, when calculating the loss between the classification result of the original classification model on the training sample set and the original label value of the training sample set, any loss function can be used, such as the cross-entropy loss function, the logarithmic loss function, the exponential loss function, etc. This embodiment does not impose specific restrictions and can be set according to actual needs.
[0101] Optionally, in this embodiment, the third preset condition can be reaching a minimum value or convergence, etc. This embodiment does not impose specific restrictions and can be set according to actual needs.
[0102] After obtaining the initial second model, the training sample set can be input into the initial second classification model. Forward inference is then performed through the initial second classification model to obtain the result of the forward inference, which is the classification result of the initial second classification model on the training sample set.
[0103] In this embodiment, the initial second classification model classifies the training sample set into a 1*N vector, where N is the number of categories. When N=5, the output is a 1*5 vector, including the probability values for the 5 categories.
[0104] After obtaining the classification results of the initial second classification model on the training sample set, the initial first classification model can be trained based on the classification results of the initial second classification model on the training sample set to obtain the intermediate first classification model.
[0105] In this embodiment, the step of training the initial first classification model based on the classification results of the initial second classification model on the training sample set to obtain the intermediate first classification model may include:
[0106] (2-1) Input the training sample set into the initial first classification model to obtain the classification result of the initial first classification model on the training sample set.
[0107] (2-2) Calculate the loss between the initial first classification model and the initial second classification model on the training sample set.
[0108] (2-3) Determine whether the loss meets the first preset condition.
[0109] (2-4) If the condition is not met, the loss is backpropagated and the weights of the initial first classification model are adjusted. This process is repeated until the loss meets the first preset condition, and an intermediate first classification model is obtained.
[0110] Specifically, when training the initial first classification model based on the classification results of the initial second classification model on the training sample set, the classification results of the initial second classification model on the training sample set are used as new label values for the training sample set and input into the initial first classification model along with the training sample set.
[0111] After inputting the training sample set into the initial first classification model and obtaining the classification result of the initial first classification model on the training sample set, the classification result of the initial first classification model on the training sample set can be compared with the classification result of the initial second classification model on the training sample set, and the loss of the classification result of the initial first classification model on the training sample set and the classification result of the initial second classification model on the training sample set can be calculated.
[0112] Understandably, when calculating the loss between the initial first classification model and the initial second classification model on the training sample set, any loss function can be used, such as the cross-entropy loss function, the logarithmic loss function, the exponential loss function, etc. This embodiment does not impose specific restrictions and can be set according to actual needs.
[0113] In this embodiment, the classification result of the initial first classification model on the training sample set is used as the predicted value, and the classification result of the initial second classification model on the training sample set is used as the label value. After calculating the loss of the classification results of the initial first classification model and the initial second classification model on the training sample set, it is determined whether the loss meets a first preset condition. If the loss meets the first preset condition, an intermediate first classification model is obtained. If the loss does not meet the first preset condition, the loss is backpropagated, the weights of the initial first classification model are adjusted, and then the loss of the classification results of the initial first classification model and the initial second classification model on the training sample set after weight adjustment is calculated again. It is then determined whether the loss meets the first preset condition again. If it does, the structure of the initial first classification model after weight adjustment is saved, and an intermediate first classification model is obtained. If it does not meet the condition, the loss calculated again is backpropagated, and the weights of the initial first classification model after weight adjustment are adjusted again. This process is iterated until the loss meets the first preset condition, and an intermediate first classification model is obtained.
[0114] Understandably, in this embodiment, the first preset condition may be reaching a minimum value or convergence, etc. This embodiment does not impose specific restrictions and can be set according to actual needs.
[0115] In this embodiment, when the classification result of the initial second classification model on the training sample set is used as the label value, the classification result of the initial second classification model on the training sample set is a soft label containing information about multiple categories. Compared with a one-hot label containing only one category information generated directly using the category result, the classification result of the initial second classification model on the training sample set is richer in information. Therefore, using the classification result of the initial second classification model on the training sample set as the label value to train the initial first classification model allows the initial first classification model to learn richer information. Understandably, in this embodiment, the classification result of the initial first classification model on the training sample set is also a 1*N vector.
[0116] After obtaining the intermediate first classification model, the validation sample set is input into the intermediate first classification model. By performing forward inference through the intermediate first classification model, the classification result of the intermediate first classification model on the validation sample set can be obtained.
[0117] After obtaining the classification results of the intermediate first classification model on the validation sample set, the first evaluation index of the initial second classification model can be calculated based on the classification results of the intermediate first classification model on the validation sample set, and the weights of the initial second classification model can be updated based on the classification results of the intermediate first classification model on the validation sample set.
[0118] In this embodiment, the step of calculating the first evaluation index of the initial second classification model based on the classification results of the intermediate first classification model on the validation sample set may include:
[0119] The validation sample set is input into the initial second classification model to obtain the classification result of the initial second classification model on the validation sample set.
[0120] The accuracy of the initial second classification model is obtained by comparing the classification results of the initial second classification model on the validation sample set with the classification results of the intermediate first classification model on the validation sample set.
[0121] Based on the accuracy of the initial second classification model, the first evaluation index of the initial second classification model is obtained.
[0122] Specifically, when calculating the first evaluation index of the initial second classification model based on the classification results of the intermediate first classification model on the validation sample set, the classification results of the intermediate first classification model on the validation sample set are used as the true values, and the classification results of the initial second classification model on the validation sample set are used as the predicted values. Then, the classification results of the intermediate first classification model on the validation sample set are compared with the classification results of the initial second classification model on the validation sample set to obtain the classification accuracy of the initial second classification model.
[0123] It should be noted that the classification result of the intermediate first classification model on the validation sample set is a 1*N vector, including probability values for N categories. Similarly, the classification result of the initial second classification model on the validation sample set is also a 1*N vector, including probability values for N categories. To compare the classification results of the intermediate first classification model and the initial second classification model on the validation sample set, the category with the highest probability value in the intermediate first classification model's result is compared with the category with the highest probability value in the initial second classification model's result. If they match, the initial second classification model is considered accurate; otherwise, it is considered incorrect. This is how the accuracy of the initial second classification model is obtained.
[0124] After obtaining the accuracy of the initial second classification model, the first evaluation index of the initial second classification model can be obtained based on the accuracy of the initial second classification model.
[0125] In one alternative implementation, after obtaining the accuracy of the initial second classification model, the accuracy of the initial second classification model can be directly used as the first evaluation index of the initial second classification model.
[0126] In an optional implementation, in order to accurately evaluate the performance of the initial second classification model, in this embodiment, the confusion matrix of the initial second classification model can be obtained based on the classification results of the initial second classification model on the validation sample set and the classification results of the intermediate first classification model on the validation sample set. The true positive rate, negative positive rate, and true negative rate of the initial second classification model are obtained based on the confusion matrix. Then, the AUC (Area Under Curve) index of the initial second classification model is calculated based on the true positive rate, negative positive rate, and true negative rate, and the AUC index is used as the evaluation index of the initial second classification model.
[0127] Accordingly, in this embodiment, the step of updating the weights of the initial second classification model based on the classification results of the validation sample set by the intermediate first classification model to obtain the intermediate second classification model includes:
[0128] (3-1) Input the validation sample set into the initial second classification model to obtain the classification result of the initial second classification model on the validation sample set.
[0129] (3-2) Calculate the loss between the initial second classification model and the intermediate first classification model on the validation sample set; where the classification result of the intermediate first classification model on the validation sample set is the label value of the validation sample set.
[0130] (3-3) Backpropagate the loss to obtain the first gradient of the initial second classification model.
[0131] (3-4) Update the weights of the initial second classification model according to the first gradient to obtain the intermediate second classification model.
[0132] Specifically, when updating the weights of the initial second classification model based on the classification results of the intermediate first classification model on the validation sample set, the classification results of the intermediate first classification model on the validation sample set can be used as the true label values and input into the initial second classification model along with the validation sample set.
[0133] After inputting the validation sample set into the initial second classification model and obtaining the classification result of the initial second classification model on the validation sample set, the loss of the classification result of the initial second classification model on the validation sample set and the classification result of the intermediate first classification model on the validation sample set can be calculated.
[0134] Understandably, when calculating the loss of the initial second classification model on the validation sample set and the intermediate first classification model on the validation sample set, any loss function can be used, such as the cross-entropy loss function, the log loss function, the exponential loss function, etc. This embodiment does not impose specific restrictions and can be set according to actual needs.
[0135] After obtaining the classification results of the initial second classification model on the validation sample set and the loss of the intermediate first classification model on the validation sample set, the loss is backpropagated to obtain the first gradient of the initial second classification model. Then, the weights of the initial second classification model can be updated based on the first gradient to obtain the intermediate second classification model.
[0136] In an optional implementation, the weights of the initial second classification model are updated based on the first gradient using the following formula:
[0137] w t+1 =w t -ng
[0138] Among them, w t+1 The weights w are the updated weights of the initial second classification model. t denoted as the current weights of the initial second classification model, n is the learning rate of the initial second classification model, and g is the first gradient.
[0139] After obtaining the gradient, directly updating the weights of the initial second classification model based on the gradient is unlikely to reach the global minimum. To quickly reach the global minimum, in this embodiment, the steps of updating the weights of the initial second classification model based on the first gradient to obtain the intermediate second classification model include:
[0140] Differentiating the first gradient yields the second gradient.
[0141] A perturbation within a defined neighborhood is applied to the second gradient to obtain the neighborhood gradient.
[0142] Based on the first gradient and the neighborhood gradient, the weights of the initial second classification model are updated to obtain the intermediate second classification model.
[0143] In this embodiment, after obtaining the first gradient, its derivative can be calculated to obtain the second gradient. After obtaining the second gradient, a perturbation within a defined neighborhood can be applied to it to obtain the neighborhood gradient. The neighborhood radius can be set according to actual needs, and can be 0.1, 0.2, 0.3, etc.; this embodiment does not impose specific limitations. By applying a perturbation within the neighborhood to the second gradient, a gradient with minimal loss can be obtained within the neighborhood radius.
[0144] After obtaining the neighborhood gradient, the weights of the initial second classification model can be updated based on the first gradient and the neighborhood gradient.
[0145] In this embodiment, after obtaining the neighborhood gradient, the target gradient can be calculated based on the first gradient and the neighborhood gradient, and then the weights of the initial second classification model can be updated based on the target gradient.
[0146] In this embodiment, the target gradient can be calculated based on the first gradient and the neighborhood gradient as follows:
[0147] g new =g+Δg′
[0148] Among them, g new Let g represent the target gradient, g represent the first gradient, Δg′ represent the neighborhood gradient, and Δ represent the perturbation applied in the neighborhood.
[0149] After obtaining the target gradient, it can be substituted into W. t+1 =w t In the -ng formula, we get w t+1 =w t -n*g new This is used to update the weights of the initial second classification model.
[0150] In this embodiment, by differentiating the first gradient to obtain the second gradient, the descent method of the gradient can be obtained. By applying a perturbation within a set neighborhood to the second gradient, the neighborhood gradient is obtained, thus the direction of the gradient is obtained. Then, by using the first gradient and the neighborhood gradient, the weights of the initial second classification model are updated, which can make the gradient smoother and reach the global minimum faster.
[0151] It should be noted that in this embodiment, when updating the weights of the second classification model based on the calculated first gradient and neighborhood gradient, the weights of the second classification model are only updated once. However, when training the first classification model based on the classification results of the training sample set using the second classification model, it is an iterative training process, and the weights of the first classification model will be updated multiple times.
[0152] After updating the weights of the initial secondary classification model, an intermediate secondary classification model is obtained. The training sample set is then input into this intermediate model, and forward inference is performed to obtain the classification result of the intermediate secondary classification model on the training sample set.
[0153] After obtaining the classification results of the intermediate second classification model on the training sample set, the intermediate first classification model can be trained again based on the classification results of the intermediate second classification model on the training sample set. That is, the classification results of the intermediate second classification model on the training sample set are used as the label values of the training sample set and input into the intermediate first classification model along with the training sample set.
[0154] After obtaining the classification results of the intermediate first classification model on the training sample set, the classification results of the intermediate first classification model on the training sample set are compared with those of the intermediate second classification model. The loss of the intermediate first classification model on the training sample set and the intermediate second classification model on the training sample set are calculated, and it is determined whether the loss meets the first preset condition. If the loss meets the first preset condition, the trained intermediate first classification model is obtained. If the loss does not meet the first preset condition, the loss is backpropagated, the weights of the intermediate first classification model are adjusted, and then the loss of the intermediate first classification model on the training sample set after weight adjustment and the intermediate second classification model on the training sample set are calculated again. The loss is again determined whether it meets the first preset condition. If it does, the structure of the intermediate first classification model after weight adjustment is saved, and the trained intermediate first classification model is obtained. If it does not meet the condition, the loss calculated again is backpropagated, the weights of the intermediate first classification model after weight adjustment are adjusted again, and this process is iterated until the loss meets the first preset condition, and the trained intermediate first classification model is obtained.
[0155] After obtaining the trained intermediate first classification model, the validation sample set is input into the trained intermediate first classification model. Forward inference is performed through the trained intermediate first classification model to obtain the classification result of the trained intermediate first classification model on the validation sample set.
[0156] After obtaining the classification results of the trained intermediate first classification model on the validation sample set, the second evaluation index of the intermediate second classification model can be calculated based on the classification results of the trained intermediate first classification model on the validation sample set.
[0157] In this embodiment, the step of calculating the second evaluation index of the intermediate second classification model based on the classification results of the intermediate first classification model on the validation sample set after training includes:
[0158] The validation sample set is input into the intermediate second classification model to obtain the classification result of the intermediate second classification model on the validation sample set.
[0159] The accuracy of the intermediate second classification model is obtained by comparing the classification results of the intermediate second classification model on the validation sample set with the classification results of the trained intermediate first classification model on the validation sample set.
[0160] Based on the accuracy of the intermediate second classification model, a second evaluation index for the intermediate second classification model is obtained.
[0161] Specifically, when calculating the second evaluation index of the intermediate second classification model based on the classification results of the intermediate first classification model on the validation sample set after training, the classification results of the intermediate first classification model on the validation sample set after training are used as the true values, and the classification results of the intermediate second classification model on the validation sample set are used as the predicted values. The accuracy of the intermediate second classification model can be obtained by comparing the classification results of the intermediate first classification model on the validation sample set and the classification results of the intermediate second classification model on the validation sample set after training.
[0162] In this embodiment, the method of obtaining the second evaluation index of the intermediate second classification model based on the accuracy of the intermediate second classification model is the same as the method of obtaining the first evaluation index of the initial second classification model based on the accuracy of the initial second classification model. It will not be elaborated on here. Please refer to the process of obtaining the first evaluation index of the initial second classification model based on the accuracy of the initial second classification model described above.
[0163] After obtaining the second evaluation index of the intermediate second classification model, the training of the model can be determined based on the first and second evaluation indices, i.e., whether the training endpoint has been reached.
[0164] In this embodiment, the steps of determining the training endpoint based on the first evaluation index and the second evaluation index, and obtaining the first classification model and the second classification model, include:
[0165] Calculate the difference between the first evaluation index and the second evaluation index.
[0166] Determine whether the difference satisfies the second preset condition.
[0167] If the conditions are met, we obtain the first classification model and the second classification model.
[0168] If the condition is not met, the weights of the intermediate second classification model are updated based on the classification results of the intermediate first classification model on the validation sample set after training. The intermediate first classification model is then trained again based on the classification results of the intermediate second classification model on the training sample set after weight update. This process is iterated until the difference between the first evaluation index and the second evaluation index meets the second preset condition, thus obtaining the first classification model and the second classification model.
[0169] Specifically, after obtaining the first evaluation index and the second evaluation index, the difference between the first evaluation index and the second evaluation index is calculated, and it is determined whether the difference between the first evaluation index and the second evaluation index meets the second preset condition.
[0170] Optionally, in this embodiment, the second preset condition can be less than a set threshold. When the difference between the first evaluation index and the second evaluation index meets the second preset condition, that is, when the difference between the first evaluation index and the second evaluation index is less than the set threshold, the intermediate second classification model and the initial second classification model have stable accuracy performance on the validation sample set, and the performance of the first classification model and the second classification model is stable. Therefore, the training process of the model can be stopped, the structure of the current intermediate second classification model can be saved to obtain the second classification model, and the structure of the current trained intermediate first classification model can be saved to obtain the first classification model. When the difference between the first evaluation metric and the second evaluation metric does not meet the second preset condition, that is, when the difference between the first evaluation metric and the second evaluation metric is not less than the set threshold, the accuracy of the intermediate second classification model and the initial second classification model on the validation sample set is unstable, and the performance of the first classification model and the second classification model is unstable. Therefore, it is necessary to continue training the model. Then, based on the classification results of the validation sample set by the trained intermediate first classification model, the weights of the intermediate second classification model are updated, and based on the classification results of the training sample set by the intermediate second classification model with updated weights, the trained intermediate first classification model is trained again. This process is repeated iteratively until the difference between the first evaluation metric and the second evaluation metric meets the second preset condition, and the first classification model and the second classification model are obtained. That is, during the iterative training process, when the evaluation metric performance of the second classification model on the validation set is stable between two iterations, training stops.
[0171] Optionally, in this embodiment, the threshold can be set according to actual needs, such as 0.05, 0.1, etc., and this embodiment does not make specific limitations.
[0172] Understandably, updating the weights of the intermediate second classification model based on the classification results of the intermediate first classification model on the validation set is the same as updating the weights of the initial second classification model based on the classification results of the intermediate first classification model on the validation set. This will not be elaborated upon further here; please refer to the process described above for updating the weights of the initial second classification model based on the classification results of the intermediate first classification model on the validation set.
[0173] The process of training the intermediate first classification model based on the classification results of the training sample set by the intermediate second classification model after weight update is the same as the process of training the initial first classification model based on the classification results of the initial second classification model on the training sample set. It will not be elaborated on here. Please refer to the above process of training the initial first classification model based on the classification results of the initial second classification model on the training sample set.
[0174] After obtaining the first classification model and the second classification model, this embodiment can utilize the first classification model to perform image classification in various fields. For example, in the logistics field, it can identify whether a deliveryman's actions while handling packages violate regulations, that is, classify the deliveryman's actions and determine whether the deliveryman has engaged in illegal actions such as throwing or kicking packages. Specifically, target images including the deliveryman's actions can be obtained from surveillance videos. These target images are then input into the trained first classification model. The first classification model performs feature processing on the target images, classifies the deliveryman's actions in the target images, and obtains the classification results of the deliveryman's actions in the target images. This allows for the detection of illegal operations such as throwing or kicking packages while handling them.
[0175] The image classification method provided in this embodiment trains the first classification model based on the output of the second classification model, and adjusts the weights of the second classification model based on the output of the first classification model, thereby entering a loop training to form a reinforcement learning loop and achieve stable improvement in model performance.
[0176] The image classification method provided in this embodiment utilizes validation sample set data for gradient backpropagation, thereby optimizing the training process using the validation sample set data and further improving model performance. By updating weights through neighborhood gradients with consistent low loss, the model can be better minimized, further optimizing model performance. Without changing the model size, the model accuracy has been steadily improved, by at least 5%.
[0177] Based on the same inventive concept, please refer to the following: Figure 4 This embodiment provides an image classification device 10, which is applied to... Figure 1The electronic devices shown, such as Figure 4 As shown, the image classification device 10 provided in this embodiment includes an image acquisition module 11 and an image classification module 12.
[0178] Image acquisition module 11 is used to acquire the target image to be classified.
[0179] The image classification module 12 is used to input the target image into a preset first classification model, and to perform feature processing on the target image through the first classification model to obtain the classification result of the target object in the target image; wherein, the first classification model is trained by using the output of the second classification model, and the second classification model is trained by using the output of the first classification model.
[0180] In an optional implementation, the image classification device 10 provided in this embodiment includes a model training module. Before inputting the target image into a preset classification model, the model training module is used to:
[0181] Obtain sample data, which includes training sample sets and validation sample sets.
[0182] Based on the classification results of the initial second classification model on the training sample set, the initial first classification model is trained to obtain the intermediate first classification model.
[0183] The first evaluation index of the initial second classification model is calculated based on the classification results of the intermediate first classification model on the validation sample set, and the weights of the initial second classification model are updated to obtain the intermediate second classification model.
[0184] Based on the classification results of the intermediate second classification model on the training sample set, the intermediate first classification model is trained, and based on the classification results of the trained intermediate first classification model on the validation sample set, the second evaluation index of the intermediate second classification model is calculated.
[0185] The training endpoint is determined based on the first and second evaluation indicators, resulting in the first classification model and the second classification model.
[0186] In an optional implementation, the training module is used for:
[0187] The validation sample set is input into the initial second classification model to obtain the classification result of the initial second classification model on the validation sample set;
[0188] Calculate the loss between the initial second classification model and the intermediate first classification model on the validation sample set; where the classification result of the intermediate first classification model on the validation sample set is the label value of the validation sample set.
[0189] Backpropagation of the loss yields the first gradient of the initial second classification model.
[0190] The weights of the initial second classification model are updated based on the first gradient to obtain the intermediate second classification model.
[0191] In an optional implementation, the model training module is used for:
[0192] Differentiating the first gradient yields the second gradient.
[0193] A perturbation within a defined neighborhood is applied to the second gradient to obtain the neighborhood gradient.
[0194] Based on the first gradient and the neighborhood gradient, the weights of the initial second classification model are updated to obtain the intermediate second classification model.
[0195] In an optional implementation, the model training module is used for:
[0196] The validation sample set is input into the initial second classification model to obtain the classification result of the initial second classification model on the validation sample set.
[0197] The accuracy of the initial second classification model is obtained by comparing the classification results of the initial second classification model on the validation sample set with the classification results of the intermediate first classification model on the validation sample set.
[0198] Based on the accuracy of the initial second classification model, the first evaluation index of the initial second classification model is obtained.
[0199] In an optional implementation, the model training module is used for:
[0200] The training sample set is input into the initial first classification model to obtain the classification result of the initial first classification model on the training sample set.
[0201] Calculate the loss between the initial first classification model and the initial second classification model on the training sample set; where the initial second classification model's classification result on the training sample set is the label value of the training sample set.
[0202] Determine whether the loss meets the first preset condition.
[0203] If the condition is not met, the loss is backpropagated, the weights of the initial first classification model are adjusted, and this process is repeated until the loss meets the first preset condition, thus obtaining an intermediate first classification model.
[0204] In an optional implementation, the model training module is used for:
[0205] The validation sample set is input into the intermediate second classification model to obtain the classification result of the intermediate second classification model on the validation sample set.
[0206] The accuracy of the intermediate second classification model is obtained by comparing the classification results of the intermediate second classification model on the validation sample set with the classification results of the intermediate first classification model after training on the validation sample set.
[0207] Based on the accuracy of the intermediate second classification model, a second evaluation index for the intermediate second classification model is obtained.
[0208] In an optional implementation, the model training module is used for:
[0209] Calculate the difference between the first evaluation index and the second evaluation index.
[0210] Determine whether the difference satisfies the second preset condition.
[0211] If the conditions are met, we obtain the first classification model and the second classification model.
[0212] If the condition is not met, the weights of the intermediate second classification model are updated based on the classification results of the intermediate first classification model on the validation sample set after training. The intermediate first classification model is then trained again based on the classification results of the intermediate second classification model on the training sample set after weight update. This process is iterated until the difference between the first evaluation index and the second evaluation index meets the second preset condition, thus obtaining the first classification model and the second classification model.
[0213] In an optional implementation, before training the initial first classification model based on the classification results of the initial second classification model on the training sample set, the model training module is used to:
[0214] The training sample set is input into the preset original classification model to obtain the classification result of the original classification model on the training sample set.
[0215] Calculate the loss of the original classification model on the training sample set compared to the original label values of the training sample set.
[0216] Determine whether the loss meets the third preset condition.
[0217] If the condition is not met, the loss is backpropagated, the weights of the original classification model are adjusted, and this process is repeated until the loss meets the third preset condition, thus obtaining the initial second classification model.
[0218] The image classification device provided in this embodiment obtains a first classification model by training it with the output of a second classification model, and updates the weights of the second classification model with the output of the first classification model. In this way, the first classification model is obtained through reinforcement loop training. After acquiring the target image to be classified, the target image is input into the first classification model for feature processing. When classifying the target objects in the target image, the classification accuracy of the target objects in the target image can be greatly improved.
[0219] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the image classification device 10 described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.
[0220] Based on the above, this embodiment provides a readable storage medium storing a computer program, which, when executed by a processor, implements the image classification method described in any of the foregoing embodiments.
[0221] The readable storage medium can be, but is not limited to, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, and other media capable of storing program code.
[0222] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the readable storage medium described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.
[0223] In summary, the image classification method, apparatus, electronic device, and readable storage medium provided in this embodiment obtain a first classification model by training it with the output of a second classification model, and update the weights of the second classification model with the output of the first classification model. In this way, the first classification model is obtained through reinforcement loop training. After acquiring the target image to be classified, the target image is input into the first classification model for feature processing. When classifying target objects in the target image, the classification accuracy of target objects in the target image can be greatly improved.
[0224] The above descriptions are merely various embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. An image classification method, characterized in that, The method includes: Acquiring the target image to be classified includes: maintaining two N-frame queues, namely a working queue and a buffer queue, wherein the frame images in the buffer queue are new frames in the surveillance video; after the buffer queue is full of N-frame images, a new working queue is formed by sampling from the working queue and the buffer queue respectively, and the target image is obtained from the new working queue, wherein the target image includes N-frame images; The target image is input into a preset first classification model, and the target image is processed by the first classification model to obtain the classification result of the target object in the target image. The process includes: processing the feature map of each frame of the target image to obtain the feature map corresponding to each frame; stacking the feature maps to obtain the stacked feature map; processing the stacked feature map; and outputting a one-dimensional vector corresponding to the number of categories, which is the classification result of the target image. The first classification model is trained using the classification results of the training sample set by the second classification model, and the second classification model updates the weights using the classification results of the validation sample set by the first classification model.
2. The image classification method according to claim 1, characterized in that, Before inputting the target image into a preset classification model, the method further includes: Acquire sample data, which includes the training sample set and the validation sample set; Based on the classification results of the initial second classification model on the training sample set, the initial first classification model is trained to obtain the intermediate first classification model; The first evaluation index of the initial second classification model is calculated based on the classification results of the validation sample set by the intermediate first classification model, and the weights of the initial second classification model are updated to obtain the intermediate second classification model. Based on the classification results of the intermediate second classification model on the training sample set, the intermediate first classification model is trained, and based on the classification results of the trained intermediate first classification model on the validation sample set, the second evaluation index of the intermediate second classification model is calculated. Based on the first evaluation index and the second evaluation index, the training endpoint is determined, and the first classification model and the second classification model are obtained.
3. The image classification method according to claim 2, characterized in that, The step of updating the weights of the initial second classification model based on the classification results of the validation sample set by the intermediate first classification model to obtain the intermediate second classification model includes: The validation sample set is input into the initial second classification model to obtain the classification result of the initial second classification model on the validation sample set; Calculate the loss between the initial second classification model's classification result on the validation sample set and the intermediate first classification model's classification result on the validation sample set; wherein, the intermediate first classification model's classification result on the validation sample set is the label value of the validation sample set; Backpropagation of the loss yields the first gradient of the initial second classification model. The weights of the initial second classification model are updated based on the first gradient to obtain an intermediate second classification model.
4. The image classification method according to claim 3, characterized in that, The step of updating the weights of the initial second classification model based on the first gradient to obtain the intermediate second classification model includes: Differentiating the first gradient yields the second gradient; A perturbation within a defined neighborhood is applied to the second gradient to obtain the neighborhood gradient; The weights of the initial second classification model are updated based on the first gradient and the neighborhood gradient to obtain an intermediate second classification model.
5. The image classification method according to claim 2, characterized in that, The step of calculating the first evaluation index of the initial second classification model based on the classification results of the validation sample set by the intermediate first classification model includes: The validation sample set is input into the initial second classification model to obtain the classification result of the initial second classification model on the validation sample set; The accuracy of the initial second classification model is obtained by comparing the classification results of the initial second classification model on the validation sample set with the classification results of the intermediate first classification model on the validation sample set. Based on the accuracy of the initial second classification model, the first evaluation index of the initial second classification model is obtained.
6. The image classification method according to claim 2, characterized in that, The step of training the initial first classification model based on the classification results of the initial second classification model on the training sample set to obtain the intermediate first classification model includes: The training sample set is input into the initial first classification model to obtain the classification result of the initial first classification model on the training sample set; Calculate the loss between the classification result of the initial first classification model on the training sample set and the classification result of the initial second classification model on the training sample set; wherein, the classification result of the initial second classification model on the training sample set is the label value of the training sample set; Determine whether the loss meets the first preset condition; If the condition is not met, the loss is backpropagated, and the weights of the initial first classification model are adjusted. This process is repeated until the loss meets the first preset condition, thus obtaining the intermediate first classification model.
7. The image classification method according to claim 2, characterized in that, The step of calculating the second evaluation index of the intermediate second classification model based on the classification results of the intermediate first classification model on the validation sample set after training includes: The validation sample set is input into the intermediate second classification model to obtain the classification result of the intermediate second classification model on the validation sample set; The accuracy of the intermediate second classification model is obtained by comparing the classification results of the intermediate second classification model on the validation sample set with the classification results of the trained intermediate first classification model on the validation sample set. Based on the accuracy of the intermediate second classification model, a second evaluation index for the intermediate second classification model is obtained.
8. The image classification method according to claim 2, characterized in that, The step of determining the training endpoint based on the first evaluation index and the second evaluation index to obtain the first classification model and the second classification model includes: Calculate the difference between the first evaluation index and the second evaluation index; Determine whether the difference satisfies the second preset condition; If the conditions are met, the first classification model and the second classification model are obtained; If the condition is not met, the weights of the intermediate second classification model are updated based on the classification results of the intermediate first classification model on the validation sample set after training. The intermediate first classification model is then trained again based on the classification results of the intermediate second classification model on the training sample set after weight update. This process is iterated until the difference between the first evaluation index and the second evaluation index meets the second preset condition, thus obtaining the first classification model and the second classification model.
9. The image classification method according to claim 2, characterized in that, Before training the initial first classification model based on the classification results of the initial second classification model on the training sample set, the method further includes: The training sample set is input into a preset original classification model to obtain the classification result of the original classification model on the training sample set; Calculate the loss between the classification result of the original classification model on the training sample set and the original label values of the training sample set; Determine whether the loss meets the third preset condition; If the condition is not met, the loss is backpropagated, the weights of the original classification model are adjusted, and this process is repeated until the loss meets the third preset condition to obtain the initial second classification model.
10. An image classification device, characterized in that, include: An image acquisition module is used to acquire a target image to be classified, including: maintaining two N-frame queues, the two N-frame queues being a working queue and a buffer queue, the frame images in the buffer queue being new frames in the surveillance video; after the buffer queue is full of N-frame images, sampling is performed from the working queue and the buffer queue to form a new working queue, and the target image is acquired from the new working queue, the target image including N-frame images; The image classification module is used to input the target image into a preset first classification model, perform feature processing on the target image through the first classification model, and obtain the classification result of the target object in the target image. This includes: performing feature processing on each frame of the target image to obtain a feature map corresponding to each frame; stacking the feature maps to obtain a stacked feature map; processing the stacked feature map; and outputting a one-dimensional vector corresponding to the number of categories, i.e., the classification result of the target image. The first classification model is trained using the classification results of a training sample set using a second classification model, and the second classification model is trained using the classification results of a validation sample set using the first classification model.
11. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image classification method as described in any one of claims 1 to 9.
12. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the image classification method as described in any one of claims 1 to 9.
Citation Information
Patent Citations
Image restoration method and device, storage medium and electronic device
CN112132766A
Neural network training method and device and computer equipment
CN112288086A
