Training method of image identification model
By preprocessing the image and embedding the logo illustration and training the image recognition model, the protection problems of the image recognition model and training data are solved, and the security and accuracy of the model and data are achieved.
Patent Information
- Application Number
- CN202311837615.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-07-14
- Filing Date
- 2023-12-28
- Publication Date
- 2025-01-14
AI Technical Summary
The prior art is difficult to effectively protect image recognition models and their training data to prevent unauthorized use, especially in applications involving confidential and sensitive data.
By preprocessing the original image, embedding the logo illustration, and embedding it into the training dataset, training using an image recognition model containing a convolutional layer, the model is finally deployed onto the integrated circuit to ensure the security of the data and model.
It effectively protects the image recognition model and its training data, prevents unauthorized use, and ensures the accuracy and security of the model, and is suitable for sensitive data applications.
Smart Images
Figure CN119314000A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of image processing and machine learning, and in particular to a method for training an image recognition model. Background Art
[0002] The application of machine learning technology in the field of image recognition is already very common. In addition to cloud computing, the application of image recognition models in Internet of Things (IoT) devices and embedded systems is also booming. For integrated circuit suppliers, their customers often need to rely on a series of machine learning-related services provided by third-party machine learning development platforms (such as Edge Impulse, Balena, TensorFlow Lite, Arduino, Microsoft Azure IoT Edge, etc.) to train image recognition models and deploy the trained models on integrated circuits.
[0003] In some applications, the training data required to optimize image recognition models may be confidential and sensitive. For example, in applications such as face recognition or medical image analysis, the required training data may be photos of human body parts containing private personal information; in applications such as defect detection, the required training data may be photos of products containing commercial secrets. In addition to training data, the trained models themselves are also at risk of being misappropriated.
[0004] Therefore, how to protect image recognition models and their training data to avoid unauthorized use is an issue that needs to be resolved. Summary of the invention
[0005] In view of the above problems, an embodiment of the present invention provides a method for training an image recognition model, the method comprising the steps of obtaining an original data set, performing a preprocessing procedure on the original image of each piece of original data in the original data set to obtain a training data set, and using the training data set to train an image recognition model. Each piece of original data in the original data set includes an original image and its corresponding label data. Each piece of training data in the training data set includes a processed image and its corresponding label data. The preprocessing procedure includes embedding a logo icon into the original image to obtain a processed image.
[0006] In one embodiment, the preprocessing procedure further includes operations of identifying the dominant color of the original image and determining the color of the logo icon embedded in the original image based on the dominant color. In a further embodiment, the preprocessing procedure further includes operations of calculating a color histogram of the original image, determining the dominant color of the original image by performing principal components analysis (PCA) on the color histogram, and using complementary colors of the dominant color as the color of the logo icon embedded in the original image.
[0007] In one embodiment, the pre-processing procedure further includes an operation of determining a size of the logo icon embedded in the original image based on a size of the original image.
[0008] In one embodiment, the image recognition model includes one or more convolution layers. Each convolution layer performs a convolution operation on the output of the previous layer to generate and output one or more feature maps having multiple channels. In a further embodiment, the method further includes adding at least one additional channel to the logo icon to embed the logo icon into the feature map generated by each convolution layer.
[0009] In one embodiment, the method further comprises the operation of deploying the trained image recognition model to an integrated circuit. In a further embodiment, the integrated circuit is a microcontroller unit (MCU).
[0010] In one embodiment, the method further includes uploading the training data set to a machine learning development platform, and using the training data set to train the image recognition model on the machine learning development platform. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The present application will be better understood from the following illustrative embodiments with accompanying drawings. In addition, it should be understood that in the flowchart of the present application, the execution order of each block can be changed, and / or some blocks can be changed, deleted or merged.
[0012] Figure 1 is a flowchart of a method for training an image recognition model according to an embodiment of the present invention.
[0013] Figure 2 Draw the adoption Figure 1 Schematic diagram of the process of obtaining a trained image recognition model using the method described above.
[0014] Figure 3 A schematic diagram of the pre-processing procedure for embedding a logo icon into an original image to obtain a processed image is drawn.
[0015] Figure 4 FIG. 4 is a flow chart illustrating the steps of determining the color of a logo icon embedded in an original image according to an embodiment of the present invention.
[0016] [Explanation of symbols]
[0017] 100:method;
[0018] S101-S103: steps;
[0019] 20: Image recognition model;
[0020] 21: original dataset;
[0021] 22: training data set;
[0022] 23: image to be identified;
[0023] 24: Identification results;
[0024] 211-21N: original data;
[0025] 211A-21NA: label data;
[0026] 211B-21NB: original images;
[0027] 221-22N: training data;
[0028] 221B-22NB: processed images;
[0029] 300: label icon;
[0030] S401-S403: steps. DETAILED DESCRIPTION
[0031] The following description lists various embodiments of the present invention, but is not intended to limit the content of the present invention. The actual scope of the invention is defined by the scope of the patent application.
[0032] In the various embodiments listed below, the same reference numerals will be used to represent the same or similar elements or components.
[0033] Serial numbers in this specification and in the scope of patent applications, such as "first", "second", etc., are only for convenience of description and there is no sequential relationship between them.
[0034] "Image recognition" is a broad technical term that covers many image-related application fields such as face recognition, animal recognition, object detection, defect detection, scene classification, font recognition, etc. Therefore, the "image recognition model" described in this specification can be applied to any of the above application fields, but the present invention is not limited to this.
[0035] Figure 1 FIG. 1 is a flow chart of a method 100 for training an image recognition model according to an embodiment of the present invention. Figure 1 As shown, the method 100 includes steps S101-S102. Figure 1 , Figure 2 Draw the adoption Figure 1 A schematic diagram of the process of obtaining the trained image recognition model 20 by the method 100 is shown. Figure 3 A schematic diagram of the pre-processing procedure for embedding a logo icon 300 into the original image 211B to obtain a processed image is shown. Figures 1 to 3 , to better understand the embodiments of the present invention.
[0036] In step S101, the original data set 21 is obtained. Then, the method 100 proceeds to step S102.
[0037] In step S102, a preprocessing procedure is performed on the original image 211B-212B of each original data 211-21N in the original data set 21 to obtain a training data set 22. The preprocessing procedure includes embedding the logo icon 300 into the original image 211B-21NB to obtain the processed image 221B-22NB. Then, the method 100 proceeds to step S103.
[0038] In step S103, the image recognition model 20 is trained using the training data set 22. The trained image recognition model 20 can accept the image to be recognized 23 as input and output a corresponding recognition result 24.
[0039] like Figure 2 As shown, the original data set 21 obtained in step S101 may include original data 211-21N, and each piece of original data may include an original image and its corresponding label data. Specifically, the original data 211 includes an original image 211B and its corresponding label data 211A, the original data 212 includes an original image 212B and its corresponding label data 212A, the original data 21N includes an original image 21NB and its corresponding label data 21NA, and so on.
[0040] The original images 211B-21NB refer to images that have not yet been trained in the preprocessing step S102, and these images are not necessarily images originally captured by the photographic device. For example, the original images 211B-21NB may be images captured by the photographic device, or images collected from open source datasets (such as the Pascal VOC dataset or the Common Objects inContext (CoCo) dataset). In addition, the original images 211B-21NB may also be images that have undergone some post-processing (such as color adjustment, light layout, or filter effects, etc.), or a portion of any of the above images (such as a portion containing a specific object), but the present invention is not limited to this.
[0041] The tag data 211A-21NA respectively correspond to the original images 211B-21NB and are used to describe the content of the original images 211B-21NB. For example, Figure 3 The original image 211B of the example depicted, and its corresponding label data 211A may describe the original image 211B as a "wine bottle". The label data 211A-21NA may be generated in various ways, such as manually annotating the category, location or other relevant attributes of the objects in the image, or using an automated method or semi-automated tool to generate the label data, but the present invention is not limited thereto.
[0042] It is worth noting that although Figure 2 Only the raw data 211-21N with corresponding label data 211A-21NA in the raw data set 21 are plotted, but the present invention does not limit the raw data set 21 to only include these raw data. In some embodiments, the raw data set 21 may further include unlabeled images or other forms of data for semi-supervised learning or self-supervised learning. In addition, each piece of raw data 211-21N may further include metadata related to the raw image 211B-21NB, such as the time or location of the image shooting, so that the image recognition model 20 trained later can be applied to a wider range of application scenarios.
[0043] like Figure 2As shown, each piece of training data 221-22N in the training data set 22 may include training data 221-22N, and each piece of training data may include a processed image and its corresponding label data, and the label data is the same as the label data in the original data corresponding to the training data. Specifically, the training data 221 includes the processed image 211B obtained after the original image 211B is preprocessed in step S102 and the corresponding label data 211A, the training data 222 includes the processed image 222B obtained after the original image 212B is preprocessed in step S102 and the corresponding label data 212A, the training data 22N includes the processed image 22NB obtained after the original image 21NB is preprocessed in step S102 and the corresponding label data 21NA, and so on.
[0044] The following about Figure 3 The description is based on the original image 211B as an example, and the pre-processing procedure of step S102 for other original images 212B-21NB can be deduced similarly. Figure 3 As shown, the processed image 221B obtained after the original image 211B is pre-processed in step S102 will contain the logo 300. The logo 300 can be an image, symbol or symbol used to identify a company, organization or individual, and its design can involve a combination of various texts or graphics. The present invention does not limit the design or commercial use of the logo 300.
[0045] In one embodiment, the logo icon 300 may be a digital watermark, which has a certain degree of concealment or transparency, so that the difference between the original image 211B and the processed image 221B is not too obvious and thus does not affect the accuracy of the image recognition model 20. In addition, the non-removable property of the digital watermark can also prevent the logo icon 300 from being removed by hackers.
[0046] In one embodiment, the size of the logo icon 300 embedded in the original image 211B depends on the size of the original image 211B. Therefore, the pre-processing procedure of step S102 further includes an operation of determining the size of the logo icon 300 embedded in the original image 211B based on the size of the original image 211B.
[0047] In one embodiment, the color of the logo icon 300 embedded in the original image 211B depends on the dominant color of the original image 211B. Therefore, the pre-processing procedure of step S102 further includes an operation of identifying the dominant color of the original image 211B, and an operation of determining the color of the logo icon 300 embedded in the original image 211B based on the dominant color of the original image 211B.
[0048] The operation of identifying the main color of the image may involve using various statistical methods to calculate the pixel color distribution, or using a convolutional neural network (CNN-based) deep learning model to extract the color features of the image and classify them. In one embodiment, the main color of the original image is identified by using a color histogram and principal components analysis (PCA). Figure 4 Describe the operation steps in more detail.
[0049] Figure 4 1 is a flowchart of operation steps S401-S403 for determining the color of a logo icon embedded in an original image according to an embodiment of the present invention. First, in step S401, a color histogram of the original image is calculated. A color histogram is a statistical tool for describing the color distribution of pixels in an image. Specifically, the calculation of a color histogram involves mapping the color value of each pixel in the image to one or more color channels, such as red (R), green (G), and blue (B), and then calculating the number of pixels in each channel.
[0050] In step S402, principal component analysis is performed on the color histogram to determine the main colors of the original image. Principal component analysis is a statistical method used to reduce the dimension of a data set and extract features. When performing principal component analysis, the color histogram of the original image is regarded as a collection of multiple feature vectors, each of which represents the distribution of different color channels in the image. Through principal component analysis, the variability direction of the color distribution in the original image can be extracted, and the main colors of the original image can be inferred based on the weights of its corresponding color features (or color channels).
[0051] In step S403, complementary colors of the main color are used as the colors of the logo icon embedded in the original image. Complementary colors are hues located at opposite ends of the diameter of the color wheel. For example, assuming Figure 3The color channels of the original image 211B are defined using the RGB color space. If the main color of the original image 211B is red (R=255, G=0, B=0), the color of the logo icon 300 in the processed image 221B is cyan (R=0, G=255, B=255); if the main color of the original image 211B is green (R=0, G=255, B=0), the color of the logo icon 300 in the processed image 221B is magenta (R=255, G=0, B=255); if the main color of the original image 211B is blue (R=0, G=0, B=255), the color of the logo icon 300 in the processed image 221B is yellow (R=255, G=255, B=0).
[0052] As previously mentioned, the image recognition model 20 can be applied to many image-related fields such as face recognition, animal recognition, object detection, defect detection, scene classification, font recognition, etc., but the present invention is not limited to this. In addition, the image recognition model 20 may involve any algorithm related to image recognition, such as convolution neural network (CNN), support vector machine (SVM), random forest, etc. In some embodiments, the image recognition model 20 may involve tasks or functions such as feature extraction, object positioning and classification. The extracted image features can be feature vectors, feature tensors, or feature maps. Feature representations represent measurable attributes or characteristics of the image. Afterwards, the category of the object (for example, Figure 3 The object in the processed image 221B is a "wine bottle"), as well as the position and range of the object in the image.
[0053] In some embodiments, a loss function such as mean square error (MSE), mean absolute error (MAE) or cross-entropy can be used to calculate a loss value representing the difference between the output result of the image recognition model 20 and the corresponding training data. Furthermore, an optimizer can be used to recursively adjust the parameters of the image recognition model 20 (e.g., the weights of each layer of the neural network) so that the loss value is minimized, thereby optimizing the image recognition model 20. The optimizer can be implemented using algorithms such as gradient descent (GD), stochastic gradient descent (SGD) or adaptive moment estimation (Adam). Taking an optimizer using the gradient descent method as an example, the gradient can be obtained by calculating the partial derivative of the loss function, and then the parameters of the image recognition model 20 can be adjusted according to the gradient to reduce the loss value. Through repeated training processes such as result feedback and parameter update, the loss value is gradually reduced until the loss value converges to a minimum value. At this point, the image recognition model 20 can more accurately identify the object in the image to be recognized 23.
[0054] In one embodiment, the image recognition model 20 may include one or more convolution layers to extract features of an image (including processed images 221B-22NB and / or an image to be recognized 23). Each convolution layer performs a convolution operation on the output of the previous layer to generate and output one or more feature maps with multiple channels. A channel is a dimension in a feature map that represents a specific type of feature in the feature map. Each channel can capture different visual features in an image, such as contour, texture, or color information. The convolution layer performs a convolution operation on the input image, multiplying and summing up the values of each filter and the corresponding position in the image one by one to calculate the convolution value of the position. By continuously sliding the position of the filter corresponding to the image, all the convolution values in the feature map can be calculated. By stacking multiple convolution layers, the image recognition model 20 can gradually extract higher-level abstract features from low-level basic features.
[0055] In a further embodiment, at least one additional channel may be added to the logo icon to embed the logo icon into the feature map generated by each convolutional layer. For example, assuming that the channels of the original image 211B are defined in the RGB color space, by adding a fourth channel to the logo icon 300, the value of each pixel of the processed image 221B can be represented as a four-dimensional vector (R, G, B, I), where I represents the pixel value mapped by the logo icon. Alternatively, three channels may be added to the logo icon 300, and the value of each pixel of the processed image 221B can be represented as a six-dimensional vector (R, G, B, R', G', B'), where R', G', B' represent the pixel values of the red, green, and blue channels of the logo icon 300, respectively. In this way, the logo icon can be embedded in the feature map generated by each convolutional layer to further enhance the protection of the image recognition model 20.
[0056] In one embodiment, the training data set 22 can be uploaded to a machine learning development platform, and the training data set 22 can be used to train the image recognition model 20 on the machine learning development platform. The above-mentioned machine learning development platform can be, for example, EdgeImpulse, Balena, TensorFlow Lite, Arduino, Microsoft Azure IoT Edge, etc., which are development platforms for training, deploying and managing embedded machine learning models, and can provide a series of machine learning related services, including data collection, data preprocessing, feature extraction, model building, training, verification, and even final model deployment. For integrated circuit suppliers, the machine learning development platform can be a third-party platform or a platform provided by the integrated circuit supplier itself, but the present invention is not limited to this.
[0057] In one embodiment, the trained image recognition model 20 can be deployed on various types of integrated circuits, including but not limited to application-specific integrated circuits (ASICs), system-on-a-chip (SoCs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), digital signal processors (DSPs), or any chips designed specifically for artificial intelligence or image recognition.
[0058] In a further embodiment, the trained image recognition model 20 can be deployed to a microcontroller unit (MCU) to expand the application scope of the image recognition model 20 by leveraging the advantages of the microcontroller unit such as low power consumption, real-time processing and local processing capabilities, and relatively high cost-effectiveness, so that it can play a key role in scenarios with limited resources and real-time requirements.
[0059] The above method can be implemented using computer executable instructions. For example, these instructions can include instructions and data that can cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or a group of functions. Some of the computer resources used can be accessed via a network. For example, the computer executable instructions can be binary or intermediate format instructions such as assembly language, firmware or source code.
[0060] The methods, or specific forms or portions thereof, of the present invention may be included in a coded form on a physical medium, such as a floppy disk, an optical disk, a hard disk, or any other machine-readable (e.g., computer-readable) storage medium, wherein when the code is loaded and executed by a machine, such as a computer, the machine becomes a device or system for participating in the present invention. The methods, systems, and devices of the present invention may also be transmitted in a coded form over some transmission medium, such as wires or cables, optical fibers, or any transmission mode, wherein when the code is received, loaded, and executed by a machine, such as a computer, the machine becomes a device or system for participating in the present invention. When implemented on a general-purpose processor, the code combines with the processor to provide a unique device that operates similarly to application-specific logic circuits.
[0061] The above paragraphs are described using multiple aspects. Obviously, the teachings of this article can be implemented in multiple ways, and any specific architecture or function disclosed in the examples is only a representative case. According to the teachings of this article, it should be understood in the art that each aspect disclosed in this article can be implemented independently, or two or more aspects can be combined to implement.
[0062] Although the present application has been described above with reference to the embodiments, it is not intended to limit the present application. Any person skilled in the art may make some changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the invention shall be determined by the appended claims.
Claims
1. A training method for an image recognition model, characterized in that: include: Obtaining an original data set, wherein each piece of original data in the original data set includes an original image and label data corresponding to the original image; A training data set is obtained by performing a preprocessing procedure on the original image of each piece of original data in the original data set, wherein each piece of training data in the training data set includes a processed image and the label data corresponding to the processed image; as well as Using the training data set to train an image recognition model; The pre-processing procedure includes embedding a logo icon into the original image to obtain the processed image.
2. The method according to claim 1, characterized in that The pre-processing procedure also includes: identifying a dominant color of the original image; and Based on the main color, the color of the logo icon embedded in the original image is determined.
3. The method according to claim 2, characterized in that The pre-processing procedure also includes: Calculating a color histogram of the original image; Performing principal component analysis on the color histogram to determine the main color of the original image; and A complementary color of the main color is used as the color of the logo icon embedded in the original image.
4. The method according to claim 1, characterized in that: The pre-processing procedure also includes: The size of the logo icon embedded in the original image is determined based on the size of the original image.
5. The method according to claim 1, characterized in that The logo is illustrated as a digital watermark.
6. The method according to claim 1, characterized in that The image recognition model includes one or more convolutional layers, wherein each convolutional layer performs a convolution operation on the output of the previous layer to generate and output one or more feature maps with multiple channels.
7. The method according to claim 6, characterized in that Also includes: At least one additional channel is added to the logo representation to embed the logo representation into the one or more feature maps generated by each convolutional layer.
8. The method according to claim 1, characterized in that Also includes: The trained image recognition model is deployed to an integrated circuit.
9. The method according to claim 8, characterized in that The integrated circuit is a microcontroller unit.
10. The method according to claim 1, characterized in that Also includes: The training data set is uploaded to a machine learning development platform, and the image recognition model is trained using the training data set on the machine learning development platform.