Method for training image recognition model, image recognition method and image recognition equipment

A training image and image recognition technology, applied in the field of image recognition, can solve problems such as unguaranteed quality and poor performance, and achieve good performance and enhanced generalization

Pending Publication Date: 2021-09-17
MEGVII BEIJINGTECH CO LTD
0 Cites 0 Cited by

AI-Extracted Technical Summary

Problems solved by technology

[0003] In model training, the image training set used is often images taken by certain cameras, but in actual use of image recognition products (such as face recognition products) or SDKs, the image...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Method used

Based on above description, according to the method for training image recognition model, image recognition method and image recognition equipment of the embodiment of the present application, by carrying out ISP augmentation process to initial training image, can realize based on a certain paragraph or some The training images taken by different cameras simulate the effect of training images taken by various cameras, so the image recognition model based on the augmented image training will enhance the generalization of different modules, so that the image recognition model It has good performance when performing image recognition on images to be recognized captured by different imaging modules.
Based on the above description, according to the method for training the image recognition model according to the embodiment of the application, by carrying out ISP augmentation processing to the initial training image, the training image simulation based on a certain paragraph or several cameras can be realized to obtain The effect of training images taken by various cameras, so the image recognition model obtained based on the augmented image training will enhance the generalization of different modules, so that the image recognition model can be used for different imaging modules. It has good performance when recognizing images for image recognition.
Based on the above description, the image recognition device according to the embodiment of the application carries out ISP augmentation processing to the initial training image through the augmentation module, and can realize the training image simulation based on a certain type or several types of cameras to obtain various The effect of the training images taken by different cameras, so the image recognition model obtained by the training module based on the augmented image training will enhance the generalization of different modules, so that the image recognition model can be used for different imaging modules. It has good performance when recognizing images for image recognition.
In an embodiment of the application, the HSV domain augmentation process can include saturation adjustment and/or contrast adjustment; wherein, the saturation parameter adopted in the saturation adjustment is to fine-tune preset parameters according to random variables The contrast parameters used in the contrast adjustment are obtained by fine-tuning preset parameters according to random variables. In the ISP processing process of different modules, the contrast and saturation are often debugged according to the aesthetics of ISP debuggers, and the saturation and contrast styles of different modules are different. Therefore, in this embodiment, saturation parameters are randomly generated (within a reasonable range) by fine-tuning the preset parameters in the HSV domain according to the random variables to saturate the initial training image or the training image processed by RGB domain augmentation. Adjustment can realize the augmentation of image saturation information. In addition, by fine-tuning the preset parameters according to random variables in the HSV domain and randomly generating contrast parameters (within a reasonable range) to adjust the contrast of the image, the image contrast information can be augmented.
In the embodiment of the present application, the HSV domain augmentation processing that augmentation module 810 carries out can comprise saturation adjustment and/or contrast adjustment; Wherein, the saturation parameter that adopts in described saturation adjustment is according to random variable It is obtained by fine-tuning a preset parameter, and the contrast parameter used in the contrast adjustment is obtained by fine-tuning a preset parameter according to a random variable. In the ISP processing process of different modules, the contrast and saturation are often debugged according to the aesthetics of ISP debuggers, and the saturation and contrast styles of different modules are different. Therefore, in this embodiment, the augmentation module 810 randomly generates saturation parameters (within a reasonable range) by fine-tuning the preset parameters in the HSV domain according to random variables. Image saturation adjustment can realize the augmentation of image saturation information. In addition, the augmentation module 810 adjusts the contrast of the image by randomly generating contrast parameters (within a reasonable range) by fine-tuning the preset parameters according to the random variables in the HSV domain, so as to realize the augmentation of the contrast information of the image.
In the embodiment of the present application, the ISP augmentation processing that augmentation module 810 carries out to initial training image can comprise following at least one: RGB domain augmentation processing, HSV domain augmentation processing and YUV domain augmentation processing . Among them, RGB, HSV, and YUV are different color spaces. Correspondingly, RGB domain augmentation processing refers to processing in RGB color space, HSV domain augmentation processing refers to processing in HSV color space, and YUV domain augmentation processing refers to processing in YUV color space. Generally, there are different ISP processing links in different color spaces. At least one ISP processing link has been augmented to realize the ISP augmentation of the initial training image. Of course, multiple ISP processing links can be enhanced by augmentation processing. The training ability of the training data, so as to enhance the generalization of the trained image recognition model for different modules.
In the embodiment of the present application, the YUV domain augmentation processing can comprise noise reduction and/or edge enhancement; Wherein, the parameter of the low-pass filter that adopts in the YUV domain noise reduction is according to random variable pair preset The parameters used in the edge enhancement are obtained by fine-tuning the parameters, and the parameters used in the edge enhancement are obtained by fine-tuning the preset parameters according to random variables. In this embodiment, the parameters of the low-pass filter (within a reasonable range) are randomly generated (within a reasonable range) by fine-tuning the preset parameters in the YUV domain according to the random variables. The training image processed by HSV domain augmentation is denoised, which can realize the augmentation of the image denoising effect. In addition, by fine-tuning the preset parameters according to the random variables in the YUV domain and randomly generating (within a reasonable range) parameters of the sharpening algorithm to enhance the edge of the image, the enhancement of the image sharpening degree can be realized.
In the embodiment of the present application, the YUV domain augmentation processing that augmentation module 810 carries out can comprise noise reduction and/or edge enhancement; Wherein, the parameter of the low-pass filter that adopts in the described YUV domain noise reduction is It is obtained by fine-tuning preset parameters according to random variables, and the parameters used in the edge enhancement are obtained by fine-tuning preset parameters according to random variables. In this embodiment, the augmentation module 810 randomly generates the parameters of the low-pass filter (within a reasonable range) by fine-tuning the preset parameters in the YUV domain according to the random variables. Noise reduction is performed on the training image or the training image processed by HSV domain augmentation, which can realize the augmentation of the image noise reduction effect. In addition, the augmentation module 810 performs edge enhancement on the image by fine-tuning the preset parameters according to random variables in the YUV domain to randomly generate (within a reasonable range) parameters of the sharpening algorithm, so as to achieve augmentation of the image sharpening degree.
In the embodiment of the present application, the image recognition model that image recognition method 700 adopts when carrying out image recognition to the image to be recognized is based on the training image obtained after performing the augmentation processing of image signal processing to the initial training image and obtains by training That is, the image recognition model adopted by the image recognition method 700 for image recognition of the image to be recognized is obtained by training according to the method for training the image recognition model according to the embodiment of the present application described above, as described above According to the method for training an image recognition model according to an embodiment of the present application, by performing ISP augmentation processing on the initial training image, it is possible to simulate the training images captured by a certain type or several types of cameras to obtain images captured by various types of cameras. Therefore, the image recognition model trained based on the augmented image will enhance the generalization of different modules. Therefore, using such an image recognition model, the image recognition method 700 according to the embodiment of the present application can perform image recognition on images to be recognized captured by different imaging modules with good performance. Those skilled in the art can understand the training method of the image recognition model adopted by the image recognition method 700 in combination with the foregoing, and for the sake of brevity, details are not repeated here.
In the embodiment of the present application, the training module 820 can also combine the initial training image and the training image through the ISP augmentation process to train the image recognition model, which can further improve the training ability of the training image set, and obtain An image recognition model with higher robust image recognition capabilities for images taken by different modules.
In this embodiment, augmentation module 810 can carry out color information adjustment (generally fine-tuning ), obtain training images with different color information, and realize the augmentation of image color information. In addition, the augmentation module 810 performs a global gamma transformation on the image by (within its reasonable range) the gamma coefficient obtained by fine-tuning the preset parameters according to the random variable, so that various random degrees of overexposure or underexposure can be performed. Correction to realize the augmentation of image exposure correction. In addition, some of different modules will perform histogram equalization, and some will not perform histogram equalization, so the augmentation module 810 performs random histogram equalization, that is, adopts a random variable (such as a value of 1 or 0 ) to control whether to perform histogram equalization processing, which can realize the enhancement of the image with or without histogram equalization effect.
In this embodiment, by adopting the color correction matrix and the offset matrix including the parameters obtained by fine-tuning the preset parameters according to random variables, the initial training image can be adjusted for color information (generally fine-tuning), to obtain different The training image of color information realizes the augmentation of image color information. In addition, by performing global gamma transformation on the image through the gamma coefficient obtained by fine-tuning the preset parameters according to the random variable, various ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Abstract

The invention discloses a method for training an image recognition model, an image recognition method and image recognition equipment, and the method comprises the steps: obtaining an initial training image, augmenting image signal processing of the initial training image, and obtaining a processed training image; and training an image recognition model based on the processed training image. According to the scheme of the invention, the augmentation processing of the image signal processing is carried out on the initial training image, so that the training images shot by various different cameras can be simulated based on the training images shot by one or more cameras. Therefore, generalization of the image recognition model obtained based on image training after augmentation processing on different modules is enhanced, so that the image recognition model has good performance when performing image recognition on to-be-recognized images shot by different imaging modules.

Application Domain

Technology Topic

Image

  • Method for training image recognition model, image recognition method and image recognition equipment
  • Method for training image recognition model, image recognition method and image recognition equipment
  • Method for training image recognition model, image recognition method and image recognition equipment

Examples

  • Experimental program(1)

Example Embodiment

[0026] In order to make the objects, technical solutions and advantages of the present application, the exemplary embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present application, rather than all embodiments of the present application, to be understood, the present application is not limited by the example embodiments described herein. Based on the present application described in the present application, those skilled in the art should fall within the scope of the application without paying creative labor.
[0027] In recent years, it has made important progress based on computer visual, deep learning, machine learning, image processing, image recognition, and other technological research. Artificial Intelligence (AI) is an emerging science technology for research, development for simulation, extension of intelligence, and emerging science techniques. Artificial Intelligence Discipline is a comprehensive discipline, involving chip, big data, cloud computing, Internet of Things, distributed storage, depth learning, machine learning, neural network, etc. As an important branch of artificial intelligence, it is specifically to identify the world. Computer vision technologies typically include face recognition, living detection, fingerprint identification and anti-counterfeiting verification, biometric identification, face testing, pedestrian testing, target detection, pedestrians Identification, image processing, image recognition, image semantic understanding, image retrieval, text identification, video processing, video content identification, behavioral identification, three-dimensional reconstruction, virtual reality, enhanced reality, synchronous positioning and map construction (SLAM), calculation photography, robot Navigation and positioning and other technologies. With the research and progress of artificial intelligence technology, this technology has launched an application in many areas, such as security, urban management, transportation management, building management, park management, face traffic, face attendance, logistics management, warehouse management, robotics , Intelligent marketing, calculation photography, mobile phone image, cloud service, smart home, wear equipment, drone, automatic driving, smart medical, face payment, face unlock, fingerprint unlock, human card, smart screen, smart TV, Camera, mobile internet, live broadcast, beauty, beauty, medical beauty, intelligent temperature measurement, etc.
[0028] Below, referring to figure 1 A method, an image recognition method, and an image recognition apparatus of the image recognition model for training the image recognition model for implementing the embodiment of the present invention are described.
[0029] like figure 1 As shown, the electronic device 100 includes one or more processors 102, one or more storage device 104, input device 106, and output device 108, which are connected by bus systems 110 and / or other forms (not shown). interconnection. It should be noted that figure 1 The components and structures of the electronic device 100 shown are merely exemplary, not limiting, and the electronic devices may also have other components and structures.
[0030] The processor 102 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and other components in the electronic device 100 can control the desired function.
[0031] The storage device 104 can include one or more computer program products, which may include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can, for example, comprise a random access memory (RAM) and / or a cache, and the like. The non-volatile memory can, for example, include a read only memory (ROM), a hard disk, a flash memory, and the like. One or more computer program instructions can be stored on the computer readable storage medium, and the processor 102 can run the program instruction to implement client functions of the present invention (implemented by the processor) described below. And / or other desired functions. Various applications and various data can also be stored in the computer readable storage medium, such as various data used and / or generated by the application.
[0032] The input device 106 can be a device for inputting an instruction, and may include one or more of a keyboard, a mouse, a microphone, and a touch screen, or the like. Furthermore, the input device 106 can also be any interface for receiving information.
[0033]The output device 108 can output various information (e.g., an image or sound) to the outside (e.g., the user), and may include one or more of the display, speaker, and the like. Further, the output device 108 can also be any other device having an output function.
[0034] Exemplarily, an example electronic device for implementing a method and apparatus for training an image recognition model in accordance with an embodiment of the present invention can be implemented such as a smartphone, a tablet, a camera and other terminals.
[0035] Below, reference figure 2 A method 200 describing a training image recognition model according to an embodiment of the present application. like figure 2 As shown, method 200 of the training image recognition model can include the following steps:
[0036] At step S210, the initial training image is acquired, and the initial training image is added to the increase in image signal processing, and the processed training image is obtained.
[0037] At step S220, the image recognition model is trained based on the processed training image.
[0038] In the embodiment of the present application, the initial training image acquired in step S210 can be an image captured by a certain or several cameras (imaging equipment, also known as a module). As mentioned above, if only a few cameras taken as a training image training image recognition model, the training image recognition model can be obtained when image recognition of the image to be identified for the camera to be identified. Good performance, but in fact, the image recognition model of training is in the process of use, especially as an SDK sell, it is uncertain, so it cannot guarantee image recognition Model performance. Therefore, in the embodiment of the present application, the initial training image acquired by step S210 will be added to an increase in image signal processing (ISP), where ISP increase is the image of the ISP process. Identify parameters of the performance of the model (such as parameters such as color conversion matrix, coefficient, saturation parameters, contrast parameters, etc. will be described in more detail below) according to the random variables The initial training image is processed based on such parameters (the processing mode depends on the parameter itself, such as the parameter of the color transform matrix is ​​used to perform color information adjustment, the coefficient of gamma transform is used to perform gamma transformation, saturation parameters Used to perform saturation adjustment, the contrast parameter is used to perform contrast adjustment, etc.), resulting in a processed training image and then used for training image recognition, and ensures image recognition of image recognition images to be identified for different imaging modules. Have a good performance. This is because the main difference between different modules is the process of the ISP algorithm, where the ISP algorithm is the process of converting the original (RAW) image collected by the image sensor into the RGB image used by the daily use of the RGB image. Therefore, in the embodiments of the present application, by performing ISP increase processing on the initial training image, a training image to be photographed based on a certain number of cameras can be achieved, and the effect of various camera captured training images can be realized. Thus, the image recognition model obtained based on the increased image training will enhance the generalization of different modules, so that the image recognition model is performed in image recognition for image recognition of different imaging modules to be identified. .
[0039] In the embodiment of the present application, the ISP increase process for the initial training image can include at least one of the following: RGB domain growth processing, HSV domain growth processing, and YUV domain growth. Among them, RGB, HSV, and YUV are different color spaces, respectively. Accordingly, RGB domain growth processing refers to the processing in the RGB color space, and the HSV domain gaining process refers to the processing in the HSV color space, and the YUV domain gaining process refers to processing in the YUV color space. In different color spaces, there are generally different ISP processing links. At least one ISP processing link has been increased, that is, the ISP increase of the initial training image, of course, the plurality of ISP processing links can be more enhanced. The training capabilities of training data have enhanced the gentlestance of different modules for different modules.
[0040] The processing link in the RGB domain generally affects the processing links in the HSV domain. The processing links in the HSV domain generally have an effect on the processing links in the YUV domain, and therefore, in one embodiment of the present application, when ISP increases Wide processing includes at least two of RGB domain gaining processing, HSV domain growth processing, and YUV domain growth processing, RGB domain growth processing is performed before the HSV domain increases, and HSV domain increases in YUV domain. Before wide processing, this can simplify the processing process and improve efficiency. For example, the RGB domain growth process can be performed, and the image after the RGB domain is increased, and the image increased by the RGB domain has increased the HSV domain, which is increased by the HSV domain. The post-processed image is an image for training image recognition model. For another example, the RGB domain growth process can be performed, and the image after the RGB domain is increased, and the YUV domain increase processing after the RGB domain is increased, and the YUV domain is increased. Widely processed images as images for training image recognition models. For example, the initial training image can be performed to increase the HSV domain growth process to obtain an image after an increase in HSV domain, and the image increased by the YUV domain after the HSV domain has increased, and the YUV domain is increased. Widely processed images as images for training image recognition models.
[0041] In yet another example, the RGB domain increase process can be performed first, and the image after the RGB domain is increased, and the image increased by the RGB domain increased processing is performed. The image after the HSV domain is increased, and the image increased by the image after the HSV domain has increased, and the image after the YUV domain is increased, as an image for training image recognition model, the process like image 3 The shown. In this example, ISP increasing processing includes both RGB domain growth processing, HSV domain growth processing, and YUV domain growth processing, which can maximize the training capabilities of the image after the increase in increased processing, maximally Enhanced image recognition model for different modules for the generalization of different modules.
[0042] In the embodiment of the present application, RGB domain growth can include at least one of color information adjustment, gamma (Gamma) transformation, and random histogram equalization; wherein the color correction matrix used in the color information adjustment. (Color Correction Matrix, referred to as CCM, typically 3 * 3 matrix) and parameters in the bias matrix (generally 3 * 1 matrix) are fine-tuning according to random variables; the gamma The gamma coefficient used in the transform is obtained according to the random variable; the random histogram equalization means that the random variable determines whether or not the histogram is equalized.
[0043] In this embodiment, the initial training image can be adjusted (generally fine tuning) by adopting a color correction matrix and a bias matrix of fine-tuning a preset parameter to preset parameters according to a random variable. Training images, achieve the increase in image color information. Further, by performing the gamma coefficient of the preset parameter on the preset parameters in accordance with the random variable, the image exposure or exposure can be made to perform various randomly different degrees of correction, and the increase in image exposure correction can be achieved. In addition, different modules will perform histogram equalization, and some will not perform histogram equalization, so the random histogram is equalized, that is, a random variable (such as value 1 or 0) is used to control whether The histogram equalization processing can be achieved, and the image has a change in the equalization effect of the image.
[0044] In the embodiment of the present application, when the RGB domain gaining process includes at least two of the color information adjustment, gamma transformation, and the random histogram equalization, the color information is adjusted before the gamma transformation. The gamma transform is performed before the random histogram is equalized, and it can obtain a better increase. For example, the initial training image can be adjusted to the initial training image, resulting in a color information adjusted, and then the image adjusted after color information is given to gamma transform, and the image after gamma shift is obtained as a RGB domain. image. For another example, the initial training image can be adjusted to obtain the color information, obtain the image adjusted after color information, and then adjust the image of the color information adjusted to equalize the random histogram, and obtain the image after the random histogram, as RGB Domain gaining image. As another example, the initial training image can be combined to obtain gamma transformation, and the image after gamma transform is obtained, and then the random histogram of the gamma transform is equalized, and the random histogram is equalized after the random histogram, as RGB Domain gaining image.
[0045] In yet another example, you can first adjust the introduction of the initial training image to obtain the image adjusted after color information, and then the image adjusted after the color information is changed to gamma transform, and then the gamma The changed image is equalized, and the random histogram is equalized after the random histogram, as an image of the RGB domain, such as Figure 4 The shown. In this example, RGB domain gaining processing includes color information adjustment, gamma transform, and random histogram equalization, and can maximize the training capabilities of the image after the RGB domain increased, and enhance training is good. The image recognition model is for the generalization of different modules.
[0046] In the embodiment of the present application, the HSV domain growth can include saturation adjustment and / or contrast adjustment; wherein the saturation parameters employed in the saturation adjustment are fine-tuning according to the random variable. The contrast parameters employed in the contrast adjustment are fine-tuning the preset parameters according to the random variable. During the ISP processing of different modules, contrast and saturation are often debugging according to the aesthetics of ISP debugers, and the saturation of different modules is different. Thus, in this embodiment, a saturation of the initial training image or the RGB domain growth process is saturated by fine-tuning the preset parameters in the HSV domain according to the random variable. Adjustment, the increase in image saturation information can be achieved. Further, by performing contrast adjustment of the image to the image in the HSV field according to the random variable, the contrast parameter is contracted (within a reasonable range) contrast adjustment, the increase in image contrast information can be achieved.
[0047] In the embodiment of the present application, the saturation adjustment is performed before the contrast adjustment is adjusted when the HSV domain gaining process includes both the saturation adjustment and the contrast adjustment. For example, it is possible to perform saturation adjustment of the initial training image or a training image of the RGB domain increasing process to obtain an image adjusted by saturated degree, and then adjust the image of the saturation adjustment to obtain an HSV domain growth. Training image, this process is like Figure 5 The shown. In this example, the HSV domain gaining process includes both saturation adjustment and contrast adjustment, and can maximize the training capabilities of the image after the HSV domain, enhanced image recognition model for different modules. Wideness.
[0048] In the embodiment of the present application, YUV domain growth can include noise reduction and / or edge enhancement; wherein the parameters of the low-pass filter used in the YUV domain noise are fine-tuning according to the random variable. It is obtained, the parameters used in the edge enhancement are obtained by fine-tuning the preset parameters according to the random variable. In this embodiment, the initial training image or the RGB domain growth process or the RGB domain increased processing image or the RGB domain increased training image or the RGB domain increased by the random variables in the YUV domain to fine-tune the preset parameters (reasonable range) The training image of the HSV domain increased treatment is noise reduction, which can achieve the increase in image noise reduction. In addition, by performing edge enhancement of the image in the yuv domain to fine-tuning the preset parameters according to the random variable, the array is enhanced to the image, and the degree of image sharpening can be achieved.
[0049]In an embodiment of the present application, when the YUV domain augmentation process comprising simultaneously both noise reduction and edge enhancement, noise reduction performed in the YUV domain prior to the edge enhancement. For example, training may be performed first initial image or the training image by the RGB domain augmented by image processing or training augmented HSV domain noise reduction processing, the image obtained after the noise reduction, noise reduction and then the image edge enhancement, to be trained by the YUV domain augmented image processing, such as the process Image 6 The shown. In this example, augmented YUV domain comprises simultaneously processing both noise reduction and edge enhancement, to maximize the capabilities of the image after the training YUV domain augmented treatment, enhanced image recognition models trained for different modules generalization.
[0050] The above example illustrates a method for processing a wide recognition model training images of an embodiment of the present application by ISP initial training images. Have better performance when image recognition model training to train the processed image based augmented by the ISP, the trained image recognition model obtained by image recognition for the identification image to be from a different camera.
[0051] In an embodiment of the present application, it may also be a combination of both initial training image and the training images processed by the ISP augmented model to train image recognition, which can further improve the ability of the training set of training images, obtained for different modules of the shooting model image having a higher image recognition robust image recognition capabilities.
[0052] In an embodiment of the present application, the foregoing model can be image recognition face recognition model to obtain recognition model based on previously described methods of training images from different cameras can obtain a highly accurate recognition result , which for some scenarios, such as security, law enforcement and other fields have a significant beneficial effect.
[0053] Based on the above description, according to the image recognition method for training a model embodiment of the present application for processing ISP augmented by the initial training images may be achieved based on a simulation of a training image or a camera to obtain several different camera models the effect of training images, which image recognition model based on image augmentation process after training get will enhance the generalization of different modules, so that the image recognition model for the image to be recognized different imaging module to capture the have good image recognition performance.
[0054] The above example illustrates a method for training model image recognition according to an embodiment of the present application. Bonded below Figure 7 The image recognition method according to an embodiment of the present application described embodiment. Figure 7 Shows a schematic flowchart of an image recognition method according to an embodiment of the present application 700. like Figure 7 , The image recognition method according to an embodiment of the present application 700 may include the steps of:
[0055] In step S710, the acquired image to be recognized.
[0056] In step S720, the image recognition models trained on the image to be recognized based on the image recognition, wherein the image recognition model training images used in the training is performed by the image signal processing on the wide initial training image processing training resulting image.
[0057] The image recognition model employed In an embodiment of the present application, the image recognition method 700 to be recognized in the image for image recognition is based on a training image by the image signal processing to the initial training images wide processing obtained and training obtained, i.e., the image recognition model using the image recognition method 700 when the image to be recognized is an image recognition training and obtained according to the method previously described for the training model image recognition application according to the embodiment of the present embodiment, as described above, according to the present application Example of the method for image recognition models trained by the initial training image ISP augmented treatment, it may be implemented based on a simulation of a training image or a camera to obtain several different camera models training image effect, thus the image based on image recognition model training after treating the resulting augmented will enhance the generalization of different modules. Therefore, such an image recognition model 700 can have a very good image recognition performance for the image to be recognized in different imaging module to capture an image recognition method according to an embodiment of the present application. Those skilled in the art can be understood that previously described in conjunction with the training method of the image recognition of the image recognition method 700 model employed, for brevity, is not repeated here.
[0058] Bonded below Figure 8 to 10 The image recognition apparatus described aspect provided herein, which may be used to perform / or image recognition methods previously described and used to train image recognition method according to an embodiment of the present model application. Those skilled in the art may be combined with the content described hereinbefore specific understanding of the structure and operation of the image recognition apparatus according to an embodiment of the present application, for brevity, specific details are not repeated here, only a description of some of the major operations.
[0059] Figure 8 Shows a schematic block diagram of an image recognition apparatus 800 according to an embodiment of the present application. like Figure 8 , The image recognition apparatus 800 includes a module 810 augmented and training module 820. Wherein the augmented initial training module 810 for acquiring images, the original training image signal processing augmented image processing, to obtain processed training images. Training module 820 is used to train image recognition model training augmented image processing based on the output module 810.
[0060] In an embodiment of the present application, augmented initial training module of the acquired image 810 may be a one or several cameras (image forming apparatus, also referred to as module) image photographed. As described above, if only several uses a camera image as a set of training images of the training image recognition model, the image recognition models trained on captured image to be recognized for that several cameras can be obtained when the image recognition is good performance, but in reality, image recognition models trained in use of the process, especially as the time of the sale of a SDK, which will identify the source (module) images is uncertain, there is no guarantee image recognition performance of the model. Thus, in an embodiment of the present application, augmented initial training module of the acquired image 810 performs image signal processing (Image Signal Process, abbreviated as ISP), the augmented treatment, wherein the treatment is augmented ISP ISP will flow parameters affecting the performance of the image recognition model (such as a color conversion matrix parameters, gamma conversion coefficients, the saturation parameter, contrast parameter and the like, will hereinafter be described in more detail in the example) of the random variable according to preset parameters obtained by trimming, for such initial training parameter based on the image processing (processing itself depends on parameters, such as color transformation matrix parameters for adjusting the color information, the gamma conversion coefficient for performing gamma conversion, saturation parameters are used to adjust the saturation, contrast parameter for contrast adjustment etc.), to obtain training images processed further for image recognition training by the training module 820, to ensure recognition model photographed image to be recognized for different imaging module the image has good image recognition performance. This is because the main difference is the different between the different modules of ISP algorithm, wherein the algorithm is a module ISP image sensor acquired raw (Raw) image into an RGB image daily use of the process. Thus, in an embodiment of the present application, the processing performed by the ISP augmented augmented initial training image module 810 may be implemented based on a simulation of a training image or a camera to obtain several different camera models training effect of the image, thus the image recognition model training module 820 based on the training image augmentation process will enhance the generalized obtained for different modules, such that image recognition for the image recognition model image to be recognized of different imaging module to capture have a very good performance.
[0061] In an embodiment of the present application, the initial training module augmented image 810 performs the processing of augmented ISP may comprise at least one of: RGB domain augmentation process, HSV-domain processing and YUV domain augmented augmentation process. Which, RGB, HSV, YUV color space are different. Accordingly, RGB fields augmentation process is a process in the RGB color space, HSV-domain processing means augmentation process in the HSV color space, YUV domain augmented processing means processing the YUV color space. Different color spaces generally have a different ISP processing chain, at least a part of ISP process has been augmented process, you can achieve ISP augmenting the initial training images, of course, handle multiple ISP links are augmented process can be further strengthened training capacity training data to enhance the generalization of the trained image recognition models for different modules.
[0062] General aspects of processing the RGB domain processing chain domain will affect HSV, HSV domain processing chain will generally affect the processing chain YUV domain, and therefore, in one embodiment of the present application, when augmented ISP augmented module 810 for processing the RGB domain comprises augmentation process, at least two processing and augmenting HSV domain augmented YUV domain processing, the RGB domain augmentation process performed before the HSV domain augmented treatment, HSV domain by wide process executed before the YUV domain augmentation treatment, which can simplify the process and improve efficiency. For example, augmentation module 810 may be performed first the RGB domain augmented process on the original training images, image can be obtained by the RGB domain augmented process, then image processing on by the RGB domain augmented for HSV domain augmented to give after HSV wide field image by processing an image for image recognition training model. As another example, module 810 may be augmented to perform processing on the RGB domain augmented the initial training images, images obtained by processing the RGB domain augmented, then the image processing performed by the RGB domain augmented YUV domain augmented to give YUV domain augmented by image processing as an image for image recognition training model. As another example, module 810 may be augmented to the initial training image augmenting HSV-domain processing to obtain the image domain by HSV augmented treatment, and then the image is augmented by the HSV domain YUV domain augmented the process proceeds to give YUV domain augmented by image processing as an image for image recognition training model.
[0063] In yet another example, the augmented module 810 may be performed first the RGB domain augmented process on the original training images, image can be obtained by the RGB domain augmented treatment, and then the image by the RGB domain augmented the process proceeds HSV domain augmented to give an image after processing by the HSV domain augmented, and then the image is augmented by the HSV domain YUV domain augmented the process proceeds to give an image after processing by the YUV domain augmented, as the training images for recognition model image. In this example, the ISP augmented process comprising simultaneously processing the RGB domain augmented, HSV-domain processing and YUV domain augmented augmenting the three processing, the ability to maximize the training of the augmented image processing, maximum enhanced image recognition models trained generalization for different modules.
[0064] In an embodiment of the present application, the RGB domain augmented module 810 for processing the augmentation may comprise adjusting at least one of color information, the gamma (the Gamma) conversion and random histogram equalization; wherein, the color information adjustment color correction matrix (color correction matrix, referred to as the CCM, usually 3 * 3 matrix) and a bias matrix (typically a matrix of 3 *) are the default parameters to fine-tune the parameters of the random variables used to obtain ; and the gamma-gamma conversion coefficient is used in accordance with preset parameters of the random variable trim obtained; histogram equalization of the random refers to determining whether the histogram equalization based on random variables.
[0065]In this embodiment, the increased distribution module 810 can be adjusted to the initial training image by using a color correction matrix and a bias matrix of fine-tuning the preset parameters including the random variable, which can be adjusted to the initial training image (generally fine tuning). Training images of different color information, realizing the increase in image color information. Further, the increased distribution module 810 is passed (in its reasonable range) to perform global gamma transitions on the preset parameters according to the random variable, which can perform various random varying degrees for exposure or exposure. Correction, realize the increase in image exposure correction. In addition, different modules have a histogram equalization, and some will not perform histogram equalization, so the increased distribution module 810 is equalized by random histogram, ie, a random variable (such as value 1 or 0) ) To control whether the histogram equalization processing is performed, the image can be achieved with the increase in the equilibrium effectiveness of the image.
[0066] In the embodiment of the present application, the color information is adjusted in the gamma when the RGB domain growth process performed by the increased distribution module 810 includes at least two of the color information adjustment, gamma transformation, and the random histogram equalization. Before the horse transformation, the gamma transformation is performed before the random histogram is equalized, and it is possible to obtain a better increase. For example, the increased distribution module 810 can first adjust the color information to the initial training image, obtain the image adjusted after color information, and then the image adjusted by the color information is given to gamma transform, and the image after gamma transform is obtained as a RGB domain. Conversion processing image. For another example, the increased distribution module 810 can first adjust the introduction of the initial training image to obtain the image-adjusted image, and then the image adjusted after color information is equalized, and the random histogram is equalized after the random histogram As an image of the RGB domain is increased. As another example, the increase in the increase in gamma converted by the initial training image, obtains the image after gamma converting, and then the random histogram is equalized to the random histogram. As an image of the RGB domain is increased.
[0067] In yet another example, the increased distribution module 810 can first adjust the color information to the initial training image, obtain the image-adjusted image, and then the color information adjusted, gamma transform, to obtain the image after gamma converting, Then the random histogram is equalized to the random histogram of the random histogram, and an image of the RGB domain is increased. In this example, the RGB domain growth process performed by the Conservation Module 810 includes color information adjustment, gamma transform, and random histogram equalization, and can maximize the training of images after RGB domain. Ability, enhanced image identification model for different modules for the generalization of different modules.
[0068] In the embodiment of the present application, the HSV domain growth process performed by the Connection Module 810 may include saturation adjustment and / or contrast adjustment; wherein the saturation parameters employed in the saturation adjustment are based on random variables. The parameters are fine-tuning, and the contrast parameters used in the contrast adjustment are fine-tuning the preset parameters based on the random variable. During the ISP processing of different modules, contrast and saturation are often debugging according to the aesthetics of ISP debugers, and the saturation of different modules is different. Thus, in this embodiment, the increased distribution module 810 is trained to the initial training image or the RGB domain growth process by fine-tuning the preset parameters in the HSV domain according to the random variable. The image is saturated, and the increase in image saturation information can be achieved. Further, the increased distribution module 810 is contracted by the image contrast information by tightening the preset parameters in the HSV domain according to the random variable, which is randomly generated (rational range) contrast adjustment, and the increase in image contrast information can be achieved.
[0069] In the embodiment of the present application, the saturation adjustment is performed before the contrast adjustment is adjusted when the HSV domain gaining process performed by the increase in the increased reservation module 810 also includes the saturation adjustment and contrast adjustment. For example, the increased distribution module 810 can first match the initial training image or a training image of the RGB domain increased training image to obtain a saturated image, and then adjust the saturation adjustment image to be contracted to obtain HSV. Domain growth processing training images. In this example, the HSV domain growth process performed by the Conservation Module 810 includes both of the saturation adjustment and contrast adjustment, and can maximize the training capabilities of the image after the HSV domain increased, enhanced the well-trained image. Identify the gentructure of different modules.
[0070] In the embodiment of the present application, the YUV domain growth process performed by the increased distribution module 810 may include noise reduction and / or edge enhancement; wherein the parameters of the low-pass filter used in the YUV domain noise are according to the random variable. The parameters employed in the edge enhancement are fine-tuning the preset parameters in accordance with the random variable. In this embodiment, the increased distribution module 810 is added to the initial training image or the RGB domain increased by the RGB domain by fine-tuning the preset parameters in the YUV domain according to the random variable. Training images or training images treated with HSV domains for noise reduction, can achieve the increase in image noise reduction effect. Further, the increased distribution module 810 is enhanced by the argument of the image sharpening algorithm to the image by fine-tuning the preset parameters in the YUV domain according to the random variable, and can achieve an increase in image sharpening.
[0071] In the embodiment of the present application, the YUV domain noise is performed before the edge enhancement is performed when the YUV domain growth process performed by the increased distribution module 810 includes noise reduction and edge enhancement. For example, the increased distribution module 810 can prior to the initial training image or a training image for increasing the RGB domain increase or a training image of the HSV domain increased, which is obtained by noise reduction, and then the image after noise reduction Perform an edge enhancement to obtain a training image for increasing the YUV domain. In this example, the YUV domain growth process performed by the Conservation Module 810 includes both noise reduction and edge enhancement, and can maximize the training capabilities of the image after the YUV domain increased, enhanced the well-trained image identification. The model is directed to the generalization of different modules.
[0072] The above exemplarily exemplarily shows the ISP increase of the initial training image in the image recognition apparatus according to the present application embodiment. The training module 820 carries the image identification model based on the training image after the ISP increase process, and the resulting training image recognition model has better performance for image recognition for image recognition from different cameras.
[0073] In the embodiment of the present application, the training module 820 can also train the image identification model in conjunction with the initial training image and the ISP increased training image, which further improves the training capabilities of training image sets to obtain different modes. The image-shot image has a more robust image recognition model.
[0074] Based on the above description, the image identification device according to the present application, the initial training image is performed by an increase in the initial training image, and the training image simulation based on one or a few cameras can be realized to obtain a variety of different cameras. The effect of the training image is taken, so the image recognition model obtained by the training module based on the increased image training will enhance the generalization of different modules, so that the image recognition model is performed for different imaging modules to be identified. There is a good performance when image recognition.
[0075] Figure 9 A schematic block diagram of the image recognition apparatus 900 according to another embodiment of the present application is shown. like Figure 9 As shown, the image recognition device 900 can include a acquisition module 910 and an identification module 920. The acquisition module 910 is used to obtain an image to be identified. The identification module 920 is configured to image the image to be identified based on the training well-trained image recognition model, wherein the image recognition model is a method for training image recognition models, according to the present application embodiment, according to the foregoing. Training is obtained. The image recognition apparatus 900 according to the present application embodiment can perform the image recognition method 700 according to the present application embodiment described above. Those skilled in the art can understand the specific operation of the image recognition apparatus 900 according to the present application embodiment, and will not be described later in order to understand the content of the image recognition apparatus 900 of the present application.
[0076] Figure 10 A schematic block diagram of the image recognition apparatus 1000 according to still another embodiment of the present application is shown. like Figure 10 As shown, the image recognition apparatus 1000 according to the present application embodiment can include a memory 1010 and a processor 1020 that stores a computer program operated by the processor 1020, the computer program, which is operated by the processor 1020, so that the processor 1020 Perform a method or image recognition method for training image recognition models according to the present application embodiment described above. Those skilled in the art can understand the specific operation of the image recognition apparatus 1000 according to the present application embodiment, for the sake of brevity, and other main operations of the processor 1020 will not be described herein.
[0077] In one embodiment of the present application, the computer program operates when the processor 1020 is operated, so that the processor 1020 performs the following steps: acquiring the initial training image, the initial training image, the increase of image signal processing, resulting The processed training image; the image recognition model is trained based on the processed training image.
[0078] In one embodiment of the present application, the computer program operates when the processor 1020 is operated, such that the processor 1020 performs an increase in image signal processing on the initial training image, including: the initial The training image is performed in at least one of the following: RGB domain gaining processing, HSV domain growth processing, and YUV domain growth process.
[0079] In one embodiment of the present application, the computer program is operated by the processor 1020 such that the RGB domain growth process performed by the processor 1020 includes at least one of color information adjustment, gamma transform, and random histogram equalization. Item; wherein the parameters in the color correction matrix used in the color information adjustment are obtained by fine-tuning the preset parameters according to the random variable; the gamma coefficients used in the gamma transformation are according to the basis The random variable is obtained by fine-tuning the preset parameters; the random histogram equalization means that the random variable determines whether or not the histogram is equalized.
[0080] In one embodiment of the present application, the computer program is operated by the processor 1020 such that the HSV domain growth process performed by the processor 1020 includes saturation adjustment and / or contrast adjustment; wherein the saturation adjustment is in The saturation parameters employed are fine-tuning according to the random variable, and the contrast parameters employed in the contrast adjustment are fine-tuning the preset parameters according to the random variable.
[0081] In one embodiment of the present application, the computer program is operated by the processor 1020 such that the YUV domain growth of processor 1020 includes noise reduction and / or edge enhancement; wherein the YUV domain is noise reduction The parameters of the low-pass filter used are obtained by fine-tuning the preset parameters according to the random variable, and the parameters used in the edge enhances are fine-tuning the preset parameters according to the random variable.
[0082] In one embodiment of the present application, the image recognition model is a face recognition model.
[0083] In one embodiment of the present application, the computer program is operated when the processor 1020 is operated, and the processor 1020 also performs the following steps: acquiring the image to be identified; the image based on the training-based image recognition model to the image to be identified Image recognition, wherein the image recognition model is obtained based on the method of training image recognition model described above.
[0084]Further, according to the embodiment of the present application, a storage medium is also provided, and a program instruction is stored on the storage medium, and is used to perform the present application embodiment for training when the program instruction is operated by the computer or processor. The method of image recognition model or the corresponding step of the image recognition method. The storage medium can include, for example, a memory card of a smartphone, a storage component of a tablet, a hard disk of a personal computer, a read-only memory (ROM), which can be erased programmable read-only memory, portable tightening disk read only memory. (CD-ROM), USB memory, or any combination of the storage medium described above. The computer readable storage medium can be any combination of one or more computer readable storage media.
[0085] Based on the above description, the image identification method, and image recognition apparatus for training image recognition model according to the present application can achieve an ISP income processing of the initial training image, which can be taken based on one or a few cameras. The training image simulates the effect of obtaining a training image captured by a variety of different cameras, so the image recognition model obtained based on the increased image training will enhance the generalization of different modules, so that the image recognition model is for different imaging. There is a good performance when the image to be identified by the module is image recognition.
[0086] Although the exemplary embodiments are described herein, it is to be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application. One of ordinary skill in the art can carry out various changes and modifications therein without departing from the scope and spirit of the present application. All of these changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.
[0087] One of ordinary skill in the art will appreciate that the unit and algorithm steps described herein described herein can be accomplished by electronic hardware, or computer software and electronic hardware. These functions are executed in hardware or software, depending on the specific application and design constraint conditions of the technical solution. Professional technicians can use different methods to implement the described functions for each particular application, but this implementation should not be considered exceeded the scope of this application.
[0088] In several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the apparatus embodiment described above is merely schematic, for example, the division of the unit is only one logical function division, and there may be additional division methods, such as a plurality of units or components, may be combined. Can be integrated into another device, or some features can be ignored, or not executed.
[0089] In the specification provided here, a large number of specific details are explained. However, it will be appreciated that the embodiments of the present application can be practiced without these specific details. In some examples, well-known methods, structures, and techniques are not shown in detail so that it is not blurred to understand the specification.
[0090] Similarly, it should be understood that in order to streaminize the application and help understand one or more of the various inventions, in the description of the exemplary embodiments of the present invention, the various features of the present application are sometimes grouped into a single embodiment. Or in the description thereof. However, the method of the present application should not be interpreted reflected in the following description: The present application claims is more features than the features explicitly described in each of the claims. More specifically, as reflected in the claims, the invention is in order to solve the corresponding technical problems with features of all features of a single embodiment. Thus, the claims of the specific embodiments will be described herein, and each of the claims itself is used as separate embodiments of the present application.
[0091] Those skilled in the art will appreciate that any combination of all features disclosed in this specification (including the appended claims, abstracts, and drawings) can be employed in addition to the features of the present specification and any of the methods or equipment of this disclosure. Process or units are combined. Each of the features disclosed in this specification (including the appended claims, abstracts, and drawings) can be replaced by an alternative feature provided by the same, equivalent or similar purpose.
[0092] Moreover, those skilled in the art will appreciate that although some of the embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments means that in the scope of the present application And different embodiments are formed. For example, in the claims, any of the claimed embodiments can be used in any combination.
[0093] Each component embodiment of the present application may be implemented in hardware, or in a software module running on one or more processors, or in their combined implementation. Those skilled in the art will appreciate that some or all of the modules in accordance with the embodiments of the present application can be implemented in practice using a microprocessor or a digital signal processor (DSP) in practice. The present application can also be implemented as a device program (e.g., computer program and computer program product) for performing a part or all of the methods described herein. Such a program for implementing the present application can be stored on a computer readable medium or may have a form of one or more signals. Such signals can be downloaded from the Internet website or is provided on the carrier signal or in any other form.
[0094] It should be noted that the above embodiments will be described with reference to the present application, and will be designed to be designed without departing from the scope of the appended claims. In the claims, any reference symbols located between parentheses should not be constructed to limit the claims. The word "contain" does not exclude the components or steps present in the claims. The word "one" or "one" in the component does not exclude multiple such components. The present application can be implemented by means of hardware including several different components and by means of appropriately programmed computers. In the unit claim, several devices are listed, and several of these devices may be embodied in the same hardware item. The use of words first, second, and third, etc. do not represent any order. These words can be interpreted as name.
[0095] As described above, only the specific embodiments of the present application or the description of the specific embodiments are not limited thereto, and any skilled in the art will be easily in the technical scope of the present invention. Think of changes or replacements should be covered within the scope of protection of this application. The scope of protection of this application should be based on the protection range of the claims.
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

no PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Similar technology patents

Unified low sample relation extraction method and device based on multi-choice matching network

PendingCN114528400AReduce computational cost and computational speedImprove generalizationSemantic analysisCharacter and pattern recognitionLanguage modelNetwork architecture
Owner:INST OF SOFTWARE - CHINESE ACAD OF SCI

Classification and recommendation of technical efficacy words

  • Good performance
  • Improve generalization

Image retrieval method and system

InactiveCN108829848AImprove accuracyImprove generalizationCharacter and pattern recognitionSpecial data processing applicationsExclusive orMultiple image
Owner:HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products