Image preprocessing and model training method, device, server and storage medium

By performing targeted pre-processing according to the image type, including noise and blurring of the optical flow map, and processing the channel weight ratio of RGB images, the problem of insufficient generalization performance in the prior art is solved, and the image is significantly improved Classification accuracy.

CN112308095BActive Publication Date: 2024-12-03SF TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910695572.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-07-30
Publication Date
2024-12-03
Estimated Expiration
2039-07-30

AI Technical Summary

Technical Problem

In the visual fields such as image classification and behavior detection, the existing data enhancement scheme has weak generalization performance, low classification accuracy, and cannot effectively extract image features, resulting in insufficient generalization capabilities of the model.

Method used

According to the type of sample pictures, different preprocessing methods are adopted, including noise processing and blurring of the optical flow diagram, channel weight ratio processing for RGB pictures, adding salt and pepper noise and environmental noise, enhancing image characteristics, and enriching training sample data.

Benefits of technology

Through rich preprocessing methods, the model's attention to features is improved, and the influence parameters and signals are ignored, which significantly improves the classification accuracy of the model. For example, the classification accuracy of the optical flow graph has been increased from 96.5% to 97.2%, and the RGB image is The classification accuracy rate increased from 97.8% to 99.1%.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112308095B_ABST
    Figure CN112308095B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a method, device, server and storage medium for image preprocessing and model training. The image preprocessing method includes: obtaining a sample image; determining the type of the sample image; determining a preprocessing method for the sample image according to the type of the sample image; preprocessing the sample image according to the preprocessing method to obtain a training sample image. The model training method includes: obtaining a training sample image according to the image preprocessing method; adding the training sample image to a training sample set; training an initial model according to the sample image in the training sample set to obtain a trained model. In the embodiment of the present invention, different preprocessing methods are used for different types of sample images, so that the sample images can show richer influencing parameters and signals. Therefore, when the sample images are used for model training later, the model can focus more on the features and ignore these influencing parameters or signals, thereby improving the classification accuracy of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, server and storage medium for image preprocessing and model training. Background Art

[0002] In deep learning of images, in order to enrich the image sample training set, better extract image features, and generalize the model (to prevent model overfitting), data images are generally enhanced. Common ways of data enhancement are to rotate images, cut images, change image color difference, distort image features, change image size, enhance image noise, mirror operations, smooth operations, contrast and brightness operations, etc. In different task contexts, the amount of sample data can be increased by using one or more combined data enhancement transformations through geometric transformations of images.

[0003] In visual fields such as image classification and behavior detection, the existing data enhancement scheme has only one basic version, which includes mirroring operations, smoothing operations, contrast and brightness operations, rotation operations, shearing operations, and adjustable parameters, etc. Before training the model, the user preprocesses the data set according to the basic version of the data enhancement scheme. The basic version supports most data set preprocessing, but for data of different objects, the model generalization performance obtained after training with sample data obtained using the basic version of the data enhancement method is weak, and the classification accuracy is not high. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, server and storage medium for image preprocessing and model training, so that sample images can show richer influencing parameters and signals. Therefore, when the sample images are subsequently used for model training, the model can focus more on the features and ignore these influencing parameters or signals, thereby improving the classification accuracy of the model.

[0005] In a first aspect, the present application provides a method for preprocessing an image, the method comprising:

[0006] Get sample images;

[0007] Determine the type of the sample image;

[0008] Determining a preprocessing method for the sample image according to the type of the sample image;

[0009] The sample images are preprocessed according to the preprocessing method to obtain training sample images.

[0010] In some embodiments of the present application, determining a preprocessing method for the sample image according to the type of the sample image includes:

[0011] If the type of the sample image is an optical flow image, it is determined that the preprocessing method for the sample image is a first preprocessing method, and the first preprocessing method includes noise processing and blur processing.

[0012] In some embodiments of the present application, preprocessing the sample image according to the preprocessing method to obtain a training sample image includes:

[0013] Performing a first basic augmentation process on the sample image to obtain a first sample image;

[0014] Performing channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the first sample image to obtain a second sample image;

[0015] The second sample image is blurred to add environmental noise to the second sample image to obtain a training sample image.

[0016] In some embodiments of the present application, determining a preprocessing method for the sample image according to the type of the sample image includes:

[0017] If the type of the sample image is an RGB image, it is determined that the preprocessing method for the sample image is a second preprocessing method, and the second preprocessing method includes channel weight matching processing.

[0018] In some embodiments of the present application, preprocessing the sample image according to the preprocessing method to obtain a training sample image includes:

[0019] Performing a second basic augmentation process on the sample image to obtain a third sample image;

[0020] Performing noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image to obtain a fourth sample image;

[0021] A Gaussian distribution weight ratio process is performed on the fourth sample image to assign different weight ratios to the three channels R, G, and B in the fourth sample image;

[0022] The three channels of R, G, and B with different weight ratios are synthesized to obtain the training sample image.

[0023] In a second aspect, the present application provides a model training method, the model training method comprising:

[0024] Get sample images;

[0025] Determine the type of the sample image;

[0026] Determining a preprocessing method for the sample image according to the type of the sample image;

[0027] Preprocessing the sample image according to the preprocessing method to obtain a training sample image;

[0028] Adding the training sample image to a training sample set;

[0029] An initial model is trained according to the sample images in the training sample set to obtain a trained model.

[0030] In some embodiments of the present application, determining a preprocessing method for the sample image according to the type of the sample image includes:

[0031] If the type of the sample image is an optical flow image, it is determined that the preprocessing method for the sample image is a first preprocessing method, and the first preprocessing method includes noise processing and blur processing.

[0032] In some embodiments of the present application, preprocessing the sample image according to the preprocessing method to obtain a training sample image includes:

[0033] Performing a first basic augmentation process on the sample image to obtain a first sample image;

[0034] Performing channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the first sample image to obtain a second sample image;

[0035] The second sample image is blurred to add environmental noise to the second sample image to obtain a training sample image.

[0036] In some embodiments of the present application, determining a preprocessing method for the sample image according to the type of the sample image includes:

[0037] If the type of the sample image is an RGB image, it is determined that the preprocessing method for the sample image is a second preprocessing method, and the second preprocessing method includes channel weight matching processing.

[0038] In some embodiments of the present application, preprocessing the sample image according to the preprocessing method to obtain a training sample image includes:

[0039] Performing a second basic augmentation process on the sample image to obtain a third sample image;

[0040] Performing noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image to obtain a fourth sample image;

[0041] A Gaussian distribution weight ratio process is performed on the fourth sample image to assign different weight ratios to the three channels R, G, and B in the fourth sample image;

[0042] The three channels of R, G, and B with different weight ratios are synthesized to obtain the training sample image.

[0043] In a third aspect, the present application provides a picture preprocessing device, the picture preprocessing device comprising:

[0044] An acquisition unit, used for acquiring a sample image;

[0045] A judging unit, used to judge the type of the sample image;

[0046] A determination unit, configured to determine a preprocessing method for the sample image according to the type of the sample image;

[0047] The preprocessing unit is used to preprocess the sample image according to the preprocessing method to obtain a training sample image.

[0048] In some embodiments of the present application, the determining unit is specifically used to:

[0049] If the type of the sample image is an optical flow image, it is determined that the preprocessing method for the sample image is a first preprocessing method, and the first preprocessing method includes noise processing and blur processing.

[0050] In some embodiments of the present application, the preprocessing unit is specifically used for:

[0051] Performing a first basic augmentation process on the sample image to obtain a first sample image;

[0052] Performing channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the first sample image to obtain a second sample image;

[0053] The second sample image is blurred to add environmental noise to the second sample image to obtain a training sample image.

[0054] In some embodiments of the present application, the determining unit is specifically used to:

[0055] If the type of the sample image is an RGB image, it is determined that the preprocessing method for the sample image is a second preprocessing method, and the second preprocessing method includes channel weight matching processing.

[0056] In some embodiments of the present application, the preprocessing unit is specifically used for:

[0057] Performing a second basic augmentation process on the sample image to obtain a third sample image;

[0058] Performing noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image to obtain a fourth sample image;

[0059] A Gaussian distribution weight ratio process is performed on the fourth sample image to assign different weight ratios to the three channels R, G, and B in the fourth sample image;

[0060] The three channels of R, G, and B with different weight ratios are synthesized to obtain the training sample image.

[0061] In a fourth aspect, the present application provides a model training device, the model training device comprising:

[0062] An acquisition unit, used for acquiring a sample image;

[0063] A judging unit, used to judge the type of the sample image;

[0064] A determination unit, configured to determine a preprocessing method for the sample image according to the type of the sample image;

[0065] A preprocessing unit, used to preprocess the sample image according to the preprocessing method to obtain a training sample image;

[0066] An adding unit, used to add the training sample image to a training sample set;

[0067] The training unit is used to train the initial model according to the sample images in the training sample set to obtain a trained model.

[0068] In some embodiments of the present application, the determining unit is specifically used to:

[0069] If the type of the sample image is an optical flow image, it is determined that the preprocessing method for the sample image is a first preprocessing method, and the first preprocessing method includes noise processing and blur processing.

[0070] In some embodiments of the present application, the preprocessing unit is specifically used for:

[0071] Performing a first basic augmentation process on the sample image to obtain a first sample image;

[0072] Performing channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the first sample image to obtain a second sample image;

[0073] The second sample image is blurred to add environmental noise to the second sample image to obtain a training sample image.

[0074] In some embodiments of the present application, the determining unit is specifically used to:

[0075] If the type of the sample image is an RGB image, it is determined that the preprocessing method for the sample image is a second preprocessing method, and the second preprocessing method includes channel weight matching processing.

[0076] In some embodiments of the present application, the preprocessing unit is specifically used for:

[0077] Performing a second basic augmentation process on the sample image to obtain a third sample image;

[0078] Performing noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image to obtain a fourth sample image;

[0079] A Gaussian distribution weight ratio process is performed on the fourth sample image to assign different weight ratios to the three channels R, G, and B in the fourth sample image;

[0080] The three channels of R, G, and B with different weight ratios are synthesized to obtain the training sample image.

[0081] In a fifth aspect, the present application provides a server, the server comprising:

[0082] one or more processors;

[0083] Memory; and

[0084] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the image preprocessing method described in any one of the first aspects, or the model training method described in the second aspect.

[0085] In a sixth aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the image preprocessing method described in any one of the first aspects, or the model training method described in the second aspect.

[0086] In the embodiment of the present invention, a sample image is obtained; the type of the sample image is determined; according to the type of the sample image, a preprocessing method for the sample image is determined; and the sample image is preprocessed according to the preprocessing method to obtain a training sample image. In the embodiment of the present invention, different preprocessing methods are used for different types of sample images, so that the sample images obtained by different processing can respectively show the influencing parameters and signals, thereby making the sample images can show richer influencing parameters and signals. Therefore, when the sample images are subsequently used for model training, the model can focus more on the features and ignore these influencing parameters or signals, thereby improving the classification accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0088] Figure 1 It is a schematic diagram of a flow chart of an embodiment of a method for preprocessing an image provided by an embodiment of the present invention;

[0089] Figure 2 is a flowchart of an embodiment of step 104 in the image preprocessing method provided by an embodiment of the present invention;

[0090] Figure 3 is a flowchart of another embodiment of step 104 in the image preprocessing method provided by an embodiment of the present invention;

[0091] Figure 4 It is a schematic diagram of an embodiment of the flow chart of the model training method provided by an embodiment of the present invention;

[0092] Figure 5 is a schematic diagram of the structure of an embodiment of a picture preprocessing device provided by an embodiment of the present invention;

[0093] Figure 6 is a schematic diagram of the structure of an embodiment of a model training device provided by an embodiment of the present invention;

[0094] Figure 7 It is a schematic diagram of the structure of an embodiment of a server provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0095] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0096] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0097] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0098] The embodiments of the present invention provide a method, device, server and storage medium for image preprocessing and model training, which are described in detail below.

[0099] First, an embodiment of the present invention provides an image preprocessing method, which includes: obtaining a sample image; determining the type of the sample image; determining a preprocessing method for the sample image according to the type of the sample image; and preprocessing the sample image according to the preprocessing method to obtain a training sample image.

[0100] like Figure 1 FIG. 1 is a flow chart of an embodiment of a method for image preprocessing in an embodiment of the present invention. The method for image preprocessing is applied to an image preprocessing device, and the image preprocessing device can be integrated in a server. Specifically, the method for image preprocessing includes:

[0101] 101. Get sample images.

[0102] In an embodiment of the present invention, the type of the sample image is determined based on the model obtained after training, and the type of the sample image corresponds to the model obtained after training. For example, if the model to be trained is a face recognition model, then the sample image is a face image. For another example, if the model to be trained is a license plate recognition model, then the sample image is a license plate image.

[0103] There are multiple ways to obtain sample images in the embodiments of the present invention. In some embodiments of the present invention, sample images can be obtained by photographing people or objects, that is, the sample images can be sample images taken by a photographing device (such as a mobile phone, camera, camcorder, and monitoring equipment, etc.), or sample images can be obtained from a sample image database, etc.

[0104] In other embodiments of the present invention, the sample image may refer to an original image (such as a sample image taken by a photographing device) that is processed to meet preset requirements. It should be noted that after the original image is obtained by photographing a person or object and the sample image database, the original image obtained may be non-standard. For example, the original image obtained has different sizes. Therefore, it is necessary to uniformly process the original images obtained to obtain sample images of uniform specifications. Furthermore, before obtaining the sample image, the original image needs to be processed so that the original image meets the preset requirements and is more conducive to the training of the model. For example, if the original image is a face image, the face can be corrected to be in the center position to obtain a sample image of uniform specifications.

[0105] 102. Determine the type of the sample image.

[0106] In an embodiment of the present invention, the type of the sample image may be an optical flow image or an RGB image. It is understandable that in some other embodiments of the present invention, the sample image may also be other types of images, such as a grayscale image, a channel image, etc.

[0107] Among them, the optical flow map is a picture with superimposed optical flow. The optical flow method is an important method for motion image analysis at present. Its concept was first proposed by James J. Gibson in the 1940s. It refers to the speed of pattern movement in time-varying images. Because when an object is moving, the brightness pattern of its corresponding point on the image is also moving. The optical flow expresses the changes in the image. Since it contains information about the movement of the target, it can be used by the observer to determine the movement of the target. The definition of optical flow can be extended to the optical flow field, which refers to a two-dimensional (2D) instantaneous velocity field composed of all pixels in the image, in which the two-dimensional velocity vector is the projection of the three-dimensional velocity vector of the visible point in the scene on the imaging surface. Therefore, the optical flow not only contains the motion information of the observed object, but also contains rich information about the three-dimensional structure of the scene.

[0108] In the embodiment of the present invention, the optical flow map may be obtained by extracting features in the captured video image frame through an optical flow method (for example, a TVL1 algorithm, an LK algorithm, etc.) to obtain the optical flow map.

[0109] 103. Determine a preprocessing method for the sample image according to the type of the sample image.

[0110] As can be seen from step 102, there are multiple types of sample images. The following describes multiple processing methods for determining the preprocessing method for the sample images according to the types of the sample images:

[0111] (1) If the type of the sample image is an optical flow image;

[0112] Specifically, determining the preprocessing method for the sample image according to the type of the sample image may include: if the type of the sample image is an optical flow map, determining that the preprocessing method for the sample image is a first preprocessing method, and the first preprocessing method includes noise processing and blur processing.

[0113] (2) If the type of the sample image is an RGB image;

[0114] At this time, determining the preprocessing method for the sample image according to the type of the sample image may specifically include: if the type of the sample image is an RGB image, determining that the preprocessing method for the sample image is a second preprocessing method, and the second preprocessing method includes channel weight ratio processing.

[0115] 104. Preprocess the sample image according to the preprocessing method to obtain a training sample image.

[0116] As can be seen from step 102, there are multiple types of sample images. The following introduces multiple processing methods for preprocessing the sample images according to the preprocessing methods to obtain training sample images:

[0117] (1) If the type of the sample image is an optical flow image;

[0118] like Figure 2 As shown, the preprocessing of the sample image according to the preprocessing method to obtain a training sample image may further include:

[0119] 201. Perform a first basic augmentation process on the sample image to obtain a first sample image.

[0120] The first basic augmentation processing may include existing common augmentation processing methods, such as rotating an image, shearing an image, changing image color difference, distorting image features, changing image size, mirroring operation, smoothing operation or contrast brightness operation, etc.

[0121] 202. Perform channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the first sample image to obtain a second sample image.

[0122] Specifically, channel Shuffle processing and Additive Laplace Noise processing are performed on the first sample image to add salt-and-pepper noise to the sample image.

[0123] Salt-and-pepper noise includes two types of noise, one is salt noise and the other is pepper noise. Salt noise = white (0), pepper noise = black (255). The former is high grayscale noise, and the latter is low grayscale noise. Generally, the two types of noise appear at the same time, which is presented as black and white noise on the image. Adding salt-and-pepper noise to the optical flow map makes the model trained by the optical flow map have better anti-interference effect on the optical flow map of black and white noise in the later stage.

[0124] 203. Perform blur processing on the second sample image to add environmental noise to the second sample image to obtain a training sample image.

[0125] Normally, pictures taken in special environments may produce blur effects, such as blurring of some pictures caused by rain, snow, and sandstorms. In order to make the sample pictures more suitable for the actual scene, the second sample pictures may be blurred, and environmental noise may be further added to the sample pictures to obtain training sample pictures. Specifically, blurring may be performed using a blur function, and environmental noise may be further added to the sample pictures.

[0126] After testing by the inventors, the training sample images obtained after processing the optical flow map in the above manner were used as the training sample images in the training sample set. The classification accuracy of the trained model increased from 96.5% of the initial model to 97.2%, showing better classification results for the optical flow map.

[0127] (2) If the type of the sample image is an RGB image;

[0128] like Figure 3 As shown, the preprocessing of the sample image according to the preprocessing method to obtain a training sample image may further include:

[0129] 301. Perform a second basic augmentation process on the sample image to obtain a third sample image.

[0130] Among them, the second basic augmentation processing can be the same as the first basic augmentation processing, or it can be different, that is, the second basic augmentation processing can also include existing common augmentation processing methods, such as rotating the image, shearing the image, changing the image color difference, distorting the image features, changing the image size, mirroring operation, smoothing operation or contrast brightness operation. In a preferred embodiment, the second basic augmentation processing adopts an augmentation processing method different from the first basic augmentation processing. For example, the first basic augmentation processing includes all operations of rotating the image, shearing the image, changing the image color difference, distorting the image features, changing the image size, mirroring operation, smoothing operation or contrast brightness operation. The second basic augmentation processing only includes rotating the image, shearing the image, changing the image color difference, distorting the image features, changing the image size, and mirroring operation. At least one of the operations is not included, and smoothing operation or contrast brightness operation is not included. In this way, different operations can make the interference factors after augmentation processing for different types of sample images different, so that the generalization performance of the trained model is better.

[0131] 302. Perform noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image to obtain a fourth sample image.

[0132] The manner of performing noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image may refer to the processing manner in steps 202 and 203, and will not be described in detail here.

[0133] 303. Perform Gaussian distribution weight matching processing on the fourth sample image, and assign different weight ratios to the three channels R, G, and B in the fourth sample image.

[0134] Among them, Gaussian distribution weight matching processing is a way of channel weight matching processing, which is used to match the weights of different channels in the picture through the probability density function of Gaussian distribution. Since the sample picture in this embodiment is an RGB picture, the RGB picture includes three channels of R (Red), G (Green), and B (Blue). Different weight ratios can be assigned to the three channels of R, G, and B in the fourth sample picture, for example, 50%, 20%, and 30%.

[0135] 304. Synthesize the three channels of R, G, and B with different weight ratios to obtain the training sample image.

[0136] In this embodiment, the three channels of R, G, and B with different weight ratios are resynthesized into a picture to obtain a training sample picture.

[0137] After testing by the inventors, the training sample images obtained by processing the RGB images in the above manner were used as the training sample images in the training sample set. The classification accuracy of the trained model increased from 97.8% of the initial model to 99.1%, showing better classification results for RGB images.

[0138] In the embodiment of the present invention, a sample image is obtained; the type of the sample image is determined; according to the type of the sample image, a preprocessing method for the sample image is determined; and the sample image is preprocessed according to the preprocessing method to obtain a training sample image. In the embodiment of the present invention, different preprocessing methods are used for different types of sample images, so that the sample images obtained by different processing can respectively show the influencing parameters and signals, thereby making the sample images can show richer influencing parameters and signals. Therefore, when the sample images are subsequently used for model training, the model can focus more on the features and ignore these influencing parameters or signals, thereby improving the classification accuracy of the model.

[0139] In order to better implement the image preprocessing method in the embodiment of the present invention, based on the image preprocessing method, the embodiment of the present invention also provides a model training method, such as Figure 4 FIG. 1 is a flow chart of an embodiment of a model training method in an embodiment of the present invention, and the model training method includes:

[0140] 401. Get sample images.

[0141] 402. Determine the type of the sample image.

[0142] 403. Determine a preprocessing method for the sample image according to the type of the sample image.

[0143] 404. Preprocess the sample image according to the preprocessing method to obtain a training sample image.

[0144] 405. Add the training sample image to a training sample set.

[0145] In this embodiment, each training sample image in the training sample set may be pre-processed using the steps in steps 401 to 404 .

[0146] 406. Train an initial model according to the sample images in the training sample set to obtain a trained model.

[0147] Among them, the initial model can be an existing neural network model, such as a convolutional neural network (CNN) model. It can be understood that the embodiment of the present invention is mainly based on the training method of the model, and the neural network model can also be a new neural network model that will appear in the future.

[0148] In this embodiment, steps 401 to 404 may be specifically referred to Figure 1 The specific process of the steps in the illustrated embodiment will not be repeated any more.

[0149] In the embodiment of the present invention, different preprocessing methods are used for different types of sample images, so that the sample images obtained by different processing can respectively display the influencing parameters and signals, thereby enabling the sample images to display richer influencing parameters and signals. The initial model is trained based on the training sample set composed of these training sample images. After training, the model can focus more on the features and ignore these influencing parameters or signals, thereby improving the classification accuracy of the model.

[0150] The following describes an embodiment of the device in the embodiment of the present invention.

[0151] See also Figure 5 , is a schematic diagram of the structure of an embodiment of a picture preprocessing device in an embodiment of the present invention, the picture preprocessing device 500 includes an acquisition unit 501, a judgment unit 502, a determination unit 503 and a preprocessing unit 504, wherein:

[0152] An acquisition unit 501 is used to acquire a sample image;

[0153] A determination unit 502, configured to determine the type of the sample image;

[0154] A determination unit 503, configured to determine a preprocessing method for the sample image according to the type of the sample image;

[0155] The preprocessing unit 504 is used to preprocess the sample image according to the preprocessing method to obtain a training sample image.

[0156] In some embodiments of the present application, the determining unit 503 is specifically used to:

[0157] If the type of the sample image is an optical flow image, it is determined that the preprocessing method for the sample image is a first preprocessing method, and the first preprocessing method includes noise processing and blur processing.

[0158] In some embodiments of the present application, the preprocessing unit 504 is specifically used for:

[0159] Performing a first basic augmentation process on the sample image to obtain a first sample image;

[0160] Performing channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the first sample image to obtain a second sample image;

[0161] The second sample image is blurred to add environmental noise to the second sample image to obtain a training sample image.

[0162] In some embodiments of the present application, the determining unit 503 is specifically used to:

[0163] If the type of the sample image is an RGB image, it is determined that the preprocessing method for the sample image is a second preprocessing method, and the second preprocessing method includes channel weight matching processing.

[0164] In some embodiments of the present application, the preprocessing unit 504 is specifically used for:

[0165] Performing a second basic augmentation process on the sample image to obtain a third sample image;

[0166] Performing noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image to obtain a fourth sample image;

[0167] A Gaussian distribution weight ratio process is performed on the fourth sample image to assign different weight ratios to the three channels R, G, and B in the fourth sample image;

[0168] The three channels of R, G, and B with different weight ratios are synthesized to obtain the training sample image.

[0169] In the embodiment of the present invention, the acquisition unit 501 acquires the sample image; the judgment unit 502 determines the type of the sample image; the determination unit 503 determines the preprocessing method for the sample image according to the type of the sample image; the preprocessing unit 504 preprocesses the sample image according to the preprocessing method to obtain the training sample image. In the embodiment of the present invention, different preprocessing methods are used for different types of sample images, so that the sample images obtained by different processing can respectively show the influencing parameters and signals, so that the sample images can show richer influencing parameters and signals. Therefore, when the sample images are used for model training later, the model can focus more on the features and ignore these influencing parameters or signals, thereby improving the classification accuracy of the model.

[0170] See also Figure 6 , is a schematic diagram of the structure of an embodiment of a model training device in an embodiment of the present invention, the model training device 600 includes an acquisition unit 601, a judgment unit 602, a determination unit 603, a preprocessing unit 604, an adding unit 605 and a training unit 606, wherein:

[0171] An acquisition unit 601 is used to acquire a sample image;

[0172] A determination unit 602, configured to determine the type of the sample image;

[0173] A determination unit 603, configured to determine a preprocessing method for the sample image according to the type of the sample image;

[0174] A preprocessing unit 604 is used to preprocess the sample image according to the preprocessing method to obtain a training sample image;

[0175] An adding unit 605, configured to add the training sample image to a training sample set;

[0176] The training unit 606 is used to train the initial model according to the sample images in the training sample set to obtain a trained model.

[0177] In some embodiments of the present application, the determining unit 603 is specifically used to:

[0178] If the type of the sample image is an optical flow image, it is determined that the preprocessing method for the sample image is a first preprocessing method, and the first preprocessing method includes noise processing and blur processing.

[0179] In some embodiments of the present application, the pre-processing unit 604 is specifically used for:

[0180] Performing a first basic augmentation process on the sample image to obtain a first sample image;

[0181] Performing channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the sample image to obtain a second sample image;

[0182] The second sample image is blurred to add environmental noise to the second sample image to obtain a training sample image.

[0183] In some embodiments of the present application, the determining unit 603 is specifically used to:

[0184] If the type of the sample image is an RGB image, it is determined that the preprocessing method for the sample image is a second preprocessing method, and the second preprocessing method includes channel weight matching processing.

[0185] In some embodiments of the present application, the pre-processing unit 604 is specifically used for:

[0186] Performing a second basic augmentation process on the sample image to obtain a third sample image;

[0187] Performing noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image to obtain a fourth sample image;

[0188] A Gaussian distribution weight ratio process is performed on the fourth sample image to assign different weight ratios to the three channels R, G, and B in the fourth sample image;

[0189] The three channels of R, G, and B with different weight ratios are synthesized to obtain the training sample image.

[0190] In the embodiment of the present invention, different preprocessing methods are used for different types of sample images, so that the sample images obtained by different processing can respectively display the influencing parameters and signals, thereby enabling the sample images to display richer influencing parameters and signals. The initial model is trained based on the training sample set composed of these training sample images. After training, the model can focus more on the features and ignore these influencing parameters or signals, thereby improving the classification accuracy of the model.

[0191] The embodiment of the present invention further provides a server, which integrates any one of the image preprocessing devices or model training devices provided in the embodiment of the present invention, and the server includes:

[0192] one or more processors;

[0193] Memory; and

[0194] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor in any of the steps in the above-mentioned image preprocessing method or model training method embodiments.

[0195] The embodiment of the present invention further provides a server, which integrates any one of the image preprocessing devices or model training devices provided by the embodiment of the present invention. Figure 7 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present invention, specifically:

[0196] The server may include one or more processing core processors 701, one or more computer-readable storage media memories 702, a power supply 703, an input unit 704 and other components. Those skilled in the art will appreciate that Figure 7 The server structure shown in the figure does not constitute a limitation on the server, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0197] The processor 701 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. By running or executing software programs and / or modules stored in the memory 702, and calling data stored in the memory 702, the processor 701 performs various functions of the server and processes data, thereby monitoring the server as a whole. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701.

[0198] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0199] The server also includes a power supply 703 for supplying power to each component. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 703 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0200] The server may further include an input unit 704, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0201] Although not shown, the server may further include a display unit, etc., which will not be described in detail here. Specifically in this embodiment, the processor 701 in the server will load the executable files corresponding to the processes of one or more applications into the memory 702 according to the following instructions, and the processor 701 will run the application stored in the memory 702, thereby realizing various functions, as follows:

[0202] Get sample images;

[0203] Determine the type of the sample image;

[0204] Determining a preprocessing method for the sample image according to the type of the sample image;

[0205] The sample images are preprocessed according to the preprocessing method to obtain training sample images.

[0206] Alternatively, the processor 701 runs the application program stored in the memory 702 to implement various functions, as follows:

[0207] Get sample images;

[0208] Determine the type of the sample image;

[0209] Determining a preprocessing method for the sample image according to the type of the sample image;

[0210] Preprocessing the sample image according to the preprocessing method to obtain a training sample image;

[0211] Adding the training sample image to a training sample set;

[0212] An initial model is trained according to the sample images in the training sample set to obtain a trained model.

[0213] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0214] To this end, an embodiment of the present invention provides a storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the image preprocessing methods or model training methods provided in the embodiments of the present invention. For example, the computer program may be loaded by a processor to execute the following steps:

[0215] Get sample images;

[0216] Determine the type of the sample image;

[0217] Determining a preprocessing method for the sample image according to the type of the sample image;

[0218] The sample images are preprocessed according to the preprocessing method to obtain training sample images.

[0219] For another example, the computer program may be loaded by the processor and the following steps may be executed:

[0220] Get sample images;

[0221] Determine the type of the sample image;

[0222] Determining a preprocessing method for the sample image according to the type of the sample image;

[0223] Preprocessing the sample image according to the preprocessing method to obtain a training sample image;

[0224] Adding the training sample image to a training sample set;

[0225] An initial model is trained according to the sample images in the training sample set to obtain a trained model.

[0226] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0227] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0228] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0229] The above is a detailed introduction to an image preprocessing and model training method, device, server and storage medium provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for image preprocessing, characterized in that: The image preprocessing method comprises: Obtain a sample image; wherein the sample image includes an optical flow image and an RGB image; Determine the type of the sample image; Determining a preprocessing method for the sample image according to the type of the sample image; wherein the preprocessing method includes adding salt and pepper noise and environmental noise; Preprocessing the sample image according to the preprocessing method to obtain a training sample image; The step of determining a preprocessing method for the sample image according to the type of the sample image includes: If the type of the sample image is an optical flow image, determining that the preprocessing method for the sample image is a first preprocessing method, wherein the first preprocessing method includes noise processing and blur processing; The preprocessing of the sample images according to the preprocessing method to obtain training sample images includes: If the type of the sample image is an optical flow image, performing a first basic augmentation process on the sample image to obtain a first sample image; Performing channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the first sample image to obtain a second sample image; The second sample image is blurred to add environmental noise to the second sample image to obtain a training sample image.

2. The image preprocessing method according to claim 1, characterized in that: The determining, according to the type of the sample image, a preprocessing method for the sample image includes: If the type of the sample image is an RGB image, it is determined that the preprocessing method for the sample image is a second preprocessing method, and the second preprocessing method includes channel weight matching processing.

3. The image preprocessing method according to claim 2, characterized in that: The preprocessing of the sample image according to the preprocessing method to obtain a training sample image includes: Performing a second basic augmentation process on the sample image to obtain a third sample image; Performing noise processing and blur processing on the third sample image to add salt and pepper noise and environmental noise to the third sample image to obtain a fourth sample image; A Gaussian distribution weight ratio process is performed on the fourth sample image to assign different weight ratios to the three channels R, G, and B in the fourth sample image; The three channels of R, G, and B with different weight ratios are synthesized to obtain the training sample image.

4. A model training method, characterized in that: The model training method comprises: Obtain a sample image; wherein the sample image includes an optical flow image and an RGB image; Determine the type of the sample image; Determining a preprocessing method for the sample image according to the type of the sample image; wherein the preprocessing method includes adding salt and pepper noise and environmental noise; Preprocessing the sample image according to the preprocessing method to obtain a training sample image; Adding the training sample image to a training sample set; Training an initial model according to sample images in the training sample set to obtain a trained model; The preprocessing of the sample images according to the preprocessing method to obtain training sample images includes: If the type of the sample image is an optical flow image, performing a first basic augmentation process on the sample image to obtain a first sample image; Performing channel shuffling and Laplace noise processing on the first sample image to add salt and pepper noise to the first sample image to obtain a second sample image; The second sample image is blurred to add environmental noise to the second sample image to obtain a training sample image.

5. A picture preprocessing device, characterized in that: The image preprocessing device comprises: An acquisition unit, used to acquire a sample image; wherein the sample image includes an optical flow image and an RGB image; A judging unit, used to judge the type of the sample image; A determination unit, configured to determine a preprocessing method for the sample image according to the type of the sample image; wherein the preprocessing method includes adding salt and pepper noise and environmental noise; specifically, if the type of the sample image is an optical flow map, the preprocessing method for the sample image is determined to be a first preprocessing method, and the first preprocessing method includes noise processing and blur processing; A preprocessing unit is used to preprocess the sample image according to the preprocessing method to obtain a training sample image; specifically, if the type of the sample image is an optical flow map, the sample image is subjected to a first basic augmentation process to obtain a first sample image; the first sample image is subjected to channel shuffling and Laplace noise processing to add salt and pepper noise to the first sample image to obtain a second sample image; the second sample image is subjected to blur processing to add environmental noise to the second sample image to obtain a training sample image.

6. A model training device, characterized in that: The model training device comprises: An acquisition unit, used to acquire a sample image; wherein the sample image includes an optical flow image and an RGB image; A judging unit, used to judge the type of the sample image; A determination unit, configured to determine a preprocessing method for the sample image according to the type of the sample image; wherein the preprocessing method includes adding salt and pepper noise and environmental noise; specifically, if the type of the sample image is an optical flow map, the preprocessing method for the sample image is determined to be a first preprocessing method, and the first preprocessing method includes noise processing and blur processing; A preprocessing unit is used to preprocess the sample image according to the preprocessing method to obtain a training sample image; specifically, if the type of the sample image is an optical flow image, the sample image is subjected to a first basic augmentation process to obtain a first sample image; the first sample image is subjected to channel shuffling and Laplace noise processing to add salt and pepper noise to the first sample image to obtain a second sample image; the second sample image is subjected to blur processing to add environmental noise to the second sample image to obtain a training sample image; An adding unit, used to add the training sample image to a training sample set; The training unit is used to train the initial model according to the sample images in the training sample set to obtain a trained model.

7. A server, characterized in that: The server comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the image preprocessing method described in any one of claims 1 to 3, or the model training method described in claim 4.

8. A storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the image preprocessing method described in any one of claims 1 to 3, or the model training method described in claim 4.