A complementary environment-adaptive image augmentation method

Abstract

The application provides a complementary environment self-adaptive weighted image augmentation method. The method comprises the following steps: firstly, the brightness, contrast and saturation of a camera snapshot picture are parameterized; the light environment and the definition are analyzed, the data is normalized and is processed in grades, the obtained image parameters are substituted into a correlation function to calculate the image parameter gain weight, and another form of optimization augmentation is obtained by using shearing, rotating and translating to change the visual angle state; then, the two forms of optimization augmentation are combined to obtain the augmentation parameters of the picture, and the parameters are reversely converted to finally obtain the picture under various environment states. The method is simple in process and high in efficiency, and is a very promising method for assisting in improving the target recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular, and especially relates to a complementary environment adaptive image augmentation method. BACKGROUND

[0002] With the rapid development of China's economy, people's material living standards have improved significantly, and the demand for goods is also increasing. Green and efficient logistics methods have begun to be explored, and sea transportation that meets the requirements has gradually been promoted. In recent years, maritime technology has become more mature, and China is a coastal country with convenient maritime transportation. Sea transportation has gradually evolved into one of the main means of goods circulation. With the rise of maritime transportation, the tonnage of ships is increasing, and the carrying capacity of goods is also increasing. How to improve the efficiency of goods loading and unloading has become a major problem in the logistics industry. It is understood that the most commonly used method at present is manual remote control of mechanical unloading combined with man and machine, but this method is inefficient and cannot quickly obtain the target of goods, and the generation of grabbing instructions is slow and needs to be tried repeatedly. If we want to achieve unmanned unloading with high intelligence, we need very high target recognition accuracy. Overcast, rainy or foggy weather will have a great impact on brightness, contrast, saturation and other aspects of recognition, and the goods themselves are stacked together or even placed randomly, which makes the difficulty of cargo recognition stage of unmanned unloading of the ship cabin increase exponentially. At this time, a large number of case pictures under various environments are needed for data reference, and real scene shooting pictures not only take a long time but also are inefficient. Through the data augmentation method, a large number of model training pictures under various environments can be obtained efficiently, which is of great significance to the cargo recognition module of unmanned unloading of the ship cabin.

[0003] Xu Xiaqing et al. of Beijing Love Pen Technology Co., Ltd. published a patent No. CN202011238928.5 in 2020, entitled "Hyperparameter optimization, target recognition model training, and target recognition method and device", which proposes a hyperparameter optimization, target recognition model training, and target recognition method. The performance of the target recognition model can be determined by using a training image set to explore the hyperparameters of the target recognition model and using a validation image set, and then the hyperparameters that meet the preset requirements are obtained. Although this method can optimize the target picture, it cannot obtain the required various training model pictures.

[0004] A target recognition method in a weak light environment based on adaptive image enhancement is disclosed in the patent CN113392702A published by Shen Shibin et al. of Nanjing Normal University in 2021. The method includes: building an adaptive image enhancement module; building a target recognition model in a weak light environment based on adaptive image enhancement; collecting target videos in a weak light environment and processing to generate a dataset of targets in a weak light environment, dividing test set images and training set images, labeling training images, and generating a label file; sending all images and label files of the training set to the target recognition model in a weak light environment based on adaptive image enhancement for training; the trained target recognition model in a weak light environment based on adaptive image enhancement completes target recognition in a weak light environment and outputs the recognition result. Although the recognition speed is fast and the recognition accuracy is high, it cannot solve other picture quality problems such as contrast and saturation. SUMMARY

[0005] The technical problem to be solved by the present application is how to efficiently obtain a large number of picture materials required for target recognition accuracy. In order to overcome the defects of the prior art, a complementary environment adaptive image augmentation method is invented. The method includes three steps: calculating the brightness, sharpness, contrast, saturation and other parameters of the obtained picture; image parameter analysis and data augmentation in different environments: according to the different environmental states of the scene, analyze the lighting environment and clarity, respectively obtain the image parameters in various environmental states, and enhance the target image parameter data; view angle augmentation: through cutting, rotating, and translating transformation, the picture realizes view angle augmentation, and finally combines the two augmentation forms to realize the final data augmentation. The method is simple, efficient, and is a very promising method to assist in improving target recognition accuracy. The technical means adopted by the present application are as follows:

[0006] A complementary environment adaptive weighted image augmentation method includes the following steps:

[0007] Step 1: Calculate the brightness, saturation and contrast parameters of the target image;

[0008] Step 2: In different environments, analyze the image parameters and data augmentation of the target image;

[0009] Step 3: Modify the target image by cutting, rotating and translating to realize view angle enhancement of the target image;

[0010] Step 4: Transform the obtained augmented parameters through the inverse image parameter association function to obtain image training sets in different environments.

[0011] Compared with the prior art, the present application has the following advantages:

[0012] The complementary environment self-adaptive image augmentation method can make the sample image acquisition process simpler and more efficient by adjusting image parameters, can greatly improve the sample set quantity, and can cover and augment multiple complex scenes under actual working conditions, and is a very promising method for improving target recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0014] Figure 1 The image augmentation method flowchart of the present application.

[0015] Figure 2 The box under actual scene of the present application.

[0016] Figure 3 The imaging parameter distribution diagram of the present application.

[0017] Figure 4 The translation example diagram of the present application.

[0018] Figure 5 The rotation example diagram of the present application.

[0019] Figure 6 The cutting example diagram of the present application. DETAILED DESCRIPTION

[0020] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0022] As shown in Figures 1-6 , the present application provides a complementary environment adaptive image augmentation method, specifically comprising the following steps: extracting the brightness, contrast, saturation parameters of the target picture. Analyze the image parameters in different environments, so as to make them as reference data for daytime, night, and foggy day, and then convert the target picture parameters into the reference data of the required environment. Perspective enhancement is performed, and the mode is selected according to the requirement: translation, rotation, and shearing, as shown in Figure 3 、 Figure 4 、 Figure 5 . That is, the imaging parameters of each carton sample are converted into the above-mentioned calculated imaging parameters to obtain new image data. Finally, the image data is inversely converted to obtain the pictures of the target image in different environments.

[0023] As a preferred embodiment, in the present application, the first step is to calculate the brightness, saturation, and contrast parameters of the target picture

[0024] First, the picture is converted with each parameter by using formula (9):

[0025]

[0026] Where L is the brightness, S is the contrast, C is the saturation, MAX and MIN are the maximum and minimum values. The average values of the red (R), green (G), and blue (B) channels in the image, respectively. (u, v) represents the horizontal and vertical coordinates of a given pixel on the image, and δ(u, v) is the gray difference value of adjacent pixels and (u, v). Pr δ (u, v) is the distribution probability of the pixel with a gray difference value of δ. The picture data in the application environment is obtained, and the obtained parameters are normalized to facilitate processing.

[0027] Further, the second step is to analyze the image parameters and data augmentation of the target image in different environments. ​

[0028] The image parameters in different environments are analyzed to generate Figure 3 The average value of the imaging parameters in each environmental state is taken as the imaging parameter of the scene, i.e. formula (10):

[0029]

[0030] Wherein is the image parameter in the sc scene (sc represents daytime, night and foggy day), respectively represent the average brightness, average saturation and average contrast in the sc scene. As follows:

[0031]

[0032] Then, the i-th image in the training set can also be calculated by formula (12) P i , i.e.

[0033] P i = (L i , S i , C i )i = (1, 2, 3)

[0034] (12) Then convert P i to to generate a new image. L i , S i are converted by formula (13):

[0035]

[0036] Wherein, f i (u, v) and f i ' -sc respectively represent (R, G, B) on the (u, v) of the i-th image in the training set and its derived image, is the contrast coefficient, and β is the brightness gain coefficient. Then convert P to S by formula (14):

[0037] S i ' -sc (u, v) = (1 + γ) S i-sc (u, v) (14)

[0038] In the formula, f' is the abbreviation of f i- ' sc (u, v),

[0039] S i ' -sc(u, v) is the saturation of the new image, and γ is the saturation adjustment coefficient. The i-th image in the training set can be converted into a new image with a set of appropriate β and γ coefficients, and an imaging parameter .

[0040] As a preferred, the third step: the target image is modified by cutting, rotating, and translating to achieve perspective enhancement of the target image.

[0041] The picture is modified by cutting, rotating, and translating to achieve perspective enhancement. Each way of changing the perspective includes two ways, i.e., Sh x or Sh y , Ro x or Ro y , Tr x or Tr y , which are collectively changed. Among them, Sh x , Sh y are the cutting along the x-axis and y-axis, respectively, Ro x , Ro y are the rotation along the x-axis and y-axis, respectively, Tr x , Tr y are the translation along the x-axis and y-axis, respectively, and W is the number of changes. The specific change formula is as follows (15), (16), (17):

[0042] (u', v') T = (u, v) T + (u t , v t ) t (15)

[0043]

[0044]

[0045] where (u', v') represents the pixel coordinates of the original (u, v) after transformation, u t and v t are the translations of (u, v) along the horizontal and vertical axes, respectively, θ γ is the rotation angle, Φ μ and Φ v are the shear angles along the horizontal and vertical axes, respectively. The above steps have data augmented and perspective enhanced the parameters of the obtained picture. Now, the two are combined to obtain image parameters in different environments. Among them, n is the number of data augmentation. Finally, the obtained augmented parameters are converted using the inverse image parameter association function.

[0046] Through the above steps, the augmented images of the images in various environments can be obtained.

[0047] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments. In the several embodiments provided in the present application, it should be understood that the disclosed technical contents can be realized by other ways.

[0048] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A complementary environment-adaptive weighted image augmentation method, characterized in that, Includes the following steps: Step 1: Calculate the brightness, saturation, and contrast parameters of the target image; Step 2: Perform image parameter analysis and data augmentation on the target image under different environments; Step 2 includes the following steps: Step 21: Take the average value of the imaging parameters under different environmental conditions to represent the imaging parameters of the scene, i.e., Equation 2: (2)； in express Image parameters in the scene, Indicates daytime, nighttime, or foggy weather; , , They represent Average brightness, average saturation, and average contrast in the scene; Step 22: For the first [item] in the training set... The parameters of the images are calculated using equation (3). ,Right now: (3)； in, Indicates the first The brightness, saturation, and contrast parameters of the image; Step 23: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the Convert to Generate a new image; Transformation using equation (4): (4) in, and They represent the first and second training sets, respectively. Zhang's image and its derivative images (R, G, B) on top. Represents the contrast ratio. Indicates the brightness gain coefficient; , They represent the first The brightness and saturation parameters of the image; Step 24: Use formula (5) to... Perform the conversion: (5)； in, = , express abbreviation, This indicates the saturation of the newly obtained image. This represents the saturation adjustment factor; Step 25: Calculate the brightness, saturation, and contrast of the new image. , , coefficients, thereby generating an imaging parameter as New images; Step 3: Modify the target image by cropping, rotating, and translating to enhance the viewpoint of the target image; Step 3 includes the following steps: Step 31: Shearing, rotation, and translation along the x and y axes, totaling... Such changes; among them, To represent the number of changes, the specific formulas for change are as follows (6), (7), and (8): (6)； (7)； (8)； in, Indicate the original Transformed pixel coordinates and They represent Translation along the horizontal and vertical axes, Indicates the rotation angle. and These represent the shear angles along the horizontal and vertical axes, respectively. Step 32: Perform data augmentation and viewpoint enhancement on the parameters of the obtained image. Combine the data augmentation and viewpoint enhancement to obtain: Image parameters under different environments; Where n represents the number of data augmentations; Step 4: Transform the obtained augmented parameters using the inverse image parameter correlation function to obtain image training sets under different environments.

2. The complementary environment adaptive weighted image augmentation method according to claim 1, characterized in that, In step 1, the target image is converted with three parameters, brightness, saturation and contrast, using formula (1) to obtain image data in the application environment, and the brightness, contrast and saturation are normalized. (1)； in, Indicates brightness. Indicates contrast. Indicates saturation. and express( The maximum and minimum values ​​of () ) represent the red color in the image. ,green and blue The average value of the channel; Represents the x and y coordinates of a given pixel in an image. Indicates adjacent pixels and The difference in grayscale values, The grayscale difference is represented as The probability distribution of pixels.

Images