Underwater halo image sharpening method based on radial gradient iterative network

Abstract

The invention discloses an underwater halo image sharpening method based on a radial gradient iterative network, and belongs to the technical field of underwater image processing, and the method mainly comprises the steps: constructing an underwater halo image sharpening model based on the radial gradient iterative network, and the model comprises a halo extraction network and an enhancement network; the halo extraction network is trained based on a first model training data set, and during training, a loss function is constructed based on radial gradient constraint iteration weight regularization to guide extraction of a halo map; training the enhanced network based on a second model training data set; and acquiring an underwater halo image to be processed, and inputting the underwater halo image to be processed into the underwater halo image sharpening model based on the radial gradient iterative network so as to acquire a clear underwater image. According to the method, the halo effect caused by an artificial intense light source can be effectively removed, the serious interference of the halo effect on a subsequent computer vision task is reduced, and the underwater image quality is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of marine environment perception, digital image processing and underwater image clarity, and in particular to an underwater halo image clarity method based on a radial gradient iterative network. Background Art

[0002] With the development of human exploration of the ocean, underwater robots have become an important tool for obtaining seabed information. However, due to the strong absorption and scattering of light by water and the complex underwater environment, there is a serious weak light phenomenon in the underwater environment. Therefore, underwater robots usually need to carry artificial lighting devices for auxiliary lighting to improve visibility of the surrounding environment. However, the introduction of artificial light sources will cause some problems such as brightness vignetting and low contrast in the captured images, resulting in poor captured images, which has a great impact on subsequent visual tasks such as feature extraction and edge detection. Although existing underwater image enhancement methods have developed rapidly, few of them have considered the serious impact of artificial light sources in real underwater operating environments, and their practicality is poor. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides a method for clarifying underwater halo images based on radial gradient iterative networks. According to the similarity of the brightness distribution characteristics of land halo images and the formation principle of land halo images, the underwater halo image is defined as a low-quality underwater image without halo. low The multiplicative combination with the halo layer V can effectively remove the halo effect caused by strong artificial light sources, reduce its serious interference to subsequent computer vision tasks, and significantly improve the quality of underwater images.

[0004] The technical means adopted by the present invention are as follows:

[0005] A method for clarifying underwater halo images based on radial gradient iterative network comprises the following steps:

[0006] S1. Constructing an underwater halo image clarity model based on a radial gradient iterative network, wherein the underwater halo image clarity model based on a radial gradient iterative network comprises a halo extraction network and an enhancement network, wherein the halo extraction network is used to decompose an underwater image with a halo into a pixel product of a halo map and a low-quality underwater image, and the enhancement network is used to enhance the low-quality underwater image output by the halo extraction network;

[0007] S2. Constructing a first model training data set, wherein the first model training data set includes underwater images with halos and real halo layers;

[0008] S3, training the halo extraction network based on the first model training data set, and constructing a loss function based on radial gradient constrained iterative weight regularization to guide the extraction of the halo image during training;

[0009] S4, constructing a second model training data set, wherein the second model training data set includes low-quality underwater images and clear underwater images output by the halo extraction network;

[0010] S5. Training the enhancement network based on the second model training data set, calculating the loss value between the image output by the enhancement network and the clear underwater image during training, performing error back propagation according to the loss value, and updating the weight of the enhancement sub-network;

[0011] S6. Obtain an underwater halo image to be processed, and input the underwater halo image to be processed into an underwater halo image clearing model based on a radial gradient iterative network, so as to obtain a clear underwater image.

[0012] Furthermore, during training, a loss function is constructed based on radial gradient constrained iterative weight regularization to guide the extraction of the halo image, including constructing the following loss function to guide model training:

[0013]

[0014] s2=p(s1) p-1

[0015] in, A radial gradient representing the true halo, represents the radial gradient of the halo in the underwater halo image obtained in the kth iteration, v gt represents the generated halo, v low represents the halo obtained from the underwater halo image, λ represents the constant coefficient that controls the importance of different items, and p represents the constraint coefficient.

[0016] Furthermore, training the halo extraction network based on the first model training data set also includes calculating a reconstruction loss, performing error back propagation according to the reconstruction loss value, and updating the weights of the halo extraction network. The calculation formula of the reconstruction loss is as follows:

[0017]

[0018] Among them, μ1 represents the constant coefficient that controls the importance of different items, It represents finding the radial gradient of a low-quality underwater image without halo. Radial gradient of an underwater image representing normal lighting.

[0019] Furthermore, training the halo extraction network based on the first model training data set also includes calculating a smoothing loss, performing error back propagation according to the smoothing loss value, and updating the weights of the halo extraction network. The calculation formula of the smoothing loss is as follows:

[0020]

[0021] in, represents the gradient of the extracted halo, A gradient representing a true glow.

[0022] Furthermore, the loss value between the image output by the enhanced network and the clear underwater image is calculated during training according to the following formula:

[0023] L EF =λ1||S pre -S normal ||2

[0024]

[0025] Among them, S pre is the image output by the enhanced network, S normal is a clear underwater image, and the reconstruction loss L EF ,λ1λ2λ3 are constant coefficients that control the importance of different items, SSIM(S pre ,S normal ) structural similarity loss, represent the radial gradients of the predicted underwater image and the real underwater image respectively.

[0026] Furthermore, each iteration of the halo extraction network goes through a five-layer network structure, the first to fourth network layers are convolution layers with a convolution kernel size of 3*3 and a Relu layer, and the fifth layer is a convolution layer with a convolution kernel size of 3*3; the input of the first network layer is a low-quality underwater image with halo, and the output result of the first network layer will be used as the input of the second, third, fourth, and fifth networks, the input of the second network layer is the output of the first layer, the input of the third network layer is the sum of the output results of the first and second networks, the input of the fourth network layer is the sum of the output results of the first, second and third networks, and the input of the fifth network layer is the sum of the output results of the first, second, third and fourth networks.

[0027] Furthermore, the enhanced network includes a shallow feature extraction block, an independent dense block, a global residual learning block and a global feature fusion block. Except for the convolution layer with a convolution kernel size of 1*1 in the global feature fusion block, the rest are convolution layers with a convolution kernel size of 3*3. The features are downsampled to different scales in the network, and then processed, desampled and fused from small scale to large scale.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] The present invention proposes a radial gradient underwater halo image sharpening method based on iterative optimization network. The method can effectively avoid the common defects of existing underwater image sharpening methods, such as overexposure of halo areas and underenhancement of background dark areas caused by artificial light source assistance. While removing the halo, the underwater image quality is improved, which is effective and robust, thus laying a theoretical and technical foundation for subsequent visual tasks such as seabed panoramic observation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0031] Figure 1 The present invention is a flowchart of a method for clearing underwater halo images based on a radial gradient iterative network in an embodiment of the present invention.

[0032] Figure 2 This is a diagram of the architecture of an underwater halo image clarity model based on a radial gradient iterative network in an embodiment of the present invention.

[0033] Figure 3 This is a halo image extracted by an underwater halo image clearing model based on a radial gradient iterative network in an embodiment of the present invention.

[0034] Figure 4 This is a clear underwater image output by the underwater halo image clearing model based on the radial gradient iterative network in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] like Figure 1 As shown, the present invention provides an underwater halo image clearing method based on radial gradient iterative network, which mainly includes the following steps:

[0038] S1. Construct an underwater halo image clarity model based on a radial gradient iterative network, the model comprising a halo extraction network and an enhancement network, the halo extraction network is used to decompose the underwater image with halo into a halo map and a low-quality underwater image with uniform illumination. The enhancement network is used to enhance the color and details of the low-quality underwater image to obtain a restored clear underwater image.

[0039] Based on the similarity of the brightness distribution characteristics between the land vignetting image and the underwater halo image, i.e., the characteristic of gradually decreasing radially from the center of the light source to the surroundings, and drawing on the formation principle of the land vignetting image and the low-light image, this application defines the underwater halo image as a halo layer and a low-quality underwater image. Therefore, we have:

[0040] S low =I low °V low

[0041] Among them, S low represents the underwater halo image, I low represents a low-quality underwater image without halo, V low Represents an underwater halo image.

[0042] The halo extraction network in this application includes: the first to fourth network layers are all convolution layers with a convolution kernel size of 3*3 and a Relu layer; the fifth layer is a convolution layer with a convolution kernel size of 3*3; the input of the first network layer is a low-quality underwater image with a halo, and the output result of the first network layer will be used as the input of the second, third, fourth, and fifth networks, and the input of the second network layer is the output of the first layer. The input of the third network layer is the sum of the output results of the first and second networks. The input of the fourth network layer is the sum of the output results of the first, second, and third networks. The input of the fifth network layer is the sum of the output results of the first, second, third, and fourth networks.

[0043] The enhanced network uses a multi-scale residual dense network as the backbone network, which can aggregate multi-scale information to obtain information in a wider range. Its basic structure is a residual dense network, which mainly consists of four parts: a shallow feature extraction network, an independent dense block, a global residual learning, and a global feature fusion. In addition to the convolution layer with a convolution kernel size of 1*1 for local and global feature fusion, the rest are convolution layers with a convolution kernel size of 3*3. In the network, the features are downsampled to different scales (1 / 8, 1 / 4, 1 / 2, and 1), and then processed, desampled, and fused from small scale to large scale.

[0044] In the present invention, it is necessary to train the halo extraction network and the enhancement network, divide the synthesized underwater image data set with halo into a training set and a test set, and use the underwater image with uniform illumination, the real halo image and the underwater halo image as the training data set to train the network model of the present invention. Specifically, the underwater image data with halo and the underwater image with uniform illumination in the training set are input into the halo extraction network for processing to generate the halo image V for training. low , low-quality underwater image without halo I low The input of the overall network model of the present invention includes: (1) a real underwater image dataset without halo, denoted as S normal ; (2) Random halo image V simulated using matlab gt (3) Underwater halo image S low (4) Input the underwater image with halo into the halo extraction sub-network and output the halo map V low , low-quality underwater image without halo I low The specific contents of network training are explained through the following steps.

[0045] S2. Construct a first model training data set, wherein the first model training data set includes underwater images with halos and real halo layers.

[0046] Specifically, the input of the halo extraction network is an underwater image S with halolow , real underwater image S normal and simulated halo map V gt Therefore, the first model training data set should include the above data.

[0047] S3. Training the halo extraction network based on the first model training data set, constructing a loss function based on radial gradient constrained iterative weight regularization to guide the extraction of the halo image during training.

[0048] In this embodiment, the specific steps of training the halo extraction network include:

[0049] S301, constructing an underwater halo extraction sub-network, setting parameters for the network model, and constructing a training set and a test set. The training set includes underwater images S with halo. low And the real underwater image S normal , simulated halo image V gt ;

[0050] S302, inputting the input image of the training set into the halo extraction network for training to obtain a halo image V low and low-quality underwater image without halo I low .

[0051] S303, calculate and output the halo image V low and the real target image V gt The loss value between the two is obtained, and the error is back-propagated according to the loss value to update the weights of the halo extraction network.

[0052] S304, determine whether the halo extraction network has been trained. If so, obtain the trained network model and execute S305, otherwise return to S302.

[0053] S305, input the test set into the trained halo extraction network model for testing, and determine whether the trained network model meets the expected requirements based on the test results. If so, execute S306, otherwise return to S302.

[0054] S306, inputting the underwater image with halo into the tested halo extraction network to obtain a halo image V low , low-quality underwater image without halo I low .

[0055] During the training of the halo extraction network, radial gradient constrained iterative weight regularization is used to improve the gradient sparsity of the convolution output image.

[0056] The radial gradient refers to the gradient of the pixel in the image along the radial direction relative to the center of the image light source. The radial gradient at any pixel (x, y) in the image can pass through the center of the light source (x0, y0). The radial gradient expression is as follows:

[0057]

[0058] In the above formula, Ψ(x,y) represents the radial gradient value of the image S at the pixel (x,y), D represents the gradient operator, and D*V is the gradient of the halo image. represents the vector formed by the pixel (x, y) and the center of the light source (x0, y0), * represents convolution, and · represents the inner product operator.

[0059] Based on the above, we constructed the loss function for extracting halo as follows:

[0060]

[0061] Inspired by the solution of non-convex optimization subproblems by iterative reweighted least squares method, we introduce radial gradient constrained iterative weight regularization to improve the gradient sparsity of CNN output images. We transform the above formula into the following iterative optimization process:

[0062]

[0063] in A radial gradient representing the true halo, represents the radial gradient of the halo in the underwater halo image obtained in the kth iteration, v gt represents the generated halo, v low represents the halo obtained from the underwater halo image, λ represents the constant coefficient that controls the importance of different items, and p represents the constraint coefficient, which is set to 0.8 according to experience.

[0064] In this application, if Figure 2 As shown, L rg Represents the radial gradient loss between the desired halo image and the real halo image, which is the value in the above formula W k ,W k+1 ,W k+2 Represents the weight factor obtained by iterative formula, W k represents the weight factor of the Kth iteration. Ψ is the radial gradient obtained according to formula (1).

[0065] The above formula uses the iterative reweighted least squares (IRLS) technique. IRLS treats the optimization as a series of standard least squares problems, each of which uses a weight factor based on the solution of the previous iteration, where w 1 =1, V gtThe ground truth of the halo image representing the constructed dataset. We use a fully convolutional network, which is iterated through a cascaded residual network. The underwater image with halo is input into the network and trained under the guidance of the above loss function to extract the halo image we need. After many experiments, it was found that the halo image obtained when the number of iterations is 3 and p=0.8 is the best. The above parameters are used in this application.

[0066] In order to preserve the details and colors of the low-quality underwater images without halo after halo removal, we also designed a reconstruction loss to make the underwater images after halo removal closer to the underwater images with uniform illumination. The reconstruction loss formula is as follows:

[0067]

[0068] Among them, μ1 represents the constant coefficient that controls the importance of different items, It represents finding the radial gradient of a low-quality underwater image without halo. Radial gradient of an underwater image representing normal lighting.

[0069] The halo image extracted from the halo extraction network should be smooth and not contain too many details, so we introduce a smoothing loss. The smoothing loss formula is as follows:

[0070]

[0071] in, represents the gradient of the extracted halo, A gradient representing a true glow.

[0072] S4. Construct a second model training data set, where the second model training data set includes low-quality underwater images and clear underwater images output by the halo extraction network.

[0073] Specifically, the second model training data set includes low-quality underwater images S obtained by the halo extraction network low , and GroundTruth's clear underwater images normal .

[0074] S5. Training the enhancement network based on the second model training data set, calculating the loss value between the image output by the enhancement network and the clear underwater image during training, performing error back propagation according to the loss value, and updating the weight of the enhancement sub-network.

[0075] Specifically, the training steps of the enhanced network include:

[0076] S501, input the input image of the training set into the enhancement sub-network for training, and obtain an enhanced and clear underwater image S pre .

[0077] S502, calculate the image S output by the enhancement sub-network pre and the real target image S normal The loss value between the two networks is determined, and error back propagation is performed according to the loss value to update the weight of the enhancement sub-network.

[0078] S504: Determine whether the enhanced sub-network is trained. If so, obtain the trained enhanced sub-network and execute S505; otherwise, return to S502.

[0079] S505, input the test set into the trained reflectance map restoration sub-network for testing, and determine whether the trained reflectance map restoration sub-network meets the expected requirements according to the test results, if so, execute S506, otherwise return to S502;

[0080] S506: The low-quality underwater image S obtained by the halo extraction sub-network low Input into the trained enhancement sub-network to obtain the enhanced underwater image S pre .

[0081] During the training process, the output image S is calculated using the following loss function: pre The distance between the target image and normal The loss value is:

[0082] L EF =λ1||S pre -S normal ||2 (6)

[0083]

[0084] Among them, S pre is the image output by the enhanced network, S normal is a clear underwater image, and the reconstruction loss L EF ,λ1λ2λ3 are constant coefficients that control the importance of different items, SSIM(S pre ,S normal ) structural similarity loss, represent the radial gradients of the predicted underwater image and the real underwater image respectively.

[0085] Formula (6) is used to constrain the enhanced underwater image S pre Compared with the real underwater image S normal Global feature difference, formula (7) uses the SSIM structural similarity loss function to calculate the enhanced underwater image S pre Compared with the real underwater image S normalThe SSIM index between them is used as the loss and in order to maintain the consistency between the enhanced underwater image and the Ground Truth, a loss function of the radial gradient distribution is constructed, and the L2 norm of the radial gradient distribution of the enhanced underwater image and the clear underwater image is calculated.

[0086] S6. Obtain an underwater halo image to be processed, and input the underwater halo image to be processed into an underwater halo image clearing model based on a radial gradient iterative network, so as to obtain a clear underwater image.

[0087] The scheme and effect of the present invention are further described below through specific application examples.

[0088] S100, selecting and constructing an existing synthetic underwater image S with halo low ;

[0089] S200, the underwater image S with halo low and the real underwater image S normal Input into the halo extraction network to obtain the estimated halo map V low , low-quality underwater image S without halo low The steps are as follows:

[0090] The halo map is calculated by using radial gradient and iterative network using iterative reweighted least squares (IRLS) technique. low And the real halo map V gt The loss between the two will result in a low-quality underwater image I low and the uniformly illuminated underwater image S normal Extract the halo using radial gradient loss constraints. Get the halo map V through network learning. low and a low-quality underwater image S without halo low .

[0091] S300: detail enhancement and color restoration, converting the low-quality underwater image S200 without halo low The image is input into the enhancement and restoration sub-network, where the underwater image is enhanced to obtain an underwater image S after detail enhancement and color correction. pre The steps are as follows:

[0092] The enhanced underwater image S pre and underwater image S normal Make a loss function and calculate the enhanced underwater image S pre and underwater image S normal Structural similarity loss.

[0093] The underwater halo image correction network is implemented based on the halo extraction network and the enhancement sub-network. The training and testing process includes the following steps:

[0094] S001. Construct the above network model based on the pytorch deep learning framework, set parameters for the network model, and construct a training set and a test set. 890 underwater images containing halos are synthesized in the training set, and 800 of them are set as the training set, and the remaining 90 are set as the test set.

[0095] S002. Input the input image of the training set into the network model for training, calculate the loss value between the output image and the real target image, perform error back propagation according to the loss value, and update the weight of the network model.

[0096] S003. Determine whether the network model has been trained. If so, obtain the trained network model and execute S004. Otherwise, return to S002.

[0097] S004, input the test set into the trained network model for testing, and determine whether the trained network model meets the expected requirements based on the test results. If so, execute S005, otherwise return to S002;

[0098] S005. Input the captured image with artificial light source into the trained network model to obtain a clear underwater image with uniform light distribution.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for clarifying underwater halo images based on radial gradient iterative network, characterized in that: The following steps are involved: S1. Constructing an underwater halo image clarity model based on a radial gradient iterative network, wherein the underwater halo image clarity model based on a radial gradient iterative network comprises a halo extraction network and an enhancement network, wherein the halo extraction network is used to decompose an underwater image with a halo into a pixel product of a halo map and a low-quality underwater image, and the enhancement network is used to enhance the low-quality underwater image output by the halo extraction network; S2. Constructing a first model training data set, wherein the first model training data set includes underwater images with halos and real halo layers; S3, training the halo extraction network based on the first model training data set, and constructing a loss function based on radial gradient constrained iterative weight regularization to guide the extraction of the halo image during training; S4, constructing a second model training data set, wherein the second model training data set includes low-quality underwater images and clear underwater images output by the halo extraction network; S5. Training the enhancement network based on the second model training data set, calculating the loss value between the image output by the enhancement network and the clear underwater image during training, performing error back propagation according to the loss value, and updating the weight of the enhancement sub-network; S6. Obtain an underwater halo image to be processed, and input the underwater halo image to be processed into an underwater halo image clearing model based on a radial gradient iterative network, so as to obtain a clear underwater image.

2. The underwater halo image clearing method based on radial gradient iterative network according to claim 1, characterized in that: During training, a loss function is constructed based on radial gradient constrained iterative weight regularization to guide the extraction of halo images, including constructing the following loss function to guide model training: s2=p(s1) p-1 in, A radial gradient representing the true halo, represents the radial gradient of the halo in the underwater halo image obtained in the kth iteration, v gt represents the generated halo, v low represents the halo obtained from the underwater halo image, λ represents the constant coefficient that controls the importance of different items, and p represents the constraint coefficient.

3. The underwater halo image clearing method based on radial gradient iterative network according to claim 2, characterized in that: The halo extraction network is trained based on the first model training data set, and the reconstruction loss is calculated. Error back propagation is performed according to the reconstruction loss value, and the weight of the halo extraction network is updated. The calculation formula of the reconstruction loss is as follows: Among them, μ1 represents the constant coefficient that controls the importance of different items, It represents the radial gradient of a low-quality underwater image without halo. Radial gradient of an underwater image representing normal lighting.

4. The underwater halo image clearing method based on radial gradient iterative network according to claim 2, characterized in that: The halo extraction network is trained based on the first model training data set, and also includes calculating a smoothing loss, performing error back propagation according to the smoothing loss value, and updating the weight of the halo extraction network. The calculation formula of the smoothing loss is as follows: in, represents the gradient of the extracted halo, A gradient representing a true glow.

5. The underwater halo image clearing method based on radial gradient iterative network according to claim 1, characterized in that: The loss between the image output by the enhanced network and the clear underwater image is calculated during training according to the following formula: L EF =λ1||S pre -S normal ||2 Among them, S pre is the image output by the enhanced network, S normal is a clear underwater image, and the reconstruction loss L EF ,λ1λ2λ3 are constant coefficients that control the importance of different items, SSIM(S pre ,S normal ) structural similarity loss, represent the radial gradients of the predicted underwater image and the real underwater image respectively.

6. The underwater halo image clearing method based on radial gradient iterative network according to claim 1, characterized in that: Each iteration of the halo extraction network goes through a five-layer network structure. The first to fourth network layers are convolution layers with a convolution kernel size of 3*3 and a Relu layer, and the fifth layer is a convolution layer with a convolution kernel size of 3*3; the input of the first network layer is a low-quality underwater image with halo, and the output result of the first network layer will be used as the input of the second, third, fourth, and fifth networks. The input of the second network layer is the output of the first layer, the input of the third network layer is the sum of the output results of the first and second networks, the input of the fourth network layer is the sum of the output results of the first, second and third networks, and the input of the fifth network layer is the sum of the output results of the first, second, third and fourth networks.

7. The underwater halo image clearing method based on radial gradient iterative network according to claim 1, characterized in that: The enhanced network includes a shallow feature extraction block, an independent dense block, a global residual learning block and a global feature fusion block. Except for the convolution layer with a convolution kernel size of 1*1 in the global feature fusion block, the rest are convolution layers with a convolution kernel size of 3*3. The features are downsampled to different scales in the network, and then processed, desampled and fused from small scale to large scale.

Images