Image processing method for underwater three-dimensional particle image velocimetry of robotic fish

Abstract

The application discloses an image processing method for underwater three-dimensional particle image velocimetry of a machine fish, which comprises the following steps: (1) acquiring a tracer particle image; (2) pre-processing the obtained tracer particle image; (3) carrying out threshold segmentation based on the pre-processed tracer particle image; and (4) extracting the bright part and the dark part of the fish body based on the threshold segmented tracer particle image to reconstruct the two-dimensional morphological features of the whole fish body. The application reduces the influence of low contrast in the image and the tracer particles under water on the subsequent extraction of the fish body, improves the precision and efficiency of the recognition and extraction of the machine fish body, realizes the extraction of the whole fish body, avoids the incomplete extraction of the fish body caused by the single threshold setting due to the different colors of the fish body, can accurately calculate the force, vorticity and velocity at the boundary of the fish body, reduces the error influence caused by the fish body in the inquiry area, solves the small space scale and high speed of the swimming behavior of the fish body, and reflects the authenticity of the fish body speed measurement.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image enhancement and image recognition, and specifically provides an image processing method for underwater three-dimensional particle image velocimetry of a robotic fish. BACKGROUND

[0002] Measuring the near-body velocity of underwater fish, the force on the fish body, and the near-body vorticity distribution has an important influence on exploring fish behavior. Due to the influence of light and water flow on the underwater particle velocimetry area, the particle field around the fish will have a large error, and the presence of the fish in the interrogation area in the PIVlab will affect the estimation of the near-body velocity. The image processing method is an effective means for shape recognition and feature extraction of underwater fish images. The existing method is mainly to directly perform threshold segmentation after image preprocessing of the underwater fish image, and the threshold setting is relatively single, and the processing effect is not ideal for fish of different colors, and it is difficult to lay a foundation for subsequent measurement of fish near-body velocity, pressure, and vorticity distribution. Therefore, it is of great significance to accurately recognize and extract the fish body in the underwater particle image.

[0003] At present, most of the researches are two-dimensional velocimetry of fish flow field in water, but the behavior of fish in water and the nature of particle flow are three-dimensional, therefore, the two-dimensional velocimetry result is greatly limited. With the development of three-dimensional particle image velocimetry technology, it is possible to measure the three-dimensional velocity vector around the fish body through high spatial and temporal resolution for two-dimensional velocimetry of the fish body.

[0004] In view of the above, there is a need for a new method for underwater three-dimensional particle image velocimetry of a robotic fish to solve the above problems. SUMMARY

[0005] In order to solve the problems in the background art, the present application provides an image processing method for underwater three-dimensional particle image velocimetry of a robotic fish, which can improve the completeness of the fish body extracted from the underwater particle image, and obtain the spatial and temporal robustness required for the flow around the underwater robotic fish.

[0006] In order to achieve the above purpose, the technical scheme of the present application is as follows:

[0007] An image processing method for underwater three-dimensional particle image velocimetry of a robotic fish, comprising the following steps:

[0008] (1) obtaining a tracer particle image;

[0009] (2) pre-processing the obtained tracer particle image;

[0010] (3) performing threshold segmentation based on the pre-processed tracer particle image;

[0011] (4) based on the threshold segmentation of the tracer particle image, the bright part and the dark part of the fish body are extracted to reconstruct the two-dimensional morphological features of the whole fish body.

[0012] As a preferred embodiment, the pre-processing in step (2) includes image channel segmentation and elimination of tracer particles.

[0013] As a preferred embodiment, the image channel segmentation includes the following steps:

[0014] (211) based on the ROI region of the tracer particle image, extraction is performed;

[0015] (212) the B, G, R three channels of the tracer particle image are sequentially segmented to obtain specific images under each single channel;

[0016] (213) contrast the tracer particle image with the images under B, G, R three single channels in terms of contrast;

[0017] (214) based on the contrast results under each channel, select the image with the highest contrast.

[0018] As a preferred embodiment, the elimination of tracer particles includes the following steps:

[0019] (221) based on the tracer particle image with the optimal contrast, median filtering is performed;

[0020] (222) fuse the tracer particle image after median filtering with the tracer particle image with the optimal contrast;

[0021] (223) realize the elimination of tracer particles by reducing the gamma parameter.

[0022] As a preferred embodiment, step (4) includes the following steps:

[0023] (41) based on the size of the gray value of the bright part and the dark part of the fish body, a suitable threshold is given for thresholding and inverse thresholding;

[0024] (42) the binary image obtained after thresholding and inverse thresholding is used as a mask for the bright part and the dark part of the fish body to realize the extraction of the bright part and the dark part of the fish body;

[0025] (43) image fusion is performed on the bright part and the dark part to determine the two-dimensional morphology of the whole fish body.

[0026] As a preferred embodiment, the calculation of the two-dimensional morphology of the whole fish body in step (43) includes the following steps:

[0027] (431) based on the pressure field around the fish body, the force at the fluid boundary is calculated.

[0028] (432)Based on the vortex field around the fish body, the vortex distribution at the fluid boundary of the fish body and the velocity circulation along the vortex ring are calculated.

[0029] (433)Based on the velocity field around the fish body, the velocity at the fluid boundary of the fish body is calculated.

[0030] As a preferred embodiment, the pre-processing of the obtained tracer particle image in step (2) further comprises synthesizing a three-dimensional cloud map through tomographic PIV processing based on the underwater robotic fish under the irradiation of a multi-path sheet light source.

[0031] As a preferred embodiment, the recording of the underwater robotic fish is measured by high time resolution.

[0032] As a preferred embodiment, the recording of the underwater robotic fish is arranged in multiple rows of cameras in space, providing sufficient viewpoints for multi-angle.

[0033] The present application has the following beneficial effects:

[0034] 1) In the image processing technical solution of underwater three-dimensional particle image velocimetry of the robotic fish in the present application, the obtained tracer particle image is pre-processed, thereby reducing the influence of low contrast in the image and the underwater tracer particles on the subsequent extraction of the fish body, and improving the precision and efficiency of identifying and extracting the robotic fish body.

[0035] 2) The bright part and the dark part of the fish body are extracted from the tracer particle image after threshold segmentation, so as to reconstruct the two-dimensional morphological features of the entire fish body. The fish body is thresholded and de-thresholded, respectively, so as to realize the extraction of the entire fish body, avoid the incomplete extraction of the fish body caused by the single threshold setting due to the different colors of the fish body, and further improve the precision of the extracted fish body.

[0036] 3) The force, vortex distribution and velocity at the boundary of the fish body can be accurately calculated through the extracted two-dimensional morphology of the complete fish body, thereby reducing the error influence caused by the fish body in the interrogation area.

[0037] 4) Based on the extracted complete two-dimensional morphology of the fish body, a three-dimensional particle flow velocity cloud map supported by time and space is synthesized through tomographic PIV processing. The small spatial scale and high speed of the swimming behavior of the fish body are solved, and the authenticity of the fish body velocity measurement is further reflected. BRIEF DESCRIPTION OF DRAWINGS

[0038] The disclosed content of the present application will become more easily understood with reference to the accompanying drawings. Those skilled in the art will easily understand that these drawings are only for illustrative purposes, and are not intended to limit the scope of protection of the present application. Among them:

[0039] Figure 1Figure 1 is a flow chart of the operation steps of the image processing method for underwater three-dimensional particle image velocimetry of a robotic fish according to an embodiment of the present application;

[0040] Figure 2 (a) is a raw gray scale image of a fish within the field of view of the image acquisition device according to an embodiment of the present application;

[0041] Figure 2 (b) is a schematic diagram of the ROI extraction region according to an embodiment of the present application;

[0042] Figure 2 (c) is a schematic diagram under the blue channel according to an embodiment of the present application;

[0043] Figure 2 (d) is a schematic diagram under the green channel according to an embodiment of the present application;

[0044] Figure 2 (e) is a schematic diagram under the red channel according to an embodiment of the present application;

[0045] Figure 3 is a three-channel histogram of the ROI region according to an embodiment of the present application;

[0046] Figure 4 (a) is a schematic diagram of the green channel result after median filtering according to an embodiment of the present application;

[0047] Figure 4 (b) is a schematic diagram of the result after image fusion according to an embodiment of the present application;

[0048] Figure 5 (a) is a schematic diagram after thresholding according to an embodiment of the present application;

[0049] Figure 5 (b) is a schematic diagram of the fish body bright part extraction result according to an embodiment of the present application;

[0050] Figure 5 (c) is a schematic diagram after inverse thresholding according to an embodiment of the present application;

[0051] Figure 5 (d) is a schematic diagram of the fish body dark part extraction result according to an embodiment of the present application;

[0052] Figure 6 is a fish body reconstruction diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] The embodiment is implemented on the premise of the technical scheme of the present application, and detailed implementation manners and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiment.

[0054] In order to solve the problems existing in the prior art, such as Figures 1 to 6 The present application provides an image processing method for underwater three-dimensional particle image velocimetry of a robotic fish, which can synthesize a three-dimensional cloud image from a two-dimensional velocity image of the fish body through high spatial and temporal resolution.

[0055] Referring to the accompanying Figure 1 , Figure 1 is a main step flow diagram of the image processing method for underwater three-dimensional particle image velocimetry of a robotic fish according to an embodiment of the present application. As shown in Figure 1 The image processing method for underwater three-dimensional particle image velocimetry of a robotic fish in the embodiment of the present application mainly includes the following steps 1-4.

[0056] Step (1): Obtain a tracer particle image. Specifically, in the present application, a high-speed camera is used to obtain raw image data, and then the raw image data is analyzed to obtain a raw image, and the raw image is grayed to obtain a raw gray image.

[0057] Step (2): Preprocess the tracer particle image. Specifically, on the basis of the existing device, although the image data collected by the high-speed camera has high accuracy, the underwater particle region will still be affected by uneven laser irradiation, etc. In addition, there are tracer particles in the water, which are artificially put into the water to measure the near-body velocity of the fish body. Uneven laser irradiation and tracer particles in the water will affect the accuracy of extracting the fish body, and further affect the estimation of the near-body velocity of the robotic fish.

[0058] For the problem of low image contrast caused by uneven laser irradiation, an image channel segmentation method is mainly used to solve it, that is, the ROI gray image of the raw data is compared and judged with the images under the three single channels of B, G and R. As shown in Figure 2 (a), which is a raw gray image. In order to achieve the best image processing effect, the present application performs ROI extraction on Figure 2 (a) to filter out unnecessary information (container wall and useless information above the water surface), and the extracted image effect is as shown in Figure 2 (b), which is an ROI gray image. The raw data is segmented into images under three channels of Figure 2 (c), Figure 2 (d), Figure 2 (e), respectively. As shown in Figure 2 (c), which is a schematic diagram showing the blue channel after three-channel segmentation. As shown in Figure 2(d) shows the schematic diagram under the green channel after the three channels are divided. As Figure 2 (e) shows the schematic diagram under the red channel after the three channels are divided. The division process can be expressed by the following formula:

[0059] mv[c](I) = src(I) c

[0060] In the formula: mv represents an array used to receive the data after separation, and src is the input image. In the present application, only the green channel data (d) is selected as the 8-bit gray image used in the PIV image filtering process, which provides higher contrast between the tracer particles and the background than the blue and red channel data (the container is irradiated by a green laser generator, and the whole water body is green, so the green image background is brighter than the blue and red ones); the green channel data also have higher contrast than the equivalent gray image (b). Figure 2 Figure 2 b) has higher contrast. As shown in Figure 3 the statistical analysis of the gray values of different pixel points under the three channels is shown, and the gray values of the green channel are mostly greater than those of the blue and red channels under the same pixel point range. The contrast difference under different channels can be more intuitively visualized through the establishment of a histogram.

[0061] In the process of removing the tracer particles, the entire image is first smoothed by using the median filter. In the operation, only the median value of the gray values of the pixels covered in the convolution kernel is taken as the gray value of the anchor point. At the same time, the edge information and feature information of the fish body in the image can be reserved in the process of smoothing the tracer particles. In the filtering process, the convolution kernel must be set to an odd number, and it is very important to select a suitable convolution kernel for the filter. If the convolution kernel is set too small, the tracer particles in the image cannot be completely smoothed. On the contrary, the image will be excessively blurred, and the edge information of the fish body will be lost. In the present application, the median filter replaces each pixel in the image with the median value of the adjacent pixels in the 101x101 pixel region. As Figure 4 (a) shows the result after the median filtering and smoothing. Based on the above method, the image as a whole has a smoothing effect, in which the tracer particles and the rest of the image are completely separated. Then, the image fusion method is used in the present application to distinguish the fish body from the image background, so as to achieve the purpose of removing the tracer particles. The calculation method of image fusion can be expressed by the following formula:

[0062] Dst = src1 * alpha + src2 * beta + gamma

[0063] ​In the formula: Dst represents the output target image; src1 and src2 represent the background and foreground images for image fusion, respectively; alpha is weight 1 and beta is weight 2, which control the fusion ratio of the images; gamma is the image offset, and by changing the pixel gamma value, the image will appear whiter or darker. In this invention, images under the green single channel are selected respectively ( Figure 2 d) The image after median filtering, used as the background image. Figure 4 a) When using the foreground image for image fusion, with alpha=1, beta=1, and gamma=-200, as shown... Figure 4 (b) shows the result after image fusion. It shows that the fish body is no longer blurry and has a clear contrast with the image background, achieving the visual removal of tracer particles and improving the accuracy of subsequent extraction of the two-dimensional morphology of the fish body.

[0064] Step (3): Threshold segmentation is performed based on the preprocessed tracer particle image. Removal of tracer particles in the water helps to achieve true two-dimensional morphological extraction of the fish. When no other objects are displayed in the background, the entire fish is detected by giving a suitable threshold. If the fish is monochrome (i.e., the fish has the same grayscale value), this step will detect the complete fish. However, in this invention, the robotic fish is multicolored. For example... Figure 4 As shown in (b), part of the fish's body has a grayscale value higher than the background (brighter), while another part has a grayscale value lower than or equal to the background (darker). For this multi-colored fish, a single threshold cannot detect the entire fish. Therefore, the brighter and darker parts of the fish are segmented and extracted separately, and then the two parts are fused to reconstruct the two-dimensional shape of the fish. First, during the threshold segmentation of the brighter parts of the fish, an appropriate threshold is set based on the grayscale values ​​of the brighter parts and the background. Pixels with grayscale values ​​greater than or equal to the threshold (brighter areas) are assigned a value of 1, represented by a grayscale value of 255 (white), while pixels with grayscale values ​​lower than the threshold (darker areas and the background) are represented by a grayscale value of 0 (black). Figure 4 As shown in (b), if the threshold is set to a low value, the brighter parts of the fish will be detected, but some tracer particles hidden around the fish with reduced brightness will also be detected. On the other hand, when the threshold is set to a high value, the problem of tracer particles hidden around the fish can be solved, but some brighter parts of the fish may not be detected. In this invention, the threshold is set to 67 to remove all portions with grayscale values ​​less than 67. The threshold is applied... Figure 4 (b) The resulting binary image is Figure 5 (a). For example Figure 5 As shown in (a), this is a schematic diagram of the thresholding result; during the thresholding segmentation process in the darker parts of the fish body, the thresholding is performed on...Figure 4 (b) Perform the inverse thresholding operation, the pixels with gray value greater than or equal to the threshold value (bright part) are assigned to gray value 0 size (black), and the pixels with gray value less than the threshold value are represented by gray value 255 (white). The inverse thresholding binary image obtained after setting the threshold value to 67 is Figure 5 (c). As shown in Figure 5 (c), which is the result of the inverse thresholding schematic diagram.

[0065] Step (4): Based on the threshold segmentation of the tracer particle image, the bright part and the dark part of the fish body are extracted to reconstruct the two-dimensional morphological features of the whole fish body. The threshold segmented binary image Figure 5 (a) is used as the mask of the PIV image Figure 2 (e), and the result is Figure 5 (b). As shown in Figure 5 (b), the bright part of the fish body is successfully extracted; the dark part of the fish body segmented Figure 5 (c) is used as the mask of the fusion image Figure 4 (b), and the result is Figure 5 (d). As shown in Figure 5 (d), since the parameter value setting in the process of reducing the brightness of the fusion image does not conform to the inverse thresholding value, the fish body contour is displayed around, which does not affect the extraction of the dark part of the fish body. Finally, the extracted bright part of the fish body Figure 5 (b) and the extracted dark part of the fish body Figure 5 (d) are fused to reconstruct the whole fish body Figure 6 ). As shown in Figure 6 , the result of gamma = 50 shows the whole fish body, and the background has no obvious tracer particles.

[0066] Based on the above steps (1)-(4), the tracer particle image is preprocessed based on the obtained tracer particle image, the image channel is segmented, the tracer particles are removed, and the low image contrast and the influence of the tracer particles on the subsequent extraction of the two-dimensional morphology of the fish body are eliminated. Furthermore, based on the preprocessing, the threshold segmentation method is used to extract the bright part and the dark part of the fish body, which avoids the omission caused by the single segmentation of the existing method and improves the extraction accuracy.

[0067] Also included is the calculation of the following based on the extracted fish body two-dimensional shape: the force at the fluid boundary of the fish body based on the pressure field around the fish body; the vorticity distribution at the fluid boundary of the fish body and the velocity circulation along the vortex ring based on the vortex field around the fish body; and the velocity at the fluid boundary of the fish body based on the velocity field around the fish body. Generally speaking, the fish body two-dimensional shape obtained based on step 4 is to lay the foundation for measuring the near-body velocity, the pressure received, and the near-body vorticity. After the fish body two-dimensional shape is extracted, the image is put into PIVlab to obtain these parameters. If the fish body is not extracted, the existing fish body will have an error impact on the estimation of these parameters. The present application reduces the error caused by measurement by extracting the fish body two-dimensional shape, and improves the accuracy of calculation. It is further illustrated that the image processing technical scheme of the underwater three-dimensional particle image velocimetry of the robotic fish in the present application can improve the accuracy of the calculation of the near-body velocity, the pressure, and the vorticity of the robotic fish. The fluid force acting on the fish body can be calculated by the following formula:

[0068] F(t) = -∫npdS + ∫τ·ndS

[0069] In the above formula, n is the normal unit vector; p is the fluid pressure; τ is the viscous stress tensor; t is time; and S is the surface area of the fish body.

[0070] The fish body vorticity distribution and the velocity circulation along the vortex ring are calculated by the following formula:

[0071]

[0072] Γ = ∮v′dl. (2)

[0073] Wherein: Ω is the vorticity (1 / s); u and v are the horizontal and vertical velocities of the particles (m / s), respectively; Γ is the velocity circulation (m2 / s); v′ is the velocity of the particles (m / s); and l is the length of the vortex ring (m).

[0074] The velocity at the fluid boundary of the fish body based on the velocity field around the fish body is calculated by the following formula:

[0075]

[0076] In the formula, v is the near-body particle velocity of the fish body, mm / ms; M is the actual length corresponding to a single pixel point, mm / pixel; x1 and y1 are the position coordinates of the particle image points of the first exposure; x2 and y2 are the position coordinates of the particle image points of the second exposure; and Δt is the interval time of the two exposures, ms.

[0077] It should be noted that although the above embodiments describe the steps in a specific order, those skilled in the art can understand that, in order to achieve the effect of the present application, the steps between different steps do not necessarily be executed in such order, they can be executed simultaneously (in parallel) or in other order, and these changes are within the protection scope of the present application.

[0078] The present application synthesizes time and space supported three-dimensional particle velocity cloud by tomographic PIV processing. The three-dimensional nature of robotic fish flow and swimming patterns has been overcome by illuminating and measuring with multiple cameras in a volume. The key point of the tomographic PIV processing method is that the two-dimensional shape of each surface of the fish needs to be synthesized, specifically, this usually needs to manually track the animal in multiple camera views. Most of the current researches are two-dimensional velocity measurement of fish flow in water, but the behavior and particle flow of fish in water is essentially three-dimensional, therefore, the two-dimensional velocity measurement result is greatly limited. The fish synthesized by the tomographic PIV processing method can directly measure the three-dimensional flow pattern. There is no need to use the spatial axis symmetry or flow stability assumption often made when using two-dimensional PIV.

[0079] The three-dimensional cloud is measured with high time resolution, specifically, the rapid oscillation of the caudal fin of the underwater robotic fish produces highly unstable flow, therefore, high time resolution is needed to solve it. In the present application, the high speed of the oscillation of the caudal fin of the robotic fish has been solved by using a high-speed camera system with high magnification.

[0080] The three-dimensional cloud arranges multiple rows of cameras in space, the purpose is to provide enough viewpoints for particle reconstruction in partially occluded areas. Specifically, multiple angle tracking measurement of the robotic fish in water is carried out by using a multi-array high-speed camera, the shape of the data at each angle is extracted, and then a three-dimensional cloud is synthesized.

[0081] So far, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical solutions after these changes or replacements will fall within the protection scope of the present application.

[0082] The basic principles and main features of the present application and the advantages of the present application have been shown and described above. Those skilled in the art should understand that the present application is not limited by the above embodiments, the above embodiments and descriptions in the specification are only to illustrate the principles of the present application, and various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the present application. The protection scope of the present application is defined by the appended claims and their equivalents.

Claims

1. An image processing method for underwater three-dimensional particle image velocimetry of robotic fish, characterized in that, Includes the following steps: (1) Acquire images of tracer particles; (2) Preprocess the obtained tracer particle images; (3) Threshold segmentation is performed based on the preprocessed tracer particle image; (4) Extract the bright and dark parts of the fish body from the tracer particle image after threshold segmentation to reconstruct the two-dimensional morphological features of the entire fish body; Step (4) includes the following steps: (41) Based on the magnitude of the gray values ​​of the bright and dark parts of the fish body, a suitable threshold is given for thresholding and inverse thresholding respectively; (42) Use the binary image obtained after thresholding and dethresholding as a mask for the bright and dark parts of the fish body to extract the bright and dark parts of the fish body; (43) The bright and dark parts are image fused to determine the two-dimensional shape of the entire fish.

2. The image processing method for underwater three-dimensional particle image velocimetry of robotic fish according to claim 1, characterized in that, The preprocessing in step (2) includes image channel segmentation and removal of tracer particles.

3. The image processing method for underwater three-dimensional particle image velocimetry of robotic fish according to claim 2, characterized in that, The image channel segmentation includes the following steps: (211) Extraction is performed based on the ROI region of the tracer particle image; (212) The B, G, and R channels of the tracer particle image are sequentially divided to obtain a specific image under each single channel; (213) The tracer particle image is converted to grayscale and its contrast is compared with the images in the three single channels of B, G, and R. (214) Based on the contrast results of each channel, select the image with the highest contrast.

4. The image processing method for underwater three-dimensional particle image velocimetry of robotic fish according to claim 2, characterized in that, The process of eliminating tracer particles includes the following steps: (221) Median filtering is performed on the tracer particle image with the best contrast. (222) The median-filtered tracer particle image is fused with the tracer particle image with the best contrast. (223) The removal of tracer particles is achieved by reducing the gamma parameter.

5. The image processing method for underwater three-dimensional particle image velocimetry of robotic fish according to claim 1, characterized in that, The calculation of determining the two-dimensional morphology of the entire fish body in step (43) includes the following steps: (431) Calculate the force at the fluid boundary based on the pressure field around the fish body; (432) Calculate the eddy current at the fluid boundary based on the eddy current field around the fish body; (433) Calculate the velocity at the fluid boundary based on the velocity field around the fish body.

6. The image processing method for underwater three-dimensional particle image velocimetry of robotic fish according to claim 1, characterized in that, The preprocessing of the obtained tracer particle image in step (2) also includes synthesizing a three-dimensional cloud map by tomographic PIV processing based on the underwater robotic fish under multi-channel light source illumination.

7. The image processing method for underwater three-dimensional particle image velocimetry of robotic fish according to claim 6, characterized in that, The underwater robotic fish records data using high temporal resolution measurements.

8. The image processing method for underwater three-dimensional particle image velocimetry of robotic fish according to claim 6, characterized in that, The underwater robotic fish uses multiple rows of cameras arranged in space to provide ample viewpoints from various angles.

Images