Image data processing method and device applied to bubbles, equipment and storage medium

Abstract

The embodiment of the invention provides an image data processing method and device applied to bubbles, equipment and a storage medium. The method comprises the steps of obtaining a to-be-processed image of a camera, and determining a mapping function corresponding to the camera; wherein the to-be-processed image comprises bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object; performing contour acquisition processing on the to-be-processed image of the camera to obtain contour information of bubbles in the to-be-processed image of the camera; determining three-dimensional form information of the bubbles according to the contour information of the bubbles in the to-be-processed images of the cameras and the mapping functions corresponding to the cameras; wherein the three-dimensional morphological information represents morphological characteristics of the bubbles. The method is used for achieving the effect of accurately obtaining the three-dimensional form of the bubble.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device, equipment and storage medium for image data processing applied to bubbles. Background Art

[0002] Bubble flow is a special multiphase flow state formed by gas being distributed in the liquid in the form of small bubbles. In the bubble flow, the liquid phase is continuous, and the gas phase is dispersed in the liquid in the form of small bubbles. Bubble flow is widely used in the fields of environmental energy, chemical engineering, and marine engineering. Among them, the shape of the bubble is an important gas phase characteristic in the bubble flow and one of the important factors affecting the bubble movement. In-depth research on bubbles can explain the relevant mechanisms of gas-liquid mixed flow and help solve related problems in many engineering fields.

[0003] At present, the measurement of bubble morphology mostly relies on probe measurement or mathematical calculation, which results in large errors in the restoration of bubble morphology and contours. When the bubble shape is more complex, it is difficult to restore the three-dimensional shape of the real bubble. Therefore, how to accurately obtain the three-dimensional shape of the bubble has become a problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment and storage medium for processing image data of bubbles, so as to achieve the effect of accurately obtaining the three-dimensional shape of the bubbles.

[0005] In a first aspect, an embodiment of the present application provides an image data processing method for bubbles, including: the method is applied to an electronic device, in which at least three cameras are deployed, and the method includes:

[0006] Acquire an image to be processed from a camera and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical spatial position of the object;

[0007] Performing contour acquisition processing on the camera's image to be processed to obtain contour information of the bubble in the camera's image to be processed;

[0008] The three-dimensional morphological information of the bubble is determined according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0009] In a possible implementation, performing contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed includes:

[0010] Performing noise reduction processing on the image to be processed of the camera to obtain the image to be processed after noise reduction;

[0011] The image to be processed after noise reduction is subjected to contour extraction processing to obtain contour information of bubbles in the image to be processed of the camera.

[0012] In a possible implementation, the image to be processed is a grayscale image; performing image noise reduction processing on the image to be processed of the camera to obtain the image to be processed after noise reduction includes:

[0013] Binarize the image to be processed by the camera according to a preset grayscale threshold to obtain a binary image corresponding to the image to be processed;

[0014] Performing an opening operation on the binary image to obtain a binary image after the opening operation; wherein the opening operation includes corrosion processing and dilation processing;

[0015] The binary image after the opening operation is subjected to area filtering to obtain a denoised image to be processed.

[0016] In a possible implementation, the image to be processed includes tracer particles; performing area filtering on the binary image after the opening operation to obtain a noise-reduced image to be processed includes:

[0017] According to a preset area filtering algorithm, the tracer particles in the binary image after the opening operation are filtered to obtain a noise-reduced image to be processed.

[0018] In a possible implementation, performing contour extraction processing on the image to be processed after noise reduction to obtain contour information of bubbles in the image to be processed of the camera includes:

[0019] Performing boundary recognition processing on the image to be processed after denoising to obtain boundary information of bubbles in the image to be processed after denoising; wherein the boundary information is initial contour information;

[0020] According to the boundary information, the bubbles in the image to be processed after noise reduction are expanded, filled, and eroded to obtain the contour information of the bubbles in the image to be processed of the camera.

[0021] In a possible implementation, determining a mapping function corresponding to a camera includes:

[0022] Determine the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point;

[0023] The mapping function corresponding to the camera is determined according to the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0024] In a possible implementation, determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point includes:

[0025] Get the physical space position of the bubble;

[0026] According to the physical space position of the bubble, a reference position and an end position are determined; wherein the reference position represents the position farthest from the camera in the physical space, and the end position represents the position closest to the camera in the physical space;

[0027] According to the distance between the reference position and the end position, the bubble is divided into equal distances to obtain the dividing points on the bubble;

[0028] The physical space position of the segmentation point and the coordinate position of the pixel point of the segmentation point in the image to be processed are obtained.

[0029] In a possible implementation, determining a mapping function corresponding to the camera according to the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point includes:

[0030] Determine the candidate function corresponding to the camera according to the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point;

[0031] The candidate functions are corrected to obtain the mapping function corresponding to the camera.

[0032] In a possible implementation, the physical space corresponding to the image to be processed includes a plurality of tracer particles, and the physical space corresponding to the image to be processed is divided into a plurality of regions, and the number of tracer particles in different regions is the same; the candidate function is corrected to obtain a mapping function corresponding to the camera, including:

[0033] Determine the coordinate position of the tracer particle in the image to be processed as the initial position;

[0034] Determining an initial mapping position of the tracer particle according to the candidate function corresponding to the camera and the initial position;

[0035] Perform fitting processing on the initial mapping positions of the tracer particles corresponding to each camera to obtain the target mapping positions of the tracer particles;

[0036] Determining a target position of the tracer particle in the image to be processed according to the target mapping position and the candidate function of the camera;

[0037] According to the initial position and the target position, a mapping function corresponding to the camera is determined based on the candidate function.

[0038] In a possible implementation, determining a mapping function corresponding to the camera based on a candidate function according to an initial position and the target position includes:

[0039] Determine a position deviation vector of the tracer particle according to the initial position and the target position; wherein the position deviation vector represents the deviation between the initial position and the target position;

[0040] Determine the region where the tracer particles are located, and determine the position deviation vector of the region according to the position deviation vectors of all tracer particles in the region;

[0041] A mapping function of the camera is obtained according to the position deviation vector of each area and the candidate function of the camera.

[0042] In a possible implementation, determining the three-dimensional morphological information of the bubble according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera includes:

[0043] Determine the physical spatial location of pixels in the image to be processed;

[0044] According to the physical spatial position of the pixel point, based on the contour information of the bubble and the mapping function, the three-dimensional grayscale information of the pixel point is determined; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space, the pixel point and the volume element have a one-to-one correspondence, and the volume element is the smallest spatial unit of the space where the bubble is located;

[0045] The three-dimensional morphological information of the bubble is determined based on the three-dimensional grayscale information of the pixel point.

[0046] In a possible implementation, determining the three-dimensional grayscale information of the pixel point according to the physical spatial position of the pixel point based on the contour information of the bubble and the mapping function includes:

[0047] According to the physical spatial position of the pixel point, the coordinate position of the pixel point, the mapping function, the preset grayscale value, and the preset point propagation function, an intermediate quantity of the pixel point is obtained; wherein the intermediate quantity represents the two-dimensional grayscale information of the pixel point, and the preset point propagation function represents the influence of the pixel point on the grayscale value of the adjacent pixel point;

[0048] According to the contour information of the bubble, the image gray value of the pixel is determined;

[0049] According to the intermediate quantity of the pixel point, the image gray value of the pixel point, and the preset gray value, a target quantity is obtained; wherein the target quantity represents the three-dimensional gray information of the pixel point;

[0050] If the intermediate quantity of the pixel is equal to the image grayscale value of the pixel, then the target quantity is determined to be the three-dimensional grayscale information of the pixel.

[0051] In a possible implementation, it further includes:

[0052] Determine the target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information;

[0053] Determine a spatial grayscale isosurface according to the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness;

[0054] Determine a first diameter of the bubble according to the spatial grayscale isosurface, wherein the first diameter represents the diameter of the bubble in the three-dimensional morphological information;

[0055] The second diameter of the bubble is determined according to the mapping function and the first diameter of the bubble; wherein the second diameter is the actual diameter of the bubble.

[0056] In a possible implementation, it further includes:

[0057] According to the second diameter, the spatial grayscale isosurface is adjusted to obtain a target isosurface;

[0058] The three-dimensional morphological information is adjusted according to the target isosurface to obtain adjusted three-dimensional morphological information.

[0059] In a second aspect, an embodiment of the present application provides an image data processing device applied to bubbles, comprising:

[0060] An acquisition unit, used to acquire an image to be processed from a camera and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object;

[0061] A processing unit, used for performing contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed;

[0062] The determination unit is used to determine the three-dimensional morphological information of the bubble according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0063] In a third aspect, an embodiment of the present application provides an electronic device, including: the electronic device is configured with at least three cameras, a high-pass filter, a water prism, a laser, a concave lens, a plano-convex cylindrical lens, an air outlet, a flow meter, a bubble generating device, an LED light source, a soft light plate, and a water tank;

[0064] The high-pass filter is located in front of the lens of the camera, each camera corresponds to a high-pass filter, the water prism is located on the side of the water tank, the camera lens faces the side where the water prism is located, the LED light source and the soft light plate are located opposite to the side where the water prism is located, the LED light source and the soft light plate are outside the water tank, the soft light plate is close to the water tank, and the LED light source is behind the soft light plate;

[0065] The laser is located on the adjacent surface of the side where the water prism is located, the laser is located behind the concave lens, the concave lens is located behind the plano-convex cylindrical lens, and the plano-convex cylindrical lens is close to the adjacent surface;

[0066] The bubble generating device is located outside the water tank and is used to provide bubbles to the water tank; one end of the flow meter is connected to the bubble generating device, and the other end of the flow meter is connected to the air outlet port and is used to measure the gas flow of the bubble generating device; the air outlet port is located at the bottom of the water tank and is used to discharge the bubbles into the water tank.

[0067] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0068] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0069] The image data processing method, device, equipment and storage medium for bubbles provided in the embodiments of the present application obtain the images to be processed of the same bubble obtained by multiple cameras and the mapping functions corresponding to each camera to obtain the contour information corresponding to the bubble, and determine the three-dimensional morphological information of the bubble according to the contour information and the mapping function. By using multiple cameras, the interference to the bubble measurement is reduced while ensuring accuracy, the contour of the bubble can be effectively identified, and the three-dimensional morphology of the bubble can be accurately reconstructed, thereby improving the accuracy of the three-dimensional measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0071] Figure 1 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure;

[0072] Figure 2 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure;

[0073] Figure 3 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure;

[0074] Figure 4 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure;

[0075] Figure 5 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure;

[0076] Figure 6 A schematic diagram of a first diameter and a second diameter of a bubble provided in an embodiment of the present disclosure;

[0077] Figure 7 A schematic diagram of three-dimensional morphological information of bubbles provided in an embodiment of the present disclosure;

[0078] Figure 8 A schematic diagram of adjusted three-dimensional morphological information of a bubble provided in an embodiment of the present disclosure;

[0079] Fig. 9 A structural block diagram of an image data processing device applied to bubbles provided by an embodiment of the present disclosure;

[0080] Fig.10 A structural block diagram of an image data processing device applied to bubbles provided by an embodiment of the present disclosure;

[0081] Fig.11 A structural block diagram of an electronic device provided in an embodiment of the present disclosure;

[0082] Fig.12 A structural block diagram of an electronic device provided in one embodiment of the present application;

[0083] Fig.13 It is a block diagram of an electronic device according to an exemplary embodiment.

[0084] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0085] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0086] First, the terms involved in this application are explained:

[0087] MART algorithm: Multiplicative Algebraic Reconstruction Techniques (MART) is an algebraic method for image reconstruction, especially used in fields such as tomography (such as CT). MART restores the structure of the original image by iteratively adjusting the image pixel values ​​to minimize the error between the reconstructed image and the observed data. Core principle: MART is based on the algebraic reconstruction method and describes the relationship between projection data and image pixels by establishing a set of linear equations. Its main feature is the use of a multiplication update rule, that is, in each iteration, the pixel value is multiplied according to the current error instead of addition. This method has better convergence and stability when processing certain types of data.

[0088] Laser-induced fluorescence: Laser-induced fluorescence refers to a method of detecting fluorescence emission after laser irradiation of a sample. High-energy lasers are used to excite electrons in a substance to an excited state. When the electrons return, they release energy in the form of photons, forming fluorescence of a specific wavelength.

[0089] PMMA Rhodamine B: Rhodamine B is a dye with fluorescent properties. Its fluorescence emission peak is around 580nm, and it has high fluorescence intensity and stability. PMMA is an acrylic material. PMMA Rhodamine B is a synthetic material produced by doping PMMA material in Rhodamine B material.

[0090] Canny operator: Canny operator is an edge detection algorithm that can effectively suppress noise while maintaining edge information. It can detect small edges. It has high robustness and can process images with various noise types.

[0091] Gaussian filter: Gaussian filtering is a linear smoothing filter that is suitable for eliminating Gaussian noise and is widely used in the noise reduction process of image processing.

[0092] Sobel operator: Sobel operator is an important processing method in the field of computer vision. It is mainly used to obtain the first-order gradient of digital images. Its common application and physical meaning is edge detection. The Sobel operator is to detect the edge by taking the weighted difference of the grayscale values ​​of the four areas of the top, bottom, left and right of each pixel in the image and reaching the extreme value at the edge.

[0093] Roberts operator: Roberts operator is an operator that uses a local difference operator to find edges. It uses the difference between two adjacent pixels in the diagonal direction to approximate the gradient amplitude to detect edges.

[0094] Prewitt operator: The Prewitt operator is a first-order differential operator for edge detection. It uses the grayscale difference between the upper and lower and left and right neighboring points of a pixel to detect the edge at the extreme value at the edge, removes some pseudo edges, and has a smoothing effect on noise.

[0095] Bubble flow is a special multiphase flow state formed by gas being distributed in the liquid in the form of small bubbles. In the bubbly flow, the liquid phase is continuous and the gas phase is dispersed in the liquid in the form of small bubbles. Bubble flow is widely used in the fields of environmental energy, chemical engineering, and marine engineering. Among them, the shape of the bubble is an important gas phase characteristic in the bubbly flow and one of the important factors affecting the bubble movement.

[0096] At present, the methods for obtaining the three-dimensional morphology of bubbles are insufficient in terms of universality. For example, contact measurement methods such as using fiber optic probes to measure bubbles will interfere with the flow field, and the bubble deformation caused by tension will result in large errors; the method of differential reconstruction of bubbles will make the reconstructed bubble contour discontinuous, making it difficult to accurately identify the bubble morphology; bubble reconstruction using neural networks requires a large amount of bubble data collection, and the cost of bubble reconstruction in many different special cases is too high. Therefore, how to accurately and quickly obtain the three-dimensional morphology of bubbles is an important difficulty.

[0097] The specific application scenario of the present application is the reconstruction of the three-dimensional morphology of bubbles, which is used to capture images of bubbles, reconstruct the three-dimensional morphology of bubbles based on information such as bubble images, and obtain the three-dimensional morphology information of bubbles.

[0098] The image data processing method for bubbles provided in the present application is intended to solve the above technical problems in the prior art.

[0099] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0100] Figure 1A flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure is provided. The method can be executed by an image data processing device applied to bubbles, wherein the device is equipped with at least three cameras.

[0101] like Figure 1 As shown, the method comprises the following steps:

[0102] S101, obtaining an image to be processed from a camera, and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object.

[0103] Exemplarily, multiple cameras are provided to capture images of the same bubble, and the camera may be a high-speed camera. For any camera, the image to be processed corresponding to the camera is obtained, and the mapping function corresponding to the camera is determined according to the image to be processed of the camera. For example, the pixel points in the image to be processed can be matched with the part of the bubble represented by the pixel points in the physical space, and the mapping relationship between the pixel in the image to be processed and the physical space is obtained, which is the mapping function. The image to be processed is the image of the bubble taken by the camera, and the image to be processed contains grayscale information. The mapping function represents the relationship between the coordinate position of the object in the image to be processed corresponding to the camera and the physical space position of the object, and the object in the image may be a pixel point in the image.

[0104] S102 , performing contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed.

[0105] Exemplarily, after obtaining the image to be processed corresponding to each camera, for any camera, according to the image to be processed of the camera, the image to be processed of the camera is subjected to contour acquisition processing, and the contour information of the bubbles in the image to be processed of the camera is obtained. For example, the image to be processed can be subjected to image denoising processing to remove the noise of the non-bubble image in the image to be processed, and then the contour of the bubbles in the denoised image to be processed is extracted to extract the contour information of the bubbles in the denoised image to be processed. In this embodiment, the method of image denoising processing and the method of extracting the contour of the bubble are not specifically limited.

[0106] S103 , determining three-dimensional morphological information of the bubbles according to contour information of the bubbles in the to-be-processed images of each camera and a mapping function corresponding to each camera; wherein the three-dimensional morphological information represents morphological features of the bubbles.

[0107] Exemplarily, the contour information of the bubble in the image to be processed includes a plurality of pixels, and each pixel corresponds to the grayscale information of the pixel. After obtaining the contour information of the bubble in the image to be processed corresponding to each camera, the three-dimensional morphological information of the bubble is determined according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera. For example, according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera, the grayscale information corresponding to any pixel in the bubble contour information can be used as the first grayscale information of the pixel. For any first grayscale information, the first grayscale information is inversely projected into the physical space through the corresponding mapping function to obtain the spatial grayscale information corresponding to each pixel. According to the corresponding mapping function, the spatial grayscale information of each pixel is projected to the corresponding position in the contour information image of the bubble, and the image grayscale information corresponding to the pixel at each corresponding position is obtained, which is the second grayscale information. The spatial grayscale information corresponding to each pixel is continuously iterated and optimized by the MART algorithm. When the difference between the first grayscale information and the second grayscale information is less than the preset grayscale difference threshold, the optimization is stopped, and the spatial grayscale information at this time is used as the three-dimensional morphological information of the bubble. The spatial grayscale information represents the grayscale information in the three-dimensional space. In this embodiment, the projection and back-projection methods are not specifically limited.

[0108] The image data processing method, device, equipment and storage medium for bubbles provided in the embodiments of the present application obtain the images to be processed of the same bubble obtained by multiple cameras and the mapping functions corresponding to each camera to obtain the contour information corresponding to the bubble, and determine the three-dimensional morphological information of the bubble according to the contour information and the mapping function. By using multiple cameras, the interference to the bubble measurement is reduced while ensuring accuracy, the contour of the bubble can be effectively identified, and the three-dimensional morphology of the bubble can be accurately reconstructed, thereby improving the accuracy of the three-dimensional measurement.

[0109] Figure 2 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure.

[0110] In this embodiment, contour acquisition processing is performed on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed, including: performing noise reduction processing on the camera's image to be processed to obtain a noise-reduced image to be processed; performing contour extraction processing on the noise-reduced image to be processed to obtain contour information of bubbles in the camera's image to be processed.

[0111] like Figure 2 As shown, the method comprises the following steps:

[0112] S201, obtaining an image to be processed from a camera, and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object.

[0113] Exemplarily, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0114] S202: performing noise reduction processing on the image to be processed of the camera to obtain the image to be processed after noise reduction.

[0115] Exemplarily, the image to be processed is subjected to noise reduction processing to obtain the image to be processed after noise reduction. Noise reduction processing is used to remove noise in the image to be processed, interference other than bubbles in the image. For example, a preset filter can be used to filter the image to be processed. The preset filter is used to filter out noise in the image to be processed, for example, it can be a median filter or a Gaussian filter. In this embodiment, no specific limitation is made on the type of preset filter.

[0116] The beneficial effect of such a setting is that noise and interference other than bubbles in the image to be processed are removed, making the bubbles in the image to be processed clearer, facilitating the subsequent extraction of bubble contour information, etc., and better retaining image detail information.

[0117] In this embodiment, the image to be processed is a grayscale image; image denoising is performed on the image to be processed of the camera to obtain the denoised image to be processed, including: binarizing the image to be processed of the camera according to a preset grayscale threshold to obtain a binary image corresponding to the image to be processed; performing an opening operation on the binary image to obtain a binary image after the opening operation; wherein the opening operation includes corrosion processing and dilation processing; performing area filtering processing on the binary image after the opening operation to obtain the denoised image to be processed.

[0118] Specifically, the image to be processed is a grayscale image, and the image to be processed is divided into multiple regions. For example, the image to be processed can be divided into 9 regions of 3×3 at equal intervals. For any region, the sensitivity information of the region is obtained. The sensitivity information represents the percentage of the number of pixels of the bubble outline in the entire image. The sensitivity information can be, for example, 50%. The grayscale value sequence in the region is obtained. The grayscale value sequence represents the order of the grayscale values ​​of the region from low to high. The grayscale value at the sensitivity information position in the grayscale value sequence is used as the grayscale threshold of the region. For example, in the grayscale value sequence, the grayscale value 125 at the 50% position of the grayscale value sequence is used as the grayscale threshold of the region.

[0119] For any region, the region is binarized according to the grayscale threshold of the region and the preset binarization formula. Binarization is used to convert the image to be processed into a black and white image. After binarization is performed on each region of the image to be processed, a binary image corresponding to the image to be processed is obtained. The preset binarization formula can be:

[0120]

[0121] Among them, I(x,y) represents the gray value of the pixel at the position (x,y) of the image to be processed, T represents the gray threshold corresponding to the area where the pixel is located, and I e (x,y) represents the grayscale value of the pixel at position (x,y) in the binary image.

[0122] After obtaining the binary image corresponding to the image to be processed, the binary image is opened, that is, the binary image is first eroded and then expanded. Erosion is used to remove noise such as burrs in the binary image and separate the adhered parts in the binary image. The formula of the erosion process is as follows:

[0123]

[0124] Among them, I ero (x, y) represents the gray value of the pixel at the position (x, y) of the binary image after the erosion. ero,x (x, y) represents the gray value of the pixel at position (x, y) after the pixel is corroded in the horizontal direction; I ero,y (x, y) represents the gray value of the position after the pixel at the position is eroded in the vertical direction.

[0125] When the grayscale values ​​of the horizontally adjacent (x-1, y) and (x+1, y) positions of the pixel at (x, y) are both 1, the grayscale value of this position is 1, otherwise, the grayscale value of this position is 0. When the grayscale values ​​of the vertically adjacent (x, y-1) and (x, y-1) positions of the pixel at (x, y) are both 1, the grayscale value of this position is 1, otherwise, the grayscale value of this position is 0. When the grayscale value of the horizontally adjacent (x, y) position is I ero,x (x, y) and the vertical erosion gray value I ero,y When (x, y) are equal, the grayscale value of this position is 1, otherwise the grayscale value of this position is 0.

[0126] After obtaining the binary image after the corrosion processing, the binary image after the corrosion processing is expanded. The expansion processing is used to increase the edge width of the bubble in the binary image after the corrosion processing, making the edge of the bubble more obvious. The formula of the expansion processing process is as follows:

[0127]

[0128] Among them, I dil (x, y) represents the gray value of the pixel at the position (x, y) in the image after the expansion process of the binary image after the corrosion process. dil,x (x, y) represents the gray value of the pixel at position (x, y) after dilation in the horizontal direction; I dil,y (x, y) represents the grayscale value of the position after the pixel at the position is expanded in the vertical direction.

[0129] When the grayscale value of the horizontally adjacent (x-1, y) position or (x+1, y) position of the pixel at (x, y) is 1, the grayscale value of this position is 1, otherwise, the grayscale value of this position is 0. When the grayscale value of the vertically adjacent (x, y-1) position or (x, y-1) position of the pixel at (x, y) is 1, the grayscale value of this position is 1, otherwise, the grayscale value of this position is 0. e (x,y), the horizontal expansion gray value I dil,x (x, y), and the vertical expansion gray value I dil,y (x, y) are added. If the added result is greater than 0, the grayscale value of the point is 1; if the added result is equal to 0, the grayscale value of the point is 0.

[0130] After obtaining the binary image after the opening operation, if the grayscale values ​​of the pixels adjacent to the pixel in the four directions of up, down, left, and right in the binary image after the opening operation are the same as the grayscale value of the pixel, then the pixel and the pixels adjacent to the pixel in the four directions of up, down, left, and right constitute a connected area, and multiple connected areas and the areas of the connected areas in the binary image after the opening operation are obtained. Among them, the connected area represents the area where the pixel in the image has the same grayscale value as the adjacent pixel. According to the connected area and the area of ​​the connected area in the binary image after the opening operation, the binary image after the opening operation is subjected to area filtering, and the connected area with an area smaller than the preset area is filtered out to obtain the image to be processed after noise reduction.

[0131] The beneficial effect of this setting is that the image to be processed is divided into multiple regions, and the grayscale threshold of each region is used to binarize the region, making the binarization process more accurate and reducing the error caused by using the same grayscale threshold for binarization. Opening and area filtering of the image can make the bubble shape clearer and eliminate other interference.

[0132] In this embodiment, the image to be processed includes tracer particles; area filtering is performed on the binary image after the opening operation to obtain the image to be processed after noise reduction, including: according to a preset area filtering algorithm, the tracer particles in the binary image after the opening operation are filtered to obtain the image to be processed after noise reduction.

[0133] Specifically, the image to be processed also includes an image of tracer particles, which are fluorescent particles added to the liquid for photographing bubbles, such as PMMA Rhodamine B type particles. The tracer particles are used to display the flow of the liquid, and the tracer particles can generate fluorescence of a specific wavelength under laser irradiation. Therefore, in addition to the image of the bubble, the image to be processed also includes an image of the tracer particles.

[0134] The bubble area mean is preset, and the bubble area mean represents the average area size of the bubbles. The preset area filtering algorithm is used to filter the image of the tracer particles in the binary image after the opening operation. For example, after determining the area of ​​the connected region in the binary image after the opening operation, the area whose connected area is less than 1 / 3 of the preset bubble area mean is determined as the tracer particle area, and the tracer particle area is deleted.

[0135] According to a preset area filtering algorithm, the tracer particles in the binary image after the opening operation are filtered to obtain a noise-reduced image to be processed.

[0136] The beneficial effect of such a setting is that the connected areas that do not belong to bubbles in the binary image after the opening operation can be filtered out, so that the image to be processed after noise reduction is purer, which is convenient for subsequent contour extraction.

[0137] S203 , performing contour extraction processing on the image to be processed after noise reduction to obtain contour information of bubbles in the image to be processed of the camera.

[0138] Exemplarily, after obtaining the denoised image to be processed, the contour of the bubbles in the denoised image to be processed is extracted to obtain the contour information of the bubbles in the image to be processed corresponding to the camera. For example, the denoised image to be processed can be firstly repaired to repair the missing parts in the denoised image to be processed due to the denoising process, reduce the distortion of the bubble shape, and then connect the outermost pixel points of the bubbles in the repaired image to obtain the contour information of the bubbles.

[0139] The beneficial effect of such a setting is that the image to be processed after noise reduction is repaired, the loss of bubble morphology caused by processing is reduced, the distortion that may be caused by bubble contour extraction is reduced, and the bubble contour is made more consistent with the real bubble shape.

[0140] In this embodiment, contour extraction processing is performed on the image to be processed after denoising to obtain contour information of bubbles in the camera's image to be processed, including: boundary recognition processing is performed on the image to be processed after denoising to obtain boundary information of bubbles in the image to be processed after denoising; wherein the boundary information is initial contour information; according to the boundary information, expansion processing, filling processing, and corrosion processing are performed on the bubbles in the image to be processed after denoising to obtain contour information of bubbles in the camera's image to be processed.

[0141] Specifically, the Canny operator is used to identify the boundaries of the image to be processed after denoising. The identification step can be, for example, using a two-dimensional Gaussian filter to smooth the image to be processed after denoising, and obtaining a smoothed image, which is a smoothed image. The Sobel operator is used to obtain the gradient amplitude of the smoothed image. The gradient represents the magnitude of the grayscale change of the image; the direction of the gradient represents the direction of the grayscale change of the image; the gradient amplitude represents the magnitude of the gradient, that is, the severity of the grayscale change of the image. The gradient amplitude is subjected to non-maximum suppression processing, that is, for any gradient direction, the gradient amplitude of the pixel points of the smoothed image is traversed along the gradient direction to obtain the location of the pixel point with the maximum gradient value, and the other pixel points in the gradient direction are deleted, and the pixel point with the maximum gradient value is used as the edge pixel point in the gradient direction.

[0142] In one example, the gradient amplitude of the pixel point in the smoothed image may also be calculated by using methods such as the Roberts operator or the Prewitt operator. In this embodiment, the method for obtaining the gradient amplitude of the pixel point in the smoothed image is not specifically limited.

[0143] A high gradient threshold and a low gradient threshold are pre-set, and the high gradient threshold and the low gradient threshold are used to classify edge points. If the gradient amplitude of an edge pixel is greater than the high gradient threshold, the edge pixel is determined to be a strong edge point; if the gradient amplitude of an edge pixel is greater than the low gradient threshold and less than the high gradient threshold, the edge pixel is determined to be a weak edge point; if the gradient amplitude of an edge pixel is less than the low gradient threshold, the edge pixel is removed. For a strong edge point, the portion of the edge point connected to the adjacent edge pixel is retained; for a weak edge point, only the portion of the weak edge point connected to the strong edge point is retained, and the boundary information of the bubble in the image to be processed after denoising is obtained.

[0144] According to the expansion process formula, the boundary information of the bubble is expanded to close the unclosed boundary information, and the boundary information of the bubble after the expansion process is obtained as the first image. The first image is filled so that the grayscale values ​​of the pixels in the image inside the boundary information are all 1, and the filled boundary information image is obtained as the second image. According to the erosion process formula, the second image is eroded to obtain the third image.

[0145] From the point P with the smallest ordinate L First, traverse each pixel point on the bubble boundary in the third image to obtain the remaining pixel points and P L The angle between the vectors formed by the two vectors relative to the horizontal direction, and the angles are sorted in ascending order to obtain P L The angle sequence of the vector. From point P L Initially, the pixel points are connected counterclockwise in the order of the angle sequence to obtain the contour information of the bubble in the camera's image to be processed.

[0146] The beneficial effect of this setting is that the Canny operator is used to obtain the boundary information of the bubble and repair the boundary information of the bubble, which can make the contour information of the bubble clearer and more accurate, remove the depressions in the bubble image, connect the pixel points on the boundary of the third image according to the vector angle, and reduce the missing contour.

[0147] S204 , determining three-dimensional morphological information of the bubbles according to contour information of the bubbles in the to-be-processed images of each camera and a mapping function corresponding to each camera; wherein the three-dimensional morphological information represents morphological features of the bubbles.

[0148] Exemplarily, this step may refer to the above-mentioned step S103 and will not be described in detail.

[0149] The image data processing method, device, equipment and storage medium for bubbles provided in the embodiments of the present application obtain the images to be processed of the same bubble obtained by multiple cameras and the mapping functions corresponding to each camera to obtain the contour information corresponding to the bubble, and determine the three-dimensional morphological information of the bubble according to the contour information and the mapping function. By using multiple cameras, the interference to the bubble measurement is reduced while ensuring accuracy, the contour of the bubble can be effectively identified, and the three-dimensional morphology of the bubble can be accurately reconstructed, thereby improving the accuracy of the three-dimensional measurement.

[0150] Figure 3 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure.

[0151] In this embodiment, determining the mapping function corresponding to the camera includes: determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point; determining the mapping function corresponding to the camera according to the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0152] like Figure 3 As shown, the method comprises the following steps:

[0153] S301, obtaining an image to be processed from a camera, and determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0154] Exemplarily, for any camera, an image to be processed corresponding to the camera is obtained, and based on the image to be processed, a coordinate system of the image to be processed is determined, which is a first coordinate system. The coordinate position of each pixel point of the image to be processed in the first coordinate system is obtained. Based on the first coordinate system, a coordinate system of the physical space is established, which is a second coordinate system.

[0155] After acquiring the image to be processed from the camera, the corresponding positions of the pixels in the image to be processed in the physical space are determined. For example, a calibration plate with obvious features can be used, and the calibration plate can be, for example, a checkerboard calibration plate, and the size and number of checkerboards in the checkerboard calibration plate are determined, and the coordinate position of the feature point in the second coordinate system, that is, the physical space position of the feature point, is matched with the coordinate position of the pixel in the image to be processed in the first coordinate system, that is, the coordinate position of the pixel in the image to be processed, and the physical space positions corresponding to the remaining pixels are determined based on the feature point and the pixel corresponding to the feature point.

[0156] The beneficial effect of such a setting is that the image to be processed is calibrated, the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point are determined, the connection between the two-dimensional image and the three-dimensional space is established, and the physical and pixel spaces are accurately matched.

[0157] In this embodiment, determining the coordinate position of a pixel point in an image to be processed and the physical space position corresponding to the pixel point includes: obtaining the physical space position of the bubble; determining a reference position and an end position according to the physical space position of the bubble; wherein the reference position represents the position farthest from the camera in the physical space position, and the end position represents the position closest to the camera in the physical space position; dividing the bubble into equal distances according to the distance between the reference position and the end position to obtain a dividing point on the bubble; obtaining the physical space position of the dividing point and the coordinate position of the dividing point at the pixel point in the image to be processed.

[0158] Specifically, for any camera, obtain the physical space position of the bubble in the image to be processed of the camera, that is, the position of the bubble in the second coordinate system. The position of the bubble in the physical space position that is farthest from the camera is used as the reference position; the position of the bubble in the physical space position that is closest to the camera is used as the end position. According to the distance between the reference position and the end position, the bubble is divided into equal distances, and the distance can be, for example, 0.1 mm. Obtain multiple sections on the bubble and the section points corresponding to each section, and obtain the coordinate position in the second coordinate system corresponding to each section point, that is, the physical space position of the section point. Use the calibration plate to traverse each section, and based on the preset calibration method, determine the coordinate position of the pixel point in the image to be processed corresponding to the section point on each section.

[0159] The beneficial effect of this setting is that a layered calibration is performed at equal intervals from the reference position to the end position, and the physical space position of the dividing point on each cut plane and the coordinate position of the pixel point of the dividing point in the image to be processed are obtained, which not only improves the accuracy of the calibration, but also makes the correspondence between the pixel point and the physical space more accurate.

[0160] S302: Determine a mapping function corresponding to the camera according to the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0161] Exemplarily, for any camera, the mapping function corresponding to the camera is determined according to the coordinate position of the pixel point in the image to be processed of the camera and the physical space position corresponding to the pixel point. For example, the coordinate position of the pixel point and the physical space position corresponding to the pixel point can be fitted using the least squares method to obtain the mapping relationship between the coordinate position of the pixel point and the physical space position corresponding to the pixel point, that is, the mapping function.

[0162] The beneficial effect of such a setting is that a connection is established between the coordinate position of a pixel point and the physical space position corresponding to the pixel point, and a conversion relationship between the physical space position corresponding to the pixel point and the coordinate position of the pixel point is obtained, which facilitates reconstruction of the bubble in subsequent steps.

[0163] In this embodiment, a mapping function corresponding to the camera is determined based on the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point, including: determining a candidate function corresponding to the camera based on the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point; and correcting the candidate function to obtain a mapping function corresponding to the camera.

[0164] Specifically, for any camera, the candidate function corresponding to the camera is determined according to the coordinate position of the pixel point in the image to be processed by the camera and the physical space position corresponding to the pixel point. The candidate function represents the mapping relationship between the coordinate position of the pixel point and the physical space position corresponding to the pixel point. The candidate function includes a candidate function in the horizontal direction and a candidate function in the vertical direction. For example, a third-order precision polynomial can be used to approximate the mapping relationship between the physical space position and the coordinates of the pixel point. The third-order precision polynomial can be, for example:

[0165] F(X)=a0+a1x+a2y+a3z+a4x 2 +a5xy+a6y 2 +a7xz+a8yz+a9z 2

[0166] +a 10 x 3 +a 11 x 2 y+a 12 xy 2 +a 13 y 3 +a 14 x 2 z+a 15 xyz+a 16 y 2 z

[0167] +a 17 xz 2 +a 18 yz 2

[0168] F(Y)=b0+b1x+b2y+b3z+b4x 2 +b5xy+b6y 2 +b7xz+b8yz+b9z 2

[0169] +b 10 x 3 +b 11 x 2 y+b 12 xy 2 +b 13 y 3 +b14 x 2 z+b 15 xyz+b 16 y 2 z

[0170] +b 17 xz 2 +b 18 yz 2

[0171] Among them, (X, Y) is the coordinate of the pixel point in the image to be processed, that is, the coordinate position of the pixel point in the first coordinate system; (x, y, z) is the physical space position corresponding to the pixel point, that is, the coordinate position of the pixel point in the second coordinate system. i and b i is the calibration coefficient, which represents the coefficient of the unknown quantity in the polynomial. For all the pixels in the image to be processed, by substituting the coordinates of the pixel points in the image to be processed and the physical space position corresponding to the pixel point into the above formula, all the calibration coefficients a can be solved. i and b i , and obtain the candidate function F(X) in the horizontal direction and the candidate function F(Y) in the vertical direction. i is 0 to 18.

[0172] For any camera, after obtaining the candidate function F(X) in the horizontal direction and the candidate function F(Y) in the vertical direction, the candidate functions in the horizontal direction and the vertical direction are corrected to obtain the mapping function corresponding to the camera. The correction process is used to correct the candidate function to reduce the deviation of the mapping relationship between the candidate function and the actual pixel coordinates and the physical space position. For example, the correction process can be to pre-set a function deviation value, the function deviation value represents the deviation of the mapping relationship between the candidate function and the actual pixel coordinates and the physical space position, and superimpose the preset function deviation value on the candidate function to obtain the mapping function.

[0173] The beneficial effect of such a setting is that the least squares method is used to fit the coordinate positions of all pixel points in the processed image and the physical space positions corresponding to the pixel points to obtain high-precision calibration coefficients, that is, to obtain a high-precision mapping relationship between the coordinate positions of the pixel points and the physical space, and to correct the candidate function to eliminate the obvious deviations that may exist between the candidate function and the actual mapping relationship caused by various factors, so that the correspondence between the coordinate positions of the pixel points and the physical space positions is more accurate.

[0174] In this embodiment, the physical space corresponding to the image to be processed includes multiple tracer particles, and the physical space corresponding to the image to be processed is divided into multiple areas, and the number of tracer particles in different areas is the same; the candidate function is corrected to obtain the mapping function corresponding to the camera, including: determining the coordinate position of the tracer particles in the image to be processed as the initial position; determining the initial mapping position of the tracer particles according to the candidate function and the initial position corresponding to the camera; fitting the initial mapping position of the tracer particles corresponding to each camera to obtain the target mapping position of the tracer particles; determining the target position of the tracer particles in the image to be processed according to the target mapping position and the candidate function of the camera; determining the mapping function corresponding to the camera based on the candidate function according to the initial position and the target position.

[0175] Specifically, for any camera, the image to be processed of the camera contains image information of multiple tracer particles, and the coordinate positions of the tracer particles in the image to be processed are obtained. According to the coordinate positions of the tracer particles in the image to be processed and the candidate function corresponding to the camera, the physical space position corresponding to the tracer particles is obtained. According to the tracer particles and the spatial positions corresponding to the tracer particles, the physical space corresponding to the image to be processed is divided to obtain multiple physical space regions, and the number of corresponding tracer particles in each region is the same.

[0176] For any tracer particle, the coordinate position of the tracer particle in the image to be processed is determined as the initial position. According to the initial position and the candidate function corresponding to each camera, the position of the tracer particle in the physical space corresponding to each camera is determined as the initial mapping position of the tracer particle in the physical space corresponding to each camera. The initial mapping position of the tracer particle in the physical space corresponding to each camera is fitted to an intersection point to obtain the target mapping position of the tracer particle. According to the target mapping position and the candidate function of each camera, the position of the tracer particle in the image to be processed of each camera is determined as the target position of the tracer particle in the image to be processed.

[0177] For any tracer particle, determine the initial position and target position of the tracer particle corresponding to each camera, and determine the vector between the initial position and target position of the tracer particle based on the initial position and target position of the tracer particle. For each region, determine the probability density distribution of the vector size between the initial position and target position of all tracer particles in the region, and use the vector corresponding to the vector size of the probability density peak in the region as the target vector of the region.

[0178] In the physical space of the area, a physical space position with the same number of tracer particles in the area is randomly selected as the first position. For any first position, based on the corresponding candidate function, the coordinate position of the pixel space corresponding to the first position is obtained as the second position. The target vector is superimposed on the second position to obtain the third position. According to the first position and the third position, the new candidate function corresponding to each camera is determined, which is the mapping function corresponding to each camera.

[0179] The beneficial effect of such a setting is that the physical space corresponding to the image to be processed is divided into regions according to the tracer particles, so that the target vector can be calculated by region, ensuring the accuracy of the deviation correction, and by using tracer particles and multiple cameras to fit the mapping position of the tracer particles and then mapping it to the image to be processed, the final mapping function can have higher accuracy.

[0180] In this embodiment, according to the initial position and the target position, based on the candidate function, the mapping function corresponding to the camera is determined, including: determining the position deviation vector of the tracer particle according to the initial position and the target position; wherein the position deviation vector represents the deviation between the initial position and the target position; determining the area where the tracer particle is located, and determining the position deviation vector of the area according to the position deviation vectors of all tracer particles in the area; and obtaining the mapping function of the camera according to the position deviation vectors of each area and the candidate function of the camera.

[0181] Specifically, for any tracer particle, the position deviation vector of the tracer particle corresponding to each camera is determined according to the initial position and target position in the to-be-processed image of each camera corresponding to the tracer particle, and the position deviation vector represents the deviation between the initial position and the target position. For each region, the position deviation vectors of all tracer particles in the region are determined, and the probability density distribution of the position deviation vector size of the region is determined according to the size of the position deviation vectors of all tracer particles in the region, and the position deviation vector corresponding to the peak value of the probability density in the region is used as the position deviation vector of the region.

[0182] For any area, determine the physical space position coordinates of the center position of the area, based on the candidate function of the camera corresponding to the area, determine the coordinate position of the pixel point of the center position in the image to be processed corresponding to the camera, superimpose the position deviation vector of the area to the coordinate position of the pixel point, and obtain the coordinate position of the superimposed pixel point. According to the physical space position coordinates of the center position of the area and the coordinate position of the superimposed pixel point, determine the new candidate function corresponding to each camera, that is, the mapping function corresponding to each camera.

[0183] The beneficial effect of such a setting is that the mapping function is determined using the position deviation vector corresponding to each area and the center position of each area, so that the mapping function has better accuracy, the deviation error is smaller, and the result is more in line with expectations.

[0184] S303 , performing contour acquisition processing on the camera's image to be processed to obtain contour information of the bubbles in the camera's image to be processed.

[0185] Exemplarily, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0186] S304 , determining three-dimensional morphological information of the bubbles according to contour information of the bubbles in the to-be-processed images of each camera and a mapping function corresponding to each camera; wherein the three-dimensional morphological information represents morphological features of the bubbles.

[0187] Exemplarily, this step may refer to the above-mentioned step S103 and will not be described in detail.

[0188] The image data processing method, device, equipment and storage medium for bubbles provided in the embodiments of the present application obtain the images to be processed of the same bubble obtained by multiple cameras and the mapping functions corresponding to each camera to obtain the contour information corresponding to the bubble, and determine the three-dimensional morphological information of the bubble according to the contour information and the mapping function. By using multiple cameras, the interference to the bubble measurement is reduced while ensuring accuracy, the contour of the bubble can be effectively identified, and the three-dimensional morphology of the bubble can be accurately reconstructed, thereby improving the accuracy of the three-dimensional measurement.

[0189] Figure 4 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure.

[0190] In this embodiment, the three-dimensional morphological information of the bubble is determined according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera, including: determining the physical spatial position of the pixel point in the image to be processed; determining the three-dimensional grayscale information of the pixel point according to the physical spatial position of the pixel point based on the contour information of the bubble and the mapping function; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space, the pixel point and the volume element have a one-to-one correspondence, and the volume element is the smallest spatial unit of the space where the bubble is located; the three-dimensional morphological information of the bubble is determined according to the three-dimensional grayscale information of the pixel point.

[0191] like Figure 4 As shown, the method comprises the following steps:

[0192] S401, obtaining an image to be processed from a camera, and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object.

[0193] Exemplarily, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0194] S402 , performing contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed.

[0195] Exemplarily, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0196] S403: Determine the physical spatial position of the pixel points in the image to be processed.

[0197] Exemplarily, for any camera, the coordinate position of each pixel point in the image to be processed corresponding to the camera is obtained, and the physical space position corresponding to each pixel point in the image to be processed of the camera is determined according to the coordinate position of the pixel point and based on the mapping function corresponding to the camera.

[0198] The beneficial effect of such a setting is that the physical space position corresponding to each pixel point is determined through a mapping function, which facilitates the subsequent acquisition of three-dimensional grayscale information.

[0199] S404. Determine the three-dimensional grayscale information of the pixel according to the physical spatial position of the pixel, based on the contour information of the bubble and the mapping function; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel in the three-dimensional space, the pixel and the volume element correspond one to one, and the volume element is the smallest spatial unit of the space where the bubble is located.

[0200] Exemplarily, according to the physical space position of the pixel point and the contour information of the bubble, the physical space corresponding to the contour information of the bubble is divided into multiple volume elements. The volume element represents the smallest spatial unit in the physical space where the bubble is located, and each volume element corresponds to a pixel point one by one.

[0201] According to the physical spatial position of the pixel, based on the contour information of the bubble of each camera and the mapping function, the three-dimensional grayscale information of the pixel is determined. For example, according to the physical spatial position of the pixel and the grayscale information of the pixel in the contour information of the bubble, based on the corresponding mapping function, the grayscale information of the pixel can be mapped to the volume element of the physical spatial position of the pixel, and the grayscale value in the three-dimensional space corresponding to the volume element can be obtained.

[0202] The beneficial effect of this setting is that the grayscale information of the pixel points in the contour information of the bubble is matched with the volume elements in the physical space to obtain the three-dimensional grayscale information corresponding to the pixel points. By using the grayscale information in the contour information of the bubble, the three-dimensional grayscale information is made purer and more consistent with the shape of the bubble.

[0203] In this embodiment, the three-dimensional grayscale information of the pixel is determined according to the physical space position of the pixel, based on the contour information of the bubble and the mapping function, including: obtaining the intermediate quantity of the pixel according to the physical space position of the pixel, the coordinate position of the pixel, the mapping function, the preset grayscale value, and the preset point propagation function; wherein the intermediate quantity represents the two-dimensional grayscale information of the pixel, and the preset point propagation function represents the influence of the pixel on the grayscale value of the adjacent pixel; determining the image grayscale value of the pixel according to the contour information of the bubble; obtaining the target quantity according to the intermediate quantity of the pixel, the image grayscale value of the pixel, and the preset grayscale value; wherein the target quantity represents the three-dimensional grayscale information of the pixel; if the intermediate quantity of the pixel is equal to the image grayscale value of the pixel, then determining the target quantity as the three-dimensional grayscale information of the pixel.

[0204] Specifically, in the process of determining the three-dimensional grayscale information of the pixel point, each volume element can be regarded as a point light source, and the point light source represents a light source that can be idealized as a particle, that is, the grayscale value of the volume element will affect the grayscale value of its adjacent volume elements like a light source. A point propagation function is preset, and the point propagation function represents the influence of the grayscale value of the point light source on the grayscale values ​​of other volume elements. The preset point propagation function can be, for example, a Gaussian function.

[0205] In an example, an error function may also be used as a point propagation function. In this embodiment, the specific type of the point propagation function is not specifically limited.

[0206] For any camera, the number of all volume elements in the corresponding physical space and the number of pixels in the contour information of the corresponding bubble are obtained, and the volume elements and pixels are numbered according to the distance from the camera. For each volume element, a preset grayscale value in the three-dimensional space is assigned, and the preset grayscale value can be, for example, 1.

[0207] Based on the preset intermediate quantity determination formula, the intermediate quantity of the pixel is obtained according to the physical space position of the pixel, the coordinate position of the pixel, the mapping function, the preset gray value, and the preset point propagation function. The preset intermediate quantity determination formula is used to determine the intermediate quantity, and the intermediate quantity represents the gray value of the pixel, that is, the two-dimensional gray information of the pixel. The preset intermediate quantity determination formula can be, for example:

[0208]

[0209] Where k represents the number of iterations. N represents the number of volume elements. m represents the camera number. W i,j Characterize the influence of the j-th volume element on the gray value of the i-th pixel; h is the point spread function, Represents pixel i, The volume element j is represented by a mapping function to obtain the corresponding pixel point in the image. Represents the distance between pixel i and the pixel corresponding to volume element j. The larger the distance, the smaller the influence of volume element j on the gray value of pixel i. j ,y j ,z j ) represents the three-dimensional grayscale value of volume element j, that is, located at (x j ,y j ,z j ) is the three-dimensional spatial grayscale value of the volume element at . In the first calculation, the spatial grayscale value is a preset grayscale value. Characterize the intermediate quantity, that is, at (X i ,Y i ) is the gray value of the pixel at .

[0210] This formula indicates that for a camera numbered m, the influence of the jth volume element on the gray value of the ith pixel is multiplied by the gray value of the kth iteration of the volume element, and all products are accumulated to obtain the sum of the gray value products, which is the intermediate quantity corresponding to the ith pixel of the kth iteration of the camera numbered m. All pixels are traversed to obtain the intermediate quantity corresponding to each pixel.

[0211] According to the contour information of the bubble, the gray value of each pixel in the contour information of the bubble is determined. Based on the preset grayscale update formula, the target amount is obtained according to the intermediate amount of the pixel, the image grayscale value of the pixel, and the preset grayscale value. The preset grayscale update formula is used to update the grayscale value of each volume element to obtain the target amount. The target amount represents the three-dimensional grayscale information of the pixel. The preset grayscale update formula can be:

[0212]

[0213] Wherein, μ represents the speed of iteration. The larger the value of μ is, the faster the iteration speed is. The value of μ can be, for example, 1. m (X i .Y i ) represents the gray value of the i-th pixel in the bubble contour information, that is, the gray value of the pixel located at (X i ,Y i ) is the gray value of the pixel at .

[0214] This formula indicates that for a camera numbered m, the gray value of the jth volume element of the kth iteration is multiplied by μ, the ratio of the gray value of the i-th pixel in the bubble contour information to the intermediate value of the i-th pixel of the kth iteration. The value of the power is multiplied, and the multiplied value is used as the grayscale value of the k+1th iteration, which is the target value.

[0215] According to the target quantity, based on the preset intermediate quantity determination formula and the preset grayscale update formula, the target quantity of all volume elements is continuously updated iteratively. When the intermediate quantity corresponding to the pixel point is equal to the grayscale value corresponding to the pixel point in the bubble contour information, that is, I m (X i .Y i )and When they are equal, the target volume at that time is obtained, and the target volume at that time is used as the three-dimensional grayscale information of the volume element corresponding to the pixel point, and the iteration of the target volume for this pixel point is stopped. The three-dimensional grayscale information of the volume elements corresponding to all the pixels is determined.

[0216] The beneficial effect of such a setting is that each volume element is regarded as a point light source, and the influence of the gray value of the volume element on the gray value of other volume elements is obtained by using the point propagation function, and the gray value of each volume element is continuously iterated by the MART algorithm, and the gray value information in the three-dimensional space is reconstructed according to the contour information of the bubble, so that the gray value information in the three-dimensional space is more accurate. The process of obtaining the gray value in the three-dimensional space can be regarded as a back projection process. The method described in this embodiment can reduce the difficulty of the back projection process.

[0217] S405 . Determine the three-dimensional morphological information of the bubble according to the three-dimensional grayscale information of the pixel point.

[0218] Exemplarily, the surface of the three-dimensional morphology of the bubble is determined based on the contour information of the bubble and the three-dimensional grayscale information corresponding to the pixel point, which is the three-dimensional morphological information of the bubble. For example, multiple isosurfaces in the three-dimensional grayscale information can be obtained, that is, the three-dimensional spatial grayscale values ​​of the three-dimensional grayscale information corresponding to the volume elements on the isosurfaces are the same, and the isosurfaces corresponding to the contour information of the bubble are used as the morphological surface of the bubble, which is the three-dimensional morphological information of the bubble. In this embodiment, the method for determining the bubble morphology is not specifically limited.

[0219] The beneficial effect of such a setting is that by obtaining the isosurface of specific three-dimensional grayscale information, the isosurface is matched with the contour information of the bubble to obtain the bubble morphology surface, thereby making the three-dimensional morphology information of the bubble more accurate.

[0220] The image data processing method, device, equipment and storage medium for bubbles provided in the embodiments of the present application obtain the images to be processed of the same bubble obtained by multiple cameras and the mapping functions corresponding to each camera to obtain the contour information corresponding to the bubble, and determine the three-dimensional morphological information of the bubble according to the contour information and the mapping function. By using multiple cameras, the interference to the bubble measurement is reduced while ensuring accuracy, the contour of the bubble can be effectively identified, and the three-dimensional morphology of the bubble can be accurately reconstructed, thereby improving the accuracy of the three-dimensional measurement.

[0221] Figure 5 A schematic flow chart of an image data processing method applied to bubbles provided in an embodiment of the present disclosure.

[0222] In this embodiment, it also includes: determining the target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information; determining the spatial grayscale isosurface according to the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness; determining the first diameter of the bubble according to the spatial grayscale isosurface; wherein the first diameter represents the diameter size of the bubble in the three-dimensional morphological information; determining the second diameter of the bubble according to the mapping function and the first diameter of the bubble; wherein the second diameter is the actual diameter size of the bubble.

[0223] like Figure 5 As shown, the method comprises the following steps:

[0224] S501, obtaining an image to be processed from a camera, and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object.

[0225] Exemplarily, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0226] S502 , performing contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed.

[0227] Exemplarily, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0228] S503 , determining three-dimensional morphological information of the bubble according to contour information of the bubble in the to-be-processed image of each camera and a mapping function corresponding to each camera; wherein the three-dimensional morphological information represents morphological features of the bubble.

[0229] Exemplarily, this step may refer to the above-mentioned step S103 and will not be described in detail.

[0230] S504. Determine target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information.

[0231] For example, according to the three-dimensional morphological information of the bubbles, the grayscale values ​​in the three-dimensional morphological information of the bubbles are sorted from low to high, and the grayscale value with the lowest grayscale brightness is used as the target grayscale brightness. The target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information.

[0232] The beneficial effect of such a setting is that the minimum grayscale value is determined as the target grayscale brightness from the three-dimensional morphological information of the bubble, which facilitates the subsequent use of the target grayscale brightness.

[0233] S505. Determine a spatial grayscale isosurface according to the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness.

[0234] Exemplarily, after obtaining the target grayscale brightness, in the three-dimensional morphological information, the isosurface with the same grayscale value as the target grayscale brightness is determined as the spatial grayscale isosurface. The size of the spatial grayscale isosurface is determined by the volume elements corresponding to the pixel points on the spatial grayscale isosurface. The grayscale values ​​of the volume elements corresponding to the pixel points in the spatial grayscale isosurface are all the target grayscale brightness.

[0235] The beneficial effect of this setting is that the isosurface of the target grayscale brightness is determined to be a spatial grayscale isosurface, which is closer to the thickness direction of the bubble. The diameter of the bubble obtained on this isosurface is more consistent with the diameter of the bubble corresponding to the current three-dimensional morphological information of the bubble.

[0236] S506. Determine a first diameter of the bubble according to the spatial grayscale isosurface; wherein the first diameter represents the diameter of the bubble in the three-dimensional morphological information.

[0237] Exemplarily, based on the spatial grayscale isosurface, the length in the direction of the adjacent surface of the spatial grayscale isosurface and the side where the camera is located is determined as the first diameter of the bubble. The first diameter characterizes the diameter of the bubble in the three-dimensional morphological information of the bubble, that is, the thickness information of the bubble compared to the camera direction.

[0238] The beneficial effect of such a setting is that the first diameter obtained is the diameter size in the three-dimensional shape of the current bubble, which is more consistent with the length of the bubble in the thickness direction, facilitating the subsequent adjustment of the three-dimensional shape of the current bubble.

[0239] S507 . Determine a second diameter of the bubble according to the mapping function and the first diameter of the bubble; wherein the second diameter is an actual diameter of the bubble.

[0240] Exemplarily, the deflection angle corresponding to the camera is determined according to the mapping function corresponding to the camera, and the camera may be the cameras at the two sides, for example, the camera numbered 1 and the camera numbered 4. For the selected camera, the camera deflection angle calculation formula is as follows:

[0241]

[0242] α=arctan(tan(α))

[0243] Among them, F(X) is the mapping function of the horizontal direction corresponding to the camera, F(Y) is the mapping function of the vertical direction corresponding to the camera, and α represents the deflection angle of the camera. Calculate the partial derivatives of F(X) with respect to x, y and z respectively, and calculate the partial derivatives of F(Y) with respect to x, y and z respectively. Subtract the product of the partial derivative of F(Y) with respect to y and the partial derivative of F(X) with respect to z from the product of the partial derivative of F(Y) with respect to z and the partial derivative of F(X) with respect to y to obtain the first difference. Subtract the product of the partial derivative of F(Y) with respect to x and the partial derivative of F(X) with respect to y from the product of the partial derivative of F(Y) with respect to y and the partial derivative of F(X) with respect to x to obtain the second difference. Compare the first difference with the second difference to obtain the tangent value of the camera deflection angle. According to the arctan function and the tangent value of the camera deflection angle, obtain α, that is, the deflection angle corresponding to the camera.

[0244] β=|α1|+|α4|

[0245] Among them, |α1| represents the absolute value of the deflection angle of camera No. 1, |α4| represents the absolute value of the deflection angle of camera No. 4, β represents the angle between cameras, and the absolute values ​​of the deflection angles corresponding to camera No. 1 and camera No. 4 are added to obtain the angle between the cameras.

[0246] The second diameter is determined based on the angle between the cameras and the first diameter. The formula for determining the second diameter is as follows:

[0247]

[0248] Among them, d′ z Characterize the first diameter, d z To characterize the second diameter, the first diameter is multiplied by the sine value of the angle between the cameras of half, and the product is the second diameter.

[0249] Figure 6 Schematic diagram of the first diameter and the second diameter of the bubble provided in the embodiment of the present disclosure. Figure 6 As shown in the figure, the circle represents the actual shape of the bubble, the dotted line represents the direction of the camera's line of sight, and the solid line represents the camera's field of view. z Characterize the first diameter, d z Characterizes the second diameter, β represents the angle between the cameras. It can be seen from the figure that compared with the second diameter of the bubble, the first diameter is severely elongated due to the viewing angle between the cameras.

[0250] The beneficial effect of this setting is that, due to the camera viewing angle and position, the reconstructed three-dimensional information of the bubble is elongated in the thickness direction, and the error in the thickness direction is large. By calculating the angle between the cameras based on the mapping function and obtaining the actual bubble diameter, that is, the second diameter, based on the angle, the error of the three-dimensional information of the bubble can be effectively reduced.

[0251] In this embodiment, the method further includes: adjusting the spatial grayscale isosurface according to the second diameter to obtain a target isosurface; and adjusting the three-dimensional morphology information according to the target isosurface to obtain adjusted three-dimensional morphology information.

[0252] Specifically, the spatial grayscale isosurface is adjusted according to the second diameter to obtain a target isosurface. For example, the second diameter may be used to replace the first diameter to obtain a new isosurface, which is the target isosurface. According to the target isosurface, the three-dimensional morphological information of the bubble is adjusted. For example, the three-dimensional morphology of the bubble may be scaled according to the direction of the diameter of the target isosurface to obtain the adjusted three-dimensional morphological information.

[0253] Figure 7 Schematic diagram of the three-dimensional morphological information of bubbles provided in the embodiment of the present disclosure. Figure 7 As shown, the coordinate system in the figure represents the physical space where the bubbles are located, the x-axis represents the horizontal direction of the physical space where the bubbles are located, the y-axis represents the vertical direction of the physical space where the bubbles are located, and the z-axis represents the thickness direction of the physical space where the bubbles are located. There is three-dimensional morphological information of two bubbles in the physical space in the figure. It can be seen from the figure that the three-dimensional morphological information of the bubbles is distorted in the thickness direction.

[0254] Figure 7 This is a schematic diagram of the adjusted three-dimensional morphological information of the bubble provided in the embodiment of the present disclosure. Figure 8 As shown, the coordinate system in the figure represents the physical space where the bubble is located. The x-axis represents the horizontal direction of the physical space where the bubble is located, the y-axis represents the vertical direction of the physical space where the bubble is located, and the z-axis represents the thickness direction of the physical space where the bubble is located. In the physical space in the figure, there is Figure 7 The adjusted three-dimensional morphological information of the two bubbles. It can be seen from the figure that after adjusting the three-dimensional morphological information of the bubbles, the distortion of the three-dimensional morphological information of the bubbles in the thickness direction is eliminated.

[0255] The beneficial effect of such an arrangement is that the distortion of the bubble produced during the measurement process is adjusted and corrected, thereby reducing the error of the three-dimensional shape of the bubble and making the three-dimensional shape of the bubble more accurate.

[0256] The image data processing method, device, equipment and storage medium for bubbles provided in the embodiments of the present application obtain the images to be processed of the same bubble obtained by multiple cameras and the mapping functions corresponding to each camera to obtain the contour information corresponding to the bubble, and determine the three-dimensional morphological information of the bubble according to the contour information and the mapping function. By using multiple cameras, the interference to the bubble measurement is reduced while ensuring accuracy, the contour of the bubble can be effectively identified, and the three-dimensional morphology of the bubble can be accurately reconstructed, thereby improving the accuracy of the three-dimensional measurement.

[0257] Fig. 9A structural block diagram of an image data processing device applied to bubbles provided in an embodiment of the present disclosure.

[0258] For the sake of convenience, only the parts related to the embodiments of the present disclosure are shown. Fig. 9 , the image data processing device 900 applied to bubbles includes: an acquisition unit 901, a processing unit 902 and a determination unit 903.

[0259] The acquisition unit 901 is used to acquire an image to be processed from a camera and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object;

[0260] The processing unit 902 is used to perform contour acquisition processing on the image to be processed of the camera to obtain contour information of the bubble in the image to be processed of the camera;

[0261] The determination unit 903 is used to determine the three-dimensional morphological information of the bubble according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0262] Figure 7 A structural block diagram of an image data processing device applied to bubbles provided in an embodiment of the present disclosure.

[0263] exist Fig. 9 Based on the embodiment shown, Fig.10 As shown, the processing unit 902 includes a noise reduction module 9021 and an extraction module 9022 .

[0264] The noise reduction module 9021 is used to perform noise reduction processing on the image to be processed of the camera to obtain the image to be processed after noise reduction;

[0265] The extraction module 9022 is used to perform contour extraction processing on the image to be processed after noise reduction to obtain contour information of bubbles in the image to be processed of the camera.

[0266] In one example, the noise reduction module 9021 includes:

[0267] A binarization submodule, used for binarizing the image to be processed of the camera according to a preset grayscale threshold value to obtain a binary image corresponding to the image to be processed;

[0268] An opening operation submodule, used for performing an opening operation on the binary image to obtain a binary image after the opening operation; wherein the opening operation includes corrosion processing and dilation processing;

[0269] The filtering submodule is used to perform area filtering on the binary image after the opening operation to obtain a denoised image to be processed.

[0270] In one example, the filter submodule is specifically used for:

[0271] According to a preset area filtering algorithm, the tracer particles in the binary image after the opening operation are filtered to obtain the image to be processed after noise reduction.

[0272] In one example, the extraction module 9022 includes:

[0273] A boundary recognition submodule is used to perform boundary recognition processing on the image to be processed after denoising to obtain boundary information of bubbles in the image to be processed after denoising; wherein the boundary information is initial contour information;

[0274] The contour acquisition submodule is used to perform expansion processing, filling processing, and corrosion processing on the bubbles in the image to be processed after noise reduction according to the boundary information, so as to obtain the contour information of the bubbles in the image to be processed of the camera.

[0275] In one example, the acquisition unit 901 includes:

[0276] A first position determination module, used to determine the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point;

[0277] The function determination module is used to determine the mapping function corresponding to the camera according to the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0278] In one example, the first position determination module includes:

[0279] The first position acquisition submodule is used to acquire the physical space position of the bubble;

[0280] A position determination submodule, used to determine a reference position and an end position according to the physical space position of the bubble; wherein the reference position represents the position farthest from the camera in the physical space position, and the end position represents the position closest to the camera in the physical space position;

[0281] A sub-slicing module, used for dividing the bubble into equal distances according to the distance between the reference position and the end position, to obtain a dividing point on the bubble;

[0282] The second position acquisition submodule is used to acquire the physical space position of the segmentation point and the coordinate position of the pixel point of the segmentation point in the image to be processed.

[0283] In one example, the function determines the module, including:

[0284] A candidate submodule, used to determine a candidate function corresponding to the camera according to the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point;

[0285] The correction submodule is used to correct the candidate function to obtain the mapping function corresponding to the camera.

[0286] In one example, the modified submodule is used to:

[0287] Determine the coordinate position of the tracer particle in the image to be processed as the initial position;

[0288] Determining an initial mapping position of the tracer particle according to the candidate function corresponding to the camera and the initial position;

[0289] Performing fitting processing on the initial mapping position of the tracer particles corresponding to each camera to obtain the target mapping position of the tracer particles;

[0290] Determining a target position of the tracer particle in the image to be processed according to the target mapping position and the candidate function of the camera;

[0291] According to the initial position and the target position, based on the candidate function, a mapping function corresponding to the camera is determined.

[0292] In one example, the correction submodule is also specifically used for:

[0293] Determine a position deviation vector of the tracer particle according to the initial position and the target position; wherein the position deviation vector represents the deviation between the initial position and the target position;

[0294] Determine the region where the tracer particles are located, and determine the position deviation vector of the region based on the position deviation vectors of all tracer particles in the region;

[0295] A mapping function of the camera is obtained according to the position deviation vector of each area and the candidate function of the camera.

[0296] In one example, the determining unit 903 includes:

[0297] A second position determination module, used to determine the physical spatial position of the pixel point in the image to be processed;

[0298] A grayscale module, used to determine the three-dimensional grayscale information of the pixel point according to the physical spatial position of the pixel point, based on the contour information of the bubble and the mapping function; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space, the pixel point and the volume element have a one-to-one correspondence, and the volume element is the smallest spatial unit of the space where the bubble is located;

[0299] The morphology module is used to determine the three-dimensional morphology information of the bubble according to the three-dimensional grayscale information of the pixel point.

[0300] In one example, the grayscale module includes:

[0301] An intermediate quantum module, used to obtain an intermediate quantity of the pixel point according to the physical spatial position of the pixel point, the coordinate position of the pixel point, a mapping function, a preset grayscale value, and a preset point propagation function; wherein the intermediate quantity represents the two-dimensional grayscale information of the pixel point, and the preset point propagation function represents the influence of the pixel point on the grayscale value of the adjacent pixel point;

[0302] A first grayscale submodule, used to determine the image grayscale value of the pixel point according to the contour information of the bubble;

[0303] A target quantum module, used to obtain a target quantity according to the intermediate quantity of the pixel point, the image grayscale value of the pixel point, and the preset grayscale value; wherein the target quantity represents the three-dimensional grayscale information of the pixel point;

[0304] The second grayscale submodule is used to determine that the target amount is the three-dimensional grayscale information of the pixel if the intermediate amount of the pixel is equal to the image grayscale value of the pixel.

[0305] In one example, the device further includes:

[0306] A grayscale unit, used to determine a target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information;

[0307] An isosurface unit is used to determine a spatial grayscale isosurface according to the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness;

[0308] A first diameter unit, used to determine a first diameter of the bubble according to the spatial grayscale isosurface; wherein the first diameter represents the diameter of the bubble in the three-dimensional morphological information;

[0309] The second diameter unit is used to determine the second diameter of the bubble according to the mapping function and the first diameter of the bubble; wherein the second diameter is the actual diameter of the bubble.

[0310] In one example, it also includes:

[0311] A first adjustment unit adjusts the spatial grayscale isosurface according to the second diameter to obtain a target isosurface;

[0312] The second adjustment unit is used to adjust the three-dimensional morphological information according to the target isosurface to obtain adjusted three-dimensional morphological information.

[0313] Fig.11 A structural block diagram of an electronic device provided in an embodiment of the present disclosure, the electronic device may be a terminal device or a server, such as Fig.11 As shown, the electronic device 1100 includes: at least one processor 1102; and a memory 1101 communicatively connected to the at least one processor 1102; wherein the memory stores instructions executable by the at least one processor 1102, and the instructions are executed by the at least one processor 1102 so that the at least one processor 1102 can execute the image data processing method applied to bubbles disclosed in the present invention.

[0314] The electronic device 1100 further includes a receiver 1103 and a transmitter 1104. The receiver 1103 is used to receive instructions and data sent by other devices, and the transmitter 1104 is used to send instructions and data to external devices.

[0315] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0316] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0317] Fig.12 This is a structural block diagram of an electronic device provided in one embodiment of the present application. The electronic device is equipped with at least three cameras, a high-pass filter, a water prism, a laser, a concave lens, a plano-convex cylindrical lens, an air outlet, a flow meter, a bubble generating device, an LED light source, a soft light plate, and a water tank.

[0318] like Fig. 9As shown, the water tank 7 is filled with water containing tracer particles. In this embodiment, four cameras are configured, and cameras 1, 2, 3, and 4 are used to photograph the bubbles in the water tank 7. The water prism 6 is located on the side of the water tank 7 to reduce the distortion caused by the refraction of light inside and outside the water tank 7. The lens of the camera faces the side where the water prism 6 is located. For each camera, there is a corresponding high-pass filter 5, which is located in front of the lens of the camera to filter out the strong reflection around the bubbles.

[0319] The LED light source 8 and the soft light plate 9 are located opposite to the side where the water prism 6 is located, the LED light source 8 and the soft light plate 9 are outside the water tank, the soft light plate 9 is close to the water tank 7, and the LED light source 8 is located behind the soft light plate 9. The LED light source 8 provides uniform background light to the water tank 7 through the soft light plate 9, which is convenient for accurate measurement of the morphology of the bubbles.

[0320] The laser 10 is located on the adjacent surface of the side where the water prism 6 is located, and the laser 10 is located behind the concave lens 11, and the concave lens 11 is located behind the plano-convex cylindrical lens 12, and the plano-convex cylindrical lens 12 is close to the adjacent surface. The laser emitted by the laser 10 is aggregated through the concave lens 11 and the plano-convex cylindrical lens 12 to form a volume light source, which evenly illuminates the measurement area of ​​the entire water tank 7. The laser irradiates the tracer particles in the water tank 7, which can cause the tracer particles to generate fluorescence.

[0321] The bubble generating device 13 is located outside the water tank, and is used to provide bubbles to the water tank 7; one end of the flow meter 15 is connected to the bubble generating device 13, and the other end of the flow meter 15 is connected to the air outlet port 14, and is used to measure the gas flow in the bubble generating device 13. The air outlet port 14 is located at the bottom of the water tank 7, and is used to discharge the bubbles into the water tank 7.

[0322] In one example, the cameras may be arranged in a cross shape, with a camera being arranged on each of the four sides of the water tank 7 .

[0323] Fig.13 It is a block diagram of an electronic device according to an exemplary embodiment, which device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0324] Device 1300 may include one or more of the following components: a processing component 1302 , a memory 1304 , a power component 1306 , a multimedia component 1308 , an audio component 1311 , an input / output (I / O) interface 1312 , a sensor component 1314 , and a communication component 1316 .

[0325] The processing component 1302 generally controls the overall operation of the device 1300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1302 may include one or more processors 1320 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1302 may include one or more modules to facilitate the interaction between the processing component 1302 and other components. For example, the processing component 1302 may include a multimedia module to facilitate the interaction between the multimedia component 1308 and the processing component 1302.

[0326] The memory 1304 is configured to store various types of data to support the operation of the device 700. Examples of such data include instructions for any application or method operating on the device 1300, contact data, phone book data, messages, pictures, videos, etc. The memory 1304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0327] The power supply component 1306 provides power to the various components of the device 1300. The power supply component 1306 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1300.

[0328] The multimedia component 1308 includes a screen that provides an output interface between the device 1300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1308 includes a front camera and / or a rear camera. When the device 1300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0329] The audio component 1310 is configured to output and / or input audio signals. For example, the audio component 1310 includes a microphone (MIC), and when the device 1300 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1304 or sent via the communication component 1316. In some embodiments, the audio component 1310 also includes a speaker for outputting audio signals.

[0330] I / O interface 1312 provides an interface between processing component 1302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0331] The sensor assembly 1314 includes one or more sensors for providing various aspects of status assessment for the device 1300. For example, the sensor assembly 1314 can detect the open / closed state of the device 1300, the relative positioning of components, such as the display and keypad of the device 1300, the sensor assembly 1314 can also detect the position change of the device 1300 or a component of the device 1300, the presence or absence of user contact with the device 1300, the orientation or acceleration / deceleration of the device 1300, and the temperature change of the device 1300. The sensor assembly 1314 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1314 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1314 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0332] The communication component 1316 is configured to facilitate wired or wireless communication between the device 1300 and other devices. The device 1300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0333] In an exemplary embodiment, device 1300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0334] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1304 including instructions, and the instructions can be executed by the processor 1320 of the device 1300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0335] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a terminal device, enables the terminal device to execute the above-mentioned image data processing method applied to bubbles.

[0336] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0337] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0338] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0339] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0340] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0341] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0342] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0343] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0344] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing image data applied to bubbles, characterized in that: The method is applied to an electronic device, wherein at least three cameras are deployed in the electronic device, and the method comprises: Acquire an image to be processed from a camera, and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical spatial position of the object; Performing contour acquisition processing on the image to be processed of the camera to obtain contour information of the bubbles in the image to be processed of the camera; The three-dimensional morphological information of the bubble is determined according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

2. The method according to claim 1, characterized in that Performing contour acquisition processing on the image to be processed of the camera to obtain contour information of bubbles in the image to be processed of the camera, including: Performing noise reduction processing on the image to be processed of the camera to obtain a noise-reduced image to be processed; The image to be processed after noise reduction is subjected to contour extraction processing to obtain contour information of bubbles in the image to be processed of the camera.

3. The method according to claim 2, characterized in that The image to be processed is a grayscale image; performing image noise reduction processing on the image to be processed by the camera to obtain the image to be processed after noise reduction, including: Binarizing the image to be processed of the camera according to a preset grayscale threshold to obtain a binary image corresponding to the image to be processed; Performing an opening operation on the binary image to obtain a binary image after the opening operation; wherein the opening operation includes corrosion processing and dilation processing; The binary image after the opening operation is subjected to area filtering to obtain a denoised image to be processed.

4. The method according to claim 3, characterized in that The image to be processed includes tracer particles; performing area filtering on the binary image after the opening operation to obtain a noise-reduced image to be processed, including: According to a preset area filtering algorithm, the tracer particles in the binary image after the opening operation are filtered to obtain the image to be processed after noise reduction.

5. The method according to claim 2, characterized in that: Performing contour extraction processing on the image to be processed after noise reduction to obtain contour information of bubbles in the image to be processed of the camera, including: Performing boundary recognition processing on the image to be processed after denoising to obtain boundary information of bubbles in the image to be processed after denoising; wherein the boundary information is initial contour information; According to the boundary information, the bubbles in the image to be processed after noise reduction are expanded, filled, and eroded to obtain the contour information of the bubbles in the image to be processed of the camera.

6. The method according to claim 1, characterized in that Determining a mapping function corresponding to the camera includes: Determine the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point; The mapping function corresponding to the camera is determined according to the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point.

7. The method according to claim 6, characterized in that Determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point includes: Get the physical space position of the bubble; Determine a reference position and an end position according to the physical space position of the bubble; wherein the reference position represents the position farthest from the camera in the physical space, and the end position represents the position closest to the camera in the physical space; According to the distance between the reference position and the end position, the bubble is divided into equal distances to obtain the dividing points on the bubble; The physical space position of the segmentation point and the coordinate position of the pixel point of the segmentation point in the image to be processed are obtained.

8. The method according to claim 6, characterized in that Determining a mapping function corresponding to the camera according to the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point includes: Determine a candidate function corresponding to the camera according to a coordinate position of a pixel point in the image to be processed and a physical space position corresponding to the pixel point; The candidate function is corrected to obtain a mapping function corresponding to the camera.

9. The method according to claim 8, characterized in that The physical space corresponding to the image to be processed includes a plurality of tracer particles, and the physical space corresponding to the image to be processed is divided into a plurality of regions, and the number of tracer particles in different regions is the same; the candidate function is corrected to obtain a mapping function corresponding to the camera, including: Determine the coordinate position of the tracer particle in the image to be processed as the initial position; Determining an initial mapping position of the tracer particle according to the candidate function corresponding to the camera and the initial position; Performing fitting processing on the initial mapping position of the tracer particles corresponding to each camera to obtain the target mapping position of the tracer particles; Determining a target position of the tracer particle in the image to be processed according to the target mapping position and the candidate function of the camera; According to the initial position and the target position, based on the candidate function, a mapping function corresponding to the camera is determined.

10. The method according to claim 9, characterized in that Determining a mapping function corresponding to the camera based on the candidate function according to the initial position and the target position includes: Determine a position deviation vector of the tracer particle according to the initial position and the target position; wherein the position deviation vector represents the deviation between the initial position and the target position; Determine the region where the tracer particles are located, and determine the position deviation vector of the region based on the position deviation vectors of all tracer particles in the region; A mapping function of the camera is obtained according to the position deviation vector of each area and the candidate function of the camera.

11. The method according to claim 1, characterized in that: According to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera, the three-dimensional morphological information of the bubble is determined, including: Determining the physical spatial position of the pixel points in the image to be processed; According to the physical spatial position of the pixel point, based on the contour information of the bubble and the mapping function, the three-dimensional grayscale information of the pixel point is determined; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space, the pixel point and the volume element have a one-to-one correspondence, and the volume element is the smallest spatial unit of the space where the bubble is located; The three-dimensional morphological information of the bubble is determined according to the three-dimensional grayscale information of the pixel point.

12. The method according to claim 11, characterized in that Determining the three-dimensional grayscale information of the pixel point according to the physical spatial position of the pixel point, based on the contour information of the bubble and the mapping function, includes: According to the physical spatial position of the pixel point, the coordinate position of the pixel point, the mapping function, the preset grayscale value, and the preset point propagation function, the intermediate quantity of the pixel point is obtained; wherein the intermediate quantity represents the two-dimensional grayscale information of the pixel point, and the preset point propagation function represents the influence of the pixel point on the grayscale value of the adjacent pixel point; Determining the image grayscale value of the pixel point according to the contour information of the bubble; Obtaining a target amount according to the intermediate amount of the pixel point, the image grayscale value of the pixel point, and the preset grayscale value; wherein the target amount represents the three-dimensional grayscale information of the pixel point; If the intermediate quantity of the pixel is equal to the image grayscale value of the pixel, the target quantity is determined to be the three-dimensional grayscale information of the pixel.

13. The method according to claim 1, characterized in that Also includes: Determine a target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information; Determine a spatial grayscale isosurface according to the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness; Determine a first diameter of the bubble according to the spatial grayscale isosurface; wherein the first diameter represents the diameter of the bubble in the three-dimensional morphological information; The second diameter of the bubble is determined according to the mapping function and the first diameter of the bubble; wherein the second diameter is the actual diameter of the bubble.

14. The method according to claim 13, characterized in that Also includes: According to the second diameter, adjusting the spatial grayscale isosurface to obtain a target isosurface; The three-dimensional morphological information is adjusted according to the target isosurface to obtain adjusted three-dimensional morphological information.

15. An image data processing device for bubbles, characterized in that: include: An acquisition unit, used for acquiring an image to be processed from a camera, and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object; A processing unit, used for performing contour acquisition processing on the image to be processed of the camera to obtain contour information of the bubbles in the image to be processed of the camera; The determination unit is used to determine the three-dimensional morphological information of the bubble according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

16. An electronic device, characterized in that: The electronic device is equipped with at least three cameras, a high-pass filter, a water prism, a laser, a concave lens, a plano-convex cylindrical lens, an air outlet, a flow meter, a bubble generating device, an LED light source, a soft light plate, and a water tank; The high-pass filter is located in front of the lens of the camera, each camera corresponds to a high-pass filter, the water prism is located on the side of the water tank, the lens of the camera faces the side where the water prism is located, the LED light source and the soft light plate are located opposite to the side where the water prism is located, the LED light source and the soft light plate are outside the water tank, the soft light plate is close to the water tank, and the LED light source is behind the soft light plate; The laser is located on the adjacent surface of the side where the water prism is located, the laser is located behind the concave lens, the concave lens is located behind the plano-convex cylindrical lens, and the plano-convex cylindrical lens is close to the adjacent surface; The bubble generating device is located outside the water tank and is used to provide bubbles to the water tank; one end of the flow meter is connected to the bubble generating device, and the other end of the flow meter is connected to the air outlet port and is used to measure the gas flow of the bubble generating device; the air outlet port is located at the bottom of the water tank and is used to discharge bubbles into the water tank; The electronic device is used to execute the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 14 when executed by a processor.

18. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 14 when being executed by a processor.

Images