Image data processing method, device, equipment and storage medium for bubbles

Abstract

The embodiments of the present application provide a method, apparatus, device, and storage medium for image data processing of bubbles. The method includes: acquiring an image to be processed from a camera and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical spatial position of the object; performing contour acquisition processing on the image to be processed from the camera to obtain contour information of the bubbles in the image to be processed from the camera; and determining three-dimensional morphological information of the bubbles based on the contour information of the bubbles in the image to be processed from each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubbles. This method is used to accurately obtain the three-dimensional morphology of the bubbles.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, apparatus, device and storage medium for processing image data of bubbles. Background Art

[0002] Bubble flow is a unique multiphase flow phenomenon characterized by the distribution of gas in the form of small bubbles within a liquid. In this flow, the liquid phase is continuous, while the gas phase disperses in the form of small bubbles. Bubble flow is widely used in fields such as environmental energy, chemical engineering, and marine engineering. Bubble shape is a key gas-phase characteristic in bubbly flow and a key factor influencing bubble motion. In-depth research on bubbles can explain the mechanisms of gas-liquid mixed flows and help solve related problems in numerous engineering fields.

[0003] Currently, bubble morphology measurements rely heavily on probe measurements or mathematical calculations, resulting in large errors in the restoration of bubble shape and contours. Complex bubble shapes can be difficult to restore to their true three-dimensional form. Therefore, accurately capturing the three-dimensional shape of bubbles has become a pressing issue. Summary of the Invention

[0004] The embodiments of the present application provide a method, apparatus, device, and storage medium for processing image data of bubbles, so as to achieve the effect of accurately obtaining the three-dimensional shape of the bubbles.

[0005] In a first aspect, an embodiment of the present application provides a method for processing image data applied to bubbles, including: the method is applied to an electronic device, wherein at least three cameras are deployed in the electronic device, and the method includes:

[0006] Obtaining an image to be processed from a camera and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical spatial position of the object;

[0007] Performing contour acquisition processing on the camera's image to be processed to obtain contour information of the bubbles in the camera's image to be processed;

[0008] The three-dimensional morphological information of the bubble is determined based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0009] In a possible implementation, performing contour acquisition processing on the camera image to be processed to obtain contour information of bubbles in the camera image to be processed includes:

[0010] Performing noise reduction on the image to be processed from the camera to obtain a noise-reduced image to be processed;

[0011] The image to be processed after noise reduction is subjected to contour extraction processing to obtain contour information of bubbles in the image to be processed of the camera.

[0012] In a possible implementation, the image to be processed is a grayscale image; performing image noise reduction processing on the image to be processed of the camera to obtain a noise-reduced image to be processed includes:

[0013] Binarize the image to be processed by the camera according to a preset grayscale threshold to obtain a binary image corresponding to the image to be processed;

[0014] Performing an opening operation on the binary image to obtain a binary image after opening operation; wherein the opening operation includes corrosion processing and dilation processing;

[0015] The binary image processed by the opening operation is subjected to area filtering to obtain a denoised image to be processed.

[0016] In a possible implementation, the image to be processed includes tracer particles; performing area filtering on the binary image after the opening operation to obtain a noise-reduced image to be processed includes:

[0017] According to a preset area filtering algorithm, the tracer particles in the binary image processed by the opening operation are filtered to obtain a noise-reduced image to be processed.

[0018] In a possible implementation, performing contour extraction on the image to be processed after noise reduction to obtain contour information of bubbles in the camera's image to be processed includes:

[0019] Performing boundary recognition processing on the image to be processed after noise reduction to obtain boundary information of bubbles in the image to be processed after noise reduction; wherein the boundary information is initial contour information;

[0020] According to the boundary information, the bubbles in the image to be processed after noise reduction are expanded, filled, and eroded to obtain the contour information of the bubbles in the image to be processed of the camera.

[0021] In one possible implementation, determining a mapping function corresponding to a camera includes:

[0022] Determine the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point;

[0023] The mapping function corresponding to the camera is determined based on the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0024] In a possible implementation, determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point includes:

[0025] Get the physical space position of the bubble;

[0026] Determine a reference position and an end position according to the physical space position of the bubble; wherein the reference position represents the position farthest from the camera in the physical space, and the end position represents the position closest to the camera in the physical space;

[0027] According to the distance between the reference position and the end position, the bubble is divided into equal distances to obtain the dividing points on the bubble;

[0028] The physical space position of the segmentation point and the coordinate position of the pixel point of the segmentation point in the image to be processed are obtained.

[0029] In a possible implementation, determining a mapping function corresponding to the camera based on the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point includes:

[0030] Determine the candidate function corresponding to the camera based on the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point;

[0031] The candidate function is corrected to obtain the mapping function corresponding to the camera.

[0032] In one possible implementation, a physical space corresponding to the image to be processed includes a plurality of tracer particles, and the physical space corresponding to the image to be processed is divided into a plurality of regions, and the number of tracer particles in different regions is the same; performing correction processing on the candidate function to obtain a mapping function corresponding to the camera includes:

[0033] Determine the coordinate position of the tracer particle in the image to be processed, which is the initial position;

[0034] Determining an initial mapping position of the tracer particle according to the candidate function corresponding to the camera and the initial position;

[0035] Perform fitting processing on the initial mapping position of the tracer particles corresponding to each camera to obtain the target mapping position of the tracer particles;

[0036] determining a target position of the tracer particle in the image to be processed according to the target mapping position and the candidate function of the camera;

[0037] According to the initial position and the target position, a mapping function corresponding to the camera is determined based on the candidate function.

[0038] In a possible implementation, determining a mapping function corresponding to the camera based on a candidate function according to the initial position and the target position includes:

[0039] Determining a position deviation vector of the tracer particle according to the initial position and the target position; wherein the position deviation vector represents the deviation between the initial position and the target position;

[0040] Determine the region where the tracer particles are located, and determine the position deviation vector of the region based on the position deviation vectors of all tracer particles in the region;

[0041] A mapping function of the camera is obtained according to the position deviation vector of each area and the candidate function of the camera.

[0042] In a possible implementation, determining the three-dimensional morphological information of the bubble based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera includes:

[0043] Determine the physical spatial location of pixels in the image to be processed;

[0044] Determining the three-dimensional grayscale information of the pixel point based on the physical spatial position of the pixel point, the contour information of the bubble, and the mapping function; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space, and the pixel point and the volume element have a one-to-one correspondence, and the volume element is the smallest spatial unit of the space where the bubble is located;

[0045] The three-dimensional morphological information of the bubble is determined based on the three-dimensional grayscale information of the pixel points.

[0046] In a possible implementation, determining the three-dimensional grayscale information of the pixel point according to the physical spatial position of the pixel point, based on the bubble contour information and the mapping function includes:

[0047] Obtaining an intermediate quantity of the pixel based on the physical spatial position of the pixel, the coordinate position of the pixel, a mapping function, a preset grayscale value, and a preset point propagation function; wherein the intermediate quantity represents the two-dimensional grayscale information of the pixel, and the preset point propagation function represents the influence of the pixel on the grayscale values ​​of adjacent pixels;

[0048] Determine the image grayscale value of the pixel point based on the contour information of the bubble;

[0049] Obtaining a target quantity based on the intermediate quantity of the pixel, the image grayscale value of the pixel, and a preset grayscale value; wherein the target quantity represents the three-dimensional grayscale information of the pixel;

[0050] If the intermediate quantity of the pixel is equal to the image grayscale value of the pixel, the target quantity is determined to be the three-dimensional grayscale information of the pixel.

[0051] In a possible implementation, the method further includes:

[0052] Determine the target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information;

[0053] Determine a spatial grayscale isosurface based on the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness;

[0054] Determine a first diameter of the bubble according to the spatial grayscale isosurface, wherein the first diameter represents the diameter of the bubble in the three-dimensional morphological information;

[0055] The second diameter of the bubble is determined according to the mapping function and the first diameter of the bubble; wherein the second diameter is the actual diameter of the bubble.

[0056] In a possible implementation, the method further includes:

[0057] According to the second diameter, the spatial grayscale isosurface is adjusted to obtain the target isosurface;

[0058] The three-dimensional morphological information is adjusted according to the target isosurface to obtain adjusted three-dimensional morphological information.

[0059] In a second aspect, an embodiment of the present application provides an image data processing device for bubbles, comprising:

[0060] an acquisition unit, configured to acquire an image to be processed from a camera and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents a relationship between a coordinate position of an object in the image and a physical spatial position of the object;

[0061] A processing unit, configured to perform contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed;

[0062] The determination unit is used to determine the three-dimensional morphological information of the bubble according to the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0063] In a third aspect, an embodiment of the present application provides an electronic device, comprising: an electronic device configured with at least three cameras, a high-pass filter, a water prism, a laser, a concave lens, a plano-convex cylindrical lens, an air outlet, a flow meter, a bubble generating device, an LED light source, a soft light panel, and a water tank;

[0064] A high-pass filter is located in front of the lens of the camera, and each camera corresponds to a high-pass filter. The water prism is located on the side of the water tank, and the camera lens faces the side where the water prism is located. The LED light source and the soft light plate are located opposite the side where the water prism is located. The LED light source and the soft light plate are outside the water tank, the soft light plate is close to the water tank, and the LED light source is behind the soft light plate.

[0065] The laser is located on the adjacent surface of the side where the water prism is located, the laser is located behind the concave lens, the concave lens is located behind the plano-convex cylindrical lens, and the plano-convex cylindrical lens is close to the adjacent surface;

[0066] The bubble generating device is located outside the water tank and is used to provide bubbles to the water tank; one end of the flow meter is connected to the bubble generating device, and the other end of the flow meter is connected to the air outlet port, which is used to measure the gas flow of the bubble generating device; the air outlet port is located at the bottom of the water tank and is used to discharge bubbles into the water tank.

[0067] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0068] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0069] The image data processing method, apparatus, device, and storage medium for bubbles provided in the embodiments of the present application obtain contour information corresponding to the bubble by acquiring images of the same bubble to be processed from multiple cameras and the mapping function corresponding to each camera. The three-dimensional morphological information of the bubble is then determined based on the contour information and the mapping function. By using multiple cameras, while ensuring accuracy and reducing interference with bubble measurement, the bubble's contour can be effectively identified and its three-dimensional morphology accurately reconstructed, thereby improving the accuracy of three-dimensional measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0071] Figure 1 A schematic flow chart of a method for processing image data of bubbles provided in an embodiment of the present disclosure;

[0072] Figure 2 A schematic flow chart of a method for processing image data of bubbles provided in an embodiment of the present disclosure;

[0073] Figure 3 A schematic flow chart of a method for processing image data of bubbles provided in an embodiment of the present disclosure;

[0074] Figure 4 A schematic flow chart of a method for processing image data of bubbles provided in an embodiment of the present disclosure;

[0075] Figure 5 A schematic flow chart of a method for processing image data of bubbles provided in an embodiment of the present disclosure;

[0076] Figure 6 A schematic diagram of a first diameter and a second diameter of a bubble provided in an embodiment of the present disclosure;

[0077] Figure 7 A schematic diagram of three-dimensional morphological information of bubbles provided in an embodiment of the present disclosure;

[0078] Figure 8 A schematic diagram of adjusted three-dimensional morphological information of a bubble provided in an embodiment of the present disclosure;

[0079] Figure 9 A structural block diagram of an image data processing device for bubbles provided in an embodiment of the present disclosure;

[0080] Figure 10 A structural block diagram of an image data processing device for bubbles provided in an embodiment of the present disclosure;

[0081] Figure 11 A structural block diagram of an electronic device provided in an embodiment of the present disclosure;

[0082] Figure 12 A structural block diagram of an electronic device provided in one embodiment of the present application;

[0083] Figure 13 It is a block diagram of an electronic device according to an exemplary embodiment.

[0084] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0085] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0086] First, let’s explain the terms involved in this application:

[0087] MART algorithm: Multiplicative Algebraic Reconstruction Techniques (MART) is an algebraic method for image reconstruction, particularly commonly used in fields such as tomography (such as CT). MART iteratively adjusts the image pixel values ​​to minimize the error between the reconstructed image and the observed data, thereby restoring the structure of the original image. Core principle: MART is based on the algebraic reconstruction method and describes the relationship between projection data and image pixels by establishing a system of linear equations. Its main feature is the use of a multiplication update rule, that is, in each iteration, the pixel value is multiplied according to the current error instead of added. This method has better convergence and stability when processing certain types of data.

[0088] Laser-induced fluorescence (LIF) is a method for detecting fluorescent emission after laser irradiation of a sample. High-energy laser light excites electrons in a substance to an excited state. When the electrons return, they release energy in the form of photons, generating fluorescence of a specific wavelength.

[0089] PMMA Rhodamine B: Rhodamine B is a fluorescent dye with a peak emission around 580nm, exhibiting high fluorescence intensity and stability. PMMA (acrylic material) is a synthetic material created by doping Rhodamine B with PMMA.

[0090] Canny operator: The Canny operator is an edge detection algorithm that effectively suppresses noise while preserving edge information. It can detect fine edges and is highly robust, capable of processing images with various noise types.

[0091] Gaussian filter: Gaussian filtering is a linear smoothing filter that is suitable for eliminating Gaussian noise and is widely used in the noise reduction process of image processing.

[0092] Sobel operator: The Sobel operator is an important processing method in computer vision. It is primarily used to obtain the first-order gradient of digital images. Its common application and physical significance is edge detection. The Sobel operator takes the weighted difference of the grayscale values ​​of each pixel in the image, and finds the maximum value at the edge.

[0093] Roberts operator: The Roberts operator is an operator that uses a local difference operator to find edges. It uses the difference between two adjacent pixels in the diagonal direction to approximate the gradient amplitude to detect edges.

[0094] Prewitt operator: The Prewitt operator is a first-order differential operator for edge detection. It uses the grayscale difference between the upper and lower, left and right neighbors of a pixel to detect the edge when it reaches the extreme value at the edge, removes some false edges, and has a smoothing effect on noise.

[0095] Bubble flow is a special multiphase flow state formed by gas in the form of small bubbles distributed in a liquid. In bubbly flow, the liquid phase is continuous, while the gas phase is dispersed in the form of small bubbles. Bubble flow is widely used in fields such as environmental energy, chemical engineering, and marine engineering. Among them, bubble shape is a key gas phase characteristic in bubbly flow and one of the important factors affecting bubble motion.

[0096] Current methods for acquiring the three-dimensional morphology of bubbles all have shortcomings in terms of universality. For example, contact measurement methods, such as using fiber optic probes to measure bubbles, interfere with the flow field and cause bubble deformation due to tension, resulting in large errors. Differential reconstruction of bubbles results in discontinuous contours, making it difficult to accurately identify bubble morphology. Bubble reconstruction using neural networks requires a large amount of bubble data collection, which is prohibitively expensive when applied to a variety of special cases. Therefore, accurately and quickly acquiring the three-dimensional morphology of bubbles is a key challenge.

[0097] The specific application scenario of this application is the reconstruction of the three-dimensional morphology of bubbles, which is used to capture images of bubbles, reconstruct the three-dimensional morphology of bubbles based on information such as bubble images, and obtain the three-dimensional morphological information of bubbles.

[0098] The image data processing method for bubbles provided in this application is intended to solve the above technical problems in the prior art.

[0099] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0100] Figure 1A flow chart of an image data processing method for bubbles provided in an embodiment of the present disclosure is provided. The method can be executed by an image data processing device for bubbles, which is equipped with at least three cameras.

[0101] like Figure 1 As shown, the method includes the following steps:

[0102] S101. Acquire an image to be processed from a camera and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object.

[0103] Exemplarily, multiple cameras are provided to capture images of the same bubble, and the cameras may be high-speed cameras. For any camera, the image to be processed corresponding to the camera is obtained, and the mapping function corresponding to the camera is determined based on the image to be processed of the camera. For example, the pixel points in the image to be processed can be matched with the part of the bubble represented by the pixel points in the physical space to obtain the mapping relationship between the pixels in the image to be processed and the physical space, which is the mapping function. The image to be processed is the image of the bubble captured by the camera, and the image to be processed contains grayscale information. The mapping function represents the relationship between the coordinate position of an object in the image to be processed corresponding to the camera and the physical space position of the object. The object in the image may be a pixel point in the image.

[0104] S102 : performing contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed.

[0105] For example, after obtaining the image to be processed corresponding to each camera, contour acquisition processing is performed on the image to be processed for each camera based on the image to be processed for that camera, thereby obtaining contour information of the bubbles in the image to be processed for that camera. For example, image noise reduction processing can be performed on the image to be processed to remove noise from non-bubble images in the image to be processed, and then contour extraction of the bubbles in the denoised image to be processed is performed to extract the contour information of the bubbles in the denoised image to be processed. In this embodiment, the image noise reduction processing method and the bubble contour extraction method are not specifically limited.

[0106] S103 , determining three-dimensional morphological information of the bubble based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0107] Exemplarily, the contour information of the bubble in the image to be processed includes multiple pixels, each pixel corresponding to the grayscale information of the pixel. After obtaining the contour information of the bubble in the image to be processed corresponding to each camera, the three-dimensional morphological information of the bubble is determined based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera. For example, based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera, the grayscale information corresponding to any pixel in the bubble contour information can be used as the first grayscale information of the pixel. For any first grayscale information, the first grayscale information is inversely projected into physical space using the corresponding mapping function to obtain the spatial grayscale information corresponding to each pixel. According to the corresponding mapping function, the spatial grayscale information of each pixel is projected to the corresponding position in the bubble contour information image to obtain the image grayscale information corresponding to the pixel at each corresponding position, which is the second grayscale information. The spatial grayscale information corresponding to each pixel is continuously iteratively optimized using the MART algorithm. When the difference between the first grayscale information and the second grayscale information is less than a preset grayscale difference threshold, the optimization is stopped, and the spatial grayscale information at this time is used as the three-dimensional morphological information of the bubble. The spatial grayscale information represents the grayscale information in the three-dimensional space. In this embodiment, the projection and back-projection methods are not specifically limited.

[0108] The image data processing method, apparatus, device, and storage medium for bubbles provided in the embodiments of the present application obtain contour information corresponding to the bubble by acquiring images of the same bubble to be processed from multiple cameras and the mapping function corresponding to each camera. The three-dimensional morphological information of the bubble is then determined based on the contour information and the mapping function. By using multiple cameras, while ensuring accuracy and reducing interference with bubble measurement, the bubble's contour can be effectively identified and its three-dimensional morphology accurately reconstructed, thereby improving the accuracy of three-dimensional measurement.

[0109] Figure 2 A flowchart of an image data processing method for bubbles provided in an embodiment of the present disclosure.

[0110] In this embodiment, contour acquisition processing is performed on the camera's image to be processed to obtain contour information of the bubbles in the camera's image to be processed, including: performing noise reduction processing on the camera's image to be processed to obtain the noise-reduced image to be processed; performing contour extraction processing on the noise-reduced image to be processed to obtain contour information of the bubbles in the camera's image to be processed.

[0111] like Figure 2 As shown, the method includes the following steps:

[0112] S201 , obtaining an image to be processed from a camera, and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object.

[0113] For example, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0114] S202: performing noise reduction processing on the image to be processed of the camera to obtain a noise-reduced image to be processed.

[0115] Exemplarily, noise reduction processing is performed on the image to be processed to obtain a noise-reduced image to be processed. Noise reduction processing is used to remove noise from the image to be processed, including interference other than bubbles in the image. For example, a preset filter can be used to filter the image to be processed. The preset filter is used to filter out noise from the image to be processed, and can be, for example, a median filter or a Gaussian filter. In this embodiment, the type of preset filter is not specifically limited.

[0116] The beneficial effect of this setting is that it removes noise and interference other than bubbles in the image to be processed, making the bubbles in the image to be processed clearer, facilitating the subsequent extraction of bubble contour information, etc., and better retaining the image detail information.

[0117] In this embodiment, the image to be processed is a grayscale image; image noise reduction processing is performed on the camera's image to be processed to obtain a noise-reduced image to be processed, including: binarizing the camera's image to be processed according to a preset grayscale threshold to obtain a binary image corresponding to the image to be processed; performing an opening operation on the binary image to obtain a binary image after the opening operation; wherein the opening operation includes corrosion processing and dilation processing; and performing area filtering processing on the binary image after the opening operation to obtain a noise-reduced image to be processed.

[0118] Specifically, the image to be processed is a grayscale image, and a region division process is performed on the image to be processed, where the image to be processed is divided into multiple regions. For example, the image to be processed can be divided into 9 3×3 regions at equal intervals. For any region, the sensitivity information of the region is obtained. The sensitivity information represents the percentage of the number of pixels of the bubble outline in the entire image. The sensitivity information can be, for example, 50%. A grayscale value sequence in the region is obtained. The grayscale value sequence represents the order of the grayscale values ​​of the region from low to high. The grayscale value at the sensitivity information position in the grayscale value sequence is used as the grayscale threshold of the region. For example, in the grayscale value sequence, the grayscale value 125 at the 50% position of the grayscale value sequence is used as the grayscale threshold of the region.

[0119] For any region, binarization is performed on the region based on the grayscale threshold of the region and a preset binarization formula. Binarization is used to convert the image to be processed into a black and white image. After binarization is performed on each region of the image to be processed, a binary image corresponding to the image to be processed is obtained. The preset binarization formula can be:

[0120]

[0121] Among them, I(x,y) represents the gray value of the pixel at the position (x,y) of the image to be processed, T represents the gray threshold corresponding to the area where the pixel is located, and I e (x,y) represents the grayscale value of the pixel at position (x,y) in the binary image.

[0122] After obtaining the binary image corresponding to the image to be processed, the binary image is opened, that is, the binary image is first eroded and then dilated. Erosion is used to remove noise such as burrs in the binary image and separate the adhered parts of the binary image. The formula for the erosion process is as follows:

[0123]

[0124] Among them, I ero (x, y) represents the grayscale value of the pixel at the position (x, y) of the binary image after the erosion process. ero,x (x, y) represents the grayscale value of the pixel at position (x, y) after the pixel is corroded in the horizontal direction; I ero,y (x, y) represents the grayscale value of the position after the pixel at the position is eroded in the vertical direction.

[0125] When the grayscale values ​​of the horizontally adjacent (x-1, y) and (x+1, y) positions of the pixel at (x, y) are both 1, the grayscale value of this position is 1, otherwise, the grayscale value of this position is 0. When the vertically adjacent (x, y-1) and (x, y-1) positions of the pixel at (x, y) are both 1, the grayscale value of this position is 1, otherwise, the grayscale value of this position is 0. When the horizontal erosion grayscale value I ero,x (x,y) and the vertical erosion gray value I ero,y When (x, y) are equal, the grayscale value of the position is 1, and in other cases the grayscale value of the position is 0.

[0126] After obtaining the binary image after the erosion process, the dilation process is performed on the binary image after the erosion process. The dilation process is used to increase the edge width of the bubble in the binary image after the erosion process, making the bubble edge more obvious. The formula of the dilation process is as follows:

[0127]

[0128] Among them, I dil (x, y) represents the grayscale value of the pixel at the position (x, y) in the image after the expansion process of the binary image after the corrosion process. dil,x (x, y) represents the grayscale value of the pixel at position (x, y) after it is expanded in the horizontal direction; I dil,y (x, y) represents the grayscale value of the position after the pixel at the position is expanded in the vertical direction.

[0129] When the grayscale value of the horizontally adjacent (x-1,y) position or (x+1,y) position of the pixel point at (x,y) is 1, the grayscale value of this position is 1, otherwise, the grayscale value of this position is 0. When the grayscale value of the vertically adjacent (x,y-1) position or (x,y-1) position of the pixel point at (x,y) is 1, the grayscale value of this position is 1, otherwise, the grayscale value of this position is 0. e (x,y), the horizontal expansion grayscale value I dil,x (x, y), and the vertical expansion grayscale value I dil,y (x, y) are added. If the addition result is greater than 0, the grayscale value of the point is 1; if the addition result is equal to 0, the grayscale value of the point is 0.

[0130] After obtaining a binary image processed by an opening operation, if the grayscale values ​​of the pixels adjacent to a pixel in the four directions of the upper, lower, left, and right in the binary image after the opening operation are the same as the grayscale value of the pixel, then the pixel and the pixels adjacent to the pixel in the four directions of the upper, lower, left, and right in the binary image after the opening operation constitute a connected region, and multiple connected regions and the areas of the connected regions in the binary image after the opening operation are obtained. The connected regions represent areas in the image where the pixel and the adjacent pixels have the same grayscale value. Based on the connected regions and the areas of the connected regions in the binary image after the opening operation, area filtering is performed on the binary image after the opening operation, and connected regions with an area smaller than a preset area are filtered out to obtain a noise-reduced image to be processed.

[0131] This setup has the beneficial effect of dividing the image to be processed into multiple equal regions and binarizing each region using a grayscale threshold, making the binarization more accurate and reducing the error caused by using a single grayscale threshold. Opening and area filtering the image can further clarify the bubble shape and eliminate other interference.

[0132] In this embodiment, the image to be processed includes tracer particles; area filtering is performed on the binary image after the opening operation to obtain a noise-reduced image to be processed, including: filtering the tracer particles in the binary image after the opening operation according to a preset area filtering algorithm to obtain a noise-reduced image to be processed.

[0133] Specifically, the image to be processed also includes an image of tracer particles. Tracer particles are fluorescent particles added to the liquid used to capture the bubbles, such as PMMA Rhodamine B particles. Tracer particles are used to visualize liquid flow and produce fluorescence at a specific wavelength under laser irradiation. Therefore, in addition to the image of the bubbles, the image to be processed also includes an image of the tracer particles.

[0134] A pre-set mean bubble area value is used to represent the average area of ​​the bubbles. A pre-set area filtering algorithm is used to filter the tracer particle image in the binary image after the opening operation. For example, after determining the area of ​​the connected regions in the binary image after the opening operation, regions with a connected area less than one-third of the pre-set mean bubble area value are identified as tracer particle regions, and these tracer particle regions are deleted.

[0135] According to a preset area filtering algorithm, the tracer particles in the binary image processed by the opening operation are filtered to obtain a noise-reduced image to be processed.

[0136] The beneficial effect of this setting is that the connected areas that do not belong to bubbles in the binary image after the opening operation can be filtered out, making the image to be processed after noise reduction purer, which is convenient for subsequent contour extraction.

[0137] S203 , performing contour extraction processing on the image to be processed after noise reduction, to obtain contour information of bubbles in the image to be processed of the camera.

[0138] For example, after obtaining the denoised image to be processed, the contours of the bubbles in the denoised image to be processed are extracted to obtain contour information of the bubbles in the image to be processed corresponding to the camera. For example, image restoration can be performed on the denoised image to repair any defects in the image to be processed caused by the denoising process, thereby reducing distortion of the bubble shape. The outermost pixels of the bubbles in the restored image can then be connected to obtain the contour information of the bubbles.

[0139] The beneficial effect of this setting is that the image to be processed after noise reduction is repaired, the loss of bubble morphology caused by processing is reduced, the distortion that may be caused by bubble contour extraction is reduced, and the bubble contour is more consistent with the real bubble shape.

[0140] In this embodiment, contour extraction processing is performed on the image to be processed after denoising to obtain contour information of bubbles in the camera's image to be processed, including: boundary recognition processing is performed on the image to be processed after denoising to obtain boundary information of bubbles in the image to be processed after denoising; wherein the boundary information is initial contour information; based on the boundary information, expansion processing, filling processing, and corrosion processing are performed on the bubbles in the image to be processed after denoising to obtain contour information of bubbles in the camera's image to be processed.

[0141] Specifically, the Canny operator is used to identify the boundaries of the image to be processed after denoising. The identification step can be, for example, using a two-dimensional Gaussian filter to smooth the image to be processed after denoising, and obtaining a smoothed image, which is a smoothed image. The Sobel operator is used to obtain the gradient amplitude of the smoothed image. The gradient represents the magnitude of the grayscale change of the image; the direction of the gradient represents the direction of the grayscale change of the image; the gradient amplitude represents the magnitude of the gradient, that is, the severity of the grayscale change of the image. The gradient amplitude is subjected to non-maximum suppression processing, that is, for any gradient direction, the gradient amplitude of the pixel points of the smoothed image is traversed along the gradient direction to obtain the location of the pixel point with the maximum gradient value, and the other pixel points in the gradient direction are deleted, and the pixel point with the maximum gradient value is used as the edge pixel point in the gradient direction.

[0142] In one example, the gradient amplitude of a pixel point in a smoothed image may be calculated using methods such as the Roberts operator or the Prewitt operator. In this embodiment, the method for obtaining the gradient amplitude of a pixel point in a smoothed image is not specifically limited.

[0143] A high gradient threshold and a low gradient threshold are pre-set and used to classify edge points. If the gradient amplitude of an edge pixel is greater than the high gradient threshold, the edge pixel is determined to be a strong edge point; if the gradient amplitude of an edge pixel is greater than the low gradient threshold and less than the high gradient threshold, the edge pixel is determined to be a weak edge point; if the gradient amplitude of an edge pixel is less than the low gradient threshold, the edge pixel is removed. For strong edge points, the portion connecting the edge point and adjacent edge pixels is retained; for weak edge points, only the portion connecting the weak edge point and the strong edge point is retained, thus obtaining the boundary information of the bubbles in the denoised image to be processed.

[0144] According to the expansion process formula, the bubble boundary information is expanded to close the open boundary information, resulting in the expanded bubble boundary information as the first image. The first image is then filled so that the grayscale value of the pixels within the boundary information is all 1, resulting in the filled boundary information image as the second image. According to the erosion process formula, the second image is eroded to obtain the third image.

[0145] From the point P with the smallest vertical coordinate L First, traverse each pixel point on the bubble boundary in the third image and obtain the remaining pixel points and P L The angle between the vectors formed by the two relative to the horizontal direction, and sort the angles in ascending order to get P L The angle sequence of the vector. From point P L Initially, the pixel points are connected counterclockwise in the order of the angle sequence to obtain the contour information of the bubble in the camera's image to be processed.

[0146] The beneficial effect of this setting is that the Canny operator is used to obtain the boundary information of the bubble and repair the boundary information of the bubble, which can make the contour information of the bubble clearer and more accurate, remove the depressions in the bubble image, connect the pixel points on the boundary of the third image according to the vector angle, and reduce the contour loss.

[0147] S204 , determining three-dimensional morphological information of the bubble based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0148] For example, this step may refer to the above-mentioned step S103 and will not be described in detail.

[0149] The image data processing method, apparatus, device, and storage medium for bubbles provided in the embodiments of the present application obtain contour information corresponding to the bubble by acquiring images of the same bubble to be processed from multiple cameras and the mapping function corresponding to each camera. The three-dimensional morphological information of the bubble is then determined based on the contour information and the mapping function. By using multiple cameras, while ensuring accuracy and reducing interference with bubble measurement, the bubble's contour can be effectively identified and its three-dimensional morphology accurately reconstructed, thereby improving the accuracy of three-dimensional measurement.

[0150] Figure 3 A flowchart of an image data processing method for bubbles provided in an embodiment of the present disclosure.

[0151] In this embodiment, determining the mapping function corresponding to the camera includes: determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point; and determining the mapping function corresponding to the camera based on the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0152] like Figure 3 As shown, the method includes the following steps:

[0153] S301: Acquire an image to be processed from a camera, and determine the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0154] Exemplarily, for any camera, an image to be processed corresponding to the camera is obtained. Based on the image to be processed, a coordinate system for the image to be processed is determined, which is a first coordinate system. The coordinate position of each pixel in the image to be processed in the first coordinate system is obtained. Based on the first coordinate system, a coordinate system in physical space is established, which is a second coordinate system.

[0155] After acquiring the image to be processed from the camera, the corresponding physical locations of the pixels in the image to be processed are determined. For example, a calibration plate with distinct features can be used, such as a checkerboard calibration plate. The size and number of checkerboard grids in the checkerboard calibration plate are determined, and the coordinate positions of the feature points in the second coordinate system, i.e., the physical locations of the feature points, are matched with the coordinate positions of the pixels in the image to be processed in the first coordinate system, i.e., the coordinate positions of the pixels in the image to be processed. The physical locations corresponding to the remaining pixels are determined based on the feature points and the corresponding pixels.

[0156] The beneficial effect of this setting is that the image to be processed is calibrated, the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point are determined, the connection between the two-dimensional image and the three-dimensional space is established, and the physical and pixel spaces are accurately matched.

[0157] In this embodiment, determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point includes: obtaining the physical space position of the bubble; determining the reference position and the end position based on the physical space position of the bubble; wherein the reference position represents the position farthest from the camera in the physical space position, and the end position represents the position closest to the camera in the physical space position; dividing the bubble into equal distances according to the distance between the reference position and the end position to obtain the dividing points on the bubble; obtaining the physical space position of the dividing point and the coordinate position of the pixel point of the dividing point in the image to be processed.

[0158] Specifically, for any camera, obtain the physical space position of the bubble in the image to be processed of the camera, that is, the position of the bubble in the second coordinate system. The position of the bubble in the physical space position that is farthest from the camera is used as the reference position; the position of the bubble in the physical space position that is closest to the camera is used as the end position. According to the distance between the reference position and the end position, the bubble is divided into equal distances, and the distance can be, for example, 0.1 mm. Obtain multiple sections on the bubble and the section points corresponding to each section, and obtain the coordinate position in the second coordinate system corresponding to each section point, that is, the physical space position of the section point. Use a calibration plate to traverse each section, and based on a preset calibration method, determine the coordinate position of the pixel point in the image to be processed corresponding to the section point on each section.

[0159] The beneficial effect of this setting is that, by performing equally spaced layered calibration from the reference position to the end position, the physical space position of the dividing point on each cut plane and the coordinate position of the pixel point of the dividing point in the image to be processed are obtained, which not only improves the accuracy of the calibration, but also makes the correspondence between the pixel point and the physical space more accurate.

[0160] S302 : Determine a mapping function corresponding to the camera according to the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0161] For example, for any camera, a mapping function corresponding to the camera is determined based on the coordinate positions of pixels in the image to be processed by the camera and the physical spatial positions corresponding to the pixels. For example, the least squares method can be used to fit the coordinate positions of the pixels and the physical spatial positions corresponding to the pixels to obtain a mapping relationship between the coordinate positions of the pixels and the physical spatial positions corresponding to the pixels, i.e., the mapping function.

[0162] The beneficial effect of this setting is that a connection is established between the coordinate position of the pixel point and the physical space position corresponding to the pixel point, and a conversion relationship is obtained between the physical space position corresponding to the pixel point and the coordinate position of the pixel point, which facilitates the reconstruction of the bubble in subsequent steps.

[0163] In this embodiment, the mapping function corresponding to the camera is determined based on the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point, including: determining the candidate function corresponding to the camera based on the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point; and correcting the candidate function to obtain the mapping function corresponding to the camera.

[0164] Specifically, for any camera, the candidate function corresponding to the camera is determined based on the coordinate position of the pixel point in the image to be processed by the camera and the physical space position corresponding to the pixel point. The candidate function represents the mapping relationship between the coordinate position of the pixel point and the physical space position corresponding to the pixel point. The candidate function includes a candidate function in the horizontal direction and a candidate function in the vertical direction. For example, a third-order precision polynomial can be used to approximate the mapping relationship between the physical space position and the coordinates of the pixel point. The third-order precision polynomial can be, for example:

[0165] F(X)=a0+a1x+a2y+a3z+a4x 2 +a5xy+a6y 2 +a7xz+a8yz+a9z 2

[0166] +a 10 x 3 +a 11 x 2 y+a 12 xy 2 +a 13 y 3 +a 14 x 2 z+a 15 xyz+a 16 y 2 z

[0167] +a 17 xz 2 +a 18 yz 2

[0168] F(Y)=b0+b1x+b2y+b3z+b4x 2 +b5xy+b6y 2 +b7xz+b8yz+b9z 2

[0169] +b 10 x 3 +b 11 x 2 y+b 12 xy 2 +b 13 y 3 +b14 x 2 z+b 15 xyz+b 16 y 2 z

[0170] +b 17 xz 2 +b 18 yz 2

[0171] Where (X, Y) is the coordinate of the pixel in the image to be processed, that is, the coordinate position of the pixel in the first coordinate system; (x, y, z) is the physical space position corresponding to the pixel, that is, the coordinate position of the pixel in the second coordinate system. i and b i is the calibration coefficient, which represents the coefficient of the unknown quantity in the polynomial. For all the pixels in the image to be processed, by substituting the coordinates of the pixel points in the image to be processed and the physical space position corresponding to the pixel point into the above formula, all the calibration coefficients a can be solved. i and b i , and obtain the candidate function F(X) in the horizontal direction and the candidate function F(Y) in the vertical direction. i ranges from 0 to 18.

[0172] For any camera, after obtaining the candidate function F(X) in the horizontal direction and the candidate function F(Y) in the vertical direction, correction processing is performed on the candidate functions in the horizontal and vertical directions to obtain the mapping function corresponding to the camera. The correction processing is used to correct the candidate function to reduce the deviation in the mapping relationship between the candidate function and the actual pixel coordinates and physical space positions. For example, the correction processing can be to pre-set a function deviation value, which represents the deviation in the mapping relationship between the candidate function and the actual pixel coordinates and physical space positions. The preset function deviation value is superimposed on the candidate function to obtain the mapping function.

[0173] The beneficial effect of this setting is that the coordinate positions of all pixel points in the processed image and the physical space positions corresponding to the pixel points are fitted using the least squares method to obtain high-precision calibration coefficients, that is, a high-precision mapping relationship between the coordinate positions of the pixel points and the physical space is obtained, and the candidate function is corrected to eliminate the obvious deviations that may exist between the candidate function and the actual mapping relationship caused by various factors, so that the correspondence between the coordinate positions of the pixel points and the physical space positions is more accurate.

[0174] In this embodiment, the physical space corresponding to the image to be processed includes multiple tracer particles, and the physical space corresponding to the image to be processed is divided into multiple areas, and the number of tracer particles in different areas is the same; the candidate function is corrected to obtain the mapping function corresponding to the camera, including: determining the coordinate position of the tracer particle in the image to be processed as the initial position; determining the initial mapping position of the tracer particle according to the candidate function and the initial position corresponding to the camera; fitting the initial mapping position of the tracer particles corresponding to each camera to obtain the target mapping position of the tracer particle; determining the target position of the tracer particle in the image to be processed according to the target mapping position and the candidate function of the camera; determining the mapping function corresponding to the camera based on the candidate function according to the initial position and the target position.

[0175] Specifically, for any camera whose image to be processed contains image information of multiple tracer particles, the coordinate positions of the tracer particles in the image to be processed are obtained. Based on the coordinate positions of the tracer particles in the image to be processed and the candidate function corresponding to the camera, the corresponding physical spatial positions of the tracer particles are obtained. Based on the tracer particles and their corresponding spatial positions, the physical space corresponding to the image to be processed is divided into multiple physical spatial regions, each of which contains the same number of tracer particles.

[0176] For any tracer particle, the coordinate position of the tracer particle in the image to be processed is determined as the initial position. Based on this initial position and the candidate function corresponding to each camera, the position of the tracer particle in the physical space corresponding to each camera is determined as the initial mapping position of the tracer particle in the physical space corresponding to each camera. The initial mapping position of the tracer particle in the physical space corresponding to each camera is fitted to an intersection point, which is the target mapping position of the tracer particle. Based on this target mapping position and the candidate function of each camera, the position of the tracer particle in the image to be processed by each camera is determined as the target position of the tracer particle in the image to be processed.

[0177] For any tracer particle, the initial and target positions of the tracer particle corresponding to each camera are determined. Based on these initial and target positions, the vector between the initial and target positions of the tracer particle is determined. For each region, the probability density distribution of the magnitudes of the vectors between the initial and target positions of all tracer particles in that region is determined. The vector corresponding to the vector magnitude at the peak of the probability density in that region is used as the target vector for that region.

[0178] In the physical space of the region, a random location is selected that has the same number of tracer particles as that region, and this location is the first location. For each first location, based on the corresponding candidate function, the pixel space coordinate location corresponding to the first location is obtained as the second location. The target vector is superimposed on the second location to obtain the third location. Based on the first and third locations, a new candidate function corresponding to each camera is determined, which is the mapping function for each camera.

[0179] The beneficial effect of this arrangement is that the physical space corresponding to the image to be processed is divided into regions according to the tracer particles, so that the target vector can be calculated in different regions, ensuring the accuracy of the deviation correction. Moreover, by using tracer particles and multiple cameras to fit the mapping position of the tracer particles and then mapping it to the image to be processed, the final mapping function can have higher accuracy.

[0180] In this embodiment, according to the initial position and the target position, based on the candidate function, the mapping function corresponding to the camera is determined, including: determining the position deviation vector of the tracer particle according to the initial position and the target position; wherein the position deviation vector represents the deviation between the initial position and the target position; determining the area where the tracer particle is located, and determining the position deviation vector of the area according to the position deviation vectors of all tracer particles in the area; and obtaining the mapping function of the camera according to the position deviation vectors of each area and the candidate function of the camera.

[0181] Specifically, for any tracer particle, the position deviation vector corresponding to each camera is determined based on the initial position and target position of the tracer particle in the processed image of each camera. The position deviation vector represents the deviation between the initial position and the target position. For each region, the position deviation vectors of all tracer particles in that region are determined. Based on the magnitude of the position deviation vectors of all tracer particles in that region, the probability density distribution of the position deviation vector magnitudes in that region is determined. The position deviation vector corresponding to the peak of the probability density in that region is used as the position deviation vector for that region.

[0182] For any area, determine the physical space position coordinates of the center position of the area, and based on the candidate function of the camera corresponding to the area, determine the coordinate position of the pixel point of the center position in the image to be processed corresponding to the camera, and superimpose the position deviation vector of the area on the coordinate position of the pixel point to obtain the coordinate position of the superimposed pixel point. According to the physical space position coordinates of the center position of the area and the coordinate position of the superimposed pixel point, determine the new candidate function corresponding to each camera, that is, the mapping function corresponding to each camera.

[0183] The beneficial effect of this setting is that the mapping function is determined using the position deviation vector corresponding to each area and the center position of each area, so that the mapping function has better accuracy, the deviation error is smaller, and the result is more in line with expectations.

[0184] S303 , performing contour acquisition processing on the camera's image to be processed to obtain contour information of the bubbles in the camera's image to be processed.

[0185] For example, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0186] S304 , determining three-dimensional morphological information of the bubble based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0187] For example, this step may refer to the above-mentioned step S103 and will not be described in detail.

[0188] The image data processing method, apparatus, device, and storage medium for bubbles provided in the embodiments of the present application obtain contour information corresponding to the bubble by acquiring images of the same bubble to be processed from multiple cameras and the mapping function corresponding to each camera. The three-dimensional morphological information of the bubble is then determined based on the contour information and the mapping function. By using multiple cameras, while ensuring accuracy and reducing interference with bubble measurement, the bubble's contour can be effectively identified and its three-dimensional morphology accurately reconstructed, thereby improving the accuracy of three-dimensional measurement.

[0189] Figure 4 A flowchart of an image data processing method for bubbles provided in an embodiment of the present disclosure.

[0190] In this embodiment, the three-dimensional morphological information of the bubble is determined based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera, including: determining the physical spatial position of the pixel point in the image to be processed; determining the three-dimensional grayscale information of the pixel point based on the physical spatial position of the pixel point, based on the contour information of the bubble and the mapping function; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space, the pixel point and the volume element have a one-to-one correspondence, and the volume element is the smallest spatial unit of the space where the bubble is located; the three-dimensional morphological information of the bubble is determined based on the three-dimensional grayscale information of the pixel point.

[0191] like Figure 4 As shown, the method includes the following steps:

[0192] S401. Acquire an image to be processed from a camera and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object.

[0193] For example, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0194] S402 : Perform contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed.

[0195] For example, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0196] S403: Determine the physical spatial position of the pixel points in the image to be processed.

[0197] Exemplarily, for any camera, the coordinate position of each pixel point in the image to be processed corresponding to the camera is obtained, and according to the coordinate position of the pixel point and based on the mapping function corresponding to the camera, the physical space position corresponding to each pixel point in the image to be processed of the camera is determined.

[0198] The beneficial effect of this setting is that the physical space position corresponding to each pixel point is determined through the mapping function, which facilitates the subsequent acquisition of three-dimensional grayscale information.

[0199] S404. Determine the three-dimensional grayscale information of the pixel point according to the physical spatial position of the pixel point, based on the contour information of the bubble and the mapping function; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space, the pixel point and the volume element have a one-to-one correspondence, and the volume element is the smallest spatial unit of the space where the bubble is located.

[0200] Exemplarily, based on the physical spatial position of the pixel point and the contour information of the bubble, the physical space corresponding to the contour information of the bubble is divided into multiple volume elements. The volume element represents the smallest spatial unit in the physical space where the bubble is located, and each volume element corresponds to a pixel point one-to-one.

[0201] Based on the physical spatial position of the pixel, the three-dimensional grayscale information of the pixel is determined based on the bubble outline information from each camera and a mapping function. For example, based on the physical spatial position of the pixel and the grayscale information of the pixel in the bubble outline information, the corresponding mapping function can be used to map the grayscale information of the pixel to the volume element at the physical spatial position of the pixel, thereby obtaining the grayscale value in the three-dimensional space corresponding to the volume element.

[0202] The beneficial effect of this setting is that the grayscale information of the pixel points in the bubble contour information is matched with the volume elements in the physical space to obtain the three-dimensional grayscale information corresponding to the pixel points. By using the grayscale information in the bubble contour information, the three-dimensional grayscale information is made purer and more consistent with the shape of the bubble.

[0203] In this embodiment, the three-dimensional grayscale information of the pixel point is determined according to the physical space position of the pixel point, based on the contour information of the bubble and the mapping function, including: obtaining the intermediate quantity of the pixel point according to the physical space position of the pixel point, the coordinate position of the pixel point, the mapping function, the preset grayscale value, and the preset point propagation function; wherein, the intermediate quantity represents the two-dimensional grayscale information of the pixel point, and the preset point propagation function represents the influence of the pixel point on the grayscale value of the adjacent pixel points; determining the image grayscale value of the pixel point according to the contour information of the bubble; obtaining the target quantity according to the intermediate quantity of the pixel point, the image grayscale value of the pixel point, and the preset grayscale value; wherein, the target quantity represents the three-dimensional grayscale information of the pixel point; if the intermediate quantity of the pixel point is equal to the image grayscale value of the pixel point, then the target quantity is determined to be the three-dimensional grayscale information of the pixel point.

[0204] Specifically, when determining the three-dimensional grayscale information of a pixel, each volume element can be considered a point light source. A point light source represents a light source that can be idealized as a particle. That is, the grayscale value of a volume element, like a light source, affects the grayscale values ​​of its neighboring volume elements. A point spread function is pre-set to represent the effect of the grayscale value of a point light source on the grayscale values ​​of other volume elements. The pre-set point spread function can be, for example, a Gaussian function.

[0205] In one example, an error function may also be used as the point spread function. In this embodiment, the specific type of the point spread function is not specifically limited.

[0206] For each camera, the number of all volume elements in the corresponding physical space and the number of pixels in the corresponding bubble outline are obtained. The volume elements and pixels are numbered based on their distance from the camera. Each volume element is assigned a preset grayscale value in three-dimensional space, such as 1.

[0207] Based on a preset intermediate quantity determination formula, the intermediate quantity of the pixel is obtained according to the physical spatial position of the pixel, the coordinate position of the pixel, the mapping function, the preset grayscale value, and the preset point propagation function. The preset intermediate quantity determination formula is used to determine the intermediate quantity, which represents the grayscale value of the pixel, that is, the two-dimensional grayscale information of the pixel. The preset intermediate quantity determination formula can be, for example:

[0208]

[0209] Where k represents the number of iterations. N represents the number of volume elements. m represents the camera number. W i,j Characterize the influence of the j-th volume element on the gray value of the i-th pixel; h is the point spread function, Representing pixel i, The volume element j is represented by a mapping function to obtain the corresponding pixel point in the image. Represents the distance between pixel i and the pixel corresponding to volume element j. The larger the distance, the smaller the influence of volume element j on the grayscale value of pixel i. j ,y j ,z j ) represents the three-dimensional grayscale value of volume element j, that is, located at (x j ,y j ,z j ) is the three-dimensional grayscale value of the volume element at . In the first calculation, the grayscale value is a preset grayscale value. Characterize the intermediate quantity, that is, at (X i ,Y i ) is the grayscale value of the pixel at .

[0210] This formula represents the intermediate quantity corresponding to the i-th pixel at the k-th iteration of camera m by multiplying the effect of the j-th volume element on the i-th pixel's grayscale value by the grayscale value of the volume element at the k-th iteration. The sum of these products is the intermediate quantity corresponding to the i-th pixel at the k-th iteration of camera m. This is repeated through all pixels to obtain the intermediate quantity corresponding to each pixel.

[0211] Based on the bubble contour information, the grayscale value of each pixel in the bubble contour information is determined. Based on the preset grayscale update formula, the target volume is obtained according to the intermediate volume of the pixel, the image grayscale value of the pixel, and the preset grayscale value. The preset grayscale update formula is used to update the grayscale value of each volume element to obtain the target volume. The target volume represents the three-dimensional grayscale information of the pixel. The preset grayscale update formula can be:

[0212]

[0213] Wherein, μ represents the speed of iteration. The larger the value of μ, the faster the iteration speed. The value of μ can be, for example, 1. m (X i .Y i ) represents the gray value of the i-th pixel in the bubble contour information, that is, the gray value of the pixel at (X i ,Y i ) is the grayscale value of the pixel at .

[0214] This formula represents that for a camera numbered m, the gray value of the jth volume element of the kth iteration is multiplied by μ, the ratio of the gray value of the i-th pixel in the bubble contour information to the intermediate value of the i-th pixel of the kth iteration. The multiplied value is multiplied, and the multiplied value is used as the grayscale value of the k+1th iteration, which is the target value.

[0215] According to the target quantity, based on the preset intermediate quantity determination formula and the preset grayscale update formula, the target quantity of all volume elements is continuously updated iteratively. When the intermediate quantity corresponding to the pixel point is equal to the grayscale value corresponding to the pixel point in the bubble contour information, that is, I m (X i .Y i )and If they are equal, the target volume at that time is obtained and used as the 3D grayscale information of the volume element corresponding to the pixel point. The iteration of the target volume for this pixel point is stopped. The 3D grayscale information of the volume elements corresponding to all pixels is determined.

[0216] This arrangement has the beneficial effect of treating each volume element as a point light source, using a point propagation function to determine the influence of that volume element's grayscale value on the grayscale values ​​of volume elements at other locations. The grayscale values ​​of each volume element are then iterated using the MART algorithm. Based on the bubble's contour information, the grayscale value information in three-dimensional space is reconstructed, making the grayscale value information in three-dimensional space more accurate. The process of obtaining grayscale values ​​in three-dimensional space can be considered a back-projection process. Using the method described in this embodiment, the difficulty of this back-projection process can be reduced.

[0217] S405 , determining the three-dimensional morphological information of the bubble according to the three-dimensional grayscale information of the pixel point.

[0218] Exemplarily, the surface of the bubble's three-dimensional morphology is determined based on the bubble's contour information and the three-dimensional grayscale information corresponding to the pixel points, i.e., the bubble's three-dimensional morphological information. For example, multiple isosurfaces can be obtained from the three-dimensional grayscale information, i.e., the three-dimensional spatial grayscale values ​​of the three-dimensional grayscale information corresponding to the volume elements on the isosurfaces are the same. The isosurfaces corresponding to the bubble's contour information are used as the bubble's morphological surface, i.e., the bubble's three-dimensional morphological information. In this embodiment, the method for determining the bubble's morphology is not specifically limited.

[0219] The beneficial effect of this setting is that by obtaining the isosurface of specific three-dimensional grayscale information, the isosurface is matched with the contour information of the bubble to obtain the bubble morphological surface, making the three-dimensional morphological information of the bubble more accurate.

[0220] The image data processing method, apparatus, device, and storage medium for bubbles provided in the embodiments of the present application obtain contour information corresponding to the bubble by acquiring images of the same bubble to be processed from multiple cameras and the mapping function corresponding to each camera. The three-dimensional morphological information of the bubble is then determined based on the contour information and the mapping function. By using multiple cameras, while ensuring accuracy and reducing interference with bubble measurement, the bubble's contour can be effectively identified and its three-dimensional morphology accurately reconstructed, thereby improving the accuracy of three-dimensional measurement.

[0221] Figure 5 A flowchart of an image data processing method for bubbles provided in an embodiment of the present disclosure.

[0222] In this embodiment, it also includes: determining the target grayscale brightness based on the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information; determining the spatial grayscale isosurface based on the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness; determining the first diameter of the bubble based on the spatial grayscale isosurface; wherein the first diameter represents the diameter size of the bubble in the three-dimensional morphological information; determining the second diameter of the bubble based on the mapping function and the first diameter of the bubble; wherein the second diameter is the actual diameter size of the bubble.

[0223] like Figure 5 As shown, the method includes the following steps:

[0224] S501 , obtaining an image to be processed from a camera, and determining a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical space position of the object.

[0225] For example, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0226] S502 : Perform contour acquisition processing on the camera's image to be processed to obtain contour information of bubbles in the camera's image to be processed.

[0227] For example, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0228] S503 : Determine three-dimensional morphological information of the bubble based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0229] For example, this step may refer to the above-mentioned step S103 and will not be described in detail.

[0230] S504 : Determine target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information.

[0231] For example, based on the three-dimensional morphological information of the bubbles, the grayscale values ​​in the three-dimensional morphological information of the bubbles are sorted from low to high, and the grayscale value with the lowest grayscale brightness is used as the target grayscale brightness. The target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information.

[0232] The beneficial effect of such a setting is that the minimum grayscale value is determined as the target grayscale brightness from the three-dimensional morphological information of the bubble, which facilitates the subsequent use of the target grayscale brightness.

[0233] S505. Determine a spatial grayscale isosurface based on the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness.

[0234] Exemplarily, after obtaining the target grayscale brightness, in the three-dimensional morphological information, the isosurface with the same grayscale value as the target grayscale brightness is determined as the spatial grayscale isosurface. The size of the spatial grayscale isosurface is determined by the volume elements corresponding to the pixel points on the spatial grayscale isosurface. The grayscale values ​​of the volume elements corresponding to the pixel points in the spatial grayscale isosurface are all the target grayscale brightness.

[0235] The beneficial effect of this setting is that the isosurface of the target grayscale brightness is determined to be the spatial grayscale isosurface, which is closer to the thickness direction of the bubble. The diameter of the bubble obtained on this isosurface is more consistent with the diameter of the bubble corresponding to the current three-dimensional morphological information of the bubble.

[0236] S506 , determining a first diameter of the bubble according to the spatial grayscale isosurface; wherein the first diameter represents the diameter of the bubble in the three-dimensional morphological information.

[0237] Exemplarily, based on the spatial grayscale isosurface, the length in the direction of the adjacent surface of the spatial grayscale isosurface and the side where the camera is located is determined as the first diameter of the bubble. The first diameter represents the diameter size of the bubble in the three-dimensional morphological information of the bubble, that is, the thickness information of the bubble compared to the camera direction.

[0238] The beneficial effect of this setting is that the first diameter obtained is the diameter size in the three-dimensional shape of the current bubble, which is more consistent with the length of the bubble in the thickness direction, making it easier to subsequently adjust the three-dimensional shape of the current bubble.

[0239] S507 : Determine a second diameter of the bubble according to the mapping function and the first diameter of the bubble; wherein the second diameter is an actual diameter of the bubble.

[0240] Exemplarily, the deflection angle corresponding to the camera is determined according to the mapping function corresponding to the camera. The camera may be the two cameras on the far side, for example, camera numbered 1 and camera numbered 4. For the selected camera, the camera deflection angle calculation formula is as follows:

[0241]

[0242] α=arctan(tan(α))

[0243] Where F(X) is the camera's horizontal mapping function, F(Y) is the camera's vertical mapping function, and α represents the camera's deflection angle. Calculate the partial derivatives of F(X) with respect to x, y, and z, and the partial derivatives of F(Y) with respect to x, y, and z. Subtract the product of the partial derivative of F(Y) with respect to y and the partial derivative of F(X) with respect to z from the product of the partial derivative of F(Y) with respect to z and the partial derivative of F(X) with respect to y to obtain the first difference. Subtract the product of the partial derivative of F(Y) with respect to x and the partial derivative of F(X) with respect to y from the product of the partial derivative of F(Y) with respect to y and the partial derivative of F(X) with respect to x to obtain the second difference. Comparing the first and second differences yields the tangent of the camera's deflection angle. α, the camera's deflection angle, is obtained from the arctan function and the tangent of the camera's deflection angle.

[0244] β=|α1|+|α4|

[0245] Where |α1| represents the absolute value of the deflection angle of camera 1, |α4| represents the absolute value of the deflection angle of camera 4, and β represents the angle between the cameras. The absolute values ​​of the deflection angles corresponding to camera 1 and camera 4 are added together to obtain the angle between the cameras.

[0246] The second diameter is determined based on the angle between the cameras and the first diameter. The formula for determining the second diameter is as follows:

[0247]

[0248] Where d′ z Characterize the first diameter, d z To characterize the second diameter, the first diameter is multiplied by the sine value of the angle between the cameras, half of which is the product of the two.

[0249] Figure 6 Schematic diagram of the first diameter and second diameter of the bubble provided in the embodiment of the present disclosure. Figure 6 As shown in the figure, the circle represents the actual shape of the bubble, the dotted line represents the direction of the camera's sight, and the solid line represents the camera's field of view. z Characterize the first diameter, d z Characterizes the second diameter, β represents the angle between the cameras. As can be seen from the figure, compared with the second diameter of the bubble, the first diameter is severely elongated due to the viewing angle between the cameras.

[0250] The beneficial effect of this setting is that due to the camera viewing angle and position, the reconstructed three-dimensional information of the bubble is elongated in the thickness direction, and the error in the thickness direction is large. By calculating the angle between the cameras based on the mapping function and obtaining the actual bubble diameter, that is, the second diameter, according to the angle, the error of the three-dimensional information of the bubble can be effectively reduced.

[0251] In this embodiment, the method further includes: adjusting the spatial grayscale isosurface according to the second diameter to obtain a target isosurface; and adjusting the three-dimensional morphology information according to the target isosurface to obtain adjusted three-dimensional morphology information.

[0252] Specifically, the spatial grayscale isosurface is adjusted based on the second diameter to obtain a target isosurface. For example, the second diameter can be used to replace the first diameter to obtain a new isosurface, namely the target isosurface. Based on the target isosurface, the three-dimensional morphological information of the bubble is adjusted. For example, the three-dimensional morphology of the bubble can be scaled according to the direction of the target isosurface diameter to obtain adjusted three-dimensional morphological information.

[0253] Figure 7 Schematic diagram of the three-dimensional morphological information of bubbles provided in the embodiment of the present disclosure. Figure 7 As shown in the figure, the coordinate system represents the physical space where the bubble is located, the x-axis represents the horizontal direction of the physical space where the bubble is located, the y-axis represents the vertical direction of the physical space where the bubble is located, and the z-axis represents the thickness direction of the physical space where the bubble is located. There is three-dimensional morphological information of two bubbles in the physical space in the figure. It can be seen from the figure that the three-dimensional morphological information of the bubble is distorted in the thickness direction.

[0254] Figure 7 Schematic diagram of the adjusted three-dimensional morphological information of the bubble provided in the embodiment of the present disclosure. Figure 8 As shown, the coordinate system in the figure represents the physical space where the bubble is located. The x-axis represents the horizontal direction of the physical space where the bubble is located, the y-axis represents the vertical direction of the physical space where the bubble is located, and the z-axis represents the thickness direction of the physical space where the bubble is located. In the physical space in the figure, there is Figure 7 The adjusted three-dimensional morphological information of the two bubbles. It can be seen from the figure that after adjusting the three-dimensional morphological information of the bubbles, the distortion of the three-dimensional morphological information of the bubbles in the thickness direction is eliminated.

[0255] The beneficial effect of such an arrangement is that the distortion of the bubble produced during the measurement process is adjusted and corrected, thereby reducing the error in the three-dimensional shape of the bubble and making the three-dimensional shape of the bubble more accurate.

[0256] The image data processing method, apparatus, device, and storage medium for bubbles provided in the embodiments of the present application obtain contour information corresponding to the bubble by acquiring images of the same bubble to be processed from multiple cameras and the mapping function corresponding to each camera. The three-dimensional morphological information of the bubble is then determined based on the contour information and the mapping function. By using multiple cameras, while ensuring accuracy and reducing interference with bubble measurement, the bubble's contour can be effectively identified and its three-dimensional morphology accurately reconstructed, thereby improving the accuracy of three-dimensional measurement.

[0257] Figure 9This is a structural block diagram of an image data processing device for bubbles provided in an embodiment of the present disclosure.

[0258] For ease of explanation, only the parts related to the embodiments of the present disclosure are shown. Figure 9 The image data processing device 900 applied to bubbles includes: an acquisition unit 901, a processing unit 902 and a determination unit 903.

[0259] An acquisition unit 901 is configured to acquire an image to be processed from a camera and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents a relationship between a coordinate position of an object in the image and a physical spatial position of the object;

[0260] The processing unit 902 is configured to perform contour acquisition processing on the image to be processed by the camera to obtain contour information of the bubbles in the image to be processed by the camera;

[0261] The determination unit 903 is configured to determine the three-dimensional morphological information of the bubble based on the contour information of the bubble in the image to be processed of each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents the morphological characteristics of the bubble.

[0262] Figure 7 This is a structural block diagram of an image data processing device for bubbles provided in an embodiment of the present disclosure.

[0263] exist Figure 9 Based on the embodiment shown, Figure 10 As shown, the processing unit 902 includes a noise reduction module 9021 and an extraction module 9022 .

[0264] The noise reduction module 9021 is used to perform noise reduction on the image to be processed by the camera to obtain a noise-reduced image to be processed;

[0265] The extraction module 9022 is configured to perform contour extraction on the image to be processed after noise reduction, so as to obtain contour information of bubbles in the image to be processed of the camera.

[0266] In one example, the noise reduction module 9021 includes:

[0267] A binarization submodule is used to perform binarization processing on the image to be processed of the camera according to a preset grayscale threshold to obtain a binary image corresponding to the image to be processed;

[0268] An opening operation submodule, configured to perform an opening operation on the binary image to obtain a binary image after the opening operation; wherein the opening operation includes an erosion process and a dilation process;

[0269] The filtering submodule is used to perform area filtering on the binary image after the opening operation to obtain a noise-reduced image to be processed.

[0270] In one example, the filtering submodule is specifically used to:

[0271] According to a preset area filtering algorithm, the tracer particles in the binary image after the opening operation are filtered to obtain the image to be processed after the noise reduction.

[0272] In one example, the extraction module 9022 includes:

[0273] a boundary recognition submodule, configured to perform boundary recognition processing on the noise-reduced image to be processed, and obtain boundary information of bubbles in the noise-reduced image to be processed; wherein the boundary information is initial contour information;

[0274] The contour acquisition submodule is used to perform expansion processing, filling processing, and corrosion processing on the bubbles in the noise-reduced image to be processed according to the boundary information, so as to obtain the contour information of the bubbles in the camera image to be processed.

[0275] In one example, the acquiring unit 901 includes:

[0276] A first position determination module is used to determine the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point;

[0277] The function determination module is used to determine the mapping function corresponding to the camera according to the coordinate position of the pixel point in the image to be processed and the physical space position corresponding to the pixel point.

[0278] In one example, the first location determination module includes:

[0279] The first position acquisition submodule is used to obtain the physical space position of the bubble;

[0280] a position determination submodule, configured to determine a reference position and an end position according to the physical spatial position of the bubble; wherein the reference position represents the position farthest from the camera in the physical spatial position, and the end position represents the position closest to the camera in the physical spatial position;

[0281] a sub-slicing module, configured to divide the bubble into equal-distance segments according to the distance between the reference position and the end position, to obtain a segmentation point on the bubble;

[0282] The second position acquisition submodule is used to acquire the physical space position of the segmentation point and the coordinate position of the pixel point of the segmentation point in the image to be processed.

[0283] In one example, the function determination module includes:

[0284] A candidate submodule, configured to determine a candidate function corresponding to the camera based on a coordinate position of a pixel point in the image to be processed and a physical space position corresponding to the pixel point;

[0285] The correction submodule is used to correct the candidate function to obtain the mapping function corresponding to the camera.

[0286] In one example, the modified submodule is used to:

[0287] Determine the coordinate position of the tracer particle in the image to be processed as the initial position;

[0288] determining an initial mapping position of the tracer particle according to the candidate function corresponding to the camera and the initial position;

[0289] Performing fitting processing on the initial mapping position of the tracer particles corresponding to each camera to obtain the target mapping position of the tracer particles;

[0290] determining a target position of the tracer particle in the image to be processed according to the target mapping position and the candidate function of the camera;

[0291] According to the initial position and the target position, a mapping function corresponding to the camera is determined based on the candidate function.

[0292] In one example, the correction submodule is also specifically used to:

[0293] Determining a position deviation vector of the tracer particle according to the initial position and the target position; wherein the position deviation vector represents a deviation between the initial position and the target position;

[0294] Determine the region where the tracer particles are located, and determine the position deviation vector of the region based on the position deviation vectors of all tracer particles in the region;

[0295] A mapping function of the camera is obtained according to the position deviation vectors of each region and the candidate function of the camera.

[0296] In one example, the determining unit 903 includes:

[0297] A second position determination module is used to determine the physical spatial position of the pixel point in the image to be processed;

[0298] a grayscale module, configured to determine three-dimensional grayscale information of the pixel point based on the physical spatial position of the pixel point, the contour information of the bubble, and the mapping function; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in three-dimensional space, the pixel point and the volume element having a one-to-one correspondence, and the volume element being the smallest spatial unit of the space where the bubble is located;

[0299] The morphology module is used to determine the three-dimensional morphology information of the bubble according to the three-dimensional grayscale information of the pixel point.

[0300] In one example, the grayscale module includes:

[0301] an intermediate quantum module, configured to obtain an intermediate quantity of the pixel point based on the physical spatial position of the pixel point, the coordinate position of the pixel point, a mapping function, a preset grayscale value, and a preset point propagation function; wherein the intermediate quantity represents the two-dimensional grayscale information of the pixel point, and the preset point propagation function represents the influence of the pixel point on the grayscale value of the adjacent pixel point;

[0302] A first grayscale submodule, configured to determine the image grayscale value of the pixel point according to the contour information of the bubble;

[0303] A target quantum module, configured to obtain a target quantity based on the intermediate quantity of the pixel point, the image grayscale value of the pixel point, and the preset grayscale value; wherein the target quantity represents the three-dimensional grayscale information of the pixel point;

[0304] The second grayscale submodule is configured to determine that the target amount is the three-dimensional grayscale information of the pixel if the intermediate amount of the pixel is equal to the image grayscale value of the pixel.

[0305] In one example, the apparatus further includes:

[0306] A grayscale unit, configured to determine a target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information;

[0307] an isosurface unit, configured to determine a spatial grayscale isosurface based on the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness;

[0308] A first diameter unit is configured to determine a first diameter of the bubble according to the spatial grayscale isosurface; wherein the first diameter represents a diameter of the bubble in the three-dimensional morphological information;

[0309] The second diameter unit is used to determine the second diameter of the bubble according to the mapping function and the first diameter of the bubble; wherein the second diameter is the actual diameter of the bubble.

[0310] In one example, it also includes:

[0311] a first adjustment unit, configured to adjust the spatial grayscale isosurface according to the second diameter to obtain a target isosurface;

[0312] The second adjustment unit is configured to adjust the three-dimensional morphological information according to the target isosurface to obtain adjusted three-dimensional morphological information.

[0313] Figure 11 This is a structural block diagram of an electronic device provided in an embodiment of the present disclosure. The electronic device may be a terminal device or a server. Figure 11 As shown, the electronic device 1100 includes: at least one processor 1102; and a memory 1101 communicatively connected to the at least one processor 1102; wherein the memory stores instructions that can be executed by the at least one processor 1102, and the instructions are executed by the at least one processor 1102 to enable the at least one processor 1102 to execute the image data processing method applied to bubbles disclosed in the present invention.

[0314] The electronic device 1100 further includes a receiver 1103 and a transmitter 1104. The receiver 1103 is used to receive instructions and data sent by other devices, and the transmitter 1104 is used to send instructions and data to external devices.

[0315] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0316] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0317] Figure 12 This is a block diagram of the structure of an electronic device provided in one embodiment of the present application. The electronic device is equipped with at least three cameras, a high-pass filter, a water prism, a laser, a concave lens, a plano-convex cylindrical lens, an air outlet, a flow meter, a bubble generator, an LED light source, a diffuser, and a water tank.

[0318] like Figure 9As shown, water tank 7 is filled with water containing tracer particles. In this embodiment, four cameras are configured: cameras 1, 2, 3, and 4, which are used to capture the bubbles within water tank 7. A water prism 6 is located on the side of water tank 7 to reduce distortion caused by light refraction inside and outside the water tank 7. The camera lenses face the side where water prism 6 is located. Each camera has a corresponding high-pass filter 5, located in front of the camera lens, to filter out strong reflections around the bubbles.

[0319] The LED light source 8 and the soft light plate 9 are located opposite the side where the water prism 6 is located. The LED light source 8 and the soft light plate 9 are outside the water tank, and the soft light plate 9 is close to the water tank 7. The LED light source 8 is located behind the soft light plate 9. The LED light source 8 provides a uniform background light to the water tank 7 through the soft light plate 9, which facilitates the accurate measurement of the bubble morphology.

[0320] Laser 10 is located adjacent to the side surface of water prism 6. Laser 10 is located behind concave lens 11, which is located behind plano-convex cylindrical lens 12, with plano-convex cylindrical lens 12 close to the adjacent surface. Laser light emitted by laser 10 is converged by concave lens 11 and plano-convex cylindrical lens 12 to form a volume light source, uniformly illuminating the entire measurement area of ​​water tank 7. Laser light irradiates tracer particles in water tank 7, causing them to produce fluorescence.

[0321] The bubble generator 13 is located outside the water tank and is used to supply bubbles to the water tank 7. A flow meter 15 is connected to the bubble generator 13 at one end and to the air outlet 14 at the other end to measure the gas flow in the bubble generator 13. The air outlet 14 is located at the bottom of the water tank 7 and is used to discharge the bubbles into the water tank 7.

[0322] In one example, the cameras may be arranged in a cross shape, with a camera being arranged on each of the four sides of the water tank 7 .

[0323] Figure 13 This is a block diagram of an electronic device according to an exemplary embodiment. The device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0324] Device 1300 may include one or more of the following components: a processing component 1302 , a memory 1304 , a power component 1306 , a multimedia component 1308 , an audio component 1311 , an input / output (I / O) interface 1312 , a sensor component 1314 , and a communication component 1316 .

[0325] Processing component 1302 generally controls the overall operation of device 1300, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. Processing component 1302 may include one or more processors 1320 to execute instructions to perform all or part of the steps of the above-described methods. In addition, processing component 1302 may include one or more modules to facilitate interaction between processing component 1302 and other components. For example, processing component 1302 may include a multimedia module to facilitate interaction between multimedia component 1308 and processing component 1302.

[0326] The memory 1304 is configured to store various types of data to support the operations of the device 700. Examples of such data include instructions for any application or method operating on the device 1300, contact data, phone book data, messages, pictures, videos, etc. The memory 1304 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0327] The power supply component 1306 provides power to the various components of the device 1300. The power supply component 1306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1300.

[0328] The multimedia component 1308 includes a screen that provides an output interface between the device 1300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1308 includes a front camera and / or a rear camera. When the device 1300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0329] The audio component 1310 is configured to output and / or input audio signals. For example, the audio component 1310 includes a microphone (MIC) that is configured to receive external audio signals when the device 1300 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 1304 or transmitted via the communication component 1316. In some embodiments, the audio component 1310 also includes a speaker for outputting audio signals.

[0330] I / O interface 1312 provides an interface between processing component 1302 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.

[0331] Sensor assembly 1314 includes one or more sensors for providing various aspects of device 1300 status assessment. For example, sensor assembly 1314 can detect the open / closed state of device 1300, the relative positioning of components, such as the display and keypad of device 1300. Sensor assembly 1314 can also detect changes in the position of device 1300 or a component of device 1300, the presence or absence of user contact with device 1300, the orientation or acceleration / deceleration of device 1300, and changes in the temperature of device 1300. Sensor assembly 1314 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1314 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1314 can also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0332] The communication component 1316 is configured to facilitate wired or wireless communication between the device 1300 and other devices. The device 1300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0333] In an exemplary embodiment, the device 1300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described methods.

[0334] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1304 including instructions, which can be executed by the processor 1320 of the device 1300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0335] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a terminal device, enables the terminal device to execute the above-mentioned image data processing method applied to bubbles.

[0336] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

[0337] It should be further noted that, although the various steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0338] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present application may also be implemented in other ways. For example, the division of units / modules in the above-described embodiments is merely a logical functional division, and actual implementations may employ other division methods. For example, multiple units, modules, or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0339] In addition, unless otherwise specified, the functional units / modules in the various embodiments of the present application may be integrated into a single unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The aforementioned integrated units / modules may be implemented in the form of hardware or software program modules.

[0340] If the integrated unit / module is implemented in hardware, the hardware may be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC. Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0341] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0342] In the above embodiments, the description of each embodiment has its own focus. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0343] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0344] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing image data of bubbles, characterized in that: The method is applied to an electronic device, wherein at least three cameras are deployed in the electronic device, and the method includes: Acquire an image to be processed from a camera, and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents the relationship between the coordinate position of an object in the image and the physical spatial position of the object; Performing contour acquisition processing on the image to be processed of the camera to obtain contour information of the bubbles in the image to be processed of the camera; Determining three-dimensional morphological information of the bubbles based on contour information of the bubbles in the images to be processed from each camera and the mapping function corresponding to each camera; wherein the three-dimensional morphological information represents morphological characteristics of the bubbles; The step of determining the three-dimensional morphological information of the bubbles based on the contour information of the bubbles in the image to be processed of each camera and the mapping function corresponding to each camera includes: Determining the physical spatial position of a pixel point in the image to be processed; dividing the physical space corresponding to the contour information of the bubble into a plurality of volume elements, wherein the pixel points correspond to the volume elements in a one-to-one manner, and the volume element represents the smallest spatial unit in the physical space where the bubble is located; Determining the image grayscale value of the pixel point according to the contour information of the bubble; Obtaining an intermediate quantity of the pixel point based on the physical spatial position of the pixel point, the coordinate position of the pixel point, a mapping function, a preset grayscale value, and a preset point propagation function; wherein the intermediate quantity represents the two-dimensional grayscale information of the pixel point, and the preset point propagation function represents the influence of the pixel point on the grayscale value of the adjacent pixel point; Iterate the MART algorithm according to the intermediate quantity of the pixel point, the image grayscale value of the pixel point, and the preset grayscale value until the target quantity corresponding to the intermediate quantity of the pixel point is equal to the image grayscale value of the pixel point, which is the three-dimensional grayscale information of the pixel point; wherein the target quantity represents the three-dimensional grayscale information of the pixel point; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space; wherein the target quantity in the first iteration process is the preset grayscale value; The three-dimensional morphological information of the bubble is determined according to the three-dimensional grayscale information of the pixel point.

2. The method according to claim 1, characterized in that Performing contour acquisition processing on the image to be processed by the camera to obtain contour information of bubbles in the image to be processed by the camera, including: Performing noise reduction processing on the image to be processed of the camera to obtain a noise-reduced image to be processed; Performing contour extraction processing on the image to be processed after noise reduction to obtain contour information of bubbles in the image to be processed of the camera.

3. The method according to claim 2, characterized in that The image to be processed is a grayscale image; performing image noise reduction processing on the image to be processed by the camera to obtain a noise-reduced image to be processed, including: Binarizing the image to be processed by the camera according to a preset grayscale threshold to obtain a binary image corresponding to the image to be processed; Performing an opening operation on the binary image to obtain a binary image after the opening operation; wherein the opening operation includes an erosion process and a dilation process; Performing area filtering on the binary image after the opening operation to obtain a noise-reduced image to be processed.

4. The method according to claim 3, characterized in that The image to be processed includes tracer particles; performing area filtering on the binary image after the opening operation to obtain a noise-reduced image to be processed, including: According to a preset area filtering algorithm, the tracer particles in the binary image after the opening operation are filtered to obtain the image to be processed after the noise reduction.

5. The method according to claim 2, characterized in that Performing contour extraction processing on the image to be processed after noise reduction to obtain contour information of bubbles in the image to be processed of the camera, including: Performing boundary recognition processing on the image to be processed after noise reduction to obtain boundary information of bubbles in the image to be processed after noise reduction; wherein the boundary information is initial contour information; According to the boundary information, the bubbles in the image to be processed after noise reduction are expanded, filled, and eroded to obtain the contour information of the bubbles in the image to be processed of the camera.

6. The method according to claim 1, characterized in that Determining a mapping function corresponding to the camera includes: Determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point; A mapping function corresponding to the camera is determined according to the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point.

7. The method according to claim 6, characterized in that Determining the coordinate position of a pixel point in the image to be processed and the physical space position corresponding to the pixel point includes: Get the physical space position of the bubble; Determine a reference position and an end position according to the physical space position of the bubble; wherein the reference position represents the position farthest from the camera in the physical space, and the end position represents the position closest to the camera in the physical space; According to the distance between the reference position and the end position, the bubble is divided into equal distances to obtain dividing points on the bubble; The physical space position of the segmentation point and the coordinate position of the pixel point of the segmentation point in the image to be processed are obtained.

8. The method according to claim 6, characterized in that Determining a mapping function corresponding to the camera according to a coordinate position of a pixel point in the image to be processed and a physical space position corresponding to the pixel point includes: Determining a candidate function corresponding to the camera according to a coordinate position of a pixel point in the image to be processed and a physical space position corresponding to the pixel point; The candidate function is corrected to obtain a mapping function corresponding to the camera.

9. The method according to claim 8, characterized in that The physical space corresponding to the image to be processed includes a plurality of tracer particles, and the physical space corresponding to the image to be processed is divided into a plurality of regions, and the number of tracer particles in different regions is the same; the candidate function is corrected to obtain a mapping function corresponding to the camera, including: Determine the coordinate position of the tracer particle in the image to be processed as the initial position; determining an initial mapping position of the tracer particle according to the candidate function corresponding to the camera and the initial position; Performing fitting processing on the initial mapping position of the tracer particles corresponding to each camera to obtain the target mapping position of the tracer particles; determining a target position of the tracer particle in the image to be processed according to the target mapping position and the candidate function of the camera; According to the initial position and the target position, a mapping function corresponding to the camera is determined based on the candidate function.

10. The method according to claim 9, characterized in that Determining a mapping function corresponding to the camera based on the candidate function according to the initial position and the target position includes: Determining a position deviation vector of the tracer particle according to the initial position and the target position; wherein the position deviation vector represents a deviation between the initial position and the target position; Determine the region where the tracer particles are located, and determine the position deviation vector of the region based on the position deviation vectors of all tracer particles in the region; A mapping function of the camera is obtained according to the position deviation vectors of each region and the candidate function of the camera.

11. The method according to claim 1, wherein Also includes: Determining a target grayscale brightness according to the three-dimensional morphological information of the bubble; wherein the target grayscale brightness represents the minimum grayscale value in the three-dimensional morphological information; Determine a spatial grayscale isosurface based on the three-dimensional morphological information and the target grayscale brightness; wherein the size of the spatial grayscale isosurface is determined by the pixels on the spatial grayscale isosurface, and the grayscale values ​​of the pixels in the spatial grayscale isosurface are all the target grayscale brightness; Determining a first diameter of the bubble according to the spatial grayscale isosurface; wherein the first diameter represents the diameter of the bubble in the three-dimensional morphological information; The second diameter of the bubble is determined according to the mapping function and the first diameter of the bubble; wherein the second diameter is the actual diameter of the bubble.

12. The method according to claim 11, characterized in that Also includes: Adjusting the spatial grayscale isosurface according to the second diameter to obtain a target isosurface; The three-dimensional morphological information is adjusted according to the target isosurface to obtain adjusted three-dimensional morphological information.

13. An image data processing device for bubbles, characterized in that: include: an acquisition unit, configured to acquire an image to be processed from a camera and determine a mapping function corresponding to the camera; wherein the image to be processed includes bubbles, and the mapping function represents a relationship between a coordinate position of an object in the image and a physical spatial position of the object; a processing unit, configured to perform contour acquisition processing on the image to be processed of the camera, and obtain contour information of bubbles in the image to be processed of the camera; a determination unit, configured to determine three-dimensional morphological information of the bubble based on contour information of the bubble in the image to be processed of each camera and a mapping function corresponding to each camera; wherein the three-dimensional morphological information represents morphological characteristics of the bubble; The determining unit includes: A second position determination module is used to determine the physical spatial position of the pixel point in the image to be processed; a grayscale module, configured to determine three-dimensional grayscale information of the pixel point based on the physical spatial position of the pixel point, the contour information of the bubble, and the mapping function; wherein the physical space corresponding to the contour information of the bubble is divided into a plurality of volume elements, the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space, the pixel point and the volume element have a one-to-one correspondence, and the volume element is the smallest spatial unit of the physical space where the bubble is located; a morphology module, configured to determine three-dimensional morphology information of the bubble based on the three-dimensional grayscale information of the pixel point; The grayscale module includes: A first grayscale submodule, configured to determine the image grayscale value of the pixel point according to the contour information of the bubble; an intermediate quantum module, configured to obtain an intermediate quantity of the pixel point based on the physical spatial position of the pixel point, the coordinate position of the pixel point, a mapping function, a preset grayscale value, and a preset point propagation function; wherein the intermediate quantity represents the two-dimensional grayscale information of the pixel point, and the preset point propagation function represents the influence of the pixel point on the grayscale value of the adjacent pixel point; The second grayscale submodule is used to iterate using the MART algorithm according to the intermediate quantity of the pixel point, the image grayscale value of the pixel point, and the preset grayscale value, until the target quantity corresponding to the intermediate quantity of the pixel point is equal to the image grayscale value of the pixel point is the three-dimensional grayscale information of the pixel point; wherein the target quantity represents the three-dimensional grayscale information of the pixel point; wherein the three-dimensional grayscale information represents the grayscale value of the volume element corresponding to the pixel point in the three-dimensional space; wherein the target quantity in the first iteration process is the preset grayscale value.

14. An electronic device, characterized in that: The electronic device is equipped with at least three cameras, a high-pass filter, a water prism, a laser, a concave lens, a plano-convex cylindrical lens, an air outlet, a flow meter, a bubble generating device, an LED light source, a soft light plate, and a water tank; The high-pass filter is located in front of the lens of the camera, and each camera corresponds to a high-pass filter. The water prism is located on the side of the water tank, and the lens of the camera faces the side where the water prism is located. The LED light source and the soft light plate are located opposite to the side where the water prism is located. The LED light source and the soft light plate are outside the water tank, the soft light plate is close to the water tank, and the LED light source is located behind the soft light plate. The laser is located on the adjacent surface of the side where the water prism is located, the laser is located behind the concave lens, the concave lens is located behind the plano-convex cylindrical lens, and the plano-convex cylindrical lens is close to the adjacent surface; The bubble generating device is located outside the water tank and is used to provide bubbles to the water tank; one end of the flow meter is connected to the bubble generating device, and the other end of the flow meter is connected to the air outlet port, and is used to measure the gas flow of the bubble generating device; the air outlet port is located at the bottom of the water tank and is used to discharge the bubbles into the water tank; The electronic device is configured to execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 12 when executed by a processor.

16. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 12 when being executed by a processor.

Images