A droplet detection method and system based on machine vision

Abstract

The present invention discloses a method, device, computer equipment and storage medium for detecting fog droplets based on machine vision, and relates to the field of data encryption technology, including classifying basic feature data of fog droplet profiles through DBSCAN algorithm, marking abnormal fog droplets, and recording the position information of abnormal fog droplets; capturing fog droplet images on water-sensitive paper at different viewing angles, adjusting camera parameter settings based on the position information of abnormal fog droplets, and collecting multi-view fog droplet images; constructing a disparity map based on multi-view fog droplet images, and generating three-dimensional point cloud data by analyzing the disparity map and camera parameters using the triangulation principle; and generating a triangular mesh model of fog droplets on the surface of water-sensitive paper based on the three-dimensional point cloud data using the Poisson surface reconstruction method, and analyzing the fog droplet morphology. The present invention improves the accuracy and practicality of fog droplet detection by constructing a disparity map, generating three-dimensional point cloud data and DBSCAN algorithm classification marking using the triangulation principle.

Description

Technical Field

[0001] The invention relates to the technical field of machine vision and image processing, and in particular to a fog droplet detection method and system based on machine vision. Background Art

[0002] In recent years, droplet detection methods based on machine vision have been widely used in agriculture, industry, and environmental monitoring. Traditional droplet detection methods mainly rely on manual measurement and microscopic observation, which are not only time-consuming and labor-intensive, but also easily affected by human factors, resulting in poor accuracy and consistency of the results. With the rapid development of computer vision and image processing technology, droplet detection methods based on machine vision have gradually become a research hotspot. Multispectral imaging technology can obtain image information in different bands, thereby improving the accuracy and reliability of droplet detection. The application of contour detection algorithms such as Canny edge detection and Sobel operator makes the extraction of droplet contours more accurate. In addition, the application of three-dimensional reconstruction techniques such as Poisson surface reconstruction and triangulation principle makes the three-dimensional morphological analysis of droplets possible.

[0003] Although the existing machine vision-based droplet detection methods have improved the accuracy and efficiency of detection to a certain extent, there are still some shortcomings. First, the existing methods usually only rely on single-view or multi-view two-dimensional images for analysis, which cannot fully reflect the three-dimensional morphological characteristics of droplets, which limits the detailed analysis of droplet shape and distribution. Second, most of the existing abnormal droplet detection methods use simple threshold methods or rule-based methods, which lack adaptability and robustness and are difficult to deal with complex backgrounds and noise interference. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a droplet detection method based on machine vision to solve the problems of incomplete three-dimensional morphological analysis of droplets and insufficient robustness of abnormal droplet detection.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a method for detecting droplets based on machine vision, which includes: photographing droplets on water-sensitive paper with a multispectral camera to obtain a multispectral droplet image, and preprocessing the multispectral droplet image; extracting droplet contours from the preprocessed multispectral droplet image using a contour detection algorithm, and analyzing basic feature data of each droplet contour based on the extracted droplet contours; classifying the basic feature data of the droplet contours using a DBSCAN algorithm, marking abnormal droplets, and recording the position information of the abnormal droplets; capturing droplet images on water-sensitive paper at different viewing angles, adjusting camera parameter settings based on the position information of the abnormal droplets, and collecting multi-view droplet images; constructing a disparity map based on the multi-view droplet image, and generating three-dimensional point cloud data by analyzing the disparity map and camera parameters using the triangulation principle; generating a triangular mesh model of droplets on the surface of water-sensitive paper based on the three-dimensional point cloud data using a Poisson surface reconstruction method, and analyzing the droplet morphology.

[0008] As a preferred solution of the method for detecting droplets based on machine vision described in the present invention, the droplets on the water-sensitive paper are photographed by a multispectral camera to obtain a multispectral droplet image, and the multispectral droplet image is preprocessed. The specific steps are as follows:

[0009] Adjust the distance between the camera and the water-sensitive paper so that all images are at the same focal length, and simultaneously shoot the droplets on the water-sensitive paper at different bands to obtain multi-spectral droplet images in the same scene;

[0010] The multispectral droplet image is subjected to denoising, brightness and contrast correction, adaptive contrast enhancement and binarization processing.

[0011] As a preferred solution of the machine vision-based droplet detection method of the present invention, wherein: the contour detection algorithm is used to extract the droplet contour from the preprocessed multispectral droplet image, and based on the extracted droplet contour, the basic feature data of each droplet contour is analyzed. The specific steps are as follows:

[0012] The Canny edge detection algorithm is used to extract clear and continuous droplet edge pixels;

[0013] According to the characteristics of multispectral droplet images, a high threshold T is defined high and low threshold T low ;

[0014] The Sobel operator is used to calculate the gradient amplitude of the multispectral droplet image. The expression is:

[0015]

[0016] Among them, G(x,y) is the gradient magnitude, G x is the horizontal gradient component, G yis the vertical gradient component, x is the horizontal position of the pixel in the image, and y is the vertical position of the pixel in the image;

[0017] The Sobel operator is used to calculate the gradient direction angle of the multispectral droplet image. The expression is:

[0018]

[0019] Among them, θ(x,y) is the gradient direction angle;

[0020] According to the gradient direction angle, the gradient amplitude of each pixel is compared with the gradient amplitude of its adjacent pixels through non-maximum suppression to refine the edge;

[0021] The gradient amplitude is greater than the high threshold T high The pixel point is marked as a strong edge, between the high threshold T high and low threshold T low The pixels between are marked as weak edges, which are less than the low threshold T low The pixel points are marked as non-edge;

[0022] Through connectivity analysis, the strong edges are connected, and the weak edges connected to the strong edges are also marked as edges to generate a binary image of the droplet edge;

[0023] Use contour detection algorithm to extract droplet contour from droplet edge binary image and record the position information of droplet contour;

[0024] Based on the extracted droplet contour, Green's formula is used to calculate the area of ​​the droplet contour, the perimeter of the droplet contour is calculated by Euclidean distance, the circularity of the droplet contour is calculated using the area and perimeter, the eccentricity of the droplet contour is calculated by the length of the major axis and minor axis, and the rectangularity is calculated by the width and height of the minimum circumscribed rectangle, and the basic characteristic data of each droplet is output.

[0025] As a preferred solution of the machine vision-based droplet detection method of the present invention, the basic feature data of the droplet profile is classified by the DBSCAN algorithm, abnormal droplets are marked, and the location information of the abnormal droplets is recorded. The specific steps are as follows:

[0026] Use the Z-score standardization method to standardize the basic feature data;

[0027] The standardized basic feature data is input into the DBSCAN algorithm, which will automatically group the data points into different clusters and use the comprehensive anomaly scoring function to score the droplet anomaly for different clusters. The expression is:

[0028]

[0029] Among them, F(S i ) is the droplet anomaly score, S i represents the i-th sample, k is the number of features, and w j is the weight of the jth feature, S ij represents the jth eigenvalue of the i-th sample, μ j is the mean of the jth feature, λ is the balance coefficient, j is the feature index, i is the sample index, LOF(S i ) is the ratio of the local density of the sample to the local density of its neighbors;

[0030] Based on the distribution of the comprehensive anomaly score, a threshold T is set;

[0031] When the comprehensive abnormality score F(S i )>threshold T, mark it as abnormal droplet and record the location information of the abnormal droplet;

[0032] When the comprehensive abnormality score F(S i )≤threshold T, marked as normal droplets.

[0033] As a preferred solution of the machine vision-based droplet detection method of the present invention, the specific steps of capturing the droplet images on the water-sensitive paper at different viewing angles, adjusting the camera parameter settings based on the position information of the abnormal droplets, and collecting the multi-view droplet images are as follows:

[0034] Use a high-resolution, multi-spectral imaging industrial camera to simultaneously capture multi-view droplet images on water-sensitive paper at the same time;

[0035] Based on the location information of abnormal fog droplets, the camera angle adjustment value and focal length adjustment value are calculated through geometric calculation, the angle and focal length of each camera are adjusted, and the parameters of each camera are reconfigured so that all cameras can shoot synchronously at the same time and re-collect multi-view fog droplet images.

[0036] As a preferred solution of the method for detecting fog droplets based on machine vision described in the present invention, wherein: the disparity map is constructed based on the multi-view fog droplet image, and the three-dimensional point cloud data is generated by analyzing the disparity map and camera parameters using the triangulation principle. The specific steps are as follows:

[0037] The SIFT feature detection algorithm is used to extract feature points in multi-view droplet images, the RANSAC algorithm is used to match feature points and the homography matrix is ​​used to register images, and the disparity map is generated through the semi-global matching algorithm;

[0038] According to the disparity map and camera parameters, the three-dimensional coordinates of each pixel are calculated using the triangulation principle;

[0039] All calculated three-dimensional coordinate points are converted into NumPy arrays by generating a three-dimensional coordinate point list, and invalid points are removed and saved in a standard point cloud format to form a complete three-dimensional point cloud data.

[0040] As a preferred solution of the method for detecting droplets based on machine vision of the present invention, the method uses the Poisson surface reconstruction method to generate a three-dimensional model of droplets on the surface of water-sensitive paper based on three-dimensional point cloud data, and analyzes the droplet morphology. The specific steps are as follows:

[0041] Preprocess the 3D point data and use the Poisson surface reconstruction method to generate a 3D model of droplets on the surface of water-sensitive paper;

[0042] Use the Marching Cubes algorithm to extract isosurfaces from implicit surface functions and generate triangular meshes consisting of vertices and triangular faces, recording the coordinates of each vertex in three-dimensional space and the vertex index information of each triangular face;

[0043] Calculate the normal direction of each vertex and each triangular face, assign material attributes and texture coordinates to the triangular mesh through the lighting algorithm, save the triangular mesh and additional information into a standard 3D model file format, load and verify it through a 3D visualization tool, and generate a 3D model of the droplet;

[0044] Based on the three-dimensional model of the droplet, the area of ​​each droplet is segmented from the three-dimensional model of the droplet through connected component analysis;

[0045] Based on the segmented droplet area, analyze the droplet morphology, including shape, volume and change trend;

[0046] The principal axis direction of the droplet is determined by PCA, the degree of roundness close to sphericity is measured, and the degree of concavity is evaluated by analyzing the difference with the actual contour. The smoothness of the boundary is measured by the standard deviation of the boundary curvature to extract and describe the droplet shape.

[0047] The volume of the droplet is calculated by integrating the triangular mesh, and the expression is:

[0048]

[0049] Where V is the total volume of the droplets, N is the number of triangles in the triangular mesh, p is the origin, and a q is the first vertex of q triangles, b q is the second vertex of the q triangles, c q is the third vertex of q triangles;

[0050] In the trend analysis, three-dimensional data of droplets are collected at different time points and aligned, the volume change curve is calculated, the shape characteristics are compared, the connected component analysis is used to detect the splitting or merging phenomenon, and the three-dimensional visualization tool is used to intuitively verify the change trend of droplets over time.

[0051] In a second aspect, the present invention provides a machine vision-based droplet detection system, comprising an image acquisition module, a contour extraction module, a feature extraction and analysis module, a detection adjustment module and a morphology analysis module; the image acquisition module is used to photograph droplets on water-sensitive paper with a multispectral camera to obtain a multispectral droplet image and pre-process the multispectral droplet image; the contour extraction module is used to extract droplet contours from the pre-processed multispectral droplet image using a contour detection algorithm, and analyze basic feature data of each droplet contour based on the extracted droplet contour; the feature extraction and analysis module is used to extract the droplet contours from the pre-processed multispectral droplet image using a contour detection algorithm and analyze basic feature data of each droplet contour based on the extracted droplet contour; the feature extraction and analysis module is used to extract the droplet contours from the pre-processed multispectral droplet image using a DBSCAN algorithm The method is used to classify the basic feature data of the droplet contour, mark the abnormal droplets, and record the position information of the abnormal droplets; the detection and adjustment module is used to capture the droplet images on the water-sensitive paper at different viewing angles, adjust the camera parameter settings based on the position information of the abnormal droplets, and collect multi-view droplet images; the three-dimensional reconstruction module is used to construct a disparity map based on the multi-view droplet images, and generate three-dimensional point cloud data by analyzing the disparity map and camera parameters using the triangulation principle; the morphology analysis module is used to generate a triangular mesh model of droplets on the surface of water-sensitive paper based on the three-dimensional point cloud data using the Poisson surface reconstruction method, and analyze the droplet morphology.

[0052] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the machine vision-based droplet detection method as described in the first aspect of the present invention is implemented.

[0053] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the machine vision-based droplet detection method as described in the first aspect of the present invention is implemented.

[0054] The beneficial effects of the present invention are as follows: by constructing a disparity map based on multi-view droplet images and generating three-dimensional point cloud data using the triangulation principle, the present invention fully reflects the three-dimensional morphological characteristics of droplets, improving the accuracy and reliability of detection; at the same time, by using the DBSCAN algorithm to classify the basic feature data of the droplet profile and mark abnormal droplets, the robustness and adaptability of abnormal detection are significantly enhanced, thereby solving the problems of incomplete three-dimensional morphological analysis and insufficient robustness of abnormal droplet detection in traditional methods. The present invention improves the accuracy and practicality of droplet detection and provides a more efficient and reliable solution for fields such as agriculture, industry and environmental monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0056] Figure 1 This is a flow chart of the droplet detection method based on machine vision in Example 1.

[0057] Figure 2 This is a flow chart of the machine vision-based droplet detection system in Example 1. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0059] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0060] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0061] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a method for detecting droplets based on machine vision, comprising the following steps:

[0062] S1. Use a multispectral camera to photograph the droplets on the water-sensitive paper to obtain a multispectral droplet image, and pre-process the multispectral droplet image. The specific steps are as follows:

[0063] Adjust the distance between the camera and the water-sensitive paper so that all images are at the same focal length, and simultaneously shoot the droplets on the water-sensitive paper at different bands to obtain multi-spectral droplet images in the same scene;

[0064] De-noising, brightness and contrast correction, adaptive contrast enhancement and binarization processing are performed on multispectral droplet images;

[0065] It should be noted that the multispectral droplet images were subjected to denoising, brightness and contrast correction, adaptive contrast enhancement, and binarization;

[0066] Convolve the image with a Gaussian kernel to smooth pixel values ​​and remove high-frequency noise;

[0067] Adjust the overall brightness and contrast of the image through linear transformation;

[0068] Adaptive histogram equalization (AHE) is used to divide the image into small blocks, and histogram equalization is performed on each small block. The results are then merged using bilinear interpolation, which can effectively enhance the local contrast while avoiding the noise amplification problem caused by over-enhancement.

[0069] Adaptive threshold method is used to dynamically select a suitable threshold according to the mean and standard deviation of the local area, and pixel values ​​greater than the threshold are set to white, and those less than the threshold are set to black.

[0070] S2. Use the contour detection algorithm to extract the droplet contour from the preprocessed multispectral droplet image. Based on the extracted droplet contour, analyze the basic feature data of each droplet contour. The specific operation steps are as follows:

[0071] The Canmy edge detection algorithm is used to extract clear and continuous droplet edge pixels;

[0072] According to the characteristics of multispectral droplet images, a high threshold T is defined high and low threshold T low ;

[0073] The Sobel operator is used to calculate the gradient amplitude of the multispectral droplet image. The expression is:

[0074]

[0075] Among them, G(x, y) is the gradient amplitude, G x is the horizontal gradient component, G y is the vertical gradient component, x is the horizontal position of the pixel in the image, and y is the vertical position of the pixel in the image;

[0076] The Sobel operator is used to calculate the gradient direction angle of the multispectral droplet image. The expression is:

[0077]

[0078] Among them, θ(x, y) is the gradient direction angle;

[0079] According to the gradient direction angle, the gradient amplitude of each pixel is compared with the gradient amplitude of its adjacent pixel points through non-maximum suppression, the local maximum value is retained, and the edge is refined;

[0080] The gradient amplitude is greater than the high threshold T high The pixel point is marked as a strong edge, between the high threshold T high and low threshold T low The pixels between are marked as weak edges, which are less than the low threshold T low The pixel points are marked as non-edge;

[0081] Through connectivity analysis, the strong edges are connected, and the weak edges connected to the strong edges are also marked as edges to generate a binary image of the droplet edge;

[0082] Use contour detection algorithm to extract droplet contour from droplet edge binary image and record the position information of droplet contour;

[0083] Based on the extracted droplet contour, the area of ​​the droplet contour is calculated using Green's formula, the perimeter of the droplet contour is calculated using the Euclidean distance, the circularity of the droplet contour is calculated using the area and perimeter, the eccentricity of the droplet contour is calculated using the length of the major axis and the minor axis, and the rectangularity is calculated using the width and height of the minimum circumscribed rectangle, and the basic characteristic data of each droplet is output;

[0084] It should be noted that the characteristics of multispectral droplet images are obtained by analyzing and counting the droplet images under different lighting conditions;

[0085] It should also be noted that the basic characteristic data calculation formula for each droplet is:

[0086]

[0087] Where A is the droplet contour area, m is the index variable for summation, and x m is the x coordinate of the mth vertex, y m is the y coordinate of the mth vertex, n is the number of vertices of the contour;

[0088]

[0089] Where P is the perimeter of the droplet contour;

[0090]

[0091] Where C is the circularity of the droplet profile and π is the circumference of a circle;

[0092]

[0093] Where e is the eccentricity of the droplet profile, a is the major axis length of the profile, and b is the minor axis length of the profile;

[0094] Calculate the rectangularity by the width and height of the minimum enclosing rectangle;

[0095]

[0096] Where R is the rectangularity of the droplet outline, w is the width of the minimum circumscribed rectangle surrounding the outline, and h is the height of the minimum circumscribed rectangle surrounding the outline.

[0097] S3. Use the DBSCAN algorithm to classify the basic feature data of the droplet profile, mark abnormal droplets, and record the location information of abnormal droplets. The specific steps are as follows:

[0098] Use the Z-score standardization method to standardize the basic feature data;

[0099] The standardized basic feature data is input into the DBSCAN algorithm, which will automatically group the data points into different clusters and use the comprehensive anomaly scoring function to score the droplet anomaly for different clusters. The expression is:

[0100]

[0101] Among them, F(S i ) is the droplet anomaly score, S i represents the i-th sample, k is the number of features, and w j is the weight of the jth feature, S ij represents the jth eigenvalue of the i-th sample, μ j is the mean of the jth feature, λ is the balance coefficient, j is the feature index, i is the sample index, LOF(S i ) is the ratio of the local density of the sample to the local density of its neighbors;

[0102] Based on the distribution of the comprehensive anomaly score, a threshold T is set;

[0103] When the comprehensive abnormality score F(S i )>threshold T, mark it as abnormal droplet and record the location information of the abnormal droplet;

[0104] When the comprehensive abnormality score F(S i )≤threshold T, marked as normal droplets;

[0105] It should be noted that the comprehensive anomaly scoring function is defined by the normalized value of droplet characteristics and the ratio of droplet local density;

[0106] The droplet local density ratio is calculated by calculating the ratio between the local density of each droplet in the neighborhood of the data point and the average local density of other data points within the threshold;

[0107] By selecting a suitable distance threshold, for each data point, the Euclidean distance between it and all other data points is calculated, and the number of all points whose distance is less than or equal to the distance threshold is counted to obtain the local density of droplets, and all points whose distance is less than or equal to the distance threshold are classified as the neighborhood of the data point;

[0108] It should also be noted that the distribution of the comprehensive anomaly score is obtained by statistically analyzing the characteristic data of all droplet profiles.

[0109] S4, capture the fog drop images on the water-sensitive paper at different viewing angles, adjust the camera parameter settings based on the location information of the abnormal fog droplets, and collect multi-view fog drop images. The specific operation steps are as follows:

[0110] Use a high-resolution, multi-spectral imaging industrial camera to simultaneously capture multi-view droplet images on water-sensitive paper at the same time;

[0111] Based on the location information of abnormal fog droplets, the camera angle adjustment value and focal length adjustment value are calculated through geometric calculation, the angle and focal length of each camera are adjusted, and the parameters of each camera are reconfigured so that all cameras can shoot synchronously at the same time and re-collect multi-view fog droplet images;

[0112] It should be noted that the camera angle adjustment value and focal length adjustment value are expressed as:

[0113]

[0114] Among them, θ opt is the camera angle adjustment value, y avg is the average y coordinate of the center of the abnormal droplet, x avg is the average x coordinate of the center of the abnormal droplets, x c and c is the current camera position coordinate, α is the fine-tuning angle coefficient;

[0115]

[0116] Among them, f opt is the focal length adjustment value, D is the distance between the camera and the water-sensitive paper, d is the diameter of the abnormal droplet area, and L is the width of the camera sensor.

[0117] S5. Construct a disparity map based on the multi-view droplet image, and generate three-dimensional point cloud data by analyzing the disparity map and camera parameters using the triangulation principle. The specific steps are as follows:

[0118] The SIFT feature detection algorithm is used to extract feature points in multi-view droplet images, the RANSAC algorithm is used to match feature points and the homography matrix is ​​used to register images, and the disparity map is generated through the semi-global matching algorithm;

[0119] The process that needs to be explained is that, first, feature points are extracted from multi-view droplet images, and the RANSAC algorithm is used to match the feature points to find consistent feature point pairs; then, based on these matched feature point pairs, the homography matrix is ​​calculated to align the images from different viewpoints to the same coordinate system; then, the semi-global matching algorithm is used to perform pixel-level matching on the aligned images to generate a disparity map;

[0120] According to the disparity map and camera parameters, the three-dimensional coordinates of each pixel are analyzed using the triangulation principle;

[0121] It should be noted that the corresponding position of each pixel in the left and right images is determined by combining the known camera intrinsic parameters (such as focal length, principal point coordinates) and extrinsic parameters (such as camera position and posture) with the disparity value in the disparity map; then, based on these corresponding positions and camera parameters, the coordinates of each pixel in three-dimensional space are determined using the triangulation formula;

[0122] All calculated three-dimensional coordinate points are converted into NumPy arrays by generating a three-dimensional coordinate point list, and invalid points are removed and saved in a standard point cloud format to form a complete three-dimensional point cloud data.

[0123] S6. Use the Poisson surface reconstruction method to generate a three-dimensional model of droplets on the surface of water-sensitive paper based on the three-dimensional point cloud data and analyze the droplet morphology. The specific steps are as follows:

[0124] Preprocess the 3D point data and use the Poisson surface reconstruction method to generate a 3D model of droplets on the surface of water-sensitive paper;

[0125] Use the Marching Cubes algorithm to extract isosurfaces from implicit surface functions and generate triangular meshes consisting of vertices and triangular faces, recording the coordinates of each vertex in three-dimensional space and the vertex index information of each triangular face;

[0126] Calculate the normal direction of each vertex and each triangular face, assign material attributes and texture coordinates to the triangular mesh through the lighting algorithm, save the triangular mesh and additional information into a standard 3D model file format, load and verify it through a 3D visualization tool, and generate a 3D model of the droplet;

[0127] It should be noted that the lighting algorithm first calculates the lighting intensity of each vertex, which includes the contributions of diffuse reflection, specular reflection and ambient light. Specifically, the intensity of diffuse light is calculated using the normal vector and the light source direction vector, the intensity of specular light is calculated using the normal vector, the light source direction vector and the viewing angle direction vector, and the total lighting intensity is obtained by combining the intensity of ambient light. Then, the final color value is assigned to each vertex based on the calculated lighting intensity and predefined material properties (such as diffuse color, specular reflection coefficient, glossiness, etc.). At the same time, texture coordinates are assigned to each vertex of the triangular mesh so that the texture image can be correctly mapped during rendering. This information is saved together in the data structure of the triangular mesh for subsequent 3D visualization and rendering;

[0128] Based on the three-dimensional model of the droplet, the area of ​​each droplet is segmented from the three-dimensional model of the droplet through connected component analysis;

[0129] Based on the segmented droplet area, analyze the droplet morphology, including shape, volume and change trend;

[0130] The main axis direction of the droplet is determined by PCA, the degree of roundness close to spherical shape is calculated, the convex hull is calculated and the difference with the actual contour is analyzed to evaluate the degree of concavity, the smoothness of the boundary is measured by the standard deviation of the boundary curvature, and the droplet shape is extracted and described;

[0131] It should be noted that droplet morphology includes shape, volume and change trend;

[0132] The volume of the droplet is calculated by integrating the triangular mesh, and the expression is:

[0133]

[0134] Where V is the total volume of the droplets, N is the number of triangles in the triangular mesh, p is the origin, and a q is the first vertex of q triangles, b q is the second vertex of the q triangles, c q is the third vertex of q triangles;

[0135] It should be noted that the expression calculation process is as follows: for each triangular face, first calculate the three vertices a q , b q and c q The vector difference between the origin p and the vector (b q -p) and (c q -p) to obtain a new vector perpendicular to the two vectors. Next, calculate the cross product vector and the vector (a q-p), and take its absolute value, divide this result by 6 to get the directed volume of the triangular face, and finally, sum the directed volumes of all triangular faces to get the total volume V of the entire 3D model.

[0136] In the trend analysis, the 3D data of droplets are collected at different time points and aligned, the volume change curve is calculated, the shape characteristics are compared, the splitting or merging phenomenon is detected by connected component analysis, and the change trend of droplets over time is visually verified using 3D visualization tools;

[0137] It should be noted that the volume change curve is calculated by collecting three-dimensional data of droplets at different time points and aligning them, calculating the volume of the droplets at each time point, and then plotting these volume values ​​as a curve in time sequence, thereby generating a volume change curve;

[0138] This embodiment also provides a fog droplet detection system based on machine vision, including: an image acquisition module, a contour extraction module, a feature extraction and analysis module, a detection adjustment module and a morphology analysis module;

[0139] An image acquisition module is used to photograph the fog droplets on the water-sensitive paper through a multispectral camera to obtain a multispectral fog droplet image and pre-process the multispectral fog droplet image;

[0140] A contour extraction module is used to extract the contour of the fog droplets from the preprocessed multispectral fog droplet image using a contour detection algorithm, and analyze the basic characteristic data of each fog droplet contour based on the extracted fog droplet contour;

[0141] The feature extraction and analysis module is used to classify the basic feature data of the droplet profile through the DBSCAN algorithm, mark abnormal droplets, and record the location information of abnormal droplets;

[0142] The detection and adjustment module is used to capture the fog drop images on the water-sensitive paper at different viewing angles, adjust the camera parameter settings based on the location information of abnormal fog droplets, and collect fog drop images from multiple viewing angles;

[0143] The 3D reconstruction module is used to construct a disparity map based on multi-view droplet images and generate 3D point cloud data by analyzing the disparity map and camera parameters using the triangulation principle;

[0144] The morphology analysis module is used to generate a triangular mesh model of droplets on the surface of water-sensitive paper based on three-dimensional point cloud data using the Poisson surface reconstruction method and analyze the droplet morphology.

[0145] This embodiment also provides a computer device, which is suitable for the case of a droplet detection method based on machine vision, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the droplet detection method based on machine vision proposed in the above embodiment.

[0146] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0147] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for detecting droplets based on machine vision as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0148] In summary, the present invention comprehensively reflects the three-dimensional morphological characteristics of fog droplets by constructing a disparity map based on multi-view fog droplet images and generating three-dimensional point cloud data using the triangulation principle, thereby improving the accuracy and reliability of detection; at the same time, by using the DBSCAN algorithm to classify the basic feature data of fog droplet contours and mark abnormal fog droplets, the robustness and adaptability of abnormal detection are significantly enhanced, thereby solving the problems of incomplete three-dimensional morphological analysis and insufficient robustness of abnormal fog droplet detection in traditional methods. The present invention improves the accuracy and practicality of fog droplet detection, and provides a more efficient and reliable solution for fields such as agriculture, industry, and environmental monitoring.

[0149] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for detecting fog droplets based on machine vision, characterized in that: include, The multispectral droplet images are obtained by photographing the droplets on the water-sensitive paper with a multispectral camera, and the multispectral droplet images are preprocessed; The contour detection algorithm is used to extract the droplet contour from the preprocessed multispectral droplet image, and the basic characteristic data of each droplet contour is analyzed based on the extracted droplet contour; The basic feature data of the droplet profile is classified through the DBSCAN algorithm, abnormal droplets are marked, and the location information of abnormal droplets is recorded; Capture the images of fog droplets on water-sensitive paper at different viewing angles, adjust the camera parameter settings based on the location information of abnormal fog droplets, and collect multi-viewing angle fog droplet images; Based on the multi-view droplet image, a disparity map is constructed, and the three-dimensional point cloud data is generated by analyzing the disparity map and camera parameters using the triangulation principle. The specific steps are as follows: The SIFT feature detection algorithm is used to extract feature points in multi-view droplet images, the RANSAC algorithm is used to match feature points and the homography matrix is ​​used to register images, and the disparity map is generated through the semi-global matching algorithm; According to the disparity map and camera parameters, the three-dimensional coordinates of each pixel are calculated using the triangulation principle; All calculated 3D coordinate points are converted into NumPy arrays by generating a 3D coordinate point list, invalid points are removed and saved in a standard point cloud format to form a complete 3D point cloud data. The Poisson surface reconstruction method is used to generate a triangular mesh model of droplets on the surface of water-sensitive paper based on 3D point cloud data to analyze the droplet morphology. The specific steps are as follows: Preprocess the 3D point cloud data and use the Poisson surface reconstruction method to generate a 3D model of droplets on the surface of water-sensitive paper; Use the Marching Cubes algorithm to extract isosurfaces from implicit surface functions and generate triangular meshes consisting of vertices and triangular faces, recording the coordinates of each vertex in three-dimensional space and the vertex index information of each triangular face; Calculate the normal direction of each vertex and each triangular face, assign material attributes and texture coordinates to the triangular mesh through the lighting algorithm, save the triangular mesh and additional information into a standard 3D model file format, load and verify it through a 3D visualization tool, and generate a 3D model of the droplet; Based on the three-dimensional model of the droplet, the area of ​​each droplet is segmented from the three-dimensional model of the droplet through connected component analysis; Analyze the droplet morphology based on the segmented droplet area; The principal axis direction of the droplet is determined by PCA, the degree of roundness close to sphericity is measured, and the degree of concavity is evaluated by analyzing the difference with the actual contour. The smoothness of the boundary is measured by the standard deviation of the boundary curvature to extract and describe the droplet shape. The volume of the droplet is calculated by integrating the triangular mesh, and the expression is: Where V is the total volume of the droplets, N is the number of triangles in the triangular mesh, p is the origin, and a q is the first vertex of q triangles, b q is the second vertex of the q triangles, c q is the third vertex of q triangles; In the trend analysis, three-dimensional data of droplets are collected at different time points and aligned, the volume change curve is calculated, the shape characteristics are compared, the connected component analysis is used to detect the splitting or merging phenomenon, and the three-dimensional visualization tool is used to intuitively verify the change trend of droplets over time.

2. The method for detecting droplets based on machine vision according to claim 1, characterized in that: The multispectral camera is used to photograph the droplets on the water-sensitive paper to obtain a multispectral droplet image, and the multispectral droplet image is preprocessed. The specific steps are as follows: Adjust the distance between the camera and the water-sensitive paper so that all images are at the same focal length, and simultaneously shoot the droplets on the water-sensitive paper at different bands to obtain multi-spectral droplet images in the same scene; The multispectral droplet image is subjected to denoising, brightness and contrast correction, adaptive contrast enhancement and binarization processing.

3. The method for detecting droplets based on machine vision according to claim 2, characterized in that: The contour detection algorithm is used to extract the droplet contour from the pre-processed multi-spectral droplet image, and based on the extracted droplet contour, the basic feature data of each droplet contour is analyzed. The specific steps are as follows: The Canny edge detection algorithm is used to extract clear and continuous droplet edge pixels; According to the characteristics of multispectral droplet images, a high threshold T is defined high and low threshold T low ; The Sobel operator is used to calculate the gradient amplitude of the multispectral droplet image. The expression is: Among them, G(x,y) is the gradient magnitude, G x is the horizontal gradient component, G y is the vertical gradient component, x is the horizontal position of the pixel in the image, and y is the vertical position of the pixel in the image; The Sobel operator is used to calculate the gradient direction angle of the multispectral droplet image. The expression is: Among them, θ(x,y) is the gradient direction angle; According to the gradient direction angle, the gradient amplitude of each pixel is compared with the gradient amplitude of its adjacent pixels through non-maximum suppression to refine the edge; The gradient amplitude is greater than the high threshold T high The pixel point is marked as a strong edge, between the high threshold T high and low threshold T low The pixels between are marked as weak edges, which are less than the low threshold T low The pixel points are marked as non-edge; Through connectivity analysis, the strong edges are connected, and the weak edges connected to the strong edges are also marked as edges to generate a binary image of the droplet edge; Use contour detection algorithm to extract droplet contour from droplet edge binary image and record the position information of droplet contour; Based on the extracted droplet contour, the area of ​​the droplet contour is calculated using Green's formula, the perimeter of the droplet contour is calculated using the Euclidean distance, the circularity of the droplet contour is calculated using the area and perimeter, the eccentricity of the droplet contour is calculated using the major and minor axis lengths, and the rectangularity is calculated using the width and height of the minimum circumscribed rectangle, and the basic characteristic data of each droplet is output.

4. The method for detecting droplets based on machine vision according to claim 3, characterized in that: The DBSCAN algorithm is used to classify the basic feature data of the droplet profile, mark abnormal droplets, and record the location information of the abnormal droplets. The specific steps are as follows: Use Z-score standardization method to standardize basic feature data; The standardized basic feature data is input into the DBSCAN algorithm, which will automatically group the data points into different clusters and use the comprehensive anomaly scoring function to score the droplet anomaly for different clusters. The expression is: Among them, F(S i ) is the droplet anomaly score, S i represents the i-th sample, k is the number of features, and w j is the weight of the jth feature, S ij represents the jth eigenvalue of the i-th sample, μ j is the mean of the jth feature, λ is the balance coefficient, j is the feature index, i is the sample index, LOF(S i ) is the ratio of the local density of the sample to the local density of its neighbors; Based on the distribution of the comprehensive anomaly score, a threshold T is set; When the comprehensive abnormality score F(S i )>threshold T, mark it as abnormal droplet, and record the location information of the abnormal droplet; When the comprehensive abnormality score F(S i )≤threshold T, marked as normal droplets.

5. The method for detecting droplets based on machine vision according to claim 4, characterized in that: The specific steps of capturing the fog droplet images on the water-sensitive paper at different viewing angles, adjusting the camera parameter settings based on the position information of the abnormal fog droplets, and collecting the fog droplet images from multiple viewing angles are as follows: Use a high-resolution, multi-spectral imaging industrial camera to simultaneously capture multi-view droplet images on water-sensitive paper at the same time; Based on the location information of abnormal fog droplets, the camera angle adjustment value and focal length adjustment value are calculated through geometric calculation, the angle and focal length of each camera are adjusted, and the parameters of each camera are reconfigured so that all cameras can shoot synchronously at the same time and re-collect multi-view fog droplet images.

6. A machine vision-based droplet detection system, based on the machine vision-based droplet detection method according to any one of claims 1 to 5, characterized in that: Including, image acquisition module, contour extraction module, feature extraction and analysis module, detection adjustment module and morphology analysis module; An image acquisition module is used to photograph the fog droplets on the water-sensitive paper through a multispectral camera to obtain a multispectral fog droplet image and pre-process the multispectral fog droplet image; A contour extraction module is used to extract the contour of the fog droplets from the preprocessed multispectral fog droplet image using a contour detection algorithm, and analyze the basic characteristic data of each fog droplet contour based on the extracted fog droplet contour; The feature extraction and analysis module is used to classify the basic feature data of the droplet profile through the DBSCAN algorithm, mark abnormal droplets, and record the location information of abnormal droplets; The detection and adjustment module is used to capture the fog drop images on the water-sensitive paper at different viewing angles, adjust the camera parameter settings based on the location information of abnormal fog droplets, and collect fog drop images from multiple viewing angles; The 3D reconstruction module is used to construct a disparity map based on multi-view droplet images and generate 3D point cloud data by analyzing the disparity map and camera parameters using the triangulation principle; The morphology analysis module is used to generate a triangular mesh model of droplets on the surface of water-sensitive paper based on three-dimensional point cloud data using the Poisson surface reconstruction method and analyze the droplet morphology.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for detecting droplets based on machine vision according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting droplets based on machine vision according to any one of claims 1 to 5 are implemented.

Images