A method, apparatus, equipment, medium, and product for three-dimensional measurement of droplets.

Abstract

This invention relates to the field of inkjet printing technology, specifically disclosing a method, apparatus, device, medium, and product for three-dimensional droplet measurement. By acquiring two images of a droplet from the same nozzle at two different positions along the optical axis of an observation device, the image radius and sharpness are extracted separately. Based on these, the geometric distance and the sharpness-based distance are calculated in parallel, and then fused to output the final three-dimensional measurement result of the droplet. This method effectively overcomes the limitations of the single geometric method being susceptible to imaging noise interference and the single sharpness method relying on calibration accuracy, significantly improving the accuracy and robustness of distance calculation. Based on the fused distance, more accurate and reliable three-dimensional spatial coordinates and physical dimensions of the droplet can be obtained, meeting the high-precision detection requirements for droplet parameters in industrial printing and other fields. Furthermore, this solution has low implementation cost and strong adaptability, providing a reliable technical foundation for comprehensive nozzle state evaluation and printing quality control.

Description

Technical Field

[0001] This invention relates to the field of inkjet printing technology, and in particular to a method, apparatus, equipment, medium, and product for three-dimensional measurement of droplets. Background Technology

[0002] In fields such as inkjet printing, accurate detection of flying droplets is crucial. Existing technologies, such as aerial static photography, typically only take a single image of the droplet at a fixed location. While this method can obtain the droplet's position information on a two-dimensional plane, it cannot directly and accurately measure the droplet's depth position along the optical axis. Consequently, it is impossible to accurately calculate the droplet's true physical dimensions (such as radius and volume), limiting the completeness and accuracy of nozzle condition assessment. Summary of the Invention

[0003] This invention provides a method, apparatus, device, medium, and product for three-dimensional measurement of droplets, in order to solve the problem of poor three-dimensional measurement results of droplets in the prior art.

[0004] According to one aspect of the present invention, a method for three-dimensional measurement of droplets is provided, the method comprising:

[0005] Acquire a first image and a second image of droplets generated by the same nozzle; wherein the first image and the second image are acquired at two different observation positions at a preset distance along the optical axis of the observation device;

[0006] Extract the first image radius and first resolution of the droplet from the first image, and extract the second image radius from the second image;

[0007] Based on a preset distance, the radius of the first image, and the radius of the second image, the object distance is calculated using a geometric method by determining the geometric optical relationship.

[0008] The object distance is calculated using the sharpness method based on the first sharpness and a pre-calibrated object distance-sharpness relationship model; the object distance-sharpness relationship model is used to characterize the mapping relationship between object distance and image sharpness.

[0009] The object distance calculated by the geometric method and the object distance calculated by the resolution method are fused to obtain the fused object distance of the droplet;

[0010] The three-dimensional measurement results of the droplet are determined based on the fused object distance and the position of the droplet in the image.

[0011] According to another aspect of the present invention, a three-dimensional measurement device for droplets is provided, the device comprising:

[0012] The image acquisition module is used to acquire a first image and a second image of the droplets generated by the same nozzle; wherein the first image and the second image are acquired at two different observation positions at a preset distance along the optical axis of the observation device;

[0013] The feature extraction module is used to extract the first image radius and first sharpness of the droplet from the first image, and to extract the second image radius from the second image;

[0014] The geometric calculation module is used to determine the object distance using geometric optical relationships based on a preset distance, a first image radius, and a second image radius.

[0015] The sharpness calculation module is used to determine the object distance calculated by the sharpness method based on the first sharpness and the pre-calibrated object distance-sharpness relationship model; the object distance-sharpness relationship model is used to characterize the mapping relationship between object distance and image sharpness;

[0016] The data fusion module is used to fuse the object distance calculated by the geometric method and the object distance calculated by the resolution method to obtain the fused object distance of the droplet;

[0017] The result determination module is used to determine the three-dimensional measurement results of the droplet based on the fused object distance and the position of the droplet in the image.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the droplet three-dimensional measurement method according to any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the droplet three-dimensional measurement method according to any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the droplet three-dimensional measurement method according to any embodiment of the present invention.

[0024] The technical solution of this invention acquires two images of a droplet from the same nozzle at two different positions along the optical axis of the observation device, extracts the image radius and sharpness respectively, and calculates the object distance using both geometric and sharpness-based methods in parallel. These two methods are then fused to output the final three-dimensional measurement result of the droplet. This method effectively overcomes the limitations of the single geometric method being susceptible to imaging noise interference and the single sharpness method relying on calibration accuracy, significantly improving the accuracy and robustness of the object distance calculation. Based on the fused object distance, more accurate and reliable three-dimensional spatial coordinates and physical dimensions of the droplet can be obtained, meeting the high-precision detection requirements for droplet parameters in fields such as industrial printing. Furthermore, this solution requires no complex hardware upgrades, has low implementation costs, and strong adaptability, providing a reliable technical foundation for comprehensive nozzle status evaluation and printing quality control.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a three-dimensional droplet measurement method provided in Embodiment 1 of the present invention;

[0028] Figure 2 This is a schematic diagram of a three-dimensional coordinate system provided according to Embodiment 1 of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of a three-dimensional droplet measuring device provided in Embodiment 2 of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the three-dimensional droplet measurement method of this invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Figure 1 This is a flowchart of a three-dimensional droplet measurement method provided in Embodiment 1 of the present invention. This embodiment is applicable to the precise three-dimensional measurement of flying droplets during inkjet printing. The method can be executed by a three-dimensional droplet measurement device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown in the figure, the three-dimensional measurement method for droplets provided in this embodiment includes the following steps:

[0035] S110. Acquire a first image and a second image of the droplets generated by the same nozzle; wherein the first image and the second image are acquired at two different observation positions at a preset distance along the optical axis of the observation device.

[0036] In this embodiment, the first image and the second image can refer to imaging data obtained by the observation device from two different observation positions along its optical axis, capturing images of droplets generated from the same nozzle. The observation device can be an imaging device used to capture droplet images, and may include, but is not limited to, image acquisition devices with optical lenses such as industrial cameras and high-speed cameras.

[0037] The optical axis direction refers to the direction pointed to by the straight line connecting the center of the lens and the center of the imaging plane in the optical lens of the observation device. It is the core imaging reference direction of the observation device. For example... Figure 2As shown, in this embodiment, the optical axis is defined as the Y-axis in the three-dimensional coordinate system, representing the depth direction, and its coordinate value is the object distance; the horizontal direction perpendicular to the optical axis is defined as the X-axis, which is usually parallel to the nozzle arrangement direction; the vertical direction perpendicular to the optical axis and the X-axis is defined as the Z-axis, which is consistent with the direction of gravity. The position of the droplet in three-dimensional space can then be represented by coordinates (X, Y, Z); correspondingly, the projection position of the droplet on the image sensor can be represented by pixel coordinates (x, z).

[0038] The droplet three-dimensional measurement device in this embodiment can be deployed as follows: the inkjet head is fixed on a high-precision fine-tuning stage, and its nozzle array plane is adjusted to be perpendicular to the optical axis of the observation device; the observation device can be a camera with a minimum exposure time of 1 microsecond and optical coupling input function, such as a charge-coupled device (CCD) camera, which is mounted on a precision one-dimensional translation stage. The translation stage's movement axis is precisely calibrated to be parallel to the camera's optical axis, i.e., parallel to the defined Y-axis (depth direction), and the camera's field of view is aligned with the droplet ejection area of ​​the nozzle; a point light source is deployed on the same side or opposite side of the camera, and its illumination direction is adjusted to ensure that the droplet flight path is fully illuminated without severe reflection or shadow.

[0039] This solution achieves high-speed, synchronized imaging of the inkjet process through a precisely integrated synchronous control mechanism that coordinates the printhead, point light source, and high-performance CCD camera. The selected CCD camera boasts extremely short exposure times (down to 1 microsecond) and is equipped with an optocoupler input interface, ensuring real-time reception of trigger signals based on the printhead's precise inkjet frequency, and instantaneous synchronization between camera capture, ink jetting, and the illumination of the point light source. By precisely controlling the illumination time of the point light source, this solution ensures sufficient and stable illumination at critical moments of ink ejection, allowing the CCD camera to capture and record clear images of the ink jetting process, providing high-quality raw data for subsequent 3D measurement and nozzle status detection.

[0040] Specifically, based on the droplet three-dimensional measurement device deployed above, the observation device can be controlled to be located at the initial observation position and synchronized with the jetting action of the nozzle to trigger the capture of the first droplet image, i.e., the first image; then, the observation device is driven to move precisely along the optical axis of its lens by a preset distance (a known fixed small distance, such as 0.1 mm), and after reaching the new observation position, it is synchronized with the jetting action of the same nozzle again to trigger the capture of the second image; finally, the first image and the second image with spatial correlation are obtained, and the basic imaging data acquisition for three-dimensional measurement is completed.

[0041] It should be noted that in actual detection, due to the extremely high speed of droplet flight, it is difficult to capture the same droplet after it has moved in a practical system. However, based on the stability and repeatability of the inkjet process, droplets continuously generated from the same nozzle under the same conditions exhibit a high degree of statistical consistency in their physical characteristics (such as size, velocity, initial orientation, etc.) and can be considered as droplets in the same state. Therefore, by photographing these continuous droplets, it is equivalent to photographing the same droplet at different locations, which fully meets the requirements of this solution for data consistency and subsequent calculations, and achieves the same technical effect. This implementation method is a conventional and reasonable conversion that can be made by those skilled in the art based on actual engineering constraints.

[0042] S120, extracting the first image radius and first resolution of the droplet from the first image, and extracting the second image radius from the second image.

[0043] The first image radius and the second image radius can refer to scalar values, determined by extracting the droplet contour in the first image and the second image respectively, used to characterize the visual size of the droplet on the imaging plane, with the unit being pixels.

[0044] First sharpness can refer to a quantitative indicator used to characterize the image sharpness of the droplet target region in the first image. Its value reflects the sharpness of the image in that region.

[0045] In this embodiment of the invention, image processing algorithms can be invoked to process the acquired first and second images respectively to obtain their image radii and sharpness. Specifically, the outline (approximately circular) of the droplet region can be extracted from the two images using algorithms such as edge detection and threshold segmentation, and the corresponding image radius can be determined based on this outline; and the sharpness of the first image can be quantified and calculated using image sharpness evaluation algorithms (such as the Laplace operator, Sobel operator, Tenengrad operator, etc.) to obtain the first sharpness.

[0046] S130. Based on the preset distance, the first image radius, and the second image radius, the object distance is calculated using a geometric method by determining the geometric optical relationship.

[0047] In this context, geometric optics relationships refer to the physical laws describing the relationship between object distance, image distance, focal length, and imaging magnification, based on the lens imaging principle. The core relationship is that the ratio of the image radius to the actual physical radius (magnification) is equal to the ratio of the image distance to the object distance. Since the actual physical radius of the same droplet remains unchanged at two different object distances, the ratio of the image radii at two locations is directly related to the geometric relationship formed by the object distance and image distance at those two locations.

[0048] The geometric method for calculating object distance can refer to the actual distance from the droplet to the first observation position, which is calculated based on geometric optical relationships and known preset distances and dual image radii.

[0049] In this embodiment of the invention, based on the obtained preset distance, first image radius and second image radius, imaging equations can be established at the two observation positions through known geometric optical relationships, such as object-image relationships based on lens imaging. Since the actual physical radius of the same droplet remains unchanged, the true radii at the two observation positions can be established as an equation relationship, and the accurate object distance of the droplet at the first observation position can be finally solved, that is, the object distance calculated by the geometric method is obtained.

[0050] S140. Based on the first sharpness and the pre-calibrated object distance-sharpness relationship model, the sharpness method is used to calculate the object distance; the object distance-sharpness relationship model is used to characterize the mapping relationship between object distance and image sharpness.

[0051] Among them, the object distance-resolution relationship model can refer to a model established through previous calibration experiments to describe the correspondence between the object distance change of the observation device and the resolution value of the acquired image. Its specific implementation forms include, but are not limited to: a quadratic curve model obtained by fitting calibration data, a lookup table or mapping matrix storing several discrete object distance points and their corresponding resolution values, and a machine learning model trained using calibration data.

[0052] The sharpness method for calculating object distance can refer to the estimated distance from the droplet to the first observation position obtained based on the first sharpness obtained, through the object distance-sharpness relationship model.

[0053] In this embodiment of the invention, a pre-calibrated object distance-resolution relationship model can be read, and the object distance can be calculated by mapping the first resolution to the corresponding resolution using the mapping relationship between object distance and image resolution built into the model.

[0054] It's important to understand that while the geometric method and the sharpness method for calculating object distance are based on different physical principles and input data, they both aim at the same physical quantity: the object distance of the droplet relative to the first observation position. The geometric method uses the size change of the droplet image at two positions and the known displacement for calculation, while the sharpness method uses the image sharpness information captured at the first position for mapping and estimation. Therefore, they are independent estimates of the same unknown quantity from different dimensions. This fundamental connection provides the logical premise and theoretical basis for subsequently fusing the two estimates to obtain a better solution (fused object distance).

[0055] S150. The object distance calculated by the geometric method and the object distance calculated by the resolution method are fused to obtain the fused object distance of the droplet.

[0056] Among them, the fused object distance can refer to the final distance estimate that characterizes the droplet to the first observation position, obtained by calculating the object distance using the fused geometry method and the sharpness method.

[0057] In this embodiment of the invention, the weights corresponding to the object distance calculated by the geometric method and the object distance calculated by the sharpness method can be obtained separately, and the object distances calculated by the two methods can be fused based on these weights to obtain a unified fused object distance, providing highly reliable droplet depth information for subsequent three-dimensional measurements. The aforementioned weights can be fixed parameters set in advance according to actual needs, or dynamic parameters determined in real time based on factors such as the goodness of fit of the object distance-sharpness relationship model and the current image quality. This embodiment does not impose any restrictions on this.

[0058] S160. Based on the fused object distance and the position of the droplet in the image, determine the three-dimensional measurement result of the droplet.

[0059] The three-dimensional measurement result can refer to a set of parameters used to characterize the spatial position and shape of a droplet, obtained through pixel coordinate transformation and physical size calculation. These parameters may include, but are not limited to, the droplet's three-dimensional spatial coordinates, physical radius, and volume.

[0060] In this embodiment of the invention, the pixel coordinates (x, z) of the droplet in the first or second image (such as the coordinates of the center point of the droplet region) can be obtained first. Then, based on the known camera calibration parameters (intrinsic and extrinsic parameters) and the fusion object distance, the pixel coordinates (x, z) are converted into X-axis and Z-axis coordinates in three-dimensional space through the camera perspective projection model, and the fusion object distance is used as the Y-axis coordinate of the droplet in three-dimensional space, thereby obtaining the complete three-dimensional spatial coordinates (X, Y, Z) of the droplet. Next, according to the imaging magnification formula in geometric optics, that is, the ratio of the size of the droplet in the image (image radius) to its actual physical size is equal to the ratio of the image distance to the object distance (i.e., the fusion object distance), and the image distance is close to the camera focal length (for fixed-focus cameras, the object distance is usually much larger than the focal length), the actual physical radius of the droplet can be accurately calculated. Furthermore, the volume of the droplet can be calculated based on the spherical assumption, thereby obtaining the complete three-dimensional measurement result of the droplet. Subsequently, the state detection of the nozzle that generates the droplet can be performed based on the three-dimensional measurement result, such as judging whether there is anomalies such as nozzle position displacement or small ink droplet.

[0061] The technical solution of this invention acquires two images of a droplet from the same nozzle at two different positions along the optical axis of the observation device, extracts the image radius and sharpness respectively, and calculates the object distance using both geometric and sharpness-based methods in parallel. These two methods are then fused to output the final three-dimensional measurement result of the droplet. This method effectively overcomes the limitations of the single geometric method being susceptible to imaging noise interference and the single sharpness method relying on calibration accuracy, significantly improving the accuracy and robustness of the object distance calculation. Based on the fused object distance, more accurate and reliable three-dimensional spatial coordinates and physical dimensions of the droplet can be obtained, meeting the high-precision detection requirements for droplet parameters in fields such as industrial printing. Furthermore, this solution requires no complex hardware upgrades, has low implementation costs, and strong adaptability, providing a reliable technical foundation for comprehensive nozzle status evaluation and printing quality control.

[0062] Furthermore, based on the above embodiments of the invention, acquiring a first image and a second image of droplets generated from the same nozzle includes:

[0063] The control observation device is positioned at the first observation position and, in response to the jetting action of the nozzle, triggers and captures the first image of the droplet at the first moment;

[0064] The control observation device is moved a preset distance along the optical axis to the second observation position;

[0065] The control observation device is positioned at the second observation position and, in response to the jetting action of the nozzle, triggers and captures a second image of the droplet at the second moment.

[0066] The jetting action of the nozzle can refer to the process by which the nozzle ejects droplets outward under preset driving conditions such as pressure and voltage. This action can be detected in real time by a sensor and a trigger signal can be generated.

[0067] In this embodiment of the invention, the observation device can be first fixed at the first observation position P1. A synchronous controller coordinates the simultaneous execution of inkjet printing, point light source illumination, and CCD camera capture. Thus, upon receiving the nozzle ejection signal, the observation device is triggered to capture the flying droplet and generate the first image. Then, a control drive mechanism (such as a high-precision one-dimensional translation stage) moves the entire observation device precisely along the optical axis of its lens by a pre-set and known fixed distance. The system reaches the second observation position P2; then, following the aforementioned image acquisition method, at the second moment after receiving the jet action signal from the nozzle, the observation device is triggered to capture the image of the flying droplet and generate the second image.

[0068] Furthermore, based on the above embodiments of the invention, the object distance is calculated using a geometric method based on a preset distance, a first image radius, and a second image radius, determined through geometric optical relationships, including:

[0069] The following formula is used to determine the geometric method for calculating object distance:

[0070]

[0071] in, Represents the geometric method for calculating object distance; Indicates the preset distance; Indicates the radius of the first image; denoted by , where is the radius of the second image; f represents the focal length of the observation device.

[0072] In one embodiment, the derivation of the above formula is as follows:

[0073] Assume the physical radius R of the actual droplet remains constant between the two shots, and the camera focal length f is fixed and known. The object distance at the first observation position P1 is... (To be determined), image distance is The radius of the first image is The camera moves a fixed distance along the optical axis. Upon reaching the first observation position P2, the object distance is... Image distance is The radius of the second image is .

[0074] According to the Gaussian lens formula:

[0075] =>

[0076] In the image, the image radius of the droplet and The relationship with the actual physical radius R is determined by the magnification:

[0077]

[0078] For fixed-focus cameras, the object distance is usually much greater than the focal length, so the image distance is close to the focal length. Therefore, we can conclude that:

[0079]

[0080] Solve using the above formula. (i.e., geometric method for calculating object distance) ):

[0081]

[0082] Furthermore, based on the above embodiments of the invention, the object distance is calculated using the sharpness method based on a first sharpness and a pre-calibrated object distance-sharpness relationship model, including:

[0083] The first sharpness is input into the object distance-sharpness relationship model to obtain the object distance calculated by the sharpness method; wherein, the object distance-sharpness relationship model includes at least a quadratic curve model.

[0084] In this embodiment of the invention, the aforementioned determined first sharpness can be... The data is input into a pre-calibrated object distance-sharpness relationship model. Utilizing the model's built-in mapping relationship between object distance and image sharpness, the first sharpness is mapped to the corresponding sharpness method to calculate the object distance. The object distance-sharpness relationship model can include at least a quadratic curve model, and is expressed as follows:

[0085]

[0086] Where S represents image sharpness; d represents object distance; and a, b, and c represent model parameters.

[0087] It's important to understand that when using a quadratic curve model for inverse solving, for the same input sharpness value... The model may provide two candidate solutions because the physical law governing the change in sharpness with object distance follows a parabolic shape, first increasing and then decreasing. Except for peak points, the same sharpness value typically corresponds to two different object distances located on opposite sides of the focal plane. To ensure the uniqueness and rationality of the measurement, one of these two solutions must be selected as the final output for calculating the object distance using the sharpness method. The selection principle can be based on known constraints or auxiliary information of the physical system. For example, a solution closer to the object distance value initially estimated by the geometric optics method can be selected, or a solution within the normal operating object distance range of the system can be selected, etc. This embodiment does not impose any restrictions on this.

[0088] Furthermore, based on the above embodiments of the invention, the object distance calculated by the geometric method and the object distance calculated by the sharpness method are fused to obtain the fused object distance of the droplet, including:

[0089] Obtain the first weight for calculating object distance using the sharpness method, and the second weight for calculating object distance using the geometric method;

[0090] Based on the first and second weights, the object distance calculated by the sharpness method and the object distance calculated by the geometric method are weighted and fused to obtain the fused object distance.

[0091] In this embodiment of the invention, the weights corresponding to the object distance calculated by the geometric method and the object distance calculated by the sharpness method can be obtained respectively. Then, the object distances calculated by the two methods are weighted and fused based on the weights to obtain the final fused object distance, so as to eliminate the errors that may exist in a single measurement method and improve the measurement accuracy and anti-interference ability of the overall system.

[0092] Furthermore, based on the above embodiments of the invention, obtaining a first weight for calculating the object distance using the sharpness method and a second weight for calculating the object distance using the geometric method includes:

[0093] Obtain the peak sharpness, scale parameters, and coefficient of determination of the object distance-sharpness relationship model;

[0094] The confidence level of the sharpness method is determined based on the first sharpness, peak sharpness, and scale parameters.

[0095] The product of the confidence score and the coefficient of determination in the clarity method is determined as the first weight;

[0096] The difference between the preset value and the first weight is determined as the second weight.

[0097] In this model, peak sharpness refers to the sharpness value at which the image sharpness reaches its maximum value in the object distance-sharpness relationship model; for example, it could be the function value corresponding to the vertex of a quadratic curve model. The scaling parameter is a quantitative parameter used to adjust the rate of decrease in the sharpness confidence score. The coefficient of determination is an index used to evaluate the goodness of fit of the object distance-sharpness relationship model, with a value ranging from [0,1]. The closer the value is to 1, the stronger the model's interpretability of the calibration data, and the more reliable the model. The sharpness confidence score is a quantitative index calculated based on the degree of matching between the current image sharpness (first sharpness) and the peak sharpness, used to characterize the reliability of the sharpness method in calculating object distance.

[0098] In this embodiment of the invention, when fusing object distance calculated using the geometric method and object distance calculated using the sharpness method, this scheme dynamically allocates the weights of the two methods based on the reliability of the object distance-sharpness relationship model and the matching degree of the current image sharpness, thereby ensuring the robustness of the fusion result. Specifically, the process of dynamically obtaining the weights may include:

[0099] (1) From the pre-calibrated object distance-sharpness relationship model, three core parameters are extracted: peak sharpness. (Best sharpness value predicted by the model), scale parameters (Used to quantify the normal fluctuation range of sharpness values) and the coefficient of determination (Used to evaluate the goodness of fit between the model and the calibration data).

[0100] (2) The first sharpness actually measured from the first image With the model's peak resolution Compare and combine scale parameters Calculations are performed to obtain the corresponding sharpness-based confidence score. Among them, the confidence level of the sharpness method This can be expressed by the formula:

[0101]

[0102] (3) Calculate the confidence level of the sharpness method in the previous step. Coefficient of determination of the model Multiplying them together, the resulting product is the first weight. .

[0103] (4) Combine the preset value with the first weight The difference is determined as the second weight. The preset value can be set to 1 to ensure the first weight. With the second weight The sum of is 1, which satisfies the normalization constraint of weight allocation.

[0104] This embodiment dynamically integrates the prior reliability of the model (coefficient of determination) with the real-time data quality (clarity confidence level), making the fusion process no longer fixed or empirical, but possessing intelligent discrimination capabilities. This ensures that when the image is clear and the model matching degree is high, the clarity method is relied upon to pursue accuracy, while when the image quality deteriorates, the more stable geometric method is automatically relied upon. Thus, highly robust object distance measurement results can be output under various complex working conditions, effectively improving the adaptability of the entire detection system to actual engineering environments and the stability of overall performance.

[0105] Furthermore, based on the above embodiments of the invention, the three-dimensional measurement results include at least the three-dimensional spatial coordinates, physical radius, and volume of the droplet; based on the fused object distance and the position of the droplet in the image, the three-dimensional measurement results of the droplet are determined, including:

[0106] Based on preset camera calibration parameters and fusion distance, the pixel coordinates of the center point of the droplet in the first image are converted into X-axis and Z-axis coordinates in three-dimensional space;

[0107] The fused object distance is used as the Y-coordinate of the droplet in three-dimensional space, and the X-axis coordinate, Y-coordinate, and Z-axis coordinate are combined into three-dimensional spatial coordinates; wherein, the optical axis direction is the Y-axis of three-dimensional space;

[0108] The physical radius and volume of the droplet are determined based on the first image radius, the fused object distance, and the focal length of the observation device.

[0109] The preset camera calibration parameters refer to the set of camera parameters obtained in advance through calibration experiments to eliminate camera distortion and establish the mapping relationship between the pixel coordinate system and the three-dimensional world coordinate system. These parameters may include intrinsic parameters (such as focal length, principal point coordinates, distortion coefficients, etc.) and extrinsic parameters (such as rotation matrix, translation vector, etc.). The center point pixel coordinates refer to the pixel coordinates of the center point of the droplet region in the first image, used to characterize the position of the droplet in the first image.

[0110] In this embodiment of the invention, the process of determining the three-dimensional measurement result of the droplet specifically includes:

[0111] (1) Obtain the pixel coordinates (u,v) of the center point of the droplet in the first image.

[0112] (2) Based on preset camera calibration parameters and fusion object distance The center point pixel coordinates (u,v) are transformed into X-axis coordinates (horizontal coordinates) and Z-axis coordinates (vertical coordinates) in three-dimensional space through perspective projection inverse transformation, while the calculated fusion object distance is also transformed. The Y-axis coordinate (depth direction coordinate) of the droplet in three-dimensional space is used to obtain the three-dimensional spatial coordinates (X, Y, Z) describing the droplet's spatial position; where the coordinates of the three axes are represented as follows:

[0113]

[0114] In the formula, Indicates the coordinates of the camera's principal point; and This indicates the equivalent focal length of the camera in the x and y directions.

[0115] (3) Based on the first image radius extracted from the first image , fusion object distance And by inversely calculating the physical radius R of the droplet in real space using the focal length f of the observation device and the magnification formula for lens imaging, we can determine the actual radius R of the droplet in real space.

[0116]

[0117] (4) After obtaining the physical radius R of the droplet, the droplet can be approximated as a sphere, and the volume of the droplet can be calculated using the formula for the volume of a sphere. Finally, a three-dimensional measurement result is obtained, which includes at least the three-dimensional spatial coordinates, physical radius, and volume of the droplet. This result can provide a reliable data basis for subsequent detection of nozzle position deviation, size anomalies, etc.

[0118] Furthermore, based on the above embodiments of the invention, the calibration process of the object distance-sharpness relationship model includes:

[0119] The observation device was moved along the optical axis and calibration images of droplets generated by a normal nozzle were captured at multiple different object distances.

[0120] Determine the sharpness of each calibration image;

[0121] Based on multiple different object distances and their corresponding sharpness, an object distance-sharpness relationship model was obtained by fitting.

[0122] Determine the coefficient of determination, peak sharpness, and scale parameters of the object distance-sharpness relationship model.

[0123] In this context, a normal nozzle refers to a nozzle that is in standard operating condition and whose parameters for ejected droplets all meet preset standards. A calibration image refers to an image collected from droplets generated by a normal nozzle at different object distances, used for model calibration.

[0124] In this embodiment of the invention, the calibration method for the object distance-resolution relationship model used for three-dimensional droplet measurement specifically includes:

[0125] (1) Select a normal nozzle as the calibration source to ensure that it can stably eject droplets with consistent shape and trajectory; then control the observation device to move along the optical axis and at a series of pre-set, known object distances. (n is the number of selected object distance positions) stop sequentially, at each object distance The nozzle is triggered to eject droplets, and the observation device is simultaneously triggered to take pictures, thereby obtaining a set of droplet calibration images at different object distances.

[0126] (2) For each calibration image acquired, an image processing algorithm (such as the Laplace operator) is used to calculate and quantify the image's sharpness. Ultimately, a one-to-one correspondence between object distance and image sharpness is obtained: .

[0127] (3) Due to the depth-of-field characteristics of optical systems, image sharpness typically increases first and then decreases with object distance, reaching its peak at the focus position. Therefore, a quadratic curve can be used to model this relationship. By fitting using the least squares method, the model coefficients a, b, and c are found, such that the curve... The overall deviation from the measured data points is minimized. The final fitted curve is the object distance-sharpness relationship model.

[0128] (4) To quantify the interpretability of the fitted model for real data, calculate the coefficient of determination. as follows:

[0129] Calculate the average sharpness:

[0130] Calculate the total sum of squares:

[0131] Calculate the model's predicted values:

[0132] Calculate the sum of squared residuals:

[0133] Calculate the coefficient of determination:

[0134] Among them, the coefficient of determination The value range is [0,1]. The closer the value is to 1, the higher the model fit, and the more the change in sharpness value can be explained by the object distance through the quadratic curve model.

[0135] (5) For the quadratic curve model The object distance (theoretical optimal focus position) corresponding to its peak point (vertex) and peak resolution It is given by the following formula:

[0136] ,

[0137] (6) In order for the model to dynamically evaluate the reliability of a single measurement in subsequent fusion calculations, a scale parameter needs to be calculated. The specific process is as follows:

[0138] ① First calculate all calibrated sharpness values With model peak resolution Absolute deviation: Then, the median of these deviations is calculated and denoted as . This value reflects the typical dispersion of clarity data around the peak.

[0139] ②Assuming that when the sharpness deviation reaches When the confidence level decreases to an acceptable level, the confidence level formula is:

[0140]

[0141] in, For the real-time detection phase (i.e., when measuring unknown droplets), the sharpness value (e.g., the aforementioned first sharpness) is calculated from the actual captured droplet image. );Other , Then the scale parameter It can be represented as:

[0142]

[0143] For example, when the acceptable change in sharpness corresponds to a confidence level hour, .

[0144] Furthermore, based on the above embodiments of the invention, the method also includes a nozzle state detection process:

[0145] If no droplets are detected in either the first or second image, it is determined that there is an ink non-dispensing abnormality in the nozzle.

[0146] If droplets are detected in both the first and second images, the presence of other types of anomalies in the droplets is determined based on the three-dimensional measurement results of the droplets.

[0147] In this embodiment of the invention, after obtaining the three-dimensional measurement results of the droplet, the working status of the corresponding nozzle can also be detected. The specific process is as follows:

[0148] (1) Analyze the first and second images acquired for the current nozzle to determine whether a valid droplet target was not identified in either of the two images.

[0149] (2) If no valid droplets are detected in either of the two images, it is determined that the nozzle has an ink-not-producing abnormality (usually corresponding to a completely blocked state), and the state detection process for the nozzle ends.

[0150] (3) If effective droplets can be detected in both images, it means that the nozzle is in working condition. Next, the system will call the three-dimensional measurement results of the droplet for multi-dimensional comparative analysis to determine whether there are other types of anomalies in the nozzle, such as positional offset or small ink droplets.

[0151] Furthermore, based on the above embodiments of the invention, the presence of other types of anomalies in the droplet is determined based on the three-dimensional measurement results of the droplet, including at least one of the following:

[0152] If the X-axis coordinate and / or Z-axis coordinate in the three-dimensional space coordinates of the three-dimensional measurement result exceed the preset standard coordinate range, it is determined that there is an abnormal positional offset of the nozzle.

[0153] If the physical radius or volume in the three-dimensional measurement results exceeds the corresponding preset standard range, it is determined that there is an abnormality in the physical quantity of the nozzle.

[0154] Among these, abnormal positional deviation refers to nozzle malfunctions where the droplet ejection position deviates from the normal range due to reasons such as misaligned nozzle installation, loose fixing, or uneven wear. Abnormal physical quantities refer to nozzle malfunctions where the droplet physical radius or volume deviates from the normal range due to reasons such as nozzle blockage, wear expansion, or abnormal injection pressure.

[0155] In this embodiment of the invention, the anomaly detection process based on the three-dimensional measurement results of droplets specifically includes:

[0156] (1) Extract the three-dimensional spatial coordinates from the three-dimensional measurement results, and compare the X-axis coordinates and Z-axis coordinates with the corresponding preset standard coordinate ranges. If any axis coordinate exceeds the preset standard coordinate range, it is determined that the nozzle has an abnormal position offset.

[0157] (2) Extract the physical radius and volume data of the droplets from the three-dimensional measurement results, and compare them with their respective preset standard ranges. If any parameter exceeds the preset standard range, it is determined that there is an abnormality in the physical quantity of the nozzle.

[0158] This embodiment significantly reduces the misjudgment rate of nozzle anomaly detection by using accurate three-dimensional measurement data, providing reliable technical support for ensuring a high-precision inkjet printing process.

[0159] Furthermore, based on the above embodiments of the invention, the method further includes:

[0160] Obtain the three-dimensional spatial coordinates of at least two droplets generated by the nozzle at consecutive moments;

[0161] The flight velocity of the droplets is determined based on the displacement of the three-dimensional spatial coordinates of at least two droplets and the corresponding time interval.

[0162] If the flight speed exceeds the preset speed range, it is determined that there is a speed anomaly in the nozzle.

[0163] In this embodiment of the invention, based on the completion of the three-dimensional coordinate measurement of the droplet, the jet velocity of the nozzle can be further evaluated. The specific process is as follows:

[0164] (1) For the same nozzle to be measured, during the continuous spraying of droplets, the three-dimensional measurement method in the above embodiment is repeatedly executed. Each measurement outputs the precise coordinates (X,Y,Z) of the droplet in the three-dimensional space, thereby obtaining the three-dimensional space coordinate data corresponding to at least two droplets.

[0165] (2) Calculate the spatial displacement of at least two droplets in three-dimensional space, that is, calculate the straight-line distance between two droplets in three-dimensional space using the distance formula between two points in space; extract the difference between the generation times of the two droplets to obtain the time interval; then divide the spatial displacement by the time interval to obtain the flight speed of the droplets.

[0166] (3) Compare the calculated droplet flight speed with the standard speed range (including the lower and / or upper speed limits) set in advance according to the normal nozzle performance. If the measured speed exceeds this preset range, it is determined that there is a speed abnormality in the nozzle.

[0167] This embodiment utilizes high-precision three-dimensional coordinate measurement results to generate dynamic monitoring capabilities for droplet flight speed, enabling a more comprehensive evaluation of nozzle performance. It can not only detect static anomalies such as position and size, but also sensitively detect key dynamic faults such as insufficient jetting power caused by partial nozzle blockage, insufficient driving voltage, or changes in ink viscosity. This provides a deeper diagnostic dimension for preventive maintenance of inkjet systems and printing quality assurance.

[0168] It is understood that the method proposed in this embodiment can be extended to efficiently batch inspect multiple nozzles in an inkjet head. In actual implementation, a predetermined number (e.g., four) of nozzles on the inkjet head can be moved sequentially or according to a set pattern to the fixed field of view center of the observation device through system coordination control. Once the target nozzle group is in place, the system controls these nozzles to simultaneously or sequentially eject droplets, and uses the observation device to capture composite images containing droplets corresponding to all nozzles in the group at two predetermined observation positions along the optical axis. Subsequently, the composite images are independently segmented and identified, and the above-mentioned three-dimensional measurement process and anomaly diagnosis process are executed in parallel for droplets belonging to each nozzle in the image. This approach fully utilizes hardware performance and algorithm flow, realizing the expansion from single-point measurement to parallel batch processing, and significantly improving the overall detection efficiency while ensuring detection accuracy.

[0169] Example 2

[0170] Figure 3 This is a schematic diagram of a three-dimensional droplet measurement device provided in Embodiment 2 of the present invention. Figure 3 As shown, the device includes:

[0171] The image acquisition module 21 is used to acquire a first image and a second image of the droplets generated by the same nozzle; wherein the first image and the second image are acquired at two different observation positions at a preset distance along the optical axis of the observation device;

[0172] The feature extraction module 22 is used to extract the first image radius and first sharpness of the droplet from the first image, and to extract the second image radius from the second image;

[0173] The geometric calculation module 23 is used to determine the object distance using geometric optical relationships based on a preset distance, a first image radius, and a second image radius.

[0174] The sharpness calculation module 24 is used to determine the object distance calculated by the sharpness method based on the first sharpness and the pre-calibrated object distance-sharpness relationship model; the object distance-sharpness relationship model is used to characterize the mapping relationship between object distance and image sharpness;

[0175] The data fusion module 25 is used to fuse the object distance calculated by the geometric method and the object distance calculated by the resolution method to obtain the fused object distance of the droplet;

[0176] The result determination module 26 is used to determine the three-dimensional measurement result of the droplet based on the fused object distance and the position of the droplet in the image.

[0177] Furthermore, based on the above embodiments of the invention, the image acquisition module 21 is specifically used for:

[0178] The control observation device is positioned at the first observation position and, in response to the jetting action of the nozzle, triggers and captures the first image of the droplet at the first moment;

[0179] The control observation device is moved a preset distance along the optical axis to the second observation position;

[0180] The control observation device is positioned at the second observation position and, in response to the jetting action of the nozzle, triggers and captures a second image of the droplet at the second moment.

[0181] Furthermore, based on the above embodiments of the invention, the geometric method calculation module 23 is specifically used for:

[0182] The following formula is used to determine the geometric method for calculating object distance:

[0183] ;

[0184] in, Represents the geometric method for calculating object distance; Indicates the preset distance; Indicates the radius of the first image; denoted by , where is the radius of the second image; f represents the focal length of the observation device.

[0185] Furthermore, based on the above embodiments of the invention, the sharpness calculation module is specifically used for:

[0186] The first sharpness is input into the object distance-sharpness relationship model to obtain the object distance calculated by the sharpness method; wherein, the object distance-sharpness relationship model includes at least a quadratic curve model.

[0187] Furthermore, based on the above embodiments of the invention, the data fusion module 25 is specifically used for:

[0188] Obtain the first weight for calculating object distance using the sharpness method, and the second weight for calculating object distance using the geometric method;

[0189] Based on the first and second weights, the object distance calculated by the sharpness method and the object distance calculated by the geometric method are weighted and fused to obtain the fused object distance.

[0190] Furthermore, based on the above embodiments of the invention, the data fusion module 25 obtains the first weight for calculating the object distance using the sharpness method and the second weight for calculating the object distance using the geometric method, including:

[0191] Obtain the peak sharpness, scale parameters, and coefficient of determination of the object distance-sharpness relationship model;

[0192] The confidence level of the sharpness method is determined based on the first sharpness, peak sharpness, and scale parameters.

[0193] The product of the confidence score and the coefficient of determination in the clarity method is determined as the first weight;

[0194] The difference between the preset value and the first weight is determined as the second weight.

[0195] Furthermore, based on the above embodiments of the invention, the three-dimensional measurement results include at least the three-dimensional spatial coordinates, physical radius, and volume of the droplet; the result determination module 26 is specifically used for:

[0196] Based on preset camera calibration parameters and fusion distance, the pixel coordinates of the center point of the droplet in the first image are converted into X-axis and Z-axis coordinates in three-dimensional space;

[0197] The fused object distance is used as the Y-coordinate of the droplet in three-dimensional space, and the X-axis coordinate, Y-coordinate, and Z-axis coordinate are combined into three-dimensional spatial coordinates; wherein, the optical axis direction is the Y-axis of three-dimensional space;

[0198] The physical radius and volume of the droplet are determined based on the first image radius, the fused object distance, and the focal length of the observation device.

[0199] Furthermore, based on the above embodiments of the invention, the calibration process of the object distance-sharpness relationship model includes:

[0200] The observation device was moved along the optical axis and calibration images of droplets generated by a normal nozzle were captured at multiple different object distances.

[0201] Determine the sharpness of each calibration image;

[0202] Based on multiple different object distances and their corresponding sharpness, an object distance-sharpness relationship model was obtained by fitting.

[0203] Determine the coefficient of determination, peak sharpness, and scale parameters of the object distance-sharpness relationship model.

[0204] Furthermore, based on the above embodiments of the invention, the device also includes a nozzle state detection module, specifically used for:

[0205] If no droplets are detected in either the first or second image, it is determined that there is an ink non-dispensing abnormality in the nozzle.

[0206] If droplets are detected in both the first and second images, the presence of other types of anomalies in the droplets is determined based on the three-dimensional measurement results of the droplets.

[0207] Furthermore, based on the above embodiments of the invention, the presence of other types of anomalies in the droplet is determined based on the three-dimensional measurement results of the droplet, including at least one of the following:

[0208] If the X-axis coordinate and / or Z-axis coordinate in the three-dimensional space coordinates of the three-dimensional measurement result exceed the preset standard coordinate range, it is determined that there is an abnormal positional offset of the nozzle.

[0209] If the physical radius or volume in the three-dimensional measurement results exceeds the corresponding preset standard range, it is determined that there is an abnormality in the physical quantity of the nozzle.

[0210] Furthermore, based on the above embodiments of the invention, the device further includes:

[0211] The coordinate acquisition module is used to acquire the three-dimensional spatial coordinates of at least two droplets generated by the nozzle at consecutive moments.

[0212] The velocity determination module is used to determine the flight velocity of a droplet based on the displacement of the three-dimensional spatial coordinates of at least two droplets and the corresponding time interval.

[0213] The speed detection module is used to determine that there is a speed abnormality in the nozzle if the flight speed exceeds the preset speed range.

[0214] The droplet three-dimensional measurement device provided in the embodiments of the present invention can execute the droplet three-dimensional measurement method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0215] Example 3

[0216] Figure 4 A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0217] like Figure 4As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0218] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0219] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as the droplet three-dimensional measurement method.

[0220] In some embodiments, the droplet three-dimensional measurement method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the droplet three-dimensional measurement method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the droplet three-dimensional measurement method by any other suitable means (e.g., by means of firmware).

[0221] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0222] In some embodiments, the droplet three-dimensional measurement method can be implemented as a computer program, which is implicitly included in a computer program product. When executed by a processor, the computer program implements the droplet three-dimensional measurement method of the present invention. The computer program product can be understood as a software product that primarily implements its solution through a computer program. The computer program used to implement the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a remote machine as a standalone software package, or entirely on a remote machine or server.

[0223] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0224] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0225] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0226] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0227] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0228] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of three-dimensional measurement of a droplet, characterized by, The method includes: Acquire a first image and a second image of droplets generated by the same nozzle; wherein the first image and the second image are acquired at two different observation positions at a preset distance along the optical axis of the observation device; Extract a first image radius and a first sharpness of the droplet from the first image, and extract a second image radius from the second image; Based on the preset distance, the first image radius, and the second image radius, the object distance is calculated using a geometric method by determining geometric optical relationships. The object distance is calculated using the sharpness method based on the first sharpness and the pre-calibrated object distance-sharpness relationship model; the object distance-sharpness relationship model is used to characterize the mapping relationship between object distance and image sharpness; The object distance calculated by the geometric method and the object distance calculated by the resolution method are fused to obtain the fused object distance of the droplet; Based on the fused object distance and the position of the droplet in the image, the three-dimensional measurement result of the droplet is determined. The step of fusing the object distance calculated by the geometric method and the object distance calculated by the resolution method to obtain the fused object distance of the droplet includes: Obtain the first weight for calculating the object distance using the sharpness method, and the second weight for calculating the object distance using the geometric method; Based on the first weight and the second weight, the object distance calculated by the sharpness method and the object distance calculated by the geometric method are weighted and fused to obtain the fused object distance; The step of obtaining the first weight for calculating the object distance using the sharpness method and the second weight for calculating the object distance using the geometric method includes: Obtain the peak sharpness, scale parameter, and coefficient of determination of the object distance-sharpness relationship model; The confidence level of the sharpness method is determined based on the first sharpness, the peak sharpness, and the scale parameter. The product of the confidence level of the sharpness method and the coefficient of determination is determined as the first weight; The difference between the preset value and the first weight is determined as the second weight; The confidence level of the sharpness method is determined using the following formula: ； in, This indicates the confidence level of the sharpness method; This indicates the first level of sharpness; This indicates the peak resolution; This represents the scale parameter.

2. The method according to claim 1, characterized in that, The acquisition of a first image and a second image of droplets generated from the same nozzle includes: The observation device is controlled to be located at a first observation position, and in response to the jetting action of the nozzle, the first image of the droplet is triggered and captured at a first moment; Control the observation device to move the preset distance along the optical axis to the second observation position; The observation device is controlled to be located at the second observation position, and in response to the jetting action of the nozzle, the second image of the droplet is triggered and captured at the second moment.

3. The method according to claim 1, characterized in that, The step of determining the object distance using a geometric method based on the preset distance, the first image radius, and the second image radius through geometric optics relationships includes: The object distance calculated using the geometric method is determined using the following formula: ； in, This indicates that the geometric method is used to calculate the object distance; This represents the preset distance; Indicates the radius of the first image; denoted by , where is the radius of the second image; and f represents the focal length of the observation device.

4. The method according to claim 1, characterized in that, The method of determining the object distance using the sharpness method based on the first sharpness and the pre-calibrated object distance-sharpness relationship model includes: The first sharpness is input into the object distance-sharpness relationship model to obtain the object distance calculated by the sharpness method; wherein, the object distance-sharpness relationship model includes at least a quadratic curve model.

5. The method according to claim 1, characterized in that, The three-dimensional measurement results include at least the three-dimensional spatial coordinates, physical radius, and volume of the droplet; determining the three-dimensional measurement results of the droplet based on the fused object distance and the position of the droplet in the image includes: Based on the preset camera calibration parameters and the fusion distance, the pixel coordinates of the center point of the droplet in the first image are converted into X-axis and Z-axis coordinates in three-dimensional space. The fusion distance is used as the Y-coordinate of the droplet in three-dimensional space, and the X-axis coordinate, the Y-coordinate, and the Z-axis coordinate are combined to form the three-dimensional spatial coordinates; wherein, the optical axis direction is the Y-axis of the three-dimensional space; The physical radius and volume of the droplet are determined based on the first image radius, the fused object distance, and the focal length of the observation device.

6. The method according to claim 1, characterized in that, The calibration process of the object distance-sharpness relationship model includes: The observation device is controlled to move along the optical axis, and calibration images of droplets generated by a normal nozzle are captured at multiple different object distances. Determine the sharpness of each of the calibration images; Based on the multiple different object distances and their corresponding sharpness, the object distance-sharpness relationship model is fitted to obtain the object distance-sharpness relationship model. Determine the coefficient of determination, peak sharpness, and scale parameter of the object distance-sharpness relationship model.

7. The method according to claim 1, characterized in that, The method also includes a nozzle status detection process: If no droplets are detected in either the first or second image, it is determined that the nozzle has an ink-not-producing abnormality. If droplets are detected in both the first and second images, it is determined whether there are other types of anomalies in the droplets based on the three-dimensional measurement results of the droplets.

8. The method according to claim 7, characterized in that, The determination of whether the droplet has other types of anomalies based on the three-dimensional measurement results includes at least one of the following: If the X-axis coordinate and / or Z-axis coordinate in the three-dimensional space coordinates of the three-dimensional measurement result exceed the preset standard coordinate range, it is determined that the nozzle has an abnormal positional offset. If the physical radius or volume in the three-dimensional measurement result exceeds the corresponding preset standard range, it is determined that the nozzle has an abnormal physical quantity.

9. The method according to claim 1 or 8, characterized in that, The method further includes: Obtain the three-dimensional spatial coordinates of at least two droplets generated by the nozzle at consecutive moments; The flight speed of the droplets is determined based on the displacement of the three-dimensional spatial coordinates of the at least two droplets and the corresponding time interval. If the flight speed exceeds the preset speed range, it is determined that the nozzle has an abnormal speed.

10. A three-dimensional measurement device for droplets, characterized in that, The device includes: An image acquisition module is used to acquire a first image and a second image of droplets generated by the same nozzle; wherein the first image and the second image are acquired at two different observation positions at a preset distance along the optical axis of the observation device; The feature extraction module is used to extract a first image radius and a first sharpness of the droplet from the first image, and to extract a second image radius from the second image; The geometric calculation module is used to determine the object distance using geometric optical relationships based on the preset distance, the first image radius, and the second image radius. The sharpness calculation module is used to determine the object distance calculated by the sharpness method based on the first sharpness and the pre-calibrated object distance-sharpness relationship model; the object distance-sharpness relationship model is used to characterize the mapping relationship between object distance and image sharpness; The data fusion module is used to fuse the object distance calculated by the geometric method and the object distance calculated by the resolution method to obtain the fused object distance of the droplet; The result determination module is used to determine the three-dimensional measurement result of the droplet based on the fused object distance and the position of the droplet in the image; Specifically, the data fusion module is used for: Obtain the first weight for calculating the object distance using the sharpness method, and the second weight for calculating the object distance using the geometric method; Based on the first weight and the second weight, the object distance calculated by the sharpness method and the object distance calculated by the geometric method are weighted and fused to obtain the fused object distance; The data fusion module obtains the first weight for calculating the object distance using the sharpness method and the second weight for calculating the object distance using the geometric method, including: Obtain the peak sharpness, scale parameter, and coefficient of determination of the object distance-sharpness relationship model; The confidence level of the sharpness method is determined based on the first sharpness, the peak sharpness, and the scale parameter. The product of the confidence level of the sharpness method and the coefficient of determination is determined as the first weight; The difference between the preset value and the first weight is determined as the second weight; The confidence level of the sharpness method is determined using the following formula: ； in, This indicates the confidence level of the sharpness method; This indicates the first level of sharpness; This indicates the peak resolution; This represents the scale parameter.

11. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the droplet three-dimensional measurement method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the droplet three-dimensional measurement method according to any one of claims 1-9.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the droplet three-dimensional measurement method according to any one of claims 1-9.

Images