Image-based proportional position correction method, control method, device and industrial vision system

Abstract

The application discloses a kind of based on imaging proportional position deviation correction method, control method, device and industrial vision system, it is related to industrial vision technical field.The application includes: determining imaging reference surface, camera height and physical center point;Obtain target height parameter and initial projection coordinate;Calculate the target position coordinate after rectification.The application does not need to be re-calibrated for different heights camera, also does not need to rely on complex three-dimensional reconstruction or depth measurement equipment, only according to the inherent installation height of camera, optical axis geometric relation and known target height parameter, the imaging position deviation caused by height difference can be directly calculated by proportional model.The calculation process is independent of specific pixel and physical unit conversion relationship, can be directly operated and compensated in the image coordinate level of existing two-dimensional vision system, reduce system complexity and implementation cost, realize efficient reuse and fast deployment in existing vision system.

Description

Technical Field

[0001] This application relates to the technical field of industrial vision, and in particular to an imaging ratio position correction method, control method, device and industrial vision system. Background Technology

[0002] In industrial vision inspection, positioning, and measurement applications, cameras are typically fixed above the equipment to image the working area below. However, in actual production, the targets to be inspected are often not on the same height plane, exhibiting varying degrees of height difference. This causes the imaging position to shift, directly affecting the measurement and positioning accuracy of the 2D vision system. Traditional solutions mainly include recalibrating the camera for different heights, employing complex 3D reconstruction or depth measurement techniques, or introducing multi-camera and depth camera systems. While these methods can alleviate the problem, they generally suffer from limitations such as system complexity, high implementation costs, strong hardware dependence, and difficulty in directly reusing them on existing 2D vision platforms. Summary of the Invention

[0003] The main purpose of this application is to provide an imaging ratio position correction method, control method, device and industrial vision system, which aims to solve the technical problem that the imaging position shift caused by the change of the height of the target object in the monocular two-dimensional vision system, and the traditional solution system is complex, costly and difficult to reuse.

[0004] To achieve the above objectives, this application proposes an imaging ratio-based position correction method for use in an industrial vision system. The industrial vision system includes a camera mounted above the target to be detected, comprising: An imaging reference plane is determined based on the optical center and optical axis of the camera; wherein, the imaging reference plane is perpendicular to the optical axis and the distance between it and the optical center is the camera height, and the intersection of the optical axis and the imaging reference plane is the physical center point; Obtain the target height parameter of the target to be detected relative to the imaging reference surface, and the initial projection coordinates of the target to be detected on the imaging reference surface; The target position coordinates after correction are calculated using the proportional relationship between the optical center of the camera, the physical coordinates of the target to be detected, and the initial projected coordinates; wherein the proportional relationship is generated based on the camera height and target height parameters.

[0005] In one embodiment, the specific steps for obtaining the camera height and physical center point include: A calibration plate is placed on the imaging reference surface, and the camera acquires images of the calibration plate to calibrate the camera and obtain the camera's intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients. The camera height is determined by the translation vector component in the camera's extrinsic parameters, and the physical center point is the perpendicular intersection of the optical axis and the imaging reference plane.

[0006] In one embodiment, the specific steps for obtaining the initial projected coordinates of the target to be detected on the imaging reference plane include: Visual processing is performed on images containing the target to be detected captured by the camera to identify and extract the image pixel coordinates of the target to be detected; By utilizing a pre-defined mapping relationship between pixel coordinates and physical coordinates of the imaging reference surface, the image pixel coordinates are converted into the initial projection coordinates on the imaging reference surface.

[0007] In one embodiment, the proportional relationship is determined based on the principle of collinear similar triangles of the camera optical center, the physical coordinates of the target to be detected, and the initial projected coordinates under the imaging geometry model, and the camera optical center, the physical coordinates of the target to be detected, and its initial projected coordinates on the imaging reference plane are always collinear in space.

[0008] In one embodiment, the formula for calculating the proportional relationship is: k = (H - h) / H; Where k is the scaling factor, H is the camera height, and h is the camera height.

[0009] In one embodiment, the formula for calculating the corrected target position coordinates is: Pq = Pc + k* (Pm - Pc); Where k is the scaling factor, Pq is the target position coordinates after correction, Pm is the initial projection coordinates, and Pc is the physical center point.

[0010] In one embodiment, the step of obtaining the target height parameter of the target to be detected relative to the imaging reference plane, and the initial projected coordinates of the target to be detected on the imaging reference plane, further includes: Lens distortion correction is performed on the initial projection coordinates to eliminate the influence of nonlinear deformation of the camera lens on the imaging geometry.

[0011] Furthermore, to achieve the above objectives, this application also proposes a control method, comprising: The pixel coordinates and physical coordinates are mapped and calibrated on the imaging reference plane to obtain the camera height and physical center point; The target height parameter is dynamically updated based on the change in the target height of the target to be detected. The above-mentioned position correction method based on imaging ratio is invoked to calculate and output the current target correction coordinates in real time. The calculated, corrected target position coordinates are transformed from the physical coordinate system of the imaging reference plane to the base coordinate system of the mechanical actuator and then output to the mechanical actuator.

[0012] In addition, to achieve the above objectives, this application also proposes a control device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method as described above.

[0013] In addition, to achieve the above objectives, this application also proposes an industrial vision system, including the control device as described above, and a mechanical actuator for performing processing or gripping, the mechanical actuator performing precise gripping or processing according to the correction coordinates.

[0014] One or more technical solutions proposed in this application have at least the following technical effects: This application eliminates the need for camera recalibration at different heights and avoids reliance on complex 3D reconstruction or depth measurement equipment. Based solely on the camera's inherent installation height, optical axis geometry, and known target height parameters, it directly calculates the imaging position deviation caused by height differences using a scale model. This calculation process is independent of specific pixel and physical unit conversions, allowing for direct computation and compensation at the image coordinate level of existing 2D vision systems. This reduces system complexity and implementation costs, enhances adaptability to targets with varying heights in production environments, and enables efficient reuse and rapid deployment within existing vision systems. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating an embodiment of an imaging ratio position correction method provided in this application; Figure 2 This is a flowchart illustrating an embodiment of a control method according to the present application. Figure 3 This is a schematic diagram of the imaging geometry model provided in Embodiment 4 of this application; Figure 4 This is another schematic diagram of the imaging geometry model provided in Embodiment 4 of this application.

[0018] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0020] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0021] In industrial vision inspection, positioning, and measurement applications, cameras are typically mounted fixedly above the equipment to image the working area below. This typical two-dimensional vision system relies on a fundamental assumption: the target being inspected lies on a plane at a fixed height parallel to the camera's imaging plane. In this case, the mapping between pixel coordinates and world coordinates established through camera calibration is stable and accurate. However, in real-world industrial scenarios, the targets being inspected are often not at the same height. For example, workpieces of varying thicknesses on a conveyor belt, protruding components in an assembly, or the top and bottom layers of stacked materials exhibit significant height differences on their surfaces. This height variation leads to perspective distortion and scale scaling during imaging, causing the pixel size and position of the same object projected onto the camera at different heights to shift, directly affecting positioning accuracy and the reliability of measurement results.

[0022] The core cause of this problem lies in the camera's imaging model itself. In commonly used pinhole camera models, the projected position of an object on the image plane is closely related to its actual position in three-dimensional space and the camera's intrinsic and extrinsic parameters. When the object shifts along the optical axis (i.e., the direction perpendicular to the imaging plane), its magnification changes accordingly, causing the "pixel-millimeter" ratio pre-calibrated on a single plane to become invalid. Therefore, directly using calibration parameters based on a fixed height to process targets of different heights will introduce systematic measurement errors, making it difficult to meet the requirements of high-precision industrial applications.

[0023] To address the challenges posed by targets at varying heights, three common traditional solutions exist. The first involves multiple calibrations, where a separate camera calibration is performed for each possible target height plane, establishing different parameter lookup tables. While simple in principle, this method is highly discrete, labor-intensive, and inflexible for scenarios with continuously changing heights. The second approach involves complex 3D vision solutions, such as binocular stereo vision, structured light, or laser scanning, to directly obtain the target's spatial 3D coordinates through 3D reconstruction. This method fundamentally solves the problem but typically leads to a significant increase in system complexity, high hardware costs, and higher demands on computing resources. The third approach involves introducing additional hardware, such as configuring multiple fixed-focal-length cameras to focus on different height planes, or directly using a depth camera. These solutions also increase system cost and the difficulty of integration and maintenance.

[0024] In summary, existing solutions generally face some common challenges. While pursuing accuracy or versatility, they often make the entire vision system complex, expensive, and highly dependent on specific hardware. Especially in industrial settings where a large number of traditional 2D vision systems have already been deployed, the above solutions cannot directly reuse existing hardware architectures and software processes. The cost of upgrades and modifications and the risk of downtime are high, thus greatly limiting their versatility and scalability.

[0025] To address the aforementioned problems, this application proposes an imaging ratio-based position correction method for industrial vision systems. The industrial vision system includes a camera mounted above the target to be inspected. Figure 1 As shown, it includes: S10: Determine the imaging reference plane based on the optical center and optical axis of the camera; wherein, the imaging reference plane is perpendicular to the optical axis and the distance between it and the optical center is the camera height, and the intersection of the optical axis and the imaging reference plane is the physical center point; S20: Obtain the target height parameter of the target to be detected relative to the imaging reference plane, and the initial projection coordinates of the target to be detected on the imaging reference plane; S30: Calculate the corrected target position coordinates using the proportional relationship between the camera's optical center, the physical coordinates of the target to be detected, and the initial projected coordinates; where the proportional relationship is generated based on the camera height and target height parameters.

[0026] This application eliminates the need for camera recalibration at different heights and avoids reliance on complex 3D reconstruction or depth measurement equipment. Based solely on the camera's inherent installation height, optical axis geometry, and known target height parameters, it directly calculates the imaging position deviation caused by height differences using a scale model. This calculation process is independent of specific pixel and physical unit conversions, allowing for direct computation and compensation at the image coordinate level of existing 2D vision systems. This reduces system complexity and implementation costs, enhances adaptability to targets with varying heights in production environments, and enables efficient reuse and rapid deployment within existing vision systems.

[0027] Example 1 This embodiment provides a position correction method based on imaging ratio. A camera is fixedly mounted above the target to be detected, used to correct the position coordinates of the target at different heights. The position correction method includes steps S10-30: In step S10, after the camera is installed, the key parameters in the camera's imaging geometry model are first determined, including the position of the camera's optical center and the direction of its optical axis. Based on the camera's optical axis direction, an imaging reference plane is constructed in three-dimensional space. The constraints of the imaging reference plane relative to the optical axis are: the imaging reference plane is perpendicular to the optical axis, and the distance between the imaging reference plane and the optical center is equal to the camera height. The camera height is the spatial distance from the optical center along the optical axis to the reference plane, which can be obtained through installation geometry or calibration. The intersection of the optical axis and the imaging reference plane is defined as the physical center point, which serves as the origin or reference point of the imaging reference plane's coordinates, used to uniformly describe the projected position of the target to be detected on the reference plane.

[0028] Through the above processing, an imaging reference plane coordinate system matching the camera's mounting attitude is established in space. The projected coordinates of the target under detection at different heights, as well as the corrected spatial position coordinates, are all referenced to this imaging reference plane. By using the physical center point as a reference marker on the reference plane, a stable mapping relationship can be established between the camera coordinate system, the pixel coordinate system, and the imaging reference plane coordinate system.

[0029] In step S20, for the current detection condition, the height information and initial projection position of the target to be detected relative to the imaging reference plane are acquired. The target height parameter is the vertical distance of the target's location relative to the imaging reference plane, measured along the optical axis or its approximate direction. The target height parameter can be obtained from an external height sensor, process setting parameters, structural design dimensions, production equipment feedback data, or other measurement methods. This height parameter is used to characterize the spatial deviation of the target to be detected relative to the reference height plane.

[0030] Simultaneously, images of the target to be detected are acquired via a camera, and the pixel coordinates of the target in the image are identified or extracted. Based on a pre-established camera calibration relationship, these pixel coordinates are converted into initial projected coordinates on the imaging reference plane. The initial projected coordinates are the position coordinates of the target on the imaging reference plane calculated based on the camera's intrinsic and extrinsic parameters, assuming the target is at the same height as the imaging reference plane. These coordinates do not consider the height difference between the actual target height and the imaging reference plane. The initial projected coordinates can be obtained by using the homography relationship between the calibrated pixel coordinates and the imaging reference plane coordinate system to transform the pixel position of the target in the image, thus obtaining the corresponding point coordinates of the target on the imaging reference plane. By obtaining the target height parameters and the initial projected coordinates, the positional offset caused by the height difference can be compensated for in subsequent steps based on the camera imaging geometry, resulting in correction coordinates that are closer to the target's true spatial position.

[0031] In step S30, based on the pinhole imaging model with imaging scale, the camera optical center, the initial projection point on the imaging reference plane, and the actual spatial position of the target are considered to be collinear. The line connecting the camera optical center to the actual position point of the target intersects the imaging reference plane at the initial projection point. Since there is a definite geometric ratio between the target height and the camera height, the ratio of the distance between the optical center and the actual position point of the target and the distance between the optical center and the initial projection point can be determined by the camera height and target height parameters.

[0032] Let the camera height be denoted as H, and the target height relative to the imaging reference plane be denoted as h. A definite scaling factor exists between the distance of the target relative to the optical center and the distance of the imaging reference plane relative to the optical center. Based on spatial similarity, the actual position coordinates of the target can be scaled or enlarged from the initial projected coordinates proportionally. That is, the initial projected coordinates are transformed proportionally using the ratio between the camera height and the target height. This proportional relationship is generated based on the camera height and target height parameters and is used to describe the geometric scaling relationship between the target's height plane and the reference height plane.

[0033] With the camera fixed in place, this embodiment requires no additional camera or complex 3D reconstruction device. It simply introduces the target height parameter and combines it with the geometric relationship between the optical center and the reference plane to perform coordinate correction based on a proportional relationship on the initial projected coordinates on the imaging reference plane. This method corrects the actual position of the target in situations with height differences, improving the accuracy of target localization and enhancing the applicability of the monocular 2D vision system in non-coplanar scenes.

[0034] Example 2 This embodiment provides a specific method for obtaining camera height and physical center point, which provides basic parameters for position correction methods. The method involves placing a calibration plate on the imaging reference plane, obtaining the camera's intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients through camera calibration, and determining the camera height and physical center point position based on the extrinsic parameter matrix. The specific steps are as follows: After the camera is installed, the calibration plate is positioned on the imaging reference plane, ensuring that the calibration plate plane coincides with or maintains a known fixed relationship with the imaging reference plane. The calibration plate has regularly distributed calibration feature points, each with known physical coordinates in the calibration plate coordinate system. Image frames containing the calibration plate are acquired using the camera, and the calibration feature points in the images are registered with their corresponding physical points in the calibration plate coordinate system, establishing the correspondence between the image plane coordinates, the calibration plate plane coordinates, and the camera coordinates.

[0035] Based on the above correspondence, a camera calibration algorithm is used to solve for the camera's intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients. The intrinsic parameter matrix describes information such as focal length, principal point position, and pixel scale during camera imaging. The distortion coefficients describe the radial and tangential distortion parameters of the lens. The extrinsic parameter matrix describes the attitude and position of the camera coordinate system relative to the imaging reference plane coordinate system, including rotation matrices and translation vectors. Through the calibration process, a mapping is established between the pixel coordinates in the camera-acquired image and the physical coordinates on the imaging reference plane. This mapping is used for subsequent conversion of image coordinates to imaging reference plane coordinates and for coordinate transformation in position correction calculations.

[0036] After obtaining the camera extrinsic parameter matrix, the translation vector and planar attitude information of the camera coordinate system relative to the imaging reference plane coordinate system are extracted from it. The imaging reference plane can be predefined as a plane in the world coordinate system, and the translation vector component in the extrinsic parameters is the position coordinate of the camera optical center in that plane coordinate system. Based on the camera's mounting structure and coordinate system definition, the vertical distance from the camera optical center to the imaging reference plane can be calculated from the normal component of the translation vector, i.e., the camera height. This height parameter describes the spatial distance between the camera optical center and the imaging reference plane, serving as a reference height in subsequent height compensation and position correction.

[0037] Simultaneously, the expression of the camera's optical axis direction in the imaging reference plane coordinate system can be determined based on the intrinsic and extrinsic parameter matrices. The optical axis is typically defined as a straight line in the camera coordinate system that passes through the optical center and is aligned with the normal to the imaging plane. After transforming the optical axis direction to the imaging reference plane coordinate system using the extrinsic parameter rotation matrix, the direction vector of the optical axis in the imaging reference plane coordinate system can be obtained. Based on the optical axis direction vector and the position of the optical center, the perpendicular intersection point between the optical axis and the imaging reference plane can be calculated in three-dimensional space. This intersection point is defined as the physical center point. The physical center point has definite coordinates in the imaging reference plane coordinate system and can serve as a reference point on the imaging reference plane to establish a unified reference between the image coordinates and the target projection coordinates.

[0038] This embodiment can directly extract the camera height and physical center point coordinates from a single camera calibration, and combine intrinsic parameters, extrinsic parameters, and distortion coefficients to achieve the transformation from pixel coordinates to imaging reference plane coordinates. The camera height is used to establish the calibration benchmark for height compensation, and the physical center point is used to define the reference position of the imaging reference plane coordinate system, providing parameter support for scaling calculations and coordinate mapping in subsequent position correction steps. This embodiment allows for accurate acquisition of camera height and physical center point without adding extra measuring equipment, reducing repetitive measurement steps during installation and debugging, and facilitating rapid deployment of the position correction algorithm in industrial settings.

[0039] Example 3 This embodiment provides a specific method for obtaining the initial projected coordinates of a target to be detected on an imaging reference plane, which provides input coordinate data for subsequent position correction calculations. The method includes performing visual processing on an image containing the target to be detected to obtain image pixel coordinates, and converting the pixel coordinates into initial projected coordinates on the imaging reference plane based on a pre-calibrated mapping relationship. The specific steps are as follows: After the camera acquires an image containing the target to be detected according to a predetermined triggering method, the image is input to the vision processing module. The vision processing module may include steps such as image preprocessing, target detection, feature extraction, contour analysis, and template matching. Depending on the specific application requirements, feature information representing the location of the target to be detected is selected from the image, such as the target's geometric center, corners, edge intersections, positioning hole centers, or other specific landmarks. By processing the target region in the image, the pixel coordinates of the target in the image coordinate system are extracted. Pixel coordinates are represented by row and column indices or by horizontal and vertical coordinates in the image coordinate system. To improve positioning accuracy, sub-pixel-level positioning algorithms can be used as needed to accurately fit feature points within a sub-pixel range to obtain more precise pixel coordinate values. These pixel coordinates represent the position of the target to be detected on the camera's imaging plane, providing input for subsequent coordinate transformation steps.

[0040] Through visual processing steps, recognition algorithms can be flexibly configured in software to suit different target shapes and detection requirements, thereby acquiring the image pixel coordinates of the target without changing the camera's installation status. Accurate extraction of pixel coordinates reduces the impact of feature extraction errors on the final position correction result.

[0041] After camera calibration, the system establishes a mapping relationship between pixel coordinates and physical coordinates on the imaging reference plane. This mapping relationship can be described by a homography matrix, perspective transformation matrix, or a coordinate transformation model obtained by combining camera intrinsic, extrinsic, and distortion parameters. This mapping relationship is used to convert pixel coordinates on the image plane into physical coordinates in the imaging reference plane coordinate system. In this step, the pixel coordinates of the target to be detected obtained in the first step are input into the mapping module. First, distortion correction is performed on the pixel coordinates. Radial and tangential distortions are compensated according to the distortion coefficients obtained from calibration to obtain the ideal imaging coordinates after distortion correction. Then, according to the intrinsic and extrinsic parameter matrices, or according to the homography matrix between the image and the imaging reference plane pre-calculated, coordinate transformation is performed on the corrected image coordinates to calculate the physical coordinates of the feature point in the imaging reference plane coordinate system. These physical coordinates are the initial projection coordinates of the target to be detected on the imaging reference plane. In the calculation of these initial projection coordinates, it is assumed that the target to be detected is at the height of the imaging reference plane, i.e., no target height parameter is introduced for correction. Therefore, the initial projected coordinates do not include compensation for deviations from the actual height of the target; they only reflect the projected position under the reference height plane.

[0042] By mapping the calibration results, the image pixel coordinates of the target to be detected are uniformly transformed into the imaging reference plane coordinate system. This eliminates the coordinate inconsistency caused by differences in camera intrinsic parameters and installation attitude, forming a unified physical coordinate expression. These initial projected coordinates are then combined with the target height parameter to calculate the corrected position coordinates of the target in space according to a proportional relationship during the position correction step. This facilitates the achievement of a unified coordinate standard in multi-batch, multi-station, and multi-equipment environments, and makes it easier to interface and compare with mechanical and process coordinates.

[0043] Example 4 This embodiment describes the calculation method of the scaling factor and the solution process of the target position correction coordinates in the imaging scaling position correction method. By introducing collinear similar triangles into the imaging geometry model, a scaling relationship is constructed between the physical coordinates of the camera optical center, the target to be detected, and the initial projected coordinates, thereby achieving unified correction calculation for the spatial position of targets at different heights.

[0044] In the ideal pinhole imaging model, any point in space, the camera's optical center, and the projection points of that point onto different height planes are always on the same straight line. The distance from the camera's optical center to the imaging reference plane is the camera height H. The imaging reference plane is set perpendicular to the camera's optical axis, and the intersection of the optical axis and the imaging reference plane is the physical center point Pc. The height of the plane containing the target to be detected relative to the imaging reference plane is h, the projection point of the target on the imaging reference plane is Pm, and the correction target point on its true height plane is Pq.

[0045] like Figure 3and Figure 4 As shown, the proportional relationship is determined based on the collinear similar triangle relationship in space between the camera optical center, the physical coordinates of the target to be detected, and the initial projected coordinates. The camera optical center, the target's real space point Pq, and the target's projected point Pm on the imaging reference plane are distributed along the same ray in space. Specifically defined: the camera height H is the vertical distance from the camera optical center along the optical axis to the imaging reference plane; the target height parameter h is the height of the target's height plane relative to the imaging reference plane; the physical center point Pc is the intersection of the camera optical axis and the imaging reference plane; the reference plane projection point Pm is the projected coordinates of the target on the imaging reference plane; and the correction target point Pq is the corresponding coordinate of the target on its real height plane.

[0046] Under this geometric relationship, the distance from the optical center to the imaging reference plane is H, and the distance from the optical center to the target's height plane is H - h. The target point Pq, the reference plane projection point Pm, and the physical center point Pc form two similar triangles relative to the optical center. According to the proportional relationship of the side lengths of similar triangles, we can obtain (Pq - Pc) / (Pm - Pc) = (H - h) / H; where (Pq - Pc) represents the vector of the target's true point relative to the physical center point, and (Pm - Pc) represents the vector of the target's projection point on the reference plane relative to the physical center point. The above proportional relationship indicates that the distance between the target's true point and the reference plane projection point in the direction of the physical center point is scaled proportionally according to the height. Therefore, the scaling factor formula is defined as: k = (H - h) / H; where k is the scaling factor. This scaling factor is determined by the camera height and target height parameters and is used to describe the relative scaling relationship between the reference plane and the target's height plane in the direction of the optical center's line of sight. A scaling factor less than or equal to 1 corresponds to common installation scenarios where the target height is greater than or equal to 0.

[0047] By constructing the above proportional relationships, without changing the camera's mounting attitude, the target's correction position coordinates can be obtained for any spatial point on the target height plane through a linear proportional transformation with the projection point on the reference plane.

[0048] Based on the above proportional relationship, the formula for calculating the target correction point coordinates is: Pq = Pc + k * (Pm - Pc). Substituting the scaling factor expression, it can be written as: Pq = Pc + (H - h) / H * (Pm - Pc). The vector (Pm - Pc) represents the spatial direction and distance of the reference plane projection point relative to the physical center point. By scaling this vector by the scaling factor k and adding it back to the physical center point coordinates Pc, the target's position coordinates Pq on the true height plane can be obtained. During the calculation, the direction of the target in the direction of the physical center point is kept consistent; scaling is only applied to the radial distance according to the height scaling factor, thus reflecting the scale change of the projected coordinates caused by changes in target height. By performing the above calculation in a unified reference coordinate system, the initial projected coordinates based on the imaging reference plane can be mapped to correction coordinates that match the actual height of the target.

[0049] The above formula can be used in any scenario employing a physical unit coordinate system for the imaging reference plane. Pq, Pc, and Pm can all be represented using the same physical unit (e.g., millimeters). The scaling factor is dimensionless and does not depend on a specific unit. This calculation relationship is only related to the height parameters H and h and the spatial geometric position, and is independent of the image resolution and pixel units. Therefore, in this embodiment, during the correction process, the direction of target position correction is always along the radial direction from the physical center point Pc to the projection point Pm on the reference plane. The correction process does not change the azimuth angle of the target relative to the physical center, but only changes the radial distance from the target to the physical center. This method conforms to the line-of-sight projection law of a monocular camera under the pinhole imaging model, which is beneficial for maintaining logical consistency and computational stability. The scaling factor k is determined only by the camera height H and the target height parameter h. For a given camera installation height, changes in the target height are directly reflected in changes in the scaling factor. By adjusting h, the position correction of targets at different heights can be performed according to a unified formula, avoiding the need to establish independent calibration models for different height planes. The correction formula is expressed in the form of vector difference and scaling factor, and the calculation steps are independent of the coordinate units used. As long as Pc, Pm, and Pq use the same physical units, they can all be calculated using the same formula. This method is decoupled from the pixel units in image coordinates and can be directly used with the calibration results of any reference surface expressed in physical units, facilitating unification with machine coordinates and control coordinates. The correction calculation steps only involve vector difference operations and scaling, requiring no additional complex matrix inversions or nonlinear optimizations. Based on the same set of camera calibration results, combined with real-time or pre-acquired target height parameters, the correction coordinate calculation can be completed, making it suitable for real-time industrial vision applications. This embodiment, based on an established imaging reference plane coordinate system, introduces the parametric relationship between camera height and target height to perform unified geometric correction of the target's position on different height planes. This method can improve positioning accuracy in monocular 2D vision systems under varying height scenarios, reduce the impact of height changes on measurement results, and provide reliable coordinate input for subsequent functions such as dimensional measurement, positioning control, and assembly guidance.

[0050] Example 5 In this embodiment, after obtaining the target height parameter of the target to be detected relative to the imaging reference plane and the initial projection coordinates of the target to be detected on the imaging reference plane, a lens distortion correction processing step is added to the initial projection coordinates to reduce the interference of camera lens distortion on the imaging geometry and improve the coordinate accuracy of subsequent position correction calculations.

[0051] The subsequent steps of obtaining the target height parameter of the target to be detected relative to the imaging reference plane, and the initial projection coordinates of the target to be detected on the imaging reference plane, also include: performing lens distortion correction processing on the initial projection coordinates to eliminate the influence of nonlinear deformation of the camera lens on the imaging geometry.

[0052] This can be understood as follows: after acquiring the height parameters of the target to be detected and calculating the initial projected coordinates, distortion correction is first performed on the image coordinates corresponding to these projected coordinates. Distortion correction is based on distortion parameters obtained during camera calibration, including radial and tangential distortion parameters. The distortion correction module maps the original image pixel coordinates from the distorted image plane to the distortion-compensated ideal imaging plane according to the camera imaging model.

[0053] In practice, the original pixel positions of the target object in the image coordinate system can be retained from the initial projection coordinate calculation process. Based on these original pixel coordinates and the calibrated camera intrinsic matrix and distortion coefficients, the distortion-corrected normalized imaging coordinates are calculated using a distortion correction function. This calculation process includes: performing anti-distortion iterative calculations on the normalized coordinates corresponding to the input pixel coordinates according to the distortion model to obtain distortion-free coordinates that conform to the ideal pinhole imaging model. The distortion-free coordinates are then converted into distortion-corrected pixel coordinates using the camera intrinsic matrix.

[0054] After obtaining the distortion-corrected pixel coordinates, the pre-established mapping relationship between pixel coordinates and the physical coordinates of the imaging reference plane is used to convert the distortion-corrected pixel coordinates into corrected projected coordinates on the imaging reference plane. These corrected projected coordinates replace the original initial projected coordinates and serve as the input coordinates in the position correction step. By inserting a distortion correction step before the pixel-to-physical coordinate transformation, the impact of lens distortion on the linearity of the coordinate mapping can be reduced. After distortion correction, the correspondence between the projected coordinates on the imaging reference plane and the actual physical position is closer to the ideal linear model. For targets near the image edge, the positional offset caused by radial distortion is compensated, and the positional error of the target in the reference plane coordinate system is reduced. This processing helps improve the overall coordinate accuracy within the imaging field of view and enhances the system's applicability in large-field-of-view, high-precision scenarios.

[0055] By adding a lens distortion correction step after obtaining the target height parameters and initial projection coordinates, this embodiment reduces the position error caused by lens nonlinear distortion through software compensation without changing the hardware structure. The distortion-corrected reference plane projection coordinates and the target height parameters are then input into the position correction calculation formula to obtain correction coordinates that are closer to the target's true spatial position, improving the measurement stability and repeatability of the imaging scale-based position correction method in engineering applications.

[0056] Example 6 This embodiment, based on the aforementioned imaging-scale position correction method, proposes a control method incorporating a mechanical actuator to achieve online correction and control of the target position under varying target height conditions. This control method is as follows: Figure 2 As shown, it includes the following steps: S100: Perform pixel coordinate and physical coordinate mapping calibration on the imaging reference plane to obtain the camera height and physical center point; S200: Dynamically updates the target height parameter based on changes in the target height of the target to be detected; S300: Call the imaging scale-based position correction method as described above to calculate and output the current target correction coordinates in real time; S400: Transform the calculated corrected target position coordinates from the physical coordinate system of the imaging reference plane to the base coordinate system of the mechanical actuator, and output them to the mechanical actuator.

[0057] This can be understood as follows: In step S100, during the system installation and debugging phase, the calibration board is placed at the imaging reference plane position, and an image containing the calibration board is acquired using a camera. The camera's intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients are solved using a calibration algorithm. Based on the calibration results, a mapping relationship from pixel coordinates to physical coordinates is established between the imaging reference plane coordinate system and the image pixel coordinate system. This mapping relationship can be described using a homography matrix or a transformation matrix calculated from a combination of intrinsic and extrinsic parameters, and is used to subsequently convert the target image coordinates into the imaging reference plane physical coordinates.

[0058] Based on the translation and rotation information in the extrinsic parameter matrix, the position and attitude parameters of the camera optical center in the reference plane coordinate system are extracted from the transformation from the camera coordinate system to the imaging reference plane coordinate system. The camera height is determined by the distance from the optical center to the imaging reference plane along the normal direction, and the physical center point is determined by the intersection of the optical axis and the imaging reference plane, with explicit coordinate values ​​given in the reference plane coordinate system. This step obtains the camera height, physical center point, and the mapping relationship from pixels to physical coordinates, providing basic parameters for subsequent target height parameter updates, position correction, and coordinate transformation.

[0059] In step S200, during system operation, the target to be detected may be at different heights or its height may change with the process. To adapt to this change, a dynamic update mechanism for the target height parameter is introduced into the control method. The system can select an appropriate method to acquire or update the target height parameter according to different application scenarios. For example: based on the production formula or process settings, the corresponding target height parameter is updated when the product model is switched or the process section changes; the height of the target or related reference surface is measured in real time by an external height sensor (such as a laser displacement sensor, encoder feedback, etc.), and the collected height data is used as the current target height parameter; in assembly or handling conditions, the height of the target relative to the imaging reference surface is calculated by combining the preset height information of structures such as fixtures, pallets, and positioning blocks with the current working status of the equipment.

[0060] Before each position correction and mechanical action, the system reads or calculates the latest target height parameters based on the current operating conditions. These parameters are then correlated with the imaging reference plane coordinate system and used as input for the height scaling factor in subsequent position correction calculations. By dynamically updating the target height parameters, the correction calculations can adapt to continuously changing or stepped changes in target height, reducing the impact of height variations on positioning accuracy.

[0061] In step S300, the vision processing module performs target recognition and feature extraction on the image to obtain the pixel coordinates of the target in the image. Through the calibrated pixel-to-physical coordinate mapping, the pixel coordinates are converted into initial projected coordinates on the imaging reference plane, and lens distortion correction is performed as needed to obtain the corrected reference plane projected coordinates. Based on this, the control method calls the aforementioned imaging ratio-based position correction method, inputting the current target height parameters, camera height, physical center point, and initial projected coordinates into the correction calculation module. The correction calculation module calculates the correction coordinates Pq of the target on its true height plane in real time according to the set geometric ratio relationship and correction formula. These correction coordinates, based on the physical coordinate system of the imaging reference plane, represent the target's spatial position after considering the target height difference.

[0062] This step can be executed after each camera acquisition trigger, forming a periodic or event-triggered real-time correction calculation process. By combining the target height parameter with the image processing results, the corrected target position can be output even when the target height changes, reducing position offset errors caused by height differences and providing highly consistent target position information for mechanical actuators.

[0063] In step S400, the coordinates are transformed from the physical coordinate system of the imaging reference plane to the base coordinate system of the mechanical actuator. For example, the base coordinate system of the mechanical actuator can be the robot base coordinate system, the motion platform coordinate system, or the mechanical coordinate system defined by other control systems. To achieve this transformation, during the system installation and debugging phase, hand-eye calibration or coordinate system alignment between the visual coordinate system and the mechanical coordinate system needs to be completed. The rotation and translation relationship between the imaging reference plane coordinate system and the mechanical base coordinate system is obtained through calibration, and a coordinate transformation model between the two coordinate systems is established. In this step, the correction coordinates Pq under the imaging reference plane coordinate system are input into the coordinate transformation module, and the target coordinates under the mechanical actuator base coordinate system are calculated according to the preset coordinate transformation matrix. The transformation process may include translation compensation, rotation transformation, and necessary coordinate axis orientation adjustment. The transformed target coordinates conform to the coordinate convention of the mechanical control system and can be directly used for trajectory planning, pose control, or grasping and positioning.

[0064] The converted target coordinates are output to the mechanical actuator controller via a communication interface. Based on the received target coordinates and the current position of the actuator, the controller generates corresponding motion commands to perform operations such as grasping, placing, assembling, or inspecting the target. Through this coordinate conversion and output process, the vision system and the mechanical actuator are linked, allowing the height-compensated visual positioning results to directly participate in motion control.

[0065] The control method described in this embodiment, based on the calibration of the imaging reference surface and the acquisition of camera parameters, introduces a dynamic update mechanism for the target height parameters. By invoking position correction and coordinate system transformation methods, the corrected target position coordinates are provided to the mechanical actuator in real time. This method reduces coordinate errors between visual measurement and mechanical execution in situations where target height changes, improving the overall positioning control accuracy and operational stability of the system. It is applicable to various industrial automation scenarios such as handling, assembly, sorting, and inspection.

[0066] Example 7 This embodiment further explains the hardware implementation of the control device and its correspondence with the control method, based on the aforementioned control device. The control device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is used to implement the various steps of the control method described in Embodiment Six. The control device can be implemented as a main controller, which can be a DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), MCU (Microcontroller Unit), SOC (System on Chip), etc.

[0067] The basic hardware structure of the control device includes a processor, memory, and interface circuits. The processor executes the computer program stored in the memory, performing comprehensive processing of visual data, target height parameters, camera calibration parameters, and mechanical coordinate transformation data. Different types of main controllers can be selected according to application requirements. The memory stores the program code and data required for the control method, including non-volatile and volatile memory. Non-volatile memory can store control programs, configuration parameters, camera calibration data, hand-eye calibration data, and system logs. Volatile memory is used to cache intermediate data such as image processing results, initial projection coordinates, current target height parameters, correction coordinates, and mechanical coordinates during operation, providing data space for the processor to execute each control step. The control device may include interface circuits for connecting to the camera, external sensors, mechanical actuators, and host computer system, such as Ethernet interfaces, serial bus interfaces, fieldbus interfaces, digital input / output interfaces, and analog-to-digital converter interfaces. The interface circuits provide hardware support for the data acquisition and command issuance functions of the control method.

[0068] When the computer program stored in the memory runs on the processor, it is configured to execute the control method described in Embodiment 6, including: mapping and calibrating pixel coordinates to physical coordinates on the imaging reference plane, and loading camera height and physical center point parameters; dynamically updating the target height parameters according to the target height changes of the target to be detected; calling the imaging ratio-based position correction algorithm module to calculate the current target correction coordinates in real time; transforming the corrected target position coordinates from the physical coordinate system where the imaging reference plane is located to the mechanical actuator base coordinate system, and outputting them to the mechanical actuator through an interface. The above computer program can be implemented in different forms depending on the type of main controller used: running as a firmware on a DSP or MCU, running as an operating system process or service on a SOC, or embedding part of the algorithm into hardware logic units in an FPGA by configuring a bitstream.

[0069] Since the control device of the present invention is used to implement the aforementioned control method, the embodiments of the control device include all the technical solutions of the embodiments of the above-mentioned control method. The control device executes the computer program in the memory through a processor, and under the drive of program instructions, completes the same steps as in Embodiment Six, including: calibration and parameter acquisition of the imaging reference plane, dynamic updating of the target height parameters, position correction calculation based on the imaging ratio, and conversion and output of the correction coordinates to the mechanical base coordinate system. The control device provides computing resources and interface resources at the hardware level, and carries all the algorithmic logic of the control method at the software level. Through the above-mentioned combination of hardware and software, the control device implements a functional flow completely corresponding to the control method during operation, achieving the same technical effects as the embodiments of the control method, including improving positioning accuracy, reducing position errors, and enhancing the system's adaptability to non-coplanar working conditions in scenarios with changing target heights.

[0070] Therefore, the control device embodiment of the present invention corresponds to the control method embodiment described above in terms of technical solutions. The control device implements all the steps of the control method through a specific hardware structure and computer program to achieve the same industrial application effect. The specific algorithm process of each control step will not be described again here.

[0071] Example 8 This embodiment also proposes an industrial vision system. The system includes a control device as described in Embodiment 7 above, and a mechanical actuator communicatively connected to the control device. The mechanical actuator performs processing or grasping operations based on the correction coordinates output by the control device, achieving precise operation on targets at different heights after position compensation.

[0072] An industrial vision system comprises: a control device, a mechanical actuator, a camera and related imaging components, and a communication interface. The control device executes a position correction control method based on imaging ratio, completing visual data processing and target coordinate output. The mechanical actuator receives the correction coordinates output by the control device and performs motion control and task execution in a mechanical coordinate system. The camera and related imaging components acquire images containing the target to be detected and provide the image data to the control device. The communication interface facilitates the exchange of coordinate data and control commands between the control device and the mechanical actuator. After installation and commissioning, these components form a closed-loop control system with visual measurement results as input and mechanical motion as output, achieving an automated process from target imaging and coordinate calculation to mechanical execution.

[0073] The specific structure and program configuration of the control device can be referred to in Embodiment 7. The control device in the industrial vision system mainly performs the following functions: During the system deployment phase, the control device loads camera calibration data, including the definition of the imaging reference plane, camera height, physical center point coordinates, and the mapping relationship from pixels to physical coordinates. During operation, the control device performs coordinate transformation and distortion correction on the acquired images based on the stored parameters, providing a basis for subsequent correction calculations. The control device obtains the target height parameters of the target to be detected based on process configuration, sensor input, or host computer instructions. For batch switching, workstation changes, or continuous changes in target height, the control device dynamically updates the height parameters and uses these parameters as input to the correction formula.

[0074] The control device performs visual processing on the image containing the target captured by the camera to obtain the target pixel coordinates. Through calibration mapping, the pixel coordinates are converted into initial projected coordinates on the imaging reference plane, and distortion correction is performed when necessary. Combining camera height, physical center point, and target height parameters, the control device invokes a position correction algorithm to calculate the corrected target position coordinates according to a preset proportional relationship. The control device transforms the corrected target position coordinates from the imaging reference plane coordinate system to the base coordinate system of the mechanical actuator. The transformation result is output in a format recognizable by the mechanical controller and sent to the mechanical actuator via a communication interface. The control device can also perform coordinate update cycle control and anomaly handling based on mechanical feedback status. In the industrial vision system, the control device realizes the measurement, compensation, and coordinated output of the target position, providing the mechanical actuator with highly compensated target coordinate input.

[0075] Mechanical actuators can be industrial robots, linear motion modules, multi-axis platforms, handling robots, assembly robots, etc. They establish a communication connection with the control device, receive correction coordinates output by the control device, and execute corresponding motion, processing, and grasping operations according to the control strategy. A mechanical actuator mainly includes a drive unit, a transmission mechanism, an end effector, and its own controller. The mechanical actuator performs motion control based on the correction coordinates provided by the control device, enabling the end effector to approach the target's true position even in conditions with height changes, thus achieving grasping or processing of the target.

[0076] By integrating the control device with the mechanical actuator to form an industrial vision system, and employing an imaging-based proportional position correction method, the system outputs correction coordinates when the target height changes, and drives the mechanical actuator to perform operations according to these correction coordinates. This reduces the impact of target height variations on positioning errors. Without adding complex 3D hardware, the system utilizes software calculations and geometric compensation to achieve unified positioning control of targets at different heights. This improves the accuracy and adaptability of automated operations in industrial scenarios and is suitable for various applications such as sorting, assembly, loading / unloading, inspection, and processing.

[0077] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for position correction based on imaging ratio, applied to an industrial vision system, the industrial vision system including a camera, characterized in that, The camera is mounted above the target to be detected, and includes: An imaging reference plane is determined based on the optical center and optical axis of the camera; wherein, the imaging reference plane is perpendicular to the optical axis and the distance between it and the optical center is the camera height, and the intersection of the optical axis and the imaging reference plane is the physical center point; the specific steps for obtaining the camera height and the physical center point include: placing a calibration plate on the imaging reference plane, acquiring an image of the calibration plate through the camera, calibrating the camera, and obtaining the camera's intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients; wherein, the camera height is determined by the translation vector component in the camera's extrinsic parameters, and the physical center point is the perpendicular intersection of the optical axis and the imaging reference plane; The specific steps for obtaining the target height parameter of the target to be detected relative to the imaging reference surface, and the initial projection coordinates of the target to be detected on the imaging reference surface, include: performing visual processing on the image containing the target to be detected captured by the camera, identifying and extracting the image pixel coordinates of the target to be detected; and converting the image pixel coordinates into the initial projection coordinates on the imaging reference surface using a pre-calibrated mapping relationship between pixel coordinates and physical coordinates of the imaging reference surface. The target position coordinates after correction are calculated using the proportional relationship between the optical center of the camera, the physical coordinates of the target to be detected, and the initial projected coordinates. This proportional relationship is determined based on the principle of collinear similar triangles in the imaging geometry model, where the optical center of the camera, the physical coordinates of the target to be detected, and the initial projected coordinates are collinear in space. Furthermore, the optical center of the camera, the physical coordinates of the target to be detected, and its initial projected coordinates on the imaging reference plane are always collinear in space. This proportional relationship is generated based on the camera height and target height parameters. The formula for calculating the proportional relationship is: k = (H - h) / H; Where k is the scaling factor, H is the camera height, and h is the target height parameter; The formula for calculating the corrected target position coordinates is as follows: Pq = Pc + k* (Pm - Pc); Where k is the scaling factor, Pq is the target position coordinates after correction, Pm is the initial projection coordinates, and Pc is the physical center point.

2. The imaging ratio-based position correction method as described in claim 1, characterized in that, The subsequent steps of obtaining the target height parameter of the target to be detected relative to the imaging reference plane, and the initial projected coordinates of the target to be detected on the imaging reference plane, further include: Lens distortion correction is performed on the initial projection coordinates to eliminate the influence of nonlinear deformation of the camera lens on the imaging geometry.

3. A control method, characterized in that, include: The pixel coordinates and physical coordinates are mapped and calibrated on the imaging reference plane to obtain the camera height and physical center point; The target height parameter is dynamically updated based on the change in the target height of the target to be detected. The imaging scale-based position correction method as described in any one of claims 1 to 2 is invoked to calculate and output the current target correction coordinates in real time. The calculated, corrected target position coordinates are transformed from the physical coordinate system of the imaging reference plane to the base coordinate system of the mechanical actuator and then output to the mechanical actuator.

4. A control device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method as claimed in claim 3.

5. An industrial vision system, characterized in that, It includes the control device as described in claim 4, and a mechanical actuator for performing processing or gripping, the mechanical actuator performing precise gripping or processing according to the correction coordinates.

Images