Intelligent pallet fork correction method based on visual inspection

Abstract

The invention discloses an intelligent pallet fork correction method based on visual inspection, and relates to the technical field of visual inspection, a real-time image of a pallet fork is collected through a visual inspection system, the real-time image is converted into a gray level image matrix, the gray level image matrix is compared with a pre-stored design image matrix pixel by pixel, a difference matrix is generated, and the difference matrix is used for correcting the pallet fork. According to the difference matrix, the pose offset of the pallet fork is calculated, the deformation compensation amount of the pallet fork is calculated through a compensation amount estimation model based on the pose offset in combination with the material parameters of the pallet fork, whether the deformation compensation amount exceeds a preset compensation amount threshold value or not is judged, and if yes, a control instruction of the pallet fork is generated based on the deformation compensation amount; and the adaptability of the system under different working environments and load conditions is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of visual detection, and in particular to an intelligent fork calibration method based on visual detection. Background Art

[0002] As a key piece of equipment in modern logistics and warehouse automation, intelligent forks' precise positioning and correction capabilities directly impact operational efficiency and safety. Traditional fork correction systems rely primarily on mechanical limits and simple sensor feedback, lacking the ability to accurately sense the fork's real-time status and intelligently adjust it. In real-world operations, forks often experience misalignment due to uneven loads, material fatigue, and environmental factors. If these misalignments are not corrected promptly, they can lead to decreased cargo handling accuracy, reduced operational efficiency, and even safety incidents.

[0003] The current technical issues in the intelligent fork calibration process are mainly reflected in the following aspects: First, traditional calibration systems lack the ability to fully monitor the fork's overall structure and can only obtain local or discrete status information. In addition, existing calibration algorithms often use simple proportional-integral-derivative (PID) control, lacking in-depth consideration of the fork's material properties and working environment, which limits calibration accuracy and stability.

[0004] To this end, the present invention proposes an intelligent fork calibration method based on visual detection. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an intelligent fork calibration method based on visual inspection, which enhances the adaptability of the system under different working environments and load conditions.

[0006] To achieve the above objectives, an intelligent fork calibration method based on visual inspection is proposed, which includes the following steps:

[0007] Step 1: Use the visual inspection system to collect a real-time image of the fork and convert it into a grayscale image matrix. The grayscale image matrix is ​​a two-dimensional array, where each element represents the grayscale value of the corresponding pixel.

[0008] Step 2: Compare the grayscale image matrix with the pre-stored design image matrix pixel by pixel to generate a difference matrix; the difference matrix is ​​a two-dimensional array, each element of which represents the grayscale difference value of the corresponding pixel point;

[0009] Step 3: Calculate the fork's posture offset based on the difference matrix; the posture offset is a vector containing horizontal displacement, vertical displacement and rotation angle;

[0010] Step 4: Based on the posture offset and the material parameters of the fork, the deformation compensation of the fork is calculated using a compensation estimation model; the deformation compensation is a vector containing compensation values ​​in each direction;

[0011] Step 5: Determine whether the deformation compensation amount exceeds a preset compensation amount threshold. If so, proceed to step 6; otherwise, repeat steps 1 to 5.

[0012] Step 6: Generate a control instruction for the fork based on the deformation compensation amount.

[0013] The method of collecting the real-time image of the fork by the visual detection system is:

[0014] The visual inspection system is installed at a fixed position on the fork and captures real-time images of the working area of ​​the fork through an optical lens;

[0015] The conversion of the real-time image into a grayscale image matrix comprises the following steps:

[0016] Step 11: preprocessing the real-time image collected by the visual inspection system to eliminate image noise and distortion, thereby generating a preprocessed real-time image;

[0017] Step 12: Convert the pre-processed real-time image into a grayscale image to generate grayscale image data;

[0018] Step 13: converting the grayscale image into a grayscale image matrix in the form of a two-dimensional array, wherein each matrix element in the grayscale image matrix corresponds to the grayscale value of a pixel;

[0019] Comparing the grayscale image matrix with the pre-stored design image matrix pixel by pixel to generate a difference matrix comprises the following steps:

[0020] Step 21: spatially aligning the grayscale image matrix with a pre-stored design image matrix to make the grayscale image matrix and the design image matrix correspond in spatial position;

[0021] Step 22: Perform grayscale value normalization on the spatially aligned grayscale image matrix and the design image matrix to eliminate the influence of different lighting conditions;

[0022] Step 23: Compare the normalized grayscale image matrix and the design image matrix pixel by pixel, calculate the grayscale difference values ​​of corresponding pixels, and generate a difference matrix;

[0023] The pixel-by-pixel comparison adopts a multi-scale weighted difference strategy. First, the grayscale difference between the corresponding pixels of the grayscale image matrix and the designed image matrix is ​​calculated at the original resolution;

[0024] A Gaussian pyramid is constructed for the grayscale image matrix and the design image matrix, and differences are calculated at multiple scale levels. Low-resolution levels capture large-scale structural changes, while high-resolution levels capture detailed changes. The differences at each scale level are fused through upsampling and weighted summation to generate the final difference matrix. Weight coefficients for this weighted summation are dynamically assigned based on the importance of different scales, with higher weights generally assigned to higher-resolution levels.

[0025] Calculating the fork's posture offset according to the difference matrix includes the following steps:

[0026] Step 31: extracting features from the difference matrix to identify key structural areas of the fork;

[0027] The feature extraction method combines region segmentation with edge detection. First, based on the grayscale distribution characteristics of the difference matrix, the adaptive threshold method is used to binarize the difference matrix and separate the significant difference areas.

[0028] The feature extraction also includes edge detection processing; that is, applying the Canny edge detection algorithm to the difference matrix to extract the structural edge lines of the significant difference area;

[0029] Step 32: Based on the extracted structural features, the position offset and angle deviation of the fork are calculated for each key structural area;

[0030] The position offset calculation uses a weighted centroid method. For each identified critical structural region Ri, its centroid coordinates (xi, yi) and area Ai are calculated. The critical structural regions are then weighted based on their area and degree of variation to calculate the overall position offset, expressed as a horizontal displacement Δx and a vertical displacement Δy, respectively, representing the difference between the weighted average of the centroid coordinates of all deformed regions and the reference position.

[0031] Step 33: Generate a pose offset vector including horizontal displacement, vertical displacement and rotation angle for each key structural area based on the position offset and angle deviation;

[0032] Calculating the deformation compensation amount of the fork by using the compensation amount estimation model includes the following steps:

[0033] Step 41: Establish a material property database based on the material parameters of the fork;

[0034] Step 42: Based on the posture offset vector, a force analysis model of each key structural area in the fork is constructed, and the stress distribution of key points in each key structural area is calculated;

[0035] Step 43: Evaluate the deformation state of each key structural area of ​​the fork according to the stress distribution and the material property database, and distinguish between elastic deformation areas and plastic deformation areas;

[0036] For each calculation point on the fork, the calculation point is the centroid coordinate of each identified critical structural area, and its von Mises equivalent stress σvm is calculated and compared with the yield strength σy of the material. When σvm<σy, the point is judged to be in an elastic deformation state; when σvm≥σy, the calculation point is judged to have entered a plastic deformation state. When the calculation point is in an elastic deformation state, the corresponding critical structural area is judged to be in an elastic deformation state; otherwise, the corresponding critical structural area is judged to be in a plastic deformation zone.

[0037] Then, based on the deformation state of each calculation point, each key structural area is divided into elastic deformation area and plastic deformation area; the area division result is represented by a binary labeling matrix, and each element in the matrix corresponds to a calculation point on the fork. The value of 0 indicates elastic deformation, and the value of 1 indicates plastic deformation.

[0038] Step 44: Based on the evaluation results of the deformation state, a compensation estimation model is constructed to calculate the deformation compensation vector of each key structural area;

[0039] Step 45: Using a weighted fusion method to optimize and adjust the deformation compensation vector to generate the final deformation compensation;

[0040] The method for determining whether the deformation compensation amount exceeds the preset compensation amount threshold is:

[0041] For any time point, a method combining component comparison and comprehensive evaluation is used to determine whether the time point needs to be corrected;

[0042] If correction is determined to be required at n consecutive time points, the deformation compensation amount is determined to exceed the preset compensation amount threshold and correction is required;

[0043] The method of generating the correction control instruction of the fork based on the deformation compensation amount is:

[0044] Pre-establishing the mapping relationship between the deformation compensation amount and the actuator control parameter vector;

[0045] Based on the mapping relationship and the deformation compensation amount, a corresponding control instruction is generated.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The visual inspection system captures real-time images of the forks and converts them into grayscale image matrices. This image matrix is ​​then compared pixel by pixel with a pre-stored design image matrix to generate a difference matrix. This matrix then calculates a pose offset vector, encompassing horizontal and vertical displacements and rotation angles. This offset is then combined with the fork's material parameters and a compensation estimation model to calculate deformation compensation. When the compensation exceeds a preset threshold, a corresponding control command is generated for correction. This significantly improves fork positioning accuracy and operational efficiency, enhancing the system's adaptability to varying working environments and load conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of an intelligent fork calibration method based on visual inspection in Example 1 of the present invention;

[0049] Figure 2 This is a flow chart of generating a difference matrix in Example 1 of the present invention;

[0050] Figure 3 This is a flowchart for calculating the posture offset of the fork in Example 1 of the present invention;

[0051] Figure 4 This is a flow chart for calculating the deformation compensation amount of the fork in Example 1 of the present invention. DETAILED DESCRIPTION

[0052] In modern intelligent logistics and warehouse automation, high-precision cargo handling and stacking are key to improving overall efficiency. However, traditional forklift systems often suffer from positional deviations when handling irregular cargo, high-density storage environments, and high-speed operations, leading to reduced efficiency and potential safety hazards. This is particularly true in cold chain logistics, precision electronics manufacturing, and hazardous materials handling, where even higher requirements are placed on fork positioning accuracy and reliability.

[0053] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] like Figure 1 As shown, an intelligent fork calibration method based on visual inspection includes the following steps:

[0055] Step 1: Use the visual inspection system to collect a real-time image of the fork and convert it into a grayscale image matrix. The grayscale image matrix is ​​a two-dimensional array, where each element represents the grayscale value of the corresponding pixel.

[0056] Step 2: Compare the grayscale image matrix with the pre-stored design image matrix pixel by pixel to generate a difference matrix; the difference matrix is ​​a two-dimensional array, each element of which represents the grayscale difference value of the corresponding pixel point;

[0057] Step 3: Calculate the fork's posture offset based on the difference matrix; the posture offset is a vector containing horizontal displacement, vertical displacement and rotation angle;

[0058] Step 4: Based on the posture offset and the material parameters of the fork, the deformation compensation of the fork is calculated using a compensation estimation model; the deformation compensation is a vector containing compensation values ​​in each direction;

[0059] Step 5: Determine whether the deformation compensation amount exceeds a preset compensation amount threshold. If so, proceed to step 6; otherwise, repeat steps 1 to 5.

[0060] Step 6: Generate a control instruction for the fork based on the deformation compensation amount.

[0061] In an embodiment of the present invention, the method of collecting the real-time image of the fork by the visual inspection system is:

[0062] The visual inspection system is installed at a fixed position on the fork and captures real-time images of the working area of ​​the fork through an optical lens;

[0063] In an embodiment of the present invention, the visual inspection system uses an industrial camera as an image acquisition device, and the fixed position is selected according to the specific physical structure of the fork to ensure that the real-time image captured by the industrial camera at the fixed position can cover the working area of ​​the fork;

[0064] It should be further noted that the resolution and frame rate of the real-time image are configured according to the movement speed and accuracy requirements of the fork, and the exposure time and gain parameters of the industrial camera are optimized through a dynamic adjustment algorithm to adapt to the image acquisition requirements under different lighting conditions.

[0065] Furthermore, converting the real-time image into a grayscale image matrix includes the following steps:

[0066] Step 11: preprocessing the real-time image collected by the visual inspection system to eliminate image noise and distortion, thereby generating a preprocessed real-time image;

[0067] Specifically, the preprocessing strategy employs a multi-stage filtering strategy. First, a Gaussian filter is used to eliminate random noise. The filter kernel size is dynamically adjusted based on the resolution of the real-time image to balance filtering effectiveness with image detail preservation. Secondly, an adaptive median filter is applied to remove impulse noise from the real-time image, such as outlier pixel values ​​generated by uneven lighting conditions. Finally, a bilateral filtering algorithm is used for edge-preserving smoothing, preserving the fork's edge contours while smoothing homogeneous areas.

[0068] In a further embodiment of the present invention, the preprocessing may also include distortion correction. For example, based on a pre-calibrated camera intrinsic parameter matrix and distortion coefficients, a polynomial distortion model is applied to geometrically correct the image to eliminate barrel or pincushion distortion caused by the lens. The distortion correction process employs a bilinear interpolation algorithm to resample pixel values ​​to adjust the geometric accuracy of the distortion-corrected image.

[0069] It should be noted that the specific parameter configuration of the pre-processing is adaptively adjusted according to the lighting conditions of the working environment and the surface characteristics of the fork. For example, by analyzing the image histogram and contrast index, the filter parameters and correction coefficients are dynamically adjusted to ensure stable image quality under different environmental conditions. The present invention will not be further described here.

[0070] Step 12: Convert the pre-processed real-time image into a grayscale image to generate grayscale image data;

[0071] Specifically, the grayscale image is converted using a weighted average method;

[0072] In an embodiment of the present invention, based on the human eye's sensitivity to different colors, the weighted averaging method calculates the corresponding grayscale value by weighting the RGB channel pixel values ​​in the real-time image according to a weight ratio of 0.299R + 0.587G + 0.114B. This conversion method preserves the image's brightness information and texture characteristics while reducing data dimensionality.

[0073] In a preferred embodiment of the present invention, the grayscale image conversion process also includes contrast enhancement. For example, histogram equalization techniques are used to stretch the dynamic range of grayscale values, enhance image contrast, and make the structural features and surface details of the forks more distinct. Histogram equalization employs an adaptive local processing strategy, applying different enhancement parameters based on the grayscale distribution characteristics of different image regions to avoid information loss caused by over-enhancement.

[0074] Step 13: converting the grayscale image into a grayscale image matrix in the form of a two-dimensional array, wherein each matrix element in the grayscale image matrix corresponds to the grayscale value of a pixel;

[0075] Specifically, the grayscale image matrix uses row-first storage, with the number of rows and columns corresponding to the image's height and width, respectively. Each matrix element stores the grayscale value of the corresponding pixel position, ranging from 0 to 255, where 0 represents pure black and 255 represents pure white. The grayscale image matrix is ​​indexed as M(i 0, j 0), where i 0 represents the row index and j 0 represents the column index, with the origin located at the upper left corner of the image.

[0076] In a preferred embodiment of the present invention, border padding can also be performed during the generation of the grayscale image matrix. Specifically, a border region of width k is added around the grayscale image matrix. The fill value of each matrix element in the border region is mirrored, i.e., the edge pixels of the grayscale image are mirrored outward. This avoids border effects during subsequent comparisons and improves comparison accuracy. The width k is determined by the window size for subsequent pixel comparisons and is typically set to twice the window radius.

[0077] In a further preferred embodiment of the present invention, the grayscale image matrix is ​​stored contiguously in memory to improve data access efficiency. A block processing mechanism is implemented for the grayscale image matrix, dividing the large image matrix into multiple overlapping sub-matrices to support parallel computing and increase processing speed. The width of the overlapping regions between the sub-matrices is determined based on the characteristics of the subsequent pixel comparison algorithm to ensure that the block processing results are consistent with the overall processing results.

[0078] Further, such as Figure 2 As shown, the grayscale image matrix is ​​compared pixel by pixel with the pre-stored design image matrix to generate a difference matrix, including the following steps:

[0079] Step 21: spatially aligning the grayscale image matrix with a pre-stored design image matrix to make the grayscale image matrix and the design image matrix correspond in spatial position;

[0080] Specifically, the pre-stored design image matrix is ​​a standard reference image of the fork in an ideal state, stored in the form of a grayscale image matrix, and is used to compare with the grayscale image matrix collected in real time to calculate the posture offset of the fork. The ideal state is, for example, the grayscale image matrix corresponding to the precise three-dimensional model of the fork generated by using a high-precision three-dimensional scanning device to perform a full-range scan of the fork before the fork leaves the factory.

[0081] The spatial alignment adopts a method combining feature point matching with affine transformation.Firstly, feature points and their descriptors are extracted from the grayscale image matrix and the design image matrix by using a scale-invariant feature transformation algorithm.

[0082] Secondly, a random sampling consensus algorithm is used to match feature points and select the expected high-quality matching point pairs. Specifically, the random sampling consensus algorithm iteratively estimates the optimal transformation model while eliminating incorrect matching points to improve matching accuracy.

[0083] In the actual matching process of the present invention, the distance ratio threshold is set to 0.7, that is, when the ratio of the nearest neighbor distance to the next nearest neighbor distance is less than 0.7, the match is considered valid.

[0084] Finally, based on the screened matching point pairs, the affine transformation matrix is ​​calculated, and the grayscale image matrix is ​​mapped to the coordinate system of the design image matrix through the affine transformation matrix to achieve alignment between the grayscale image matrix and the design image matrix.

[0085] In a preferred embodiment of the present invention, the spatial alignment may also include sub-pixel precision registration. Specifically, the sub-pixel displacement between the grayscale image matrix and the design image matrix is ​​calculated using a phase correlation algorithm, and then accurately corrected using an interpolation method to achieve an alignment accuracy of less than 0.1 pixel.

[0086] Step 22: Perform grayscale value normalization on the spatially aligned grayscale image matrix and the design image matrix to eliminate the influence of different lighting conditions;

[0087] Specifically, the grayscale value normalization employs a piecewise linear mapping method. Specifically, the grayscale histograms of the grayscale image matrix and the design image matrix are calculated, and statistical features, including mean, standard deviation, maximum, minimum, and quantiles, are extracted. Next, based on these statistical features, a piecewise linear mapping function is designed to adjust the grayscale value distributions of the grayscale image matrix and the design image matrix to similar ranges and distribution shapes.

[0088] It should be noted that the normalization parameters are optimized based on the fork's material properties and ambient lighting conditions. For areas with highly reflective materials, a nonlinear mapping function is used to reduce the impact of highlights; for shadowed areas, an adaptive gain is used to enhance detail visibility. This intelligent normalization strategy ensures consistent comparison results under various lighting conditions.

[0089] Step 23: Compare the normalized grayscale image matrix and the design image matrix pixel by pixel, calculate the grayscale difference values ​​of corresponding pixels, and generate a difference matrix;

[0090] Specifically, the pixel-by-pixel comparison uses a multi-scale weighted difference strategy. First, the grayscale difference between the corresponding pixels in the grayscale image matrix and the design image matrix is ​​calculated at the original resolution. The difference is defined as D(i 1, j 1) = |L1(i 1, j 1) - L2(i 1, j 1)|, where L1 and L2 represent the grayscale image matrix and the design image matrix, respectively, and i 1 and j 1 are the pixel coordinates.

[0091] Next, a Gaussian pyramid is constructed using the grayscale image matrix and the design image matrix, and differences are calculated at multiple scale levels. Low-resolution levels capture large-scale structural changes, while high-resolution levels capture detailed changes. The differences at each scale level are fused through upsampling and weighted summation to generate the final difference matrix. Weight coefficients for this weighted summation are dynamically assigned based on the importance of different scales, with higher weights generally assigned to higher-resolution levels.

[0092] In a preferred embodiment of the present invention, the pixel-by-pixel comparison can also consider pixel neighborhood information. A structural similarity index (SSIM) is used as a measure of difference in local regions. It comprehensively considers similarities in brightness, contrast, and structure, and better reflects the perceived differences in images than simple grayscale differences. The SSIM value range is [-1, 1], with values ​​closer to 1 indicating higher similarity. The SSIM values ​​are mapped to the range [0, 255] and used as element values ​​in the difference matrix.

[0093] Further, such as Figure 3 As shown, the calculation of the fork's posture offset according to the difference matrix includes the following steps:

[0094] Step 31: extracting features from the difference matrix to identify key structural areas of the fork;

[0095] Specifically, feature extraction utilizes a method that combines region segmentation with edge detection. First, based on the grayscale distribution characteristics of the difference matrix, an adaptive thresholding method is used to binarize the difference matrix and isolate regions of significant difference. The adaptive threshold is dynamically calculated based on the grayscale mean and standard deviation of the local region to adapt to contrast variations across different regions.

[0096] Secondly, morphological processing is applied to the binarized difference map, including opening operation to remove noise and small connected areas, and closing operation to fill small holes in the area, so as to obtain a more complete and coherent structural area.

[0097] In an embodiment of the present invention, the morphologically processed structural element is disk-shaped, with its radius dynamically adjusted based on image resolution and fork structural characteristics, typically set to 0.5% to 1% of the image width. The processed binary image is labeled using connected domain analysis, with each connected region representing a potential region of structural deformation. Geometric features are extracted from each connected region, including area, perimeter, center of mass position, principal axis orientation, and eccentricity.

[0098] In a preferred embodiment of the present invention, feature extraction can also include edge detection. For example, the Canny edge detection algorithm is applied to the difference matrix to extract structural edge lines in areas of significant difference. The Canny algorithm uses a high-low threshold ratio of 2:1, with the specific threshold value adaptively determined based on the gradient statistics of the difference matrix. The edge detection results are combined with the region segmentation results to further enhance the characterization of the fork structure.

[0099] Step 32: Based on the extracted structural features, the position offset and angle deviation of the fork are calculated for each key structural area;

[0100] Specifically, the position offset calculation uses a weighted centroid method. For each identified critical structural region Ri, its centroid coordinates (xi, yi) and area Ai are calculated. The key structural regions are then weighted based on their area and degree of variation to calculate the overall position offset, expressed as a horizontal displacement Δx and a vertical displacement Δy, respectively, representing the difference between the weighted average of the centroid coordinates of all deformed regions and the reference position.

[0101] In this embodiment of the present invention, the weighted centroid method takes into account the influence of the degree of difference. For each structurally deformed region Ri, the average difference value Di is calculated and used as a component of the weighting factor. The weighting calculation comprehensively considers the region area and the degree of difference, so that regions with large areas and significant differences have a greater influence on the position offset calculation.

[0102] The angle deviation calculation uses the principal direction analysis method. First, a linear fit is performed on the extracted edge segments to obtain a series of straight line segments and their orientation angles. Next, the principal orientation angle θm is calculated by weighting the line segment length and fitting accuracy. Finally, the principal orientation angle is compared with the reference orientation angle θr to obtain the rotation angle deviation θ = θm - θr.

[0103] Step 33: Generate a pose offset vector including horizontal displacement, vertical displacement and rotation angle for each key structural area based on the position offset and angle deviation;

[0104] Specifically, the pose offset vector is represented as a triplet: O = [Δx, Δy, θ], where Δx represents the horizontal displacement (in millimeters), Δy represents the vertical displacement (in millimeters), and θ represents the rotation angle (in degrees), i.e., the angular deviation. The pose offset vector is represented in a right-handed coordinate system, with the origin at the fork's reference point (usually the center of the fork), the x-axis pointing in the direction of the fork's forward motion, the y-axis pointing vertically, and the z-axis following the right-hand rule.

[0105] In an embodiment of the present invention, the pose offset vector may also be accompanied by a confidence index C, forming an extended vector O_ext = [Δx, Δy, θ, C]. The confidence index C ranges from [0, 1] and reflects the reliability of the current pose estimate. The C value is calculated based on multiple factors, including the signal-to-noise ratio of the difference matrix, the stability of feature extraction, the temporal consistency of the pose estimate, and the consistency of multi-method estimation. When the C value falls below a preset index threshold, typically set to 0.7, a warning signal is triggered, alerting the operator to the uncertainty of the current pose estimate.

[0106] In a preferred embodiment of the present invention, the process for generating the pose offset vector may also include a precision control mechanism. Regular calibration using a standard test specimen establishes a mapping relationship between visual measurements and actual physical displacements, compensating for system errors. The calibration process utilizes a multi-point sampling strategy, evenly distributing sampling points within the fork's workspace. Each sampling point records both a visual measurement and a high-precision reference measurement acquired via external measurement equipment. The visual measurement is then mapped to actual physical displacements using a least-squares fit to a transfer function.

[0107] Further, such as Figure 4 As shown, the calculation of the deformation compensation amount of the fork by the compensation amount estimation model includes the following steps:

[0108] Step 41: Establish a material property database based on the material parameters of the fork;

[0109] Specifically, the material property database includes material parameters of various fork components, including but not limited to elastic modulus E (unit: GPa), yield strength σy (unit: MPa), Poisson's ratio ν (dimensionless) and hardening coefficient H (unit: MPa).

[0110] It should be noted that the elastic modulus characterizes the stiffness characteristics of the material in the elastic deformation stage, the yield strength defines the critical stress value at which the material transitions from elastic deformation to plastic deformation, the Poisson's ratio describes the ratio of the transverse to axial strain of the material under uniaxial tension or compression, and the hardening coefficient characterizes the stress-strain relationship of the material in the plastic deformation stage.

[0111] In this embodiment of the present invention, the material property database is organized in a hierarchical structure. The top layer contains the main components of the fork, such as the fork body, fork arm, and connecting frame; the middle layer contains the material types of each component, such as high-strength steel, aluminum alloy, and composite materials; and the bottom layer contains specific material parameter values ​​and their temperature dependence. Material parameter values ​​are obtained through standard material testing methods, including tensile testing, compression testing, and hardness testing.

[0112] In a preferred embodiment of the present invention, the material property database may also include uncertainty information about material parameters, represented as a probability distribution. For example, the elastic modulus E follows a normal distribution with a mean of 210 GPa and a standard deviation of 5 GPa, while the yield strength σy follows a normal distribution with a mean of 355 MPa and a standard deviation of 10 MPa. It will be appreciated that representing these parameters in a probabilistic manner can more accurately reflect actual engineering material parameter fluctuations and improve the reliability of compensation estimation.

[0113] Step 42: Based on the posture offset vector, a force analysis model of each key structural area in the fork is constructed, and the stress distribution of key points in each key structural area is calculated;

[0114] Specifically, the force analysis model adopts a simplified finite element method. First, the fork is simplified into a mechanical model consisting of beam elements and shell elements. The beam elements are mainly used to simulate slender components such as the fork arm, and the shell elements are used to simulate plate-like components such as the fork body.

[0115] Secondly, based on the pose offset vector O = [Δx, Δy, θ] of the key structural area, the boundary conditions and load distribution of the corresponding key structural area under the working state are calculated.

[0116] The boundary conditions include fixed constraints and displacement constraints. In an embodiment of the present invention, for example, fixed constraints are applied to the hinge point connecting the fork to the forklift, and displacement constraints are applied to the key contact points of the fork based on the posture offset vector. The load distribution includes gravity load, working load, and correction force. The gravity load is generated by the weight of the fork, the working load is determined by the weight and distribution of the cargo, and the correction force is applied by the posture correction mechanism to adjust the posture of the fork.

[0117] The stress distribution calculation uses the principle of linear superposition. For each load type, the resulting stress distribution is calculated separately. The stresses generated by each load type are then superimposed to obtain the total stress distribution. Stress calculation uses the generalized Hooke's law: σ = D·ε, where σ is the stress tensor, ε is the strain tensor, and D is the elastic matrix, which is determined by the material's elastic modulus E and Poisson's ratio ν.

[0118] Step 43: Evaluate the deformation state of each key structural area of ​​the fork according to the stress distribution and the material property database, and distinguish between elastic deformation areas and plastic deformation areas;

[0119] Specifically, the deformation state is evaluated using the von Mises yield criterion. For each calculation point on the fork (the centroid coordinates of each identified critical structural area), its von Mises equivalent stress σvm is calculated and compared with the yield strength σy of the material. When σvm<σy, the point is judged to be in an elastic deformation state; when σvm≥σy, the calculation point is judged to have entered a plastic deformation state. When the calculation point is in an elastic deformation state, the corresponding critical structural area is judged to be in an elastic deformation state; otherwise, the corresponding critical structural area is judged to be in a plastic deformation area.

[0120] It can be understood that the deformation in the elastic deformation region of the fork can be completely recovered by unloading, while the deformation in the plastic deformation region includes an irreversible permanent deformation component.

[0121] Based on the deformation state of each calculation point, each key structural area is divided into elastic deformation areas and plastic deformation areas. The regional division results are represented by a binary label matrix. Each element in the matrix corresponds to a calculation point on the fork. A value of 0 indicates elastic deformation, and a value of 1 indicates plastic deformation.

[0122] Step 44: Based on the evaluation results of the deformation state, a compensation estimation model is constructed to calculate the deformation compensation vector of each key structural area;

[0123] Specifically, the compensation estimation model adopts a piecewise function form. For the elastic deformation area, the linear elastic model is used to calculate the compensation; for the plastic deformation area, the elastic-plastic model is used to calculate the compensation.

[0124] In an embodiment of the present invention, the linear elastic model is based on Hooke's law, and the compensation amount is proportional to the stress. The displacement compensation vector Ce=[Δxe, Δye, θe] of the elastic region is calculated as follows:

[0125] Δxe=-kx·σx·L / E;

[0126] Δye=-ky·σy·L / E;

[0127] θe=-kθ·τxy·L / (G·r);

[0128] where σx and σy are the normal stresses in the x- and y-directions, respectively, τxy is the shear stress, L is the characteristic length, E is the elastic modulus, G is the shear modulus, G = E / [2(1+ν)], r is the characteristic radius, and kx, ky, and kθ are correction coefficients determined by experimental calibration.

[0129] The elastic-plastic model is based on the Ramberg-Osgood constitutive equation and takes into account the nonlinear hardening characteristics of the material. The displacement compensation vector Cp=[Δxp, Δyp, θp] in the plastic region is calculated as follows:

[0130] Δxp=-kx·[σx·L / E+α·(σx / σy)^n·L / E];

[0131] Δyp=-ky·[σy·L / E+α·(σy / σy)^n·L / E];

[0132] θp=-kθ·[τxy·L / (G·r)+β·(τxy / τy)^m·L / (G·r)];

[0133] Among them, α and β are material constants, n and m are hardening exponents, which are determined by the hardening coefficient H of the material, and τy is the shear stress yield value.

[0134] Step 45: Using a weighted fusion method to optimize and adjust the deformation compensation vector to generate the final deformation compensation;

[0135] The weighted fusion method is:

[0136] The elastic deformation compensation vector Ce and the plastic deformation compensation vector Cp are weightedly fused to obtain the preliminary compensation vector C0 = we·Ce+wp·Cp, where the weight coefficients we and wp are determined according to the relative areas of the elastic region and the plastic region.

[0137] The preset compensation threshold adopts the same vector form as the deformation compensation, and is determined based on the comprehensive consideration of the fork's accuracy requirements, safety standards, and operating efficiency;

[0138] For example, in an embodiment of the present invention, the preset thresholds are dynamically adjusted based on the fork's operating mode. For high-precision operation mode, lower thresholds are set, such as Tx = 2.0 mm, Ty = 1.5 mm, and Tθ = 0.5°, to ensure high position accuracy. For standard operation mode, medium thresholds are set, such as Tx = 3.5 mm, Ty = 2.5 mm, and Tθ = 1.0°, to balance accuracy and efficiency. For high-speed operation mode, higher thresholds are set, such as Tx = 5.0 mm, Ty = 4.0 mm, and Tθ = 1.5°, prioritizing operational efficiency.

[0139] Furthermore, the method for determining whether the deformation compensation amount exceeds the preset compensation amount threshold is:

[0140] For any time point, a method combining component comparison and comprehensive evaluation is used to determine whether the time point needs to be corrected;

[0141] If correction is determined to be required at n consecutive time points, the deformation compensation amount is determined to exceed the preset compensation amount threshold and correction is required;

[0142] Specifically, the way to combine component comparison with comprehensive evaluation is as follows:

[0143] First, each component of the deformation compensation vector C = [Δxc, Δyc, θc] is compared with the corresponding preset threshold components:

[0144] If |Δxc|>Txe, the horizontal compensation amount exceeds the corresponding threshold;

[0145] If |Δyc|>Tye, the vertical compensation amount exceeds the corresponding threshold;

[0146] If |θc|>Tθe, the angle compensation amount exceeds the corresponding threshold.

[0147] Secondly, Txe, Tye, and Tθe are the values ​​of each component in the vector corresponding to the preset compensation threshold, and the comprehensive deviation index CDI of the deformation compensation amount is calculated. The calculation formula of CDI is CDI=[(Δxc / Tx)²+(Δyc / Ty)²+(θc / Tθ)²];

[0148] When CD I>1, it means that the deformation compensation amount exceeds the preset threshold in a comprehensive sense, that is, correction is required at this time point;

[0149] Furthermore, the method of generating the correction control instruction of the fork based on the deformation compensation amount is:

[0150] Pre-establishing the mapping relationship between the deformation compensation amount and the actuator control parameter vector;

[0151] generating corresponding control instructions based on the mapping relationship and the deformation compensation amount;

[0152] In an embodiment of the present invention, the mapping relationship between the deformation compensation amount and the actuator control parameter vector is constructed as follows:

[0153] The mapping relationship is represented by the Jacobian matrix J;

[0154] Specifically, the Jacobian matrix J is determined based on the fork's mechanical structure and actuator layout. For a typical three-degree-of-freedom fork system, where the actuators include horizontal, vertical, and rotational actuators, the Jacobian matrix is ​​simplified to a diagonal matrix form, with the mapping relationships: a1 = k1·Δxc; a2 = k2·Δyc; and a3 = k3·θc. a1, a2, and a3 represent the adjustment values ​​for the horizontal, vertical, and rotational actuators, respectively. k1, k2, and k3 are proportional coefficients, representing the actuator response coefficients for horizontal, vertical, and rotational displacements, respectively, with units of pulses / mm, pulses / mm, and pulses / degree, respectively.

[0155] Furthermore, the method of generating the corresponding control instruction is:

[0156] By substituting the value of each component in the deformation compensation amount into the corresponding formula of the mapping relationship, the rated adjustment amount of the actuator corresponding to each component is obtained, and the adjustment amount of each actuator constitutes the control instruction;

[0157] It is understandable that by respectively adjusting the respective actuators of the fork, the deformation of the fork can be corrected.

[0158] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An intelligent fork calibration method based on visual inspection, characterized in that: The following steps are involved: Step 1: Collect the real-time image of the fork through the visual inspection system and convert the real-time image into a grayscale image matrix; Step 2: Compare the grayscale image matrix with the pre-stored design image matrix pixel by pixel to generate a difference matrix; Step 3: Calculate the fork's posture offset based on the difference matrix; Step 4: Based on the posture offset and the material parameters of the fork, the deformation compensation of the fork is calculated using a compensation estimation model; Step 5: Determine whether the deformation compensation amount exceeds a preset compensation amount threshold. If so, proceed to step 6; otherwise, repeat steps 1 to 5. Step 6: Generate a control instruction for the fork based on the deformation compensation amount.

2. The intelligent fork calibration method based on visual inspection according to claim 1, characterized in that: The method of collecting the real-time image of the fork by the visual detection system is: The visual inspection system is installed at a fixed position on the fork and captures real-time images of the working area covering the fork through an optical lens.

3. The intelligent fork calibration method based on visual inspection according to claim 2, characterized in that: The conversion of the real-time image into a grayscale image matrix comprises the following steps: Step 11: preprocessing the real-time image collected by the visual inspection system to eliminate image noise and distortion, thereby generating a preprocessed real-time image; Step 12: Convert the pre-processed real-time image into a grayscale image to generate grayscale image data; Step 13: Convert the grayscale image into a grayscale image matrix in the form of a two-dimensional array, where each matrix element in the grayscale image matrix corresponds to the grayscale value of a pixel.

4. The intelligent fork calibration method based on visual inspection according to claim 3, characterized in that: Comparing the grayscale image matrix with the pre-stored design image matrix pixel by pixel to generate a difference matrix comprises the following steps: Step 21: spatially aligning the grayscale image matrix with a pre-stored design image matrix to make the grayscale image matrix and the design image matrix correspond in spatial position; Step 22: Perform grayscale value normalization on the spatially aligned grayscale image matrix and the design image matrix to eliminate the influence of different lighting conditions; Step 23: Compare the normalized grayscale image matrix and the design image matrix pixel by pixel, calculate the grayscale difference values ​​of corresponding pixels, and generate a difference matrix; The pixel-by-pixel comparison adopts a multi-scale weighted difference strategy; first, the grayscale difference between the corresponding pixels of the grayscale image matrix and the designed image matrix is ​​calculated at the original resolution; A Gaussian pyramid of grayscale image matrices and design image matrices is constructed, and difference values ​​are calculated at multiple scale levels. Low-resolution levels capture large-scale structural changes, while high-resolution levels capture detail changes. The difference values ​​at each scale level are fused through upsampling and weighted summation to generate a final difference matrix. The weight coefficients corresponding to the weighted summation method are dynamically assigned based on the importance of different scales, with higher weights generally assigned to the high-resolution level.

5. The intelligent fork calibration method based on visual inspection according to claim 4, characterized in that: Calculating the fork's posture offset according to the difference matrix includes the following steps: Step 31: extracting features from the difference matrix to identify key structural areas of the fork; Step 32: Based on the extracted structural features, the position offset and angle deviation of the fork are calculated for each key structural area; Step 33: Based on the position offset and angle deviation, a pose offset vector including horizontal displacement, vertical displacement and rotation angle is generated for each key structure area.

6. The intelligent fork calibration method based on visual inspection according to claim 5, characterized in that: The feature extraction adopts a method combining region segmentation and edge detection. First, based on the grayscale distribution characteristics of the difference matrix, the difference matrix is ​​binarized using an adaptive threshold method to separate the significant difference regions. The feature extraction further includes edge detection processing; that is, applying the Canny edge detection algorithm to the difference matrix to extract the structural edge lines of the significant difference areas.

7. The intelligent fork calibration method based on visual inspection according to claim 6, characterized in that: The position offset calculation adopts the weighted centroid method; for each identified key structural area Ri, its centroid coordinates (xi, yi) and area Ai are calculated; then, based on the regional area and degree of difference of the key structural area, weighting is performed to calculate the overall position offset, which is expressed as the horizontal displacement Δx and vertical displacement Δy, respectively, which are the difference between the weighted average of the centroid coordinates of all structural deformation areas and the reference position.

8. The intelligent fork calibration method based on visual inspection according to claim 7, characterized in that: Calculating the deformation compensation amount of the fork by using the compensation amount estimation model includes the following steps: Step 41: Establish a material property database based on the material parameters of the fork; Step 42: Based on the posture offset vector, a force analysis model of each key structural area in the fork is constructed, and the stress distribution of key points in each key structural area is calculated; Step 43: Evaluate the deformation state of each key structural area of ​​the fork according to the stress distribution and the material property database, and distinguish between elastic deformation areas and plastic deformation areas; For each calculation point on the fork, the calculation point is the centroid coordinate of each identified key structural area, and its von Mises equivalent stress σvm is calculated and compared with the yield strength σy of the material; when σvm<σy, the point is judged to be in an elastic deformation state; when σvm≥σy, the calculation point is judged to have entered a plastic deformation state; when the calculation point is in an elastic deformation state, the corresponding key structural area is judged to be in an elastic deformation state, otherwise, the corresponding key structural area is judged to be in a plastic deformation area; Based on the deformation state of each calculation point, each key structural area is divided into elastic deformation area and plastic deformation area. The area division result is represented by a binary labeling matrix. Each element in the matrix corresponds to a calculation point on the fork. The value of 0 indicates elastic deformation, and the value of 1 indicates plastic deformation. Step 44: Based on the evaluation results of the deformation state, a compensation estimation model is constructed to calculate the deformation compensation vector of each key structural area; Step 45: Use a weighted fusion method to optimize and adjust the deformation compensation vector to generate the final deformation compensation.

9. The intelligent fork calibration method based on visual inspection according to claim 8, characterized in that: The method for determining whether the deformation compensation amount exceeds the preset compensation amount threshold is: For any time point, a method combining component comparison and comprehensive evaluation is used to determine whether the time point needs to be corrected; If correction is determined to be required at n consecutive time points, it is determined that the deformation compensation amount exceeds the preset compensation amount threshold and correction is required.

10. The intelligent fork calibration method based on visual inspection according to claim 9, characterized in that: The method of generating the correction control instruction of the fork based on the deformation compensation amount is: Pre-establishing the mapping relationship between the deformation compensation amount and the actuator control parameter vector; Based on the mapping relationship and the deformation compensation amount, a corresponding control instruction is generated.

Images