A target positioning method and system based on machine vision

Abstract

The application discloses a target positioning method and system based on machine vision, relates to the technical field of machine vision and target positioning, and comprises the following steps: key hardware selection of machine vision is performed, and image information is collected; image preprocessing is performed, and key points and feature points of the image are automatically detected by using a SIFT algorithm; an inner parameter matrix is generated by establishing a coordinate system, an outer parameter matrix is generated by establishing a camera coordinate system, and coordinate conversion is performed in combination with the inner parameter matrix and the outer parameter matrix; and target positioning calibration is performed by using a camera calibration method.The SIFT algorithm used in the method has strong robustness to scale change, rotation, illumination change and a certain degree of view angle change of an image, and is suitable for a complex environment in an industrial scene. In the calibration process, the calculation precision of the inner and outer parameters is improved by using a chessboard calibration plate and a least square method, the calibration error is effectively reduced, and the positioning precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision and target positioning technology, in particular to a target positioning method and system based on machine vision. BACKGROUND

[0002] With the rapid development of industrial automation, intelligent manufacturing and artificial intelligence, machine vision technology has become a key component in industrial applications, and is widely used in robot navigation, quality detection, target recognition and other fields. Machine vision collects and processes image data to give the computer "vision" ability, enabling it to recognize and locate target objects. In recent years, the improvement of hardware performance and the continuous innovation of image processing algorithms have provided strong support for the popularization and development of machine vision technology. However, in the actual application of high-precision target positioning, there are still many challenges in the stability of image acquisition, the accuracy of feature extraction and the conversion precision of multi-dimensional space coordinates. Different application scenarios have different requirements for target positioning, usually requiring optimization of image acquisition hardware configuration and selection of appropriate algorithms to extract key features in the image, so as to achieve stable and accurate target positioning.

[0003] The existing machine vision target positioning technology mainly relies on a variety of image processing algorithms and camera calibration techniques to realize the conversion from two-dimensional image data to three-dimensional space. Common feature extraction algorithms include SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features) and others, which show high robustness in complex scenes such as rotation and scale change. However, these methods still have certain deficiencies in practical application. First, the traditional hardware configuration scheme lacks flexibility and is difficult to adapt to complex and variable industrial environments, resulting in unstable image acquisition quality; second, although the feature extraction algorithm has strong robustness, its stability and accuracy may be limited in the presence of noise interference and significant changes in lighting conditions. In addition, the existing coordinate system establishment and internal and external parameter matrix generation methods often result in reduced positioning accuracy due to the instability of camera calibration. Accurate and fast camera calibration is crucial for improving target positioning accuracy, but existing technologies have certain limitations in calibration methods and parameter optimization, and have not effectively solved the calibration error problem in dynamic application scenarios. SUMMARY

[0004] In view of the above problems, the present application is proposed.

[0005] Therefore, the technical problem solved by the present application is that the existing machine vision target positioning technology has problems in feature extraction stability, image acquisition hardware adaptability and coordinate system conversion precision.

[0006] To solve the above technical problems, the present application provides the following technical scheme: a target positioning method based on machine vision, comprising

[0007] Carrying out machine vision key hardware selection, collecting image information;

[0008] Carrying out image preprocessing, using SIFT algorithm to automatically detect key points and feature points of the image;

[0009] Establishing a coordinate system to generate an intrinsic matrix, establishing a camera coordinate system to generate an extrinsic matrix, and combining the intrinsic and extrinsic matrices to perform coordinate conversion;

[0010] Through the method of camera calibration, target positioning calibration is carried out.

[0011] As a preferred scheme of the target positioning method based on machine vision, wherein: the machine vision key hardware selection includes selecting light source and lighting method, and determining resolution and lens of the camera;

[0012] For lens selection, the main consideration factor is focal length; assuming that the size of the field of view is FOV, the horizontal and vertical directions are FOV H and FOV V , the horizontal and vertical sizes of the camera chip target surface are S H and S V , and the detection distance is d w , then the horizontal component calculation formula and the vertical component calculation formula of the focal length f are represented as:

[0013]

[0014] Wherein, f H represents the horizontal focal length, and f V represents the vertical focal length.

[0015] As a preferred scheme of the target positioning method based on machine vision, wherein: the image information collection includes using two cameras to collect images; the first camera collects the end face image of the pull rod round hole surface and the pull rod rear hanger shaft from the front, detects the centering condition of the pull rod and the hanger shaft; the second camera collects the side face image of the pull rod and the hanger shaft, detects the centering condition of the pull rod and the degree of the hanger shaft passing through the pin.

[0016] As a preferred scheme of the target positioning method based on machine vision, wherein: the image preprocessing includes gray processing based on color features, image filtering, and binary processing of the image to realize ROI rough positioning;

[0017] Selecting a suitable gray scale transformation method to transform a three-channel color RGB image into a single-channel gray scale image with the same value range; assuming that the gray scale value of the pixel point P(x, y) after gray scale transformation is Gray(x, y);

[0018] The average gray processing method is that the average value of the RGB three-channel component values is taken as the gray value of the pixel point after the gray transformation, and the formula is expressed as:

[0019]

[0020] The weighted average gray processing method is a generalization of the average gray processing method, which calculates the gray value of the pixel point after the gray transformation by assigning appropriate weights to the RGB three-channel component values, and the formula is expressed as:

[0021] Gray(x,y)=r1R(x,y)+r2G(x,y)+r3B(x,y)

[0022] Wherein, R(x,y) represents the pixel point of R channel, G(x,y) represents the pixel point of G channel, and B(x,y represents the pixel point of B channel; r1, r2, r3 represent the initial weight assigned to each channel component;

[0023] The ROI coarse positioning process is based on the gray image after the smoothing processing in the previous section, and the inverse binary threshold processing is performed, wherein the threshold thresh is selected, and the formula is expressed as:

[0024]

[0025] Wherein, src(x,y) represents the pixel value of the input image at the coordinate (x,y); dst(x,y) represents the pixel value of the output image at the coordinate (x,y); maxval represents the maximum value to be assigned to the pixels meeting the condition;

[0026] The maximum connected domain center point is obtained, and it can be seen from the binary image that the position of the maximum connected domain is the position of the target to be detected, and the center point is also the approximate center point of the region. After obtaining the center point position, the ROI coarse positioning can be performed according to the size information of the circular hole;

[0027] The ROI coarse positioning region is obtained, based on the approximate position of the target center point obtained above, the rectangular frame obtained by expanding a certain number of pixel points from the center point to the surrounding according to the size information is the coarse positioning region containing the target to be detected;

[0028] The ROI coarse positioning region is cropped, and the subsequent processing process only needs to process the coarse positioning region.

[0029] As a preferred scheme of the target positioning method based on machine vision, the SIFT algorithm is used to automatically detect the key points and feature points of the image, including scale space extreme value detection, searching for image positions in all scales, and identifying potential interest points that are invariant to scale and rotation through a Gaussian differential function;

[0030] The key points are precisely located, and in each candidate position, the position and scale are determined by fitting a fine model, and the key points are selected according to their stability;

[0031] The main direction of the key points is determined, one or more directions are assigned to each key point position based on the gradient direction of the image local part, and all subsequent operations on the image data are transformed relative to the direction, scale and position of the key points, thereby providing invariance to these transformations;

[0032] The SIFT feature vector is generated, and the gradient of the image local part is measured in the neighborhood around each key point at the selected scale;

[0033] When performing scale space extremum detection, a dynamic feedback adjustment mechanism is added to monitor the number and distribution of detected key points in real time; if too few or too many key points are detected at a certain scale, the scale parameter of the Gaussian blur is automatically adjusted.

[0034] As a preferred scheme of the target positioning method based on machine vision, the establishment of the coordinate system generates an intrinsic matrix, the pixel coordinate system is a coordinate system about the image of the determined target point, the basic unit is the pixel coordinate system established on the photosensitive element of the camera, and the image coordinate system is a coordinate system with physical length as the unit, which is also established on the photosensitive element of the camera;

[0035] The image coordinates (u, v) are converted into normalized camera coordinates (x n ,y n ), and the formula is as follows:

[0036]

[0037] Where f x and f y represent the pixel units of the camera focal length; c x and c y represent the pixel coordinates of the image principal point;

[0038] The normalized camera coordinates are converted into the camera coordinate system, the normalized camera coordinates are mapped into the camera coordinate system, and the depth Z c of the object is known, and the formula is as follows:

[0039] X c =x n ·Z c ,Y c =y n ·Z c ,Z c =Z c

[0040] The camera coordinate system is a reference system based on a camera as a reference object to express the position of a target, with physical length as a unit, the axis direction and the axis direction are defined respectively in the same axis and axis, the axis is perpendicular to the photosensitive element of the camera, and is positive away from the photosensitive element, and the center of the lens is the origin of the coordinate system;

[0041] In summary, the complete formula of the image coordinates (u, v) converted into the world coordinates X w ,Y w ,Z w ) through intrinsic and extrinsic parameters is as follows:

[0042]

[0043] Wherein, K represents the intrinsic matrix

[0044]

[0045] Wherein, s represents a scale factor, which is calculated according to actual depth information.

[0046] As a preferred scheme of the target positioning method based on machine vision, wherein: the calibration combining the intrinsic and extrinsic parameter matrices comprises camera calibration, selecting a suitable chessboard calibration plate for printing, and determining the physical size of the related points of the calibration plate to ensure the accuracy of the calibration plate.

[0047] The calibration plate is fixed on a certain plane plate, and the position and angle of the calibration plate are changed for multiple shooting within the field of view of the camera.

[0048] The algorithm is used to detect the corner feature points of the chessboard calibration plate in the image.

[0049] Distortion coefficient solving and optimization, after preliminary solving of the camera intrinsic and extrinsic parameters, considering the image distortion factors, establishing a radial distortion and tangential distortion model:

[0050] x' = x + (x-c x )(k1r 2 +k2r 4 )+2p1xy+p2(r 2 +2x 2 )

[0051] y' = y + (y-c y )(k1r 2 +k2r 4 )+p1(r 2 +2y 2 )+2p2xy

[0052] Wherein, k1, k2 are radial distortion coefficients, p1, p2 are tangential distortion coefficients, r 2 =x2 + y 2 is the square distance to the optical axis;

[0053] The least square method and the maximum likelihood method optimization, using the least square method to solve the initial distortion coefficient k1, k2, p1, p2, minimize the following error function:

[0054]

[0055] Wherein, x ij is the pixel coordinate under ideal non-distortion condition, x' ij is the actual pixel coordinate after distortion; the maximum likelihood method is used to further optimize the estimated distortion parameter to obtain more accurate internal and external parameters, and the parameter value is solved according to the pixel coordinate and physical coordinate of the detected feature point.

[0056] As a preferred scheme of the target positioning system based on machine vision, wherein:

[0057] The image acquisition module selects appropriate machine vision key hardware to acquire image information meeting accuracy and clarity requirements, installs multiple pull rod alignment vision cameras and suspension shaft hole alignment vision cameras, and accurately detects the deviation distance of the two pull rods and the suspension shaft.

[0058] The image preprocessing module pre-processes the acquired image to improve image quality and the accuracy of feature extraction; the preprocessing includes denoising, grayscale, edge enhancement, feature point detection, and feature point extraction.

[0059] The coordinate system establishment and internal and external parameter matrix generation module establishes a pixel coordinate system and an image coordinate system, establishes a camera coordinate system and a world coordinate system, realizes coordinate conversion, and accurately maps the target position in the image to the actual space position.

[0060] The automatic calibration module calibrates the camera by using a calibration board, calculates the internal and external parameter matrix through image acquisition at multiple angles and distances, corrects the geometric distortion of the camera, and ensures accurate camera imaging; the system processes and feeds back the received vision detection data through PLC control, and realizes automatic calibration and positioning.

[0061] A computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps of the target positioning method based on machine vision.

[0062] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the target positioning method based on machine vision.

[0063] The beneficial effects of the present application: the SIFT algorithm adopted by the target positioning method based on machine vision provided by the present application has strong robustness to the scale change, rotation, illumination change and certain degree of view angle change of the image, and is suitable for the complex environment in the industrial scene. By establishing the pixel coordinate system, the image coordinate system and the conversion from the normalized camera coordinate to the camera coordinate system, the intrinsic and extrinsic parameter matrices are generated, and combined with the camera calibration method, the conversion difficulty from the image coordinate to the world coordinate is solved. In the calibration process, the chessboard calibration board and the least square method are used to improve the calculation accuracy of the intrinsic and extrinsic parameters, effectively reduce the calibration error, and improve the positioning accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0065] Fig. 1 The overall flowchart of the target positioning method based on machine vision provided for the first embodiment of the present application.

[0066] Fig. 2 The camera device layout diagram of the target positioning method based on machine vision provided for the first embodiment of the present application.

[0067] Fig. 3 The coordinate conversion schematic diagram of the camera coordinate system and the image coordinate system of the target positioning method based on machine vision provided for the first embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0069] Embodiment 1, refer to Figs. 1-3 For an embodiment of the present application, a target positioning method based on machine vision is provided, comprising:

[0070] S1: Selecting machine vision key hardware, collecting image information.

[0071] Further, the machine vision key hardware selection includes selecting light source and lighting method, and determining camera resolution and lens.

[0072] Further, for lens selection, the main consideration factor is focal length; assuming the field of view size of shooting is FOV, the horizontal and vertical directions are FOV H and FOV V , the horizontal and vertical sizes of the camera chip target surface are S H and S V , and the detection distance is d w , then the horizontal component calculation formula and the vertical component calculation formula of the focal length f are represented as:

[0073]

[0074] Wherein, f H represents the horizontal focal length size, and f V represents the vertical focal length size.

[0075] Further, as shown in Fig. 2 , the collected image information includes using two cameras to collect images; the first camera collects the end face image of the pull rod round hole surface and the pull rod rear hanger shaft from the front, detects the centering condition of the pull rod to the hole and the pull rod to the hanger shaft; the second camera collects the side face image of the pull rod and the hanger shaft, detects the pull rod to the hole condition and the hanger shaft to the pin degree.

[0076] It should be noted that the specific image collection method is that the pull plate is supported on the locking beam, the shift shaft device is driven to walk and close to the locking beam; the shift shaft device and the locking beam are fixed with the locking pin; the hoisting is placed at the front shaft storage groove, the lifting platform is gradually jacked up, the shaft to be installed is aligned with the shaft hole; the telescopic horizontal moving oil cylinder drives the sliding table, the electric push rod and the hanger shaft on the sliding table to translate, and the up and down position of the platform is adjusted at the same time, so that the shaft to be installed is completely aligned with the shaft hole; the shift shaft transfer device moves the pin shaft from the storage position to the pin insertion position through the clamp; the electric push rod is started, the electric push rod and the shaft to be installed are connected, and the hanger shaft is gradually pushed until the pin is inserted in place; the electric push rod and the hanger shaft connecting device are removed, the electric push rod is retracted, and the lifting platform is lowered to the lowest position; the locking pin of the shift shaft device and the locking beam is removed, and the shift shaft device is retreated and pulled away from the locking beam.

[0077] S2: image preprocessing is performed, and SIFT algorithm is used to automatically detect key points and feature points of the image.

[0078] Further, the image preprocessing includes gray processing based on color features, image filtering, and binary processing of the image to realize ROI rough positioning.

[0079] Further, a suitable gray scale transformation method is selected to transform the three-channel color RGB image into a single-channel gray scale image with the same value range; the gray scale value of the pixel point P(x, y) after gray scale transformation is set as Gray(x, y).

[0080] Further, the average gray processing method takes the average value of the RGB three channel component values as the gray value of the pixel point after gray scale conversion, and the formula is expressed as:

[0081]

[0082] Further, the weighted average gray processing method is used to generalize the average gray processing method, and the gray value of the pixel point after gray scale conversion is calculated by assigning appropriate weights to the RGB three channel component values, and the formula is expressed as:

[0083] Gray(x,y)=r1R(x,y)+r2G(x,y)+r3B(x,y)

[0084] Wherein, R(x,y) represents the pixel point of R channel, G(x,y) represents the pixel point of G channel, and B(x,y represents the pixel point of B channel; r1, r2, r3 represent the initial weight assigned to each channel component.

[0085] Further, the ROI coarse positioning process is based on the gray scale image after smoothing processing in the previous section, and the inverse binary threshold processing is performed, wherein the threshold thresh is selected, and the formula is expressed as:

[0086]

[0087] Further, the center point of the largest connected domain is obtained, and it can be seen from the binary image that the position of the largest connected domain is the position of the target to be detected, and the center point is also the approximate center point of the region. After obtaining the center point position, the ROI coarse positioning can be performed according to the size information of the circular hole.

[0088] Further, the ROI coarse positioning region is obtained, based on the approximate position of the target center point obtained above, and the rectangular frame obtained by expanding a certain number of pixel points from the center point to the surrounding according to the size information is the coarse positioning region containing the target to be detected.

[0089] The ROI coarse positioning region is cropped, and the subsequent processing process only needs to process the coarse positioning region.

[0090] Further, the use of SIFT algorithm to automatically detect the key points and feature points of the image includes performing scale space extreme value detection, searching for image positions in all scales, and identifying potential interest points that are invariant to scale and rotation through a Gaussian differential function.

[0091] All the initial detected key points are counted to calculate the total number of key points.

[0092] If the number of key points is far beyond the preset threshold range, adjust the Gaussian blur parameter to reduce redundant key points. If the number of key points is too small, the detection standard may need to be relaxed to obtain more feature points.

[0093] Perform spatial analysis on the distribution of key points in the image to calculate the distribution of key points in the image area. Grid division or clustering algorithm can be used to evaluate the uniformity of the distribution. The distribution uniformity analysis formula divides the image into m x n grid areas, and the number of key points in each grid area is N ij , calculate the uniformity of key point distribution. Variance is used to measure the uniformity of the distribution:

[0094]

[0095] Where μ is the average number of key points per grid, and the calculation formula is:

[0096]

[0097] If the variance exceeds the set threshold, it means that the distribution is uneven and needs to be adjusted.

[0098] Adjust the Gaussian blur scale. If the number of key points is small or the distribution is uneven, you can refine the detection by increasing the number of Gaussian blur scales to allow more features to be identified. If the number of key points is too large and redundant, you can reduce the Gaussian blur scale or increase the interval between scales to reduce the detection of details and prevent excessive redundant features.

[0099] According to the new Gaussian blur parameter and threshold setting, re-construct the scale space and detect key points.

[0100] Recount and analyze the newly detected key points to verify whether the adjustment achieves the expected effect. If the result is still not ideal, continue to trigger feedback adjustment to further adjust the parameters.

[0101] Iterative feedback formula, in the i-th feedback iteration, assuming the number of detected key points is The ideal key point number range is [ N min ,N max] . The feedback adjustment formula is:

[0102]

[0103] The iteration termination condition can be set to reach the target key point number range or exceed the maximum iteration number iter max :

[0104] Stop if or i≥itermax

[0105] The target key point number range is set as [ N min ,N max] The feedback condition is:

[0106] N min ≤N keypoints ≤N max

[0107] The distribution uniformity condition is that the distribution variance threshold is set as The feedback condition is:

[0108]

[0109] The key points are accurately positioned, in each candidate position, a fitting fine model is used to determine the position and scale, and the key points are selected according to their stability.

[0110] The main direction of the key point is determined, one or more directions are assigned to each key point position based on the gradient direction of the image local, and all subsequent operations on the image data are transformed relative to the direction, scale and position of the key point, thereby providing invariance to these transformations;

[0111] The SIFT feature vector is generated, and the gradient of the image local is measured in the neighborhood around each key point at the selected scale.

[0112] It should be explained that the present application uses an industrial camera to collect the image between the pull rod and the hanging shaft, and then transmits it to an industrial computer, detects and describes the feature points and edge detection through the OpenCV vision algorithm, obtains the spatial position and edge of the shaft hole, and realizes the conversion between different vision coordinate systems and the mechanical arm coordinate system through camera calibration, finally transmits the vision information to the industrial control center, and controls the multi-degree-of-freedom platform through the PLC to work on the shaft, and completes the automatic and rapid alignment between the pull rod and the hanging shaft and the pull rod.

[0113] S3: Establishing a coordinate system generates an internal parameter matrix, establishing a camera coordinate system generates an external parameter matrix, and combining the internal parameter matrix and the external parameter matrix to perform coordinate conversion.

[0114] Further, the establishment of the coordinate system generates the internal parameter matrix, which includes that the pixel coordinate system is the coordinate of the image about the determined target point, the coordinate system with the basic unit of pixels is established on the photosensitive element of the camera, and there is a point (u0, v0) in the pixel coordinate system.

[0115] The image coordinate system is a coordinate system with physical length as the unit, which is also established on the photosensitive element of the camera, and the image coordinate is (u, v):

[0116]

[0117] Convert the image coordinates (u,v) to normalized camera coordinates (x,v). n ,y n The formula is expressed as:

[0118]

[0119] Among them, f x and f y The pixel unit representing the camera's focal length; c x and c y Represents the pixel coordinates of the principal point of the image;

[0120] Normalized camera coordinates are transformed into the camera coordinate system by mapping the normalized camera coordinates to the camera coordinate system. Assume the object's depth Z... c Given that, the formula is expressed as:

[0121] X c =x n ·Z c ,Y c =y n ·Z c Z c =Z c

[0122] The camera coordinate system is a reference system that uses the camera as a reference to describe the location of a target, with physical length as the unit. The x and y axes are defined on the same axis as the x and y axes, respectively. The x axis is perpendicular to the camera's image sensor, and the axis furthest from the image sensor is considered positive. The center of the lens is the origin of this coordinate system. Therefore, the coordinates of a point P(x,y) in the camera coordinate system are p c (x c ,y c ,z c ).

[0123] like Fig. 3 As shown, if there exists a point in the camera coordinate system, and that point is in the image coordinate system, then by the similarity theorem, we can obtain the following equation:

[0124]

[0125] From the above formula, we can see that the coordinate transformation involves a negative sign, so the imaging plane can be transformed, such as... Fig. 3 As shown in the dashed plane, the transformation relationship is as follows.

[0126]

[0127] For ease of calculation, the above equation is transformed into a homogeneous coordinate matrix form, as shown in the following equation:

[0128]

[0129] The inverse transformation is shown in the following formula:

[0130]

[0131] If the relationship of the above formula is substituted into the arrangement, it is shown in the following formula.

[0132]

[0133] The world coordinate system is introduced in the vision system, only the position of the target in the world coordinate system is needed, and the external parameter matrix is obtained by the corresponding formula; it is assumed that there is a point in the world coordinate system, and the corresponding coordinates are and the relationship between the two is shown in the following formula.

[0134] p c =Rp w +T

[0135] Wherein, R represents the rotation matrix, T represents the translation, and p w represents the coordinates of the point in the world coordinate system;

[0136]

[0137] T=(t x ,t y ,t z ) T

[0138] For convenient calculation, the above three formulas can be jointly expressed in the form of homogeneous coordinate matrix, as shown in the following formula.

[0139]

[0140] Wherein, the pose of the camera coordinate system is determined, which is called external parameter, that is, the pose of the camera, and the matrix M is called external parameter matrix, and the inverse transformation is shown in the following formula.

[0141]

[0142] Comprehensive the whole process, the image coordinates (u, v) are converted into the world coordinate system through the intrinsic parameter and the external parameter.

[0143] It should be noted that the pixel coordinate system is a coordinate system with pixel as the basic unit for determining the coordinates of target points, and is established on the light sensing element of the camera. The horizontal direction is defined as the Opu axis, with the right direction as positive, and the vertical direction is defined as the Opv axis, with the downward direction as positive. The upper left corner of the image is defined as the origin Op of the coordinate system. The image coordinate system is a coordinate system with physical length as the unit for determining the coordinates of different points, and is also established on the light sensing element of the camera. The Oix axis and the Oiy axis are defined in the same direction as the Opu axis and the Opv axis, respectively. The projection point of the optical axis on the light sensing element is defined as the origin Oi of the coordinate system, representing the position of the image coordinate system origin in the pixel coordinate system, which is generally 1 / 2 of the image pixel size.

[0144] S4: Calibrate the target positioning by the camera calibration method.

[0145] Further, the calibration by combining the intrinsic and extrinsic matrices includes camera calibration, selection of a suitable chessboard calibration board for printing, and determination of the physical size of the related points of the calibration board to ensure the accuracy of the calibration board.

[0146] The calibration board is fixed on a certain plane board, and the position and angle of the calibration board are changed for multiple shootings within the field of view of the camera.

[0147] The corner feature points of the chessboard calibration board in the image are detected by an algorithm.

[0148] Distortion coefficient solving and optimization: After the initial solving of the camera intrinsic and extrinsic parameters, the image distortion factors are considered. The radial distortion and tangential distortion models are established as follows:

[0149] x' = x + (x - c x )(k1r 2 +k2r 4 )+2p1xt+p2(r 2 +2x 2 )

[0150] y' = y + (y - c y )(k1r 2 +k2r 4 )+p1(r 2 +2y 2 )+2p2xy

[0151] where k1, k2 are the radial distortion coefficients, p1, p2 are the tangential distortion coefficients, r 2 = x 2 +y 2 is the square distance to the optical axis.

[0152] Least squares method and maximum likelihood method optimization: the least squares method is used to solve the preliminary distortion coefficients k1, k2, p1, p2, and the following error function is minimized:

[0153]

[0154] where x ij is the pixel coordinate under ideal distortion-free condition, x' ij is the pixel coordinate after actual distortion. The estimated distortion parameters are further optimized by using the maximum likelihood method to obtain more accurate intrinsic and extrinsic parameters. According to the pixel coordinates and physical coordinates of the detected feature points, the parameter values are solved.

[0155] It should be noted that the improved traditional calibration method not only has the low error advantage of the camera self-calibration method, but also does not have the cumbersome process and high requirement disadvantage of other traditional calibration methods. It combines the characteristics of traditional methods and self-calibration methods.

[0156] Embodiment 2, which is an embodiment of the present application, provides a target positioning method based on machine vision. In order to verify the beneficial effects of the present application, economic benefit calculation and simulation experiments are carried out for scientific demonstration.

[0157] This embodiment takes the alignment and detection of parts in an automated assembly line as the test object, and the purpose is to realize high-precision target position detection and alignment through the target positioning method based on machine vision. The experimental system includes two high-resolution industrial cameras (camera 1 and camera 2), a variety of optional lenses, an LED light source and a control computer. The selection of system hardware is based on the size of the target part, surface features, environmental light and other conditions. Camera 1 is used to capture the overall outline of the part from the front, and camera 2 is used to capture key feature points from the side. The focal length of the lens is determined by calculating the field of view size and the detection distance to ensure the accuracy and clarity of the captured image.

[0158] In the experiment, first, the selected parts are image collected. In the image preprocessing process, the collected color image is grayed and filtered to remove noise, and the target detection area is further determined through ROI (region of interest) coarse positioning. In the preprocessed image, the SIFT (scale-invariant feature transform) algorithm is used to extract key points and feature points. The feature point extraction process includes scale space extreme value detection, key point precise positioning and main direction assignment steps to ensure stability under different viewing angles and lighting conditions. Subsequently, the system establishes a pixel coordinate system and an image coordinate system, and generates a corresponding intrinsic matrix to describe the imaging characteristics of the camera. The relationship between the camera coordinate system and the world coordinate system is described by an extrinsic matrix, which realizes accurate mapping from image coordinates to camera coordinates in combination with the intrinsic matrix.

[0159] Finally, camera calibration was performed using a standard chessboard calibration plate. After taking multiple images of the calibration plate from different angles, the algorithm detected the pixel coordinates of the checkerboard grid points and the known physical coordinates, generated and optimized the internal and external parameter matrices. Through error correction by least squares method and maximum likelihood method, high precision and low error performance of the system in target positioning was ensured. Part of the results are shown in Table 1.

[0160] Table 1 Data Record Table

[0161]

[0162]

[0163] From the table data, it can be seen that through this embodiment, the target positioning method based on machine vision realizes a high key point matching success rate and a low positioning error. Compared with the existing system based on a simple feature extraction algorithm, this embodiment exhibits extremely high stability and robustness on different experimental objects. In the data, the positioning error of part C is 0.4 mm, and the key point matching success rate reaches 96%, while the traditional system can only achieve a matching rate of about 90% and a positioning error of 0.7-1.0 mm in a similar scenario, which indicates the advantages of using SIFT algorithm in complex environments. The imaging clarity index of each part generally remains at a high level with the support of high-precision hardware selection, ensuring the stability of image acquisition.

[0164] Embodiment 3, as an embodiment of the present application, provides a target positioning system based on machine vision, comprising:

[0165] An image acquisition module selects appropriate machine vision key hardware to acquire image information meeting accuracy and clarity requirements, installs multiple pull rod alignment vision cameras and suspension shaft hole alignment vision cameras to accurately detect the deviation distance of the two pull rods and the suspension shaft.

[0166] An image preprocessing module preprocesses the acquired images to improve image quality and feature extraction accuracy; preprocessing includes denoising, grayscale, edge enhancement, feature point detection, and feature point extraction.

[0167] A coordinate system establishment and internal and external parameter matrix generation module establishes a pixel coordinate system and an image coordinate system, and establishes a camera coordinate system and a world coordinate system to realize coordinate conversion and accurately map the target position in the image to the actual space position.

[0168] An automatic calibration module uses a calibration plate to calibrate the camera, calculates the internal and external parameter matrices through image acquisition from multiple angles and distances, corrects the geometric distortion of the camera to ensure accurate imaging, and realizes automatic calibration and positioning through PLC control and processing and feedback of received vision detection data.

[0169] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions of the present application can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0170] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered a list of executable instructions for implementing logic functions, and can be specifically embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with these instructions execution systems, apparatuses, or devices. For the purpose of this specification, the "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport programs for use by an instruction execution system, apparatus, or device, or in conjunction with these instruction execution systems, apparatuses, or devices.

[0171] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by editing, interpreting, or otherwise processing, if necessary, in other suitable ways, to be electronically obtained, and then stored in the computer memory.

[0172] It should be understood that portions of the present application can be implemented with hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, implementation can be with any or a combination of the following technologies, which are all well known in the art: a discrete logic circuit having logic gates for implementing logic functions upon an application of data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be understood that the foregoing embodiments are merely illustrative of the present application and are not to be used to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, it will be apparent to those skilled in the art that various changes and modifications can be contributed to the present application without departing from the spirit and scope of the present application, and such changes and modifications should be encompassed within the scope of the appended claims.

[0173] It should be understood that the foregoing embodiments are merely illustrative of the present application and are not to be used to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, it will be apparent to those skilled in the art that various changes and modifications can be contributed to the present application without departing from the spirit and scope of the present application, and such changes and modifications should be encompassed within the scope of the appended claims.

Claims

1. A target localization method based on machine vision, characterized in that, include: Select key hardware for machine vision and acquire image information; Image preprocessing is performed, and the SIFT algorithm is used to automatically detect key points and feature points in the image; Establish a coordinate system to generate an intrinsic parameter matrix, establish a camera coordinate system to generate an extrinsic parameter matrix, and perform coordinate transformation by combining the intrinsic and extrinsic parameter matrices; Target positioning and calibration are performed using camera calibration methods; The image preprocessing includes grayscale processing based on color features, image filtering, and binarization of the image to achieve coarse ROI localization. Choose an appropriate grayscale transformation method to transform the three-channel color RGB image into a single-channel grayscale image with the same value range; set the grayscale value at pixel P(x,y) after grayscale transformation to Gray(x,y); The average grayscale processing method uses the average value of the RGB three-channel components as the grayscale value of the corresponding pixel after grayscale transformation. The formula is expressed as: The weighted average grayscale processing method is used, which is generalized by assigning appropriate weights to the RGB three-channel component values ​​and then calculating the grayscale value of the corresponding pixel after grayscale transformation. The formula is expressed as: Gray(x,y)=r1R(x,y)+r2G(x,y)+r3B(x,y) Where R(x,y) represents the pixel in the R channel, G(x,y) represents the pixel in the G channel, and B(x,y) represents the pixel in the B channel; r1, r2, and r3 represent the initial weights assigned to each channel component. The ROI coarse localization process is based on the smoothed grayscale image from the previous section, and undergoes inverse binarization thresholding. During this process, a threshold of thresh is selected, and the formula is expressed as: Where src(x,y) represents the pixel value at coordinates (x,y) in the input image; dst(x,y) represents the pixel value at coordinates (x,y) in the output image; and maxval represents the maximum value to be assigned to the pixels that meet the conditions. Obtain the center point of the largest connected component. From the binary image, it can be seen that the location of the largest connected component is the location of the target to be detected, and its center point is also the approximate center point of the region. After obtaining the center point location, the ROI can be coarsely located based on the hole size information. To obtain the coarse ROI location area, based on the approximate location of the target center point obtained above, the rectangle obtained by expanding several pixels outward from the center point according to the size information is the coarse location area containing the target to be detected. Perform coarse ROI positioning region clipping; subsequent processing only needs to process the coarse positioning region. The automatic detection of key points and feature points in an image using the SIFT algorithm includes performing scale-space extremum detection, searching for image locations at all scales, and identifying potential scale- and rotation-invariant interest points using the Gaussian differential function. When performing scale-space extremum detection, a dynamic feedback adjustment mechanism is added to monitor the number and distribution of detected key points in real time; if too few or too many key points are detected at a specific scale, the scale parameters of the Gaussian blur are automatically adjusted. To achieve precise localization of key points, a finely fitted model is used to determine the location and scale at each candidate location, with the selection of key points based on their stability. The principal orientation of keypoints is determined based on the gradient orientation of the local image, and one or more orientations are assigned to each keypoint location. All subsequent operations on the image data are transformed relative to the orientation, scale, and location of the keypoints, thus providing invariance to these transformations. Generate SIFT feature vectors and measure the local gradient of the image at a selected scale within the neighborhood around each keypoint.

2. The target localization method based on machine vision as described in claim 1, characterized in that: The selection of key hardware for machine vision includes selecting the light source and lighting method, and determining the camera resolution and lens. When selecting a lens, the primary consideration is focal length; let the field of view be FOV, with FOV for the horizontal and vertical directions respectively. H and FOV V The horizontal and vertical dimensions of the camera chip target surface are S respectively. H and S V The detection distance is d w Then the formulas for calculating the horizontal and vertical components of the focal length f are expressed as follows: Among them, f H f represents the horizontal focal length. V This indicates the vertical focal length.

3. The target localization method based on machine vision as described in claim 2, characterized in that: The image information acquisition includes image acquisition using two cameras; the first camera acquires images of the front of the pull rod's circular hole and the end face of the pull rod's rear suspension shaft, detecting the alignment of the pull rod with the hole and the alignment of the pull rod with the suspension shaft; the second camera acquires side images of the pull rod and suspension shaft, detecting the alignment of the pull rod with the hole and the degree of pin penetration of the suspension shaft.

4. The target localization method based on machine vision as described in claim 3, characterized in that: The coordinate transformation combining intrinsic and extrinsic parameter matrices includes: a pixel coordinate system, which is the coordinates of the image with respect to a defined target point, with the basic unit being pixels, established on the camera's photosensitive element; and an image coordinate system, which is a coordinate system with physical length as the unit, also established on the camera's photosensitive element. Convert the image coordinates (u,v) to normalized camera coordinates (x,v). n ,y n The formula is expressed as: Among them, f x and f y The pixel unit representing the camera's focal length; c x and c y Represents the pixel coordinates of the principal point of the image; Normalized camera coordinates are transformed into the camera coordinate system by mapping the normalized camera coordinates to the camera coordinate system. Assume the object's depth Z... c Given that, the formula is expressed as: X c =x n ·Z c ,Y c =y n ·Z c ,Z c =Z c The camera coordinate system is a reference system that uses the camera as a reference to describe the location of a target, with physical length as the unit. The x and y axes are defined on the same axis as the x and y axes, respectively. The x axis is perpendicular to the camera's image sensor, and the axis furthest from the image sensor is considered positive. The center of the lens is the origin of this coordinate system. Therefore, the coordinates of a point P(x,y) in the camera coordinate system are p c (x c ,y c ,z c ); Introducing a world coordinate system into a vision system allows us to determine the extrinsic parameter matrix simply by knowing the target's position within that system and using a corresponding formula. Assuming a point exists in the world coordinate system with coordinates [x, y], the relationship between the two is shown in the following equation: p c =Rp w +T Where R represents the rotation matrix, T represents the translation amount, and p w Represents the coordinates of a point in the world coordinate system; In summary, the image coordinates (u,v) are converted into world coordinates through intrinsic and extrinsic parameters.

5. The target localization method based on machine vision as described in claim 4, characterized in that: The camera calibration method includes performing camera calibration, selecting a suitable chessboard calibration board for printing, and determining the physical dimensions of relevant points on the calibration board to ensure the accuracy of the calibration board. Fix the calibration plate to a flat plate, and take multiple pictures by changing the position and angle of the calibration plate within the camera's field of view; Algorithms were used to detect the corner feature points of a chessboard calibration board in an image; Distortion coefficient solution and optimization: After initially solving the camera's intrinsic and extrinsic parameters, image distortion factors are considered; radial and tangential distortion models are established. x'=x+(x-cx)(k1r 2 +k2r 4 )+2p1xy+p2(r 2 +2x 2 ) y'=y+(y-cy)(k1r 2 +k2r 4 )+p1(r 2 +2y 2 +2p2xy Where k1,k2 are radial distortion coefficients, p1,o2 are tangential distortion coefficients, and r 2 =x 2 +y 2 The squared distance to the optical axis; The least squares method and maximum likelihood method are used for optimization. The least squares method is used to solve for the initial distortion coefficients k1, k2, p1, p2, and the following error function is minimized: Where, x ij x represents the pixel coordinates under ideal, distortion-free conditions. ' ij The actual distorted pixel coordinates are used; the maximum likelihood method is used to further optimize the estimated distortion parameters to obtain more accurate intrinsic and extrinsic parameters. The parameter values ​​are solved based on the pixel coordinates and physical coordinates of the detected feature points.

6. A system employing the machine vision-based target localization method as described in any one of claims 1 to 5, characterized in that: It includes an image acquisition module, an image preprocessing module, a coordinate system establishment and intrinsic / extrinsic parameter matrix generation module, and an automatic calibration module; The image acquisition module selects appropriate key machine vision hardware to acquire image information that meets the accuracy and clarity requirements. Multiple tie rod alignment vision cameras and hanger shaft hole alignment vision cameras are installed to accurately detect the deviation distance between the two tie rods and hanger shafts. The image preprocessing module preprocesses the acquired images to improve image quality and the accuracy of feature extraction. Preprocessing includes denoising, grayscale conversion, edge enhancement, feature point detection, and feature point extraction; The coordinate system establishment and intrinsic / extrinsic parameter matrix generation module establishes pixel coordinate system and image coordinate system, camera coordinate system and world coordinate system, realizes coordinate transformation, and accurately maps the target position in the image to the actual spatial position. The automatic calibration module uses a calibration board to calibrate the camera. It calculates the intrinsic and extrinsic parameter matrices by acquiring images from multiple angles and distances, thereby correcting the camera's geometric distortion and ensuring accurate imaging. The system is controlled by a PLC to process and feed back the received visual inspection data, achieving automatic calibration and positioning.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the target localization method based on machine vision as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the target localization method based on machine vision as described in any one of claims 1 to 5.

Images