A visual positioning method, system, device and medium based on three-dimensional model key size

Abstract

The application provides a three-dimensional model key size-based visual positioning method, system, device and medium, and belongs to the technical field of industrial automation. The method comprises the following steps: acquiring three-dimensional model data of a workpiece, calculating a theoretical center of the workpiece, and extracting key geometric dimensions to generate a size matrix; a global image of the workpiece is shot by a camera, edge detection and contour extraction are performed, a center coordinate is preliminarily estimated, a large field-of-view camera is controlled to move to a predicted position of a first feature point, and then the distance between the camera and the workpiece is shortened to a close distance; a local image of the first feature point is re-shot at the close distance, the actual coordinate of the first feature point is extracted, and the error between the actual coordinate and the model size is calculated; the second and third feature points are resampled to obtain errors; the preliminarily estimated center coordinate is corrected according to the errors, and a final high-precision center coordinate is obtained. The application realizes low-cost high-precision positioning, overcomes machining errors and environmental interference, and improves industrial production efficiency and product quality.

Description

Technical Field

[0001] This application belongs to the field of industrial automation technology, specifically relating to a visual positioning method, system, device, and medium based on key dimensions of a three-dimensional model. Background Technology

[0002] In industrial production, high-precision visual positioning is a key element in achieving production automation, intelligent manufacturing, and quality control. With the development of industrial automation, more and more production processes require precise positioning systems to complete complex tasks, such as robotic assembly, precision machining, and automated inspection. These tasks demand that positioning systems be able to quickly and accurately identify and locate workpieces to ensure the efficiency of the production process and product quality.

[0003] Visual positioning systems typically acquire images of target workpieces using cameras, and then identify and locate the workpieces based on the pixel positions within the images. However, firstly, in practical applications, cameras with large fields of view often have lower positioning accuracy because pixel density decreases as the field of view expands. Secondly, workpiece machining errors, variations in lighting conditions, and limitations of image extraction algorithms further affect positioning accuracy. Thirdly, traditional visual positioning methods usually rely on high-resolution cameras or complex hardware configurations to improve accuracy, which not only increases costs but also limits the applicability of the technology.

[0004] To address these issues, several improved methods have been proposed, such as multi-feature point matching or introducing 3D models to assist localization. However, these methods still suffer from high computational complexity and strong hardware dependence when achieving high-precision localization. Therefore, there is an urgent need for a method that can achieve high-precision visual localization under low-cost hardware conditions, while effectively overcoming the effects of manufacturing errors and environmental interference. Summary of the Invention

[0005] In a first aspect, embodiments of this application provide a visual positioning method based on key dimensions of a three-dimensional model, comprising the following steps: S1. Obtain the 3D model data of the workpiece, and calculate the theoretical center coordinates of the workpiece based on the 3D model data. And extract key geometric dimensions to generate a dimension matrix. ; S2. Take a global image of the workpiece using a wide-view camera, perform edge detection and contour extraction on the global image of the workpiece, and obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Preliminary location of the pixel coordinates of the geometric center of the workpiece contour ; Based on the intrinsic parameter matrix calibrated by the wide field-of-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Converting to world coordinates yields a preliminary estimate of the center coordinates. ; S3. Based on the dimension matrix and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; S4. At close range The first feature of the reshoot The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error ; S5. Repeat steps S3-S4, sequentially processing the second feature point. The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; S6. Based on error , , Preliminary estimation of center coordinates After correction, the final high-precision center coordinates are obtained. .

[0006] Furthermore, the specific steps of step S1 are as follows: S11. Obtain and parse the STEP or IGES file of the workpiece's 3D model to obtain the vertex coordinate set of the workpiece's bounding box. , where n represents the number of vertices; S12. Based on vertex coordinate set Calculate the coordinates of the theoretical center : ; S13. Feature points of the workpiece Perform localization and calculate feature points. Relative to the theoretical center coordinates Geometric distance, generate size matrix ; S14. Calculate feature points Relative to the theoretical center coordinates Unit direction vector in the model coordinate system : ; unit direction vector With size matrix Associated storage.

[0007] Furthermore, the specific steps in step S2 are as follows: S21. Use a wide-view camera to photograph the workpiece and obtain a global image containing the entire workpiece; S22. Perform Gaussian filtering to remove noise from the global image, and use the Canny operator to detect the workpiece edges; S23. Fill the edge gaps using morphological closing operations to generate a continuous closed workpiece contour. Then, extract the minimum bounding rectangle based on the pixel set of the continuous closed workpiece contour, and calculate the pixel coordinates of the center of the minimum bounding rectangle as the geometric center of the workpiece contour. ; S24. Pixel coordinates of the geometric center of the workpiece contour Using a predetermined region around the initial point as the starting point, extract the image from the global image and calculate the accurate pixel center coordinates using the gray-level weighted centroid method. ; S25. Obtain the actual distance from the wide-field-of-view camera to the workpiece using a laser rangefinder. Combined with camera intrinsic parameter matrix Accurate pixel coordinates Preliminary estimate of center coordinates in world coordinate system .

[0008] Furthermore, the specific steps of step S3 are as follows: S31. Select any one of the feature points of the workpiece as the first feature point. ; S32. According to the dimension matrix The first feature point in Model size and preliminary estimation of center coordinates And based on the first feature point Unit direction vector in the model coordinate system Calculate the first feature point Predicted world coordinates : ; S33. Control the six-axis robotic arm to carry the camera and move the wide-field camera linearly to the first feature point according to the preset precision. Predicted world coordinates ; S34. The wide-view camera is moved along the normal direction of the workpiece surface by a servo motor, increasing the working distance from... shorten to close distance :

[0009] in, Focal length It is the pixel size. It is the minimum size of the feature point.

[0010] Furthermore, the specific steps of step S4 are as follows: S41. At close range The following steps involve acquiring local images and performing illumination equalization processing: ; in, The original image at coordinate points grayscale value at that location The image after illumination equalization is at coordinate points grayscale value at that location Based on coordinate points The average gray value of all pixels within a local image window centered on the image. Based on coordinate points The standard deviation of gray levels of all pixels within a local image window centered on the image. S42. Employ a sub-pixel edge detection algorithm based on Zernike moments to calculate the normal angle of edge points. ; S43. Based on edge point normal angle Fit an edge line within a range of ±N pixels along the normal direction, and set the intersection point as the first feature point. coordinate:

[0011] in, It is the first feature point The center of the line is located at sub-pixel coordinates in the image pixel coordinate system, and k represents the k-th edge line. It is a point on the k-th edge line. It is the normal angle of the k-th edge line. It is used as the first feature point The actual coordinates of the precise center coordinates; S44. Obtain the first feature point Model size Calculate the first feature point The error between the actual coordinates and the model size .

[0012] Furthermore, the specific steps of step S5 are as follows: S51. Select a second feature point from the feature points of the workpiece according to the following spatial symmetry conditions. :

[0013] in, ; S52. Repeat steps S31-S34 to move the wide-field camera to the second feature point. Predicted world coordinates :

[0014] in, Based on the second feature point The unit direction vector in the model coordinate system; S53. Repeat steps S41-S44 to obtain the second feature point. actual coordinates With model size error ; S54. Select a third feature point from the feature points of the workpiece according to the following non-coplanar conditions. :

[0015] in, ; S55. Repeat steps S31-S34 to move the wide-view camera to the third feature point. Predicted world coordinates :

[0016] in, Based on the third feature point The unit direction vector in the model coordinate system; S56. Repeat steps S41-S44 to obtain the third feature point. actual coordinates With model size error ; S57. Apply the following formula to the second feature point. Third feature point Adjust the working distance during sampling:

[0017] Where k=2 corresponds to the second feature point. k=3 corresponds to the third feature point , This is the adjusted working distance of the wide-view camera.

[0018] Furthermore, step S6 is detailed as follows: S61. Calculate the overall error using the model size of each feature point as the weight. : ; S62. Dynamically adjust the preset weighting coefficients according to the error amplitude. : ; S63. Calculate the final high-precision center coordinates. :

[0019] S64. Verify the final high-precision center coordinates. Does it satisfy one of the following convergence conditions:

[0020]

[0021] in, This is a preset proportional coefficient. This is a preset distance error threshold; Output when any one of the convergence conditions is met. Perform visual positioning; When neither of the two convergence conditions is satisfied, let Return to step S3.

[0022] Secondly, embodiments of this application also provide a visual positioning system based on key dimensions of a three-dimensional model, comprising: The model parsing and dimension construction module is used to acquire the 3D model data of the workpiece and calculate the theoretical center coordinates of the workpiece based on the 3D model data. And extract key geometric dimensions to generate a dimension matrix. ; The global vision coarse localization module is used to capture a global image of the workpiece using a wide-field-of-view camera, perform edge detection and contour extraction on the global image of the workpiece, and obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Preliminary location of the pixel coordinates of the geometric center of the workpiece contour ; Based on the intrinsic parameter matrix calibrated by the wide field-of-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Convert to coordinates in the world coordinate system to obtain a preliminary estimate of the center coordinates; The feature point dynamic sampling module is used to sample points based on the size matrix. and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; Local visual precision positioning module, used for close-range positioning. The first feature of the reshoot The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error ; The multi-feature point collaborative calibration module is used to sequentially calibrate the second feature point based on the feature point dynamic sampling module and the local visual fine localization module. The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; The error feedback iterative correction module is used to correct errors. , , Preliminary estimation of center coordinates After correction, the final high-precision center coordinates are obtained. .

[0023] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the visual positioning method based on key dimensions of a three-dimensional model as described in the first aspect.

[0024] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the visual positioning method based on key dimensions of a three-dimensional model as described in the first aspect.

[0025] As can be seen from the above technical solutions, this application has the following advantages: The visual positioning method, system, device, and medium based on key dimensions of a 3D model provided in this application improve positioning accuracy and reduce dependence on high-resolution cameras by dynamically adjusting the camera position and performing multiple samplings, combined with 3D model dimensions and error feedback, making it suitable for low-cost hardware environments. By introducing 3D model dimensions, the influence of workpiece processing errors and environmental interference is overcome. Through multi-feature point collaborative calibration and error feedback iterative correction, the stability and reliability of the system are improved. It is applicable to various industrial scenarios, including robot assembly, precision machining, and automated inspection. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating the visual positioning method based on key dimensions of a 3D model according to the present invention.

[0028] Figure 2This is a schematic diagram of the visual positioning system based on key dimensions of a three-dimensional model according to the present invention. Detailed Implementation

[0029] The various embodiments of this disclosure will be described more fully in the following detailed description of the specific steps of the visual positioning method based on the key dimensions of a 3D model. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0030] For example, in today's industrial production, high-precision visual positioning technology plays an indispensable and crucial role in promoting production automation, intelligent manufacturing, and quality control. With the acceleration of industrial automation, many production processes increasingly rely on precise positioning systems to perform increasingly complex operations. Typical examples include robotic assembly operations, precision parts machining, and automated product inspection. These tasks place stringent demands on positioning systems, requiring them to possess rapid and accurate workpiece identification and positioning capabilities to ensure the efficient operation of production processes and the stability of product quality.

[0031] Existing visual positioning systems primarily rely on cameras to acquire images of workpieces, then use the positional information of pixels within those images to identify and locate the workpiece. However, this technology faces numerous challenges in practical applications: firstly, when a camera needs to cover a large field of view, a decrease in pixel density is inevitable, directly leading to a drop in positioning accuracy; secondly, factors such as processing errors in actual workpiece production, fluctuations in ambient lighting conditions, and the inherent limitations of image extraction algorithms can also negatively impact the accuracy of the positioning results. Furthermore, traditional visual positioning solutions typically rely on high-resolution cameras or complex hardware to improve accuracy, which not only significantly increases system costs but also limits the wider application and promotion of this technology.

[0032] While some improvements have been proposed, such as using multi-feature point matching strategies or introducing 3D models to assist in localization, these solutions still suffer from significant drawbacks when achieving high-precision positioning, including excessive computational complexity and strong dependence on hardware performance. Therefore, the industry urgently needs a method that can achieve high-precision visual positioning with low-cost hardware configurations. This method must also be able to effectively resist processing errors and environmental interference to meet the growing demand for precise positioning in industrial production.

[0033] To address the aforementioned issues, this embodiment provides a visual positioning method based on key dimensions of a 3D model. By dynamically adjusting the camera position and performing multiple samplings, it significantly improves positioning accuracy, reduces costs, overcomes processing errors and environmental interference, is applicable to various industrial scenarios, enhances system robustness, and improves production efficiency and product quality.

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figure 1 The diagram shows a flowchart of a visual positioning method based on key dimensions of a 3D model in a specific embodiment. The method includes the following steps: S1. Obtain the 3D model data of the workpiece, and calculate the theoretical center coordinates of the workpiece based on the 3D model data. And extract key geometric dimensions to generate a dimension matrix. ; It should be noted that by parsing the 3D model file, the set of vertex coordinates of the workpiece's bounding box is obtained, providing basic data for subsequent calculation of the theoretical center coordinates; based on the vertex coordinate set, the theoretical center coordinates of the workpiece are accurately calculated, providing a theoretical benchmark for subsequent positioning; key geometric dimensions are extracted and a dimension matrix is ​​generated, providing accurate dimensional data for subsequent feature point positioning and error calculation; the unit direction vector of the feature points relative to the theoretical center coordinates in the model coordinate system is calculated, providing directional information for subsequent calculation of the predicted position of the feature points; S2. Take a global image of the workpiece using a wide-view camera, perform edge detection and contour extraction on the global image of the workpiece, and obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Preliminary location of the pixel coordinates of the geometric center of the workpiece contour ; Based on the intrinsic parameter matrix calibrated by the wide field-of-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Converting to world coordinates yields a preliminary estimate of the center coordinates. ; It should be noted that capturing a global image of the workpiece using a wide-view camera provides basic image data for subsequent edge detection and contour extraction; edge detection and contour extraction improve image quality and the robustness of edge detection; the pixel coordinates of the workpiece's geometric center are initially located to improve the accuracy of the initial center coordinates; and the distance from the wide-view camera to the workpiece, combined with the camera's intrinsic parameter matrix, accurately transforms the pixel coordinates to the world coordinate system. S3. Based on the dimension matrix and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; It should be noted that the first feature point is selected from the feature points of the workpiece to provide a target for subsequent feature point positioning; based on the size matrix and the preliminary estimated center coordinates, combined with the unit direction vector of the feature point in the model coordinate system, the estimated world coordinates of the feature point are calculated; the camera is controlled to move to the estimated world coordinates of the feature point to ensure that the camera accurately reaches the target position; and the pixel density is increased by shortening the camera working distance. S4. At close range The first feature of the reshoot The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error ; It should be noted that by acquiring local images of feature points at close range, high-resolution image data is provided for subsequent sub-pixel level localization; a sub-pixel edge detection algorithm is used to accurately extract the actual coordinates of feature points; and by calculating the error between the actual coordinates of feature points and the model size, data is provided for subsequent error feedback correction. S5. Repeat steps S3-S4, sequentially processing the second feature point. The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; It should be noted that by selecting the second and third feature points, the spatial distribution of the feature points is ensured to be reasonable; by repeating the movement process in step S3, the camera is moved to the estimated positions of the second and third feature points; by repeating the image acquisition and processing process in step S4, the actual coordinates and errors of the second and third feature points are obtained; and the working distance is adjusted according to the sampling results of the feature points to optimize the pixel density and positioning accuracy. S6. Based on error , , Preliminary estimation of center coordinates After correction, the final high-precision center coordinates are obtained. ; It should be noted that by using the model size of each feature point as a weight to calculate the comprehensive error, comprehensive error data is provided for subsequent error feedback correction; through error feedback correction, the final high-precision center coordinates are calculated, thereby improving the accuracy of positioning.

[0036] This embodiment improves positioning accuracy by dynamically adjusting the camera position and performing multiple samplings, combined with 3D model dimensions and an error feedback mechanism; it reduces reliance on high-resolution cameras, making it suitable for low-cost hardware environments; and by introducing 3D model dimensions, it overcomes the influence of workpiece machining errors and environmental interference. The stability and reliability of the system are improved by multi-feature point collaborative calibration and error feedback iterative correction.

[0037] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, another visual positioning method based on key dimensions of a three-dimensional model is provided. This method includes the following steps: Take the following scenario as an example: The workpiece to be positioned is a rectangular metal plate. The 3D model is stored as a STEP file, and the robot needs to grasp it through visual positioning (positioning accuracy required ±0.05mm). A large field-of-view camera (resolution 1920×1080, focal length f=16mm, pixel size=3.45μm) is used, along with a six-axis robotic arm and a laser rangefinder. S1. Obtain the 3D model data of the workpiece, and calculate the theoretical center coordinates of the workpiece based on the 3D model data. And extract key geometric dimensions to generate a dimension matrix. ; The specific steps of step S1 are as follows: S11. Obtain and parse the STEP or IGES file of the workpiece's 3D model to obtain the vertex coordinate set of the workpiece's bounding box. , where n represents the number of vertices; For example, the workpiece STEP file is parsed to obtain the set of bounding box vertex coordinates (taking the 4 vertices of the upper surface). (Unit: mm); S12. Based on vertex coordinate set Calculate the coordinates of the theoretical center : ; For example, calculate the coordinates of the theoretical center. :

[0038] S13. Feature points of the workpiece Perform localization and calculate feature points. Relative to the theoretical center coordinates Geometric distance, generate size matrix ; For example, feature points are determined and the size matrix is ​​calculated: Feature points: (Center of the left circular hole, model coordinates (50,50,0)) (Center of the right circular hole, model coordinates (150, 50, 0)) (Protruding vertex, model coordinates (100, 80, 0)); Geometric distance (relative to) ): , , ; Size matrix ; S14. Calculate feature points Relative to the theoretical center coordinates Unit direction vector in the model coordinate system : ; unit direction vector With size matrix Associated storage; For example, calculate the unit direction vector: ( exist (Left side) ( exist (Right side) ( exist (Above) Related storage ; S2. Take a global image of the workpiece using a wide-view camera, perform edge detection and contour extraction on the global image of the workpiece, and obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Preliminary location of the pixel coordinates of the workpiece geometric contour center ; Based on the intrinsic parameter matrix calibrated by the wide field-of-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Converting to world coordinates yields a preliminary estimate of the center coordinates. ; The specific steps in step S2 are as follows: S21. Use a wide-view camera to photograph the workpiece and obtain a global image containing the entire workpiece; For example, a wide-view camera captures a global image (working distance d=500mm) containing the entire workpiece; S22. Perform Gaussian filtering to remove noise from the global image, and use the Canny operator to detect the workpiece edges; For example, the image is subjected to Gaussian filtering (kernel size 5×5), and edges are detected using the Canny operator (threshold [50,150]). S23. Fill the edge gaps using morphological closing operations to generate a continuous closed workpiece contour. Then, extract the minimum bounding rectangle based on the pixel set of the continuous closed workpiece contour, and calculate the pixel coordinates of the center of the minimum bounding rectangle as the geometric center of the workpiece contour. ; For example, a morphological closing operation (3×3 structuring element) fills the edge gaps to generate a closed contour, extracts the minimum bounding rectangle (pixel coordinate range: top left corner (200, 300), bottom right corner (1800, 900)), and calculates the center pixel coordinates of the rectangle. ; S24. Pixel coordinates of the geometric center of the workpiece contour Using a predetermined region around the initial point as the starting point, extract the image from the global image and calculate the accurate pixel center coordinates using the gray-level weighted centroid method. ; For example, in Within a pre-defined 5x5 pixel area, the precise center is calculated using the grayscale weighted centroid method:

[0039] in, Represents the grayscale value of a pixel; For example, in The precise pixel coordinates of the surrounding 5×5 pixel area (pixel coordinates (998-1002, 598-602)) are calculated using the gray-level weighted centroid method: Assume the grayscale matrix of this region is as follows:

[0040] The calculation yields: ,

[0041] S25. Obtain the actual distance from the wide-field-of-view camera to the workpiece using a laser rangefinder. Combined with camera intrinsic parameter matrix Accurate pixel coordinates Preliminary estimate of center coordinates in world coordinate system ; For example, a laser rangefinder measures the actual distance. Camera intrinsic parameter matrix Convert to world coordinates: ; S3. Based on the dimension matrix and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; The specific steps of step S3 are as follows: S31. Select any one of the feature points of the workpiece as the first feature point. ; S32. According to the dimension matrix The first feature point in Model size and preliminary estimation of center coordinates And based on the first feature point Unit direction vector in the model coordinate system Calculate the first feature point Predicted world coordinates : ; S33. Control the six-axis robotic arm to carry the camera and move the wide-field camera linearly to the first feature point according to the preset precision. Predicted world coordinates ; S34. The wide-view camera is moved along the normal direction of the workpiece surface by a servo motor, increasing the working distance from... shorten to close distance :

[0042] in, Focal length It is the pixel size. It is the minimum size of the feature point; For example, F1 is selected as the first feature point; Calculate F1 predicted world coordinates:

[0043] Control the six-axis robotic arm (positioning accuracy ±0.1mm) to move the camera to ; Adjust working distance to close range (Minimum size of feature points) : ; S4. At close range The first feature of the reshoot The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error ; The specific steps of step S4 are as follows: S41. At close range The following steps involve acquiring local images and performing illumination equalization processing: ; in, The original image at coordinate points grayscale value at that location The image after illumination equalization is at coordinate points grayscale value at that location Based on coordinate points The average gray value of all pixels within a local image window centered on the image. Based on coordinate points The standard deviation of gray levels of all pixels within a local image window centered on the image. S42. Employ a sub-pixel edge detection algorithm based on Zernike moments to calculate the normal angle of edge points. ; S43. Based on edge point normal angle Fit an edge line within a range of ±N pixels along the normal direction, and set the intersection point as the first feature point. coordinate:

[0044] in, It is the first feature point The center of the line is located at sub-pixel coordinates in the image pixel coordinate system, and k represents the k-th edge line. It is a point on the k-th edge line. It is the normal angle of the k-th edge line. It is used as the first feature point The actual coordinates of the precise center coordinates; S44. Obtain the first feature point Model size Calculate the first feature point The error between the actual coordinates and the model size ; For example, a close-up shot of a partial image at F1 resolution is taken, and then illumination equalization processing is performed (taking pixel (50,50) as an example): Mean grayscale value of a local window (9×9) Standard deviation Original grayscale ; Zernike subpixel edge detection is used to calculate the normal angle of edge points. (Radial normal to the edge of the circular hole); Fit two edge lines within a range of ±5 pixels along the normal direction, with the intersection point being... Actual coordinates ; Calculation error ; S5. Repeat steps S3-S4, sequentially processing the second feature point. The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; The specific steps of step S5 are as follows: S51. Select a second feature point from the feature points of the workpiece according to the following spatial symmetry conditions. :

[0045] in, (For example Take 0.3); For example, select (and exist Both sides, satisfy It meets the conditions of spatial symmetry. ; S52. Repeat steps S31-S34 to move the wide-field camera to the second feature point. Predicted world coordinates :

[0046] in, Based on the second feature point The unit direction vector in the model coordinate system; S53. Repeat steps S41-S44 to obtain the second feature point. actual coordinates With model size error ; For example, Predicted coordinates Adjust working distance The test results ,error ; S54. Select a third feature point from the feature points of the workpiece according to the following non-coplanar conditions. :

[0047] in, (For example Take 0.7); Select (keep away - The lines meet the condition of being non-coplanar. ); S55. Repeat steps S31-S34 to move the wide-view camera to the third feature point. Predicted world coordinates :

[0048] in, Based on the third feature point The unit direction vector in the model coordinate system; S56. Repeat steps S41-S44 to obtain the third feature point. actual coordinates With model size error ; For example, Predicted coordinates Adjust working distance The test results ,error ; S57. Apply the following formula to the second feature point. Third feature point Adjust the working distance during sampling:

[0049] Where k=2 corresponds to the second feature point. k=3 corresponds to the third feature point , This is the adjusted working distance of the wide-view camera; S6. Based on error , , Preliminary estimation of center coordinates After correction, the final high-precision center coordinates are obtained. ,in; The specific steps of step S6 are as follows: S61. Calculate the overall error using the model size of each feature point as the weight. : ; For example, the overall error is calculated (weighted by model size).

[0050] S62. Dynamically adjust the preset weighting coefficients according to the error amplitude. : ; For example, because ,Pick ; S63. Calculate the final high-precision center coordinates. :

[0051] For example, correct the center coordinates ; S64. Verify the final high-precision center coordinates. Does it satisfy one of the following convergence conditions:

[0052]

[0053] in, For example, a preset scaling factor. Take 0.1, For example, a preset distance error threshold. Take 0.05mm; Output when any one of the convergence conditions is met. Perform visual positioning; When neither of the two convergence conditions is satisfied, let Return to step S3; For example, deviation from the theoretical center: Original deviation ,satisfy ; Deviation from coarse positioning: Satisfying the threshold ; Output As a positioning point for grasping, the positioning accuracy meets the standards.

[0054] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0055] like Figure 2 As shown, the following are embodiments of the visual positioning system based on key dimensions of a 3D model provided in this disclosure. This system and the visual positioning methods based on key dimensions of a 3D model in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the visual positioning system based on key dimensions of a 3D model, please refer to the embodiments of the visual positioning methods based on key dimensions of a 3D model described above.

[0056] The system includes: The model parsing and dimension construction module is used to acquire the 3D model data of the workpiece and calculate the theoretical center coordinates of the workpiece based on the 3D model data. And extract key geometric dimensions to generate a dimension matrix. ; The global vision coarse localization module is used to capture a global image of the workpiece using a wide-field-of-view camera, perform edge detection and contour extraction on the global image of the workpiece, and obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Preliminary location of the pixel coordinates of the geometric center of the workpiece contour ; Based on the intrinsic parameter matrix calibrated by the wide field-of-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Convert to coordinates in the world coordinate system to obtain a preliminary estimate of the center coordinates; The feature point dynamic sampling module is used to sample points based on the size matrix. and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; Local visual precision positioning module, used for close-range positioning. The first feature of the reshoot The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error ; The multi-feature point collaborative calibration module is used to sequentially calibrate the second feature point based on the feature point dynamic sampling module and the local visual fine localization module. The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; The error feedback iterative correction module is used to correct errors. , , Preliminary estimation of center coordinates After correction, the final high-precision center coordinates are obtained. .

[0057] This embodiment achieves low-cost, high-precision positioning by the interactive collaboration of a model analysis and size construction module, a global visual coarse positioning module, a feature point dynamic sampling module, a local visual fine positioning module, a multi-feature point collaborative calibration module, and an error feedback iterative correction module. This overcomes processing errors and environmental interference, and improves industrial production efficiency and product quality.

[0058] The visual positioning method based on key dimensions of a 3D model provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0059] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0060] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0061] A processor may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0062] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0063] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0064] The aforementioned electronic device realizes the acquisition of the workpiece's three-dimensional model data using the visual positioning method based on the key dimensions of the three-dimensional model, and calculates the theoretical center coordinates of the workpiece based on the three-dimensional model data. And extract key geometric dimensions to generate a dimension matrix. A global image of the workpiece is captured using a wide-view camera. Edge detection and contour extraction are then performed on the global image to obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Pixel coordinates for preliminary positioning of the geometric center of the workpiece contour Based on the intrinsic parameter matrix calibrated by the wide-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Convert to world coordinates to obtain preliminary estimated center coordinates; based on the size matrix... and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; at close range The first feature of the reshoot The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error Repeat steps S3-S4, sequentially processing the second feature point. The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; based on the error... , , Preliminary estimation of center coordinates After correction, the final high-precision center coordinates are obtained. ,in, The technical solution of pre-setting weighting coefficients has achieved the beneficial effects of improving positioning accuracy, reducing dependence on high-resolution cameras, and overcoming the influence of workpiece processing errors and environmental interference.

[0065] The storage medium provided in this application stores a program product capable of implementing a visual positioning method based on key dimensions of a 3D model.

[0066] The visual positioning method based on key dimensions of a 3D model includes: acquiring the 3D model data of the workpiece, and calculating the theoretical center coordinates of the workpiece based on the 3D model data. And extract key geometric dimensions to generate a dimension matrix. A global image of the workpiece is captured using a wide-view camera. Edge detection and contour extraction are then performed on the global image to obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Pixel coordinates for preliminary positioning of the geometric center of the workpiece contour Based on the intrinsic parameter matrix calibrated by the wide-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Convert to world coordinates to obtain preliminary estimated center coordinates; based on the size matrix... and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; at close range The first feature of the reshoot The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error Repeat for the second feature point The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; based on the error... , , Preliminary estimation of center coordinates After correction, the final high-precision center coordinates are obtained. ,in, These are the preset weighting coefficients.

[0067] In some possible implementations, the visual positioning method based on key dimensions of a 3D model disclosed herein can be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0068] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0069] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A visual positioning method based on key dimensions of a 3D model, characterized in that, The steps include the following: S1. Obtain the 3D model data of the workpiece, and calculate the theoretical center coordinates of the workpiece based on the 3D model data. And extract key geometric dimensions to generate a dimension matrix. ; S2. Take a global image of the workpiece using a wide-view camera, perform edge detection and contour extraction on the global image of the workpiece, and obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Preliminary location of the pixel coordinates of the geometric center of the workpiece contour ; Based on the intrinsic parameter matrix calibrated by the wide field-of-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Converting to world coordinates yields a preliminary estimate of the center coordinates. ; S3. Based on the dimension matrix and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; S4. At close range Next, re-shoot the first feature point. The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error ; S5. Repeat steps S3-S4, sequentially processing the second feature point. The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; S6. Based on the error , , Preliminary estimate of center coordinates After correction, the final high-precision center coordinates are obtained. ; The specific steps in step S2 include: S21. Use a wide-view camera to photograph the workpiece and obtain a global image containing the entire workpiece; S22. Perform Gaussian filtering to remove noise from the global image, and use the Canny operator to detect the workpiece edges; S23. Fill the edge gaps using morphological closing operations to generate a continuous closed workpiece contour. Then, extract the minimum bounding rectangle based on the pixel set of the continuous closed workpiece contour, and calculate the pixel coordinates of the center of the minimum bounding rectangle as the geometric center of the workpiece contour. .

2. The visual positioning method based on key dimensions of a 3D model according to claim 1, characterized in that, The specific steps of step S1 are as follows: S11. Obtain and parse the STEP or IGES file of the workpiece's 3D model to obtain the vertex coordinate set of the workpiece's bounding box. , where n represents the number of vertices; S12. Based on vertex coordinate set Calculate the coordinates of the theoretical center : ； S13. Feature points of the workpiece Perform localization and calculate feature points. Relative to the theoretical center coordinates Geometric distance, generate size matrix ; S14. Calculate feature points Relative to the theoretical center coordinates Unit direction vector in the model coordinate system : ； unit direction vector With size matrix Associated storage.

3. The visual positioning method based on key dimensions of a three-dimensional model according to claim 2, characterized in that, The specific steps in step S2 also include: S24. Pixel coordinates of the geometric center of the workpiece contour Using a predetermined region around the initial point as the starting point, extract the image from the global image and calculate the accurate pixel center coordinates using the gray-level weighted centroid method. ; S25. Obtain the actual distance from the wide-field-of-view camera to the workpiece using a laser rangefinder. Combined with camera intrinsic parameter matrix Accurate pixel coordinates Preliminary estimate of center coordinates in world coordinate system .

4. The visual positioning method based on key dimensions of a three-dimensional model according to claim 3, characterized in that, The specific steps of step S3 are as follows: S31. Select any one of the feature points of the workpiece as the first feature point. ; S32. According to the dimension matrix The first feature point in Model size and preliminary estimation of center coordinates And based on the first feature point Unit direction vector in the model coordinate system Calculate the first feature point Predicted world coordinates : ； S33. Control the six-axis robotic arm to carry the camera and move the wide-field camera linearly to the first feature point according to the preset precision. Predicted world coordinates ; S34. The wide-view camera is moved along the normal direction of the workpiece surface by a servo motor, increasing the working distance from... shorten to close distance : in, Focal length It is the pixel size. It is the minimum size of the feature point.

5. The visual positioning method based on key dimensions of a three-dimensional model according to claim 4, characterized in that, The specific steps of step S4 are as follows: S41. At close range The following steps involve acquiring local images and performing illumination equalization processing: ； in, The original image at coordinate points grayscale value at that location The image after illumination equalization is at coordinate points grayscale value at that location Based on coordinate points The average gray value of all pixels within a local image window centered on the image. Based on coordinate points The standard deviation of gray levels of all pixels within a local image window centered on the image. S42. Employ a sub-pixel edge detection algorithm based on Zernike moments to calculate the normal angle of edge points. ; S43. Based on edge point normal angle Fit an edge line within a range of ±N pixels along the normal direction, and set the intersection point as the first feature point. coordinate: in, It is the first feature point The center of the line is located at sub-pixel coordinates in the image pixel coordinate system, and k represents the k-th edge line. It is a point on the k-th edge line. It is the normal angle of the k-th edge line. It is used as the first feature point The actual coordinates of the precise center coordinates; S44. Obtain the first feature point Model size Calculate the first feature point The error between the actual coordinates and the model size .

6. The visual positioning method based on key dimensions of a three-dimensional model according to claim 5, characterized in that, The specific steps of step S5 are as follows: S51. Select a second feature point from the feature points of the workpiece according to the following spatial symmetry conditions. : in, ; S52. Repeat steps S31-S34 to move the wide-field camera to the second feature point. Predicted world coordinates : in, Based on the second feature point The unit direction vector in the model coordinate system; S53. Repeat steps S41-S44 to obtain the second feature point. actual coordinates With model size error ; S54. Select a third feature point from the feature points of the workpiece according to the following non-coplanar conditions. : in, ; S55. Repeat steps S31-S34 to move the wide-view camera to the third feature point. Predicted world coordinates : in, Based on the third feature point The unit direction vector in the model coordinate system; S56. Repeat steps S41-S44 to obtain the third feature point. actual coordinates With model size error ; S57. Apply the following formula to the second feature point. The third feature point Adjust the working distance during sampling: Where k=2 corresponds to the second feature point. k=3 corresponds to the third feature point , This is the adjusted working distance of the wide-view camera.

7. The visual positioning method based on key dimensions of a three-dimensional model according to claim 5, characterized in that, The specific steps of step S6 are as follows: S61. Calculate the overall error using the model size of each feature point as the weight. : ； S62. Dynamically adjust the preset weighting coefficients according to the error amplitude. : ； S63. Calculate the final high-precision center coordinates. : S64. Verify the final high-precision center coordinates. Does it satisfy one of the following convergence conditions: in, This is a preset proportional coefficient. This is the preset distance error threshold; Output when any one of the convergence conditions is met. Perform visual positioning; When neither of the two convergence conditions is satisfied, let Return to step S3.

8. A visual positioning system based on key dimensions of a 3D model, characterized in that, include: The model parsing and dimension construction module is used to acquire the 3D model data of the workpiece and calculate the theoretical center coordinates of the workpiece based on the 3D model data. And extract key geometric dimensions to generate a dimension matrix. ; The global vision coarse localization module is used to capture a global image of the workpiece using a wide-field-of-view camera, perform edge detection and contour extraction on the global image of the workpiece, and obtain the theoretical center coordinates. The pixel coordinates in the global image are then converted into preliminary estimated center coordinates. : Preliminary location of the pixel coordinates of the geometric center of the workpiece contour ; Based on the intrinsic parameter matrix calibrated by the wide field-of-view camera and the distance from the wide-view camera to the workpiece The pixel coordinates of the geometric center of the workpiece contour Convert to coordinates in the world coordinate system to obtain a preliminary estimate of the center coordinates; The feature point dynamic sampling module is used to sample points based on the size matrix. and preliminary estimation of center coordinates Control the wide-view camera to move to the first feature point. The estimated location, the moving distance is the first feature point. Model size Subsequently, the distance between the wide-view camera and the workpiece was shortened to close range. The first feature point Model size The first feature point Compared with the preliminary estimated center coordinates Key geometric dimensions; Local visual precision positioning module, used for close-range positioning. Next, re-shoot the first feature point. The first feature point is extracted from a local image using a sub-pixel method. actual coordinates And calculate the first feature point. actual coordinates With the first feature point Model size error ; The multi-feature point collaborative calibration module is used to sequentially calibrate the second feature point based on the feature point dynamic sampling module and the local visual fine localization module. The third feature point Sampling is performed to obtain the error. , The second feature point With the first feature point Located at the preliminary estimated center coordinates Both sides, and the third feature point Distance from the second feature point With the first feature point The distance between the lines is greater than the set distance threshold; The error feedback iterative correction module is used to correct errors. , , Preliminary estimate of center coordinates After correction, the final high-precision center coordinates are obtained. ; The global visual coarse localization module is configured to perform the following steps: S21. Use a wide-view camera to photograph the workpiece and obtain a global image containing the entire workpiece; S22. Perform Gaussian filtering to remove noise from the global image, and use the Canny operator to detect the workpiece edges; S23. Fill the edge gaps using morphological closing operations to generate a continuous closed workpiece contour. Then, extract the minimum bounding rectangle based on the pixel set of the continuous closed workpiece contour, and calculate the pixel coordinates of the center of the minimum bounding rectangle as the geometric center of the workpiece contour. .

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the visual positioning method based on the key dimensions of a three-dimensional model as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the visual positioning method based on the key dimensions of a three-dimensional model as described in any one of claims 1 to 7.

Images