Precise workpiece positioning method and system based on three-dimensional vision

Abstract

The invention provides a precise workpiece positioning method and system based on three-dimensional vision, and the method comprises the steps: obtaining a scene three-dimensional vision data set containing a target workpiece, carrying out the multi-modal data fusion processing of the scene three-dimensional vision data set, generating a fusion data set with aligned space coordinates, extracting a workpiece feature identifier set from the fusion data set, and carrying out the recognition of the workpiece feature identifier set. And feature matching association processing is carried out on the workpiece feature identifier set and a pre-stored standard workpiece feature template, a feature matching corresponding relation set is generated, three-dimensional positioning coordinate information of the target workpiece in the scene is calculated based on the feature matching corresponding relation set, and the three-dimensional positioning coordinate information comprises workpiece centroid coordinates and space attitude angle parameters. The precision of precision workpiece machining is improved, and the stability of the positioning process is ensured.

Description

Technical Field

[0001] The present invention relates to the field of intelligent processing technology, and in particular to a precision workpiece positioning method and system based on three-dimensional vision. Background Art

[0002] With the development of 3D vision technology, precision workpiece positioning technology has been proposed. By analyzing a workpiece's visual data to determine its position and posture in 3D space, it is widely used in scenarios requiring high-precision positioning, such as intelligent manufacturing and precision assembly. Currently, common precision workpiece positioning technologies typically acquire visual data, extract features such as surface key points or contour lines, and then perform matching and positioning calculations based on these features. However, existing technologies often suffer from incomplete feature extraction, resulting in inaccurate and unstable positioning results. Summary of the Invention

[0003] In view of this, an embodiment of the present invention provides a method and system for precision workpiece positioning based on three-dimensional vision. The technical solution of the embodiment of the present invention is implemented as follows: On the one hand, an embodiment of the present invention provides a precision workpiece positioning method based on three-dimensional vision, the method comprising: acquiring a scene three-dimensional vision data set including a target workpiece, the scene three-dimensional vision data set consisting of workpiece surface point cloud data collected synchronously from multiple perspectives and two-dimensional image data from corresponding perspectives; performing multimodal data fusion processing on the scene three-dimensional vision data set to generate a fusion data group with spatial coordinate alignment, the fusion data group including a mapping relationship between point cloud coordinates and two-dimensional image pixel coordinates; extracting a workpiece feature identification set from the fusion data group, the workpiece feature identification set including key feature point position information and continuous contour line feature description on the workpiece surface; performing feature matching association processing on the workpiece feature identification set and a pre-stored standard workpiece feature template to generate a feature matching correspondence relationship set, the feature matching correspondence relationship set including position correspondences of key feature points and shape matching relationships of contour lines; and calculating the three-dimensional positioning coordinate information of the target workpiece in the scene based on the feature matching correspondence relationship set, the three-dimensional positioning coordinate information including workpiece center of mass coordinates and spatial attitude angle parameters.

[0004] On the other hand, an embodiment of the present invention provides a computer system including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above method when executing the program.

[0005] The present invention's precision workpiece positioning method based on three-dimensional vision uses multi-perspective synchronous acquisition of point cloud data and two-dimensional image data to generate a spatially aligned fusion data set, integrating the three-dimensional coordinate information of the point cloud with the texture and color information of the image, thus avoiding the limitations of simple modal data in spatial information or detail expression. It extracts dual feature identification of key feature point position information and continuous contour line feature description, providing both local position constraints through key feature points and global shape constraints through continuous contour lines, thus compensating for the lack of positioning constraints. Through the composite feature association of the position correspondence of key feature points and the shape matching relationship of contour lines, it achieves the coordination of local position verification and global shape verification, reducing the risk of misjudgment caused by insufficient feature extraction. Finally, based on the dual feature correspondence, it calculates the three-dimensional positioning coordinates, combining the local coordinate transformation constraint with the global shape consistency constraint, so that the positioning result not only reflects the positional relationship of discrete features on the workpiece surface, but also conforms to the shape regularity of the overall workpiece contour, thereby improving the accuracy and reliability of precision workpiece positioning. At the same time, the fusion processing of multimodal data and the composite matching of dual features enhance the adaptability of the method to complex surface workpieces or partially occluded scenes through information complementarity and constraint superposition, ensuring the stability of the positioning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 A schematic diagram of the implementation flow of a three-dimensional vision-based precision workpiece positioning method provided in an embodiment of the present invention.

[0007] Figure 2 A schematic diagram of a hardware entity of a computer system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0008] An embodiment of the present invention provides a method for positioning a precision workpiece based on three-dimensional vision, which can be executed by a processor of a computer system. The computer system can refer to a computer system in a control system on a precision workpiece processing line. For example, the precision workpiece can be the housing of a mobile phone camera module, and the processing process can include applying film to it.

[0009] Figure 1 A schematic diagram of the implementation process of a three-dimensional vision-based precision workpiece positioning method provided in an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes: Step S100: Acquire a scene three-dimensional visual data set containing a target workpiece, wherein the scene three-dimensional visual data set is composed of workpiece surface point cloud data collected synchronously from multiple perspectives and two-dimensional image data from corresponding perspectives.

[0010] The scene's 3D visual data set serves as the foundation for subsequent precision workpiece positioning. It integrates point cloud data and 2D image data of the workpiece surface from multiple viewpoints. Point cloud data is a collection of points representing the spatial position of an object's surface. Each point has 3D coordinate information, accurately reflecting the workpiece's surface geometry. 2D image data, on the other hand, contains texture and color information on the workpiece surface, providing rich visual features. Multi-view simultaneous acquisition aims to comprehensively and accurately capture workpiece information, avoiding occlusion or information loss caused by a single viewpoint.

[0011] For example, in a scenario where a mobile phone camera automatically positions and applies a film, the target workpiece is the phone's camera module. To acquire a 3D visual data set of the scene containing this target workpiece, specialized acquisition equipment and technology are required. For example, a high-precision 3D visual acquisition system might be integrated with a LiDAR sensor and an RGB camera. In practice, the phone is placed on a workbench, and the 3D visual acquisition equipment is controlled to capture data from multiple perspectives around the phone's camera module.

[0012] As an implementation manner, step S100 can be specifically implemented as the following steps S110 to S150: Step S110: Control the three-dimensional vision acquisition device to collect data at multiple preset observation positions around the workpiece, and set fixed device posture parameters for each observation position.

[0013] 3D vision acquisition equipment is a hardware device used to obtain three-dimensional and two-dimensional image information of workpieces. It can observe and collect data from workpieces from different angles and positions. Multiple preset observation positions are pre-planned based on the shape, size, and characteristics of the workpiece. These positions ensure comprehensive and accurate acquisition of workpiece information. Device pose parameters describe the device's position and orientation in space, including the device's origin coordinates and optical axis direction vector. Setting fixed device pose parameters ensures consistent device status each time data is collected, improving data accuracy and repeatability.

[0014] As an implementation manner, step S110 can be specifically implemented as the following steps S111 to S116: Step S111: Pre-scanning the workpiece surface to generate preliminary point cloud data including surface feature distribution, where the surface feature distribution includes texture complexity and curvature change rate information.

[0015] Pre-scan processing is a quick scan of the workpiece surface before formal data acquisition. Its purpose is to obtain general feature information of the workpiece surface. The preliminary point cloud data obtained from the pre-scan contains the basic geometry and feature distribution information of the workpiece surface. Texture complexity reflects the richness and complexity of the workpiece surface texture, while curvature change rate indicates the rate of change of the workpiece surface curvature. These two parameters can help determine feature-rich and feature-sparse areas on the workpiece surface.

[0016] For example, in the scenario of automatic positioning and film application for mobile phone cameras, a low-resolution LiDAR sensor can be used to pre-scan the surface of the mobile phone camera module. For example, a LiDAR sensor is set to quickly scan the surface of the camera module with low precision to obtain a series of point cloud data. Then, these point cloud data are processed and analyzed to calculate the texture complexity and curvature change rate of each point. The curvature change rate can be calculated by counting the distance and angle changes between adjacent points in the point cloud data, and the texture complexity can be evaluated by analyzing the local variance of the grayscale image corresponding to the point cloud data. For the lens part of the mobile phone camera, due to its smooth surface, the curvature change rate is small and the texture complexity is relatively low; while for the logos and fixed structures around the lens, the curvature change rate is large and the texture complexity is also high.

[0017] Step S112: constructing an observation position planning model based on the surface feature distribution information. The observation position planning model takes coverage integrity of feature-rich areas and redundancy control of feature-sparse areas as optimization goals.

[0018] The observation position planning model is a mathematical model used to determine the optimal observation position. It plans the observation position of the acquisition equipment based on the feature distribution information on the workpiece surface. Feature-rich areas are those with high texture complexity and large curvature variation on the workpiece surface. These areas contain more feature information and are crucial for accurate identification and positioning of the workpiece. Therefore, it is necessary to ensure that these areas are fully covered. Feature-sparse areas are those with low texture complexity and small curvature variation. To avoid unnecessary data redundancy, observations in these areas need to be properly controlled to reduce duplicate acquisition.

[0019] For example, an observation position planning model can be constructed based on the surface feature distribution information obtained from the pre-scan. Optimization algorithms such as genetic algorithms or particle swarm algorithms can be used to implement model construction. Taking the genetic algorithm as an example, the coordinates and angles of the observation positions are first used as genes, and each possible combination of observation positions is treated as an individual to form a population. Then, a fitness function is defined, which uses the coverage completeness of feature-rich areas and the control of redundancy in feature-sparse areas as optimization objectives. By continuously performing selection, crossover, and mutation operations, the population is iteratively updated until the optimal combination of observation positions is found. For example, for feature-rich areas around the lens of a mobile phone camera module, the model will plan more observation positions to ensure that the feature information of these areas can be fully collected; while for feature-sparse areas on the lens surface, the number of observation positions will be appropriately reduced to avoid data redundancy.

[0020] Step S113: Dynamically determine a target observation position set around the workpiece through the observation position planning model, wherein the density of observation points in the feature-rich area of ​​the target observation position set is higher than that in the feature-sparse area.

[0021] The target observation position set is the set of final observation positions determined according to the observation position planning model. It can ensure that when collecting data, the feature-rich areas on the workpiece surface are observed more densely, while the feature-sparse areas are observed relatively sparsely, thereby reducing unnecessary data collection while ensuring data integrity.

[0022] In the scenario of automatic positioning and film application using a mobile phone camera, the surface feature distribution information obtained from pre-scanning is input into the observation position planning model. Through calculation and optimization of the model, the target observation position set is dynamically determined. For example, for the feature-rich area around the lens of a mobile phone camera module, 5-10 observation positions are set, while for the feature-sparse area of ​​the lens surface, 2-3 observation positions are set. This ensures that key surface feature information of the workpiece can be more effectively obtained during data collection, improving positioning accuracy.

[0023] Step S114: At each target observation position, the orientation of the lens of the three-dimensional vision acquisition device is adjusted by the posture calibration module so that the optical axis of the lens maintains a preset angle relationship with the normal vector of the corresponding area of ​​the workpiece surface.

[0024] The pose calibration module is used to precisely adjust the pose of 3D vision acquisition equipment. It accurately adjusts the device's lens orientation based on the target observation position. A normal vector is a vector perpendicular to a point on a workpiece surface, indicating the surface orientation at that point. The preset angle relationship is a pre-set angle between the lens optical axis and the workpiece surface normal vector, based on actual requirements. Maintaining this angle ensures optimal quality and accuracy of captured images and point cloud data.

[0025] In the scenario of automatic positioning and film application for a mobile phone camera, the pose calibration module begins after the 3D vision acquisition device moves to each target observation position. This module utilizes a high-precision angle sensor and motor control system to adjust the lens orientation. For example, the angle sensor monitors the angle between the lens optical axis and the normal vector of the corresponding area on the workpiece surface in real time. Based on the preset angle relationship, the motor is controlled to rotate and tilt the lens until the preset angle is achieved.

[0026] Step S115: Record the device space coordinates and lens orientation parameters of each target observation position as fixed device pose parameters, where the device pose parameters include the device origin coordinates and the optical axis direction vector.

[0027] Device space coordinates refer to the device's position in three-dimensional space, while lens orientation parameters describe the lens's orientation and angle. Recording these parameters as fixed device pose parameters allows accurate determination of the device's state at each observation location during subsequent data collection and processing, ensuring data consistency and traceability. The device origin coordinates are the coordinates of the origin of the device coordinate system in the global coordinate system, while the optical axis direction vector indicates the direction of the lens's optical axis.

[0028] In the scenario of automatic positioning and film application for mobile phone cameras, a high-precision positioning system and angle sensor are used to record the device's spatial coordinates and lens orientation parameters for each target observation location. For example, a laser tracker or GPS positioning system is used to obtain the device's spatial coordinates, and an electronic compass or gyroscope is used to obtain the lens orientation parameters. These parameters are recorded and stored in a computer database as a reference for subsequent data collection and processing. Each target observation location is associated with a unique set of device pose parameters that accurately describe the device's state at that location.

[0029] Step S116: Based on the device posture parameters, the three-dimensional vision acquisition device is controlled to synchronously acquire point cloud data and two-dimensional image data of each observation position to form a scene three-dimensional vision data set covering the surface feature distribution of the workpiece.

[0030] Controlling 3D vision acquisition equipment based on device pose parameters means precisely controlling the acquisition equipment to collect data at each observation location based on previously recorded device pose parameters. Synchronous acquisition involves collecting both point cloud data and 2D image data at the same time, ensuring an accurate correspondence between the two types of data. A scene-wide 3D vision data set covering the distribution of workpiece surface features means that the collected data comprehensively and accurately reflects the distribution of features on the workpiece surface, including both feature-rich and feature-sparse areas.

[0031] In the scenario of automatically positioning and applying film to a mobile phone camera, the 3D visual acquisition device is controlled to synchronously collect data at each target observation position based on the recorded device pose parameters. For example, at the observation position directly above the lens, the device adjusts its position and orientation according to the preset pose parameters, and then simultaneously activates the LiDAR sensor and RGB camera to collect data. The LiDAR sensor obtains point cloud data of the mobile phone camera module surface at that position, and the RGB camera obtains the corresponding 2D image data. This process is repeated, collecting data at each target observation position. Finally, all collected point cloud data and 2D image data are integrated to form a 3D visual data set for the scene that covers the distribution of surface features of the mobile phone camera module.

[0032] Step S120: At each observation position, point cloud data of the workpiece surface is obtained by the laser radar sensor, where the point cloud data includes the three-dimensional spatial coordinate information of each point.

[0033] LiDAR sensors can accurately measure the 3D coordinates of every point on an object's surface, generating point cloud data. Each point in the point cloud represents a location on the workpiece's surface, and its 3D coordinate information accurately describes the workpiece's geometry.

[0034] In the scenario of automatically positioning and applying a film to a mobile phone camera, at each observation position, the LiDAR sensor emits a laser beam toward the surface of the mobile phone camera module. For example, the LiDAR sensor emits laser pulses at a certain frequency. When the laser pulses encounter the surface of the camera module, they are reflected and received by the sensor. By measuring the time difference between the emission and reception of the laser pulses and combining it with the propagation speed of the laser, the distance between each reflection point and the sensor can be calculated. At the same time, based on the position and angle information of the sensor, this distance information is converted into three-dimensional spatial coordinates. For the lens portion of the mobile phone camera, the LiDAR sensor can obtain a series of precise point cloud data, which can accurately depict the circular shape and surface curvature of the lens.

[0035] Step S130: At the same observation position, two-dimensional image data of corresponding viewing angles are synchronously collected by an RGB camera, where the two-dimensional image data includes texture and color information of the workpiece surface.

[0036] RGB cameras record image color information through three color channels (red, green, and blue). Simultaneously acquiring 2D image data from the same observation position ensures perspective consistency between the point cloud and 2D image data, facilitating subsequent multimodal data fusion processing. Texture and color information in 2D image data provides additional visual features, helping to more accurately identify and locate workpieces.

[0037] In the scenario of automatically positioning and applying film using a mobile phone camera, at each observation position, while the LiDAR sensor collects point cloud data, the RGB camera simultaneously collects 2D image data from the corresponding perspective. For example, the RGB camera lens maintains the same orientation and position as the LiDAR sensor lens to ensure that the collected 2D image data and the point cloud data have the same perspective. For the surface of the mobile phone camera module, the RGB camera can capture information such as the glass gloss of the lens and the color and texture of surrounding markings. This texture and color information can be combined with the geometric information in the point cloud data to provide richer features for subsequent workpiece positioning.

[0038] Step S140: Record the device posture parameters of each observation position as a spatial reference benchmark, where the spatial reference benchmark includes the position and orientation information of the device relative to the scene fixed coordinate system.

[0039] A spatial reference is a standard used to determine the position and orientation of data in three-dimensional space. It records the device pose parameters at each observation location, including the device's position and orientation relative to the scene-fixed coordinate system. The scene-fixed coordinate system is a predefined global coordinate system to which all data can be converted using the spatial reference, facilitating unified processing and analysis. In a scenario where a mobile phone camera automatically positions and applies a protective film, the device pose parameters at each observation location are recorded as the spatial reference. High-precision positioning systems and angle sensors can be used to obtain the device's position and orientation relative to the scene-fixed coordinate system. For example, a mobile phone can be placed on a fixed work surface, and a scene-fixed coordinate system can be established with one corner of the work surface as the origin. At each observation location, a laser tracker or GPS positioning system is used to measure the device's origin coordinates in the scene-fixed coordinate system. An electronic compass or gyroscope is used to measure the angle of the device's optical axis vector relative to the scene-fixed coordinate system. This information is recorded as the spatial reference for subsequent data storage and processing.

[0040] Step S150: The point cloud data and two-dimensional image data collected at each observation position are stored according to the corresponding spatial reference datum to form a scene three-dimensional visual data set consisting of multiple groups of point cloud-image data pairs.

[0041] The point cloud data and 2D image data collected at each observation location are stored according to the corresponding spatial reference datum to ensure spatial consistency and traceability of the data. The point cloud data and 2D image data of each observation location constitute a set of point cloud-image data pairs, which contain the geometric information and texture and color information of the workpiece surface from the same perspective. Combining all point cloud-image data pairs together forms a set of 3D visual data for the scene. In the scene where the mobile phone camera automatically positions the film, the point cloud data and 2D image data collected at each observation location are stored according to the corresponding spatial reference datum.

[0042] Step S200: performing multimodal data fusion processing on the scene three-dimensional visual data set to generate a fused data set with aligned spatial coordinates, wherein the fused data set includes a mapping relationship between point cloud coordinates and two-dimensional image pixel coordinates.

[0043] Multimodal data fusion integrates and processes different types of data (such as point cloud data and 2D image data) to obtain more comprehensive and accurate information. Spatial coordinate alignment refers to unifying point cloud data and 2D image data from different observation locations into a common spatial coordinate system, ensuring spatial consistency. A fused data set is the resulting data set after multimodal data fusion. It contains a mapping relationship between point cloud coordinates and 2D image pixel coordinates. This mapping relationship converts 3D coordinates in the point cloud data into pixel coordinates in the 2D image, and vice versa. In a scenario where a mobile phone camera automatically positions and applies a protective film, multimodal data fusion is performed on the collected 3D visual data set of the scene. First, the spatial alignment of data from different observation locations must be addressed. Since the acquisition devices at different observation locations may have different poses, the collected data may have different spatial coordinate systems. Multimodal data fusion can unify this data into a common coordinate system. A mapping relationship between point cloud coordinates and 2D image pixel coordinates is then established, facilitating the subsequent correlation of geometric information in the point cloud data with texture and color information in the 2D image data. For example, for a point cloud data point of a mobile phone camera module, its corresponding pixel point in the two-dimensional image can be found through the mapping relationship.

[0044] As an implementation manner, step S200 can be specifically implemented as the following steps S210 to S250: Step S210: Select any set of point cloud-image data pairs in the scene 3D visual data set as a reference data set, and extract feature points of the 2D image of the reference data set.

[0045] The benchmark dataset is a representative set of data selected from the scene's 3D visual data set. It serves as a reference for subsequent data fusion and processing. Feature points are distinct points in a 2D image, such as corners and edges. These points contain important structural information about the image and are crucial for image matching and coordinate alignment.

[0046] In the scenario of automatically positioning and applying a film to a mobile phone camera, a set of point cloud-image data pairs is randomly selected from the collected 3D visual data set of the scene as the reference data set. For example, a point cloud-image data pair is selected from the observation position directly above the lens. Feature extraction algorithms are then used to extract feature points from the 2D image of the reference data set. These algorithms can be used, such as the SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) algorithm. Taking the SIFT algorithm as an example, a Gaussian difference pyramid is first constructed on the 2D image. Extreme points are then found in each layer of the pyramid. These extreme points are then precisely located and filtered to remove unstable points. Finally, a feature descriptor is calculated for each feature point for subsequent feature matching. A large number of feature points are extracted from locations such as the logo and edges of the mobile phone camera, and these feature points can represent key image structures.

[0047] Step S220: performing feature matching on the two-dimensional image data of other observation positions and the two-dimensional image data of the reference data set through a feature matching algorithm to obtain the transformation relationship between the images of different observation positions.

[0048] Feature matching algorithms are used to find matching feature points in different images. They determine which feature points correspond by comparing their feature descriptors. Feature matching can be used to find correspondences between images at different observation positions and then calculate the transformation relationship between them. The transformation relationship describes how to transform a point in one image to a corresponding position in another image, expressed using transformation parameters such as rotation, translation, and scaling.

[0049] In the scenario of automatic positioning and film application for mobile phone cameras, a feature matching algorithm is used to match the 2D image data from other observation positions with the 2D image data from the baseline dataset. This can be achieved by combining the RANSAC (Random Sample Consensus) algorithm with SIFT feature matching. The specific steps are as follows: First, feature point extraction and feature descriptor calculation are performed on the 2D image data from other observation positions, using the same method as for the baseline dataset. Then, SIFT feature descriptors are used for preliminary feature matching to identify possible matching point pairs. These matching point pairs are then screened using the RANSAC algorithm to remove false matches. The RANSAC algorithm randomly selects a set of matching point pairs and calculates the transformation relationship between them. This transformation relationship is then used to verify other matching point pairs, retaining those that meet the criteria. Finally, based on the screened matching point pairs, the transformation relationship between images from different observation positions is calculated. For example, for 2D image data from a position diagonally above the lens, feature matching and the RANSAC algorithm can be used to calculate its rotation, translation, and scale parameters relative to the 2D image data from the baseline dataset, thereby determining the transformation relationship between them.

[0050] As an implementation manner, step S220 may be specifically implemented as the following steps S221 to S227: Step S221: extracting feature points from the two-dimensional image data of the reference data set, and synchronously obtaining the three-dimensional coordinates of each feature point in the corresponding point cloud data as a feature space anchor point.

[0051] Feature point extraction involves finding points with significant features from a two-dimensional image using a feature extraction algorithm. Feature space anchor points are the corresponding points of feature points in three-dimensional space. By associating feature points with point cloud data, the three-dimensional coordinates of each feature point can be obtained. These three-dimensional coordinates serve as reference points for subsequent spatial matching and transformation.

[0052] In the scenario of automatic positioning and film application for mobile phone cameras, feature points are extracted from the two-dimensional image data of the benchmark data set. The SIFT algorithm can be used for feature point extraction, which can find feature points with scale invariance and rotation invariance in the image. For each extracted feature point, the three-dimensional coordinates in the corresponding point cloud data are found through the previously established mapping relationship between the point cloud coordinates and the two-dimensional image pixel coordinates. For example, for a feature point on the edge of the lens in the two-dimensional image of the benchmark data set, the corresponding three-dimensional coordinates can be found in the point cloud data through the mapping relationship. This three-dimensional coordinate is the feature space anchor point of the feature point.

[0053] Step S222: performing the same feature point extraction and feature space anchor point acquisition operations on the two-dimensional image data of other observation positions to generate a feature point-space anchor point association set corresponding to the observation position.

[0054] Performing the same operations on the 2D image data from other observation locations ensures consistency and comparability across all observation locations. The feature point-spatial anchor point association set records the relationship between each feature point at each observation location and its corresponding feature space anchor point. This serves as the basis for subsequent feature matching and spatial transformation.

[0055] In the scenario of automatic positioning and film application for mobile phone cameras, the SIFT algorithm is also used to extract feature points from the two-dimensional image data of other observation positions besides the baseline data set. Then, the feature space anchor point of each feature point is obtained based on the mapping relationship between the corresponding point cloud and image data pair. The feature points of each observation position are associated with the corresponding feature space anchor point to generate a feature point-space anchor point association set. For example, for the two-dimensional image data of the side observation position of the lens, after extracting a series of feature points, their three-dimensional coordinates in the point cloud data are found through the mapping relationship to form the feature point-space anchor point association set of the observation position.

[0056] Step S223: Calculate the similarity between the image feature descriptors of the feature points of the reference data set and the feature points of other observation positions to generate an initial similarity matching matrix.

[0057] Image feature descriptors are vectors used to describe the characteristics of feature points, containing local image information about those feature points. Similarity refers to the degree of similarity between two feature descriptors. By calculating similarity, we can determine whether two feature points match. The initial similarity matching matrix records the similarity between feature points in the baseline dataset and feature points at other observation locations, providing preliminary information for subsequent feature matching.

[0058] In the scenario of automatic positioning and film application using a mobile phone camera, the similarity between the image feature descriptors of the feature points in the baseline dataset and those at other observed locations is calculated. Methods such as Euclidean distance or cosine similarity can be used. For example, smaller Euclidean distances indicate higher similarity. The similarity calculation results for all feature points in the baseline dataset and those at other observed locations are stored in a matrix to form the initial similarity matching matrix.

[0059] Step S224: Based on the three-dimensional coordinate information of the feature space anchor point, calculate the spatial Euclidean distance between the feature point of the reference data set and the feature points of other observation positions to generate a spatial distance matching matrix.

[0060] Spatial Euclidean distance refers to the distance between two feature space anchor points in three-dimensional space, reflecting the relative position of feature points in space. The spatial distance matching matrix records the spatial Euclidean distances between feature points in the baseline dataset and feature points at other observation locations. It is used together with the initial similarity matching matrix for subsequent comprehensive matching.

[0061] In the scenario of automatic positioning and film application of mobile phone cameras, the spatial Euclidean distance between the feature points of the benchmark data set and the feature points of other observation locations is calculated based on the 3D coordinate information of the feature space anchor point. For the feature space anchor point of a feature point in the benchmark data set and the feature space anchor point of a feature point at other observation locations, the distance between them is calculated using the Euclidean distance formula based on their 3D coordinates (x1, y1, z1) and (x2, y2, z2): The spatial Euclidean distance calculation results of all the feature points of the benchmark data set and the feature points of other observation positions are stored in a matrix to form a spatial distance matching matrix.

[0062] Step S225: performing weighted fusion processing on the initial similarity matching matrix and the spatial distance matching matrix to generate a comprehensive matching score matrix. The weight parameters of the weighted fusion are dynamically adjusted according to the texture complexity of the area where the feature points are located.

[0063] The weight parameter controls the weights assigned to the initial similarity matching matrix and the spatial distance matching matrix during the fusion process. It is dynamically adjusted based on the texture complexity of the feature point region to accommodate feature matching requirements in different areas. In the scenario of automatic positioning and film application for a mobile phone camera, the initial similarity matching matrix and the spatial distance matching matrix are weightedly fused. First, the weight parameter is determined based on the texture complexity of the feature point region. Regions with high texture complexity may prioritize image feature similarity, so a higher weight is assigned to the initial similarity matching matrix. Regions with low texture complexity may prioritize spatial position consistency, so a higher weight is assigned to the spatial distance matching matrix. The initial similarity matching matrix and the spatial distance matching matrix are then weighted and summed according to the weights to produce a comprehensive matching score matrix.

[0064] Step S226: Screening feature point pairs with comprehensive matching scores higher than a preset threshold as candidate matching pairs, where the candidate matching pairs simultaneously meet the requirements of image feature similarity and spatial location proximity.

[0065] The preset threshold is a pre-set score criterion used to screen out feature point pairs with a high degree of matching. Candidate matching pairs are feature point pairs with a comprehensive matching score higher than the preset threshold. These feature point pairs meet the requirements of both image feature similarity and spatial proximity, forming the basis for subsequent topological consistency verification.

[0066] Step S227: Perform topological consistency verification on the candidate matching pairs to ensure that the relative position relationship between the feature points of the reference data group is consistent with the relative position relationship between the feature points of other observation positions, retain the candidate matching pairs that pass the topological consistency verification as valid matching pairs, and solve the transformation relationship parameters between images at different observation positions based on the coordinate information of the valid matching pairs.

[0067] Topological consistency verification further screens and verifies candidate matching pairs. It determines the validity of matching pairs by comparing the relative positional relationships between feature points in the baseline dataset with those at other observation locations. Valid matching pairs are candidate matching pairs that have passed topological consistency verification and have high matching accuracy. The coordinate information of these matching pairs can be used to accurately determine the transformation parameters between images at different observation locations.

[0068] In the scenario of automatic positioning and film application for mobile phone cameras, candidate matching pairs are verified for topological consistency. A graph-based matching method can be used, treating the feature points of the baseline dataset and the feature points at other observation locations as graph nodes, and the relative positional relationships between the feature points as graph edges. By comparing the structures and edge connectivity of the two graphs, it is determined whether the candidate matching pairs satisfy topological consistency. For example, for the three feature points A, B, and C in the baseline dataset, the relative distances and angles between them form the structure of one graph; for the three corresponding candidate matching feature points A', B', and C' at other observation locations, the relative distances and angles between them form the structure of another graph. If the structures of the two graphs are similar, that is, the relative positional relationships between the feature points are consistent, the candidate matching pairs are considered to pass the topological consistency verification. The candidate matching pairs that pass the verification are retained as valid matching pairs. Then, using methods such as least squares or singular value decomposition, the transformation parameters between the images at different observation locations, including rotation, translation, and scaling, are calculated based on the coordinate information of these valid matching pairs.

[0069] Step S230: Based on the transformation relationship and the spatial reference datum of each observation position, coordinate transformation parameters of the point cloud data of other observation positions relative to the point cloud data of the reference data set are calculated.

[0070] Coordinate transformation parameters are used to transform point cloud data from other observation locations into the spatial coordinate system of the reference dataset point cloud data. These parameters are calculated based on the transformation relationship between images from different observation locations and the spatial reference of each observation location. This coordinate transformation allows spatial alignment of point cloud data from different observation locations. In the mobile phone camera automatic positioning and film application scenario, the coordinate transformation parameters of the point cloud data from other observation locations relative to the point cloud data from the reference dataset are calculated based on the transformation relationship between images from different observation locations obtained in step S220 and the spatial reference of each observation location. First, the transformation relationship is represented as a transformation matrix, which contains information about rotation, translation, and scaling. Then, based on the spatial reference of each observation location, the coordinate system of the point cloud data from the other observation locations is transformed into the coordinate system of the point cloud data from the reference dataset. For example, for the point cloud data from the side of the lens, a coordinate transformation matrix is ​​calculated based on its transformation relationship matrix with the reference dataset and its own spatial reference. Multiplying the coordinates of each point in the point cloud data from this observation location by this coordinate transformation matrix converts it into the spatial coordinate system of the reference dataset point cloud data.

[0071] Step S240: using coordinate conversion parameters to convert the point cloud data of other observation positions into the spatial coordinate system of the reference data set, and splicing them with the point cloud data of the reference data set to generate global point cloud data.

[0072] The point cloud data from other observation locations is transformed using coordinate transformation parameters to make the point cloud data from different observation locations spatially consistent and capable of being spliced ​​and fused. Global point cloud data is the point cloud data obtained by splicing the point cloud data from all observation locations together, and contains the complete geometric information of the workpiece surface.

[0073] In the scenario where the mobile phone camera automatically positions and applies film, the coordinate conversion parameters calculated in step S230 are used to convert the point cloud data of other observation positions into the spatial coordinate system of the reference data set. For example, for the point cloud data of the observation position obliquely above the lens, the coordinates of each point are converted into the coordinate system of the reference data set through the coordinate conversion matrix. Then, the converted point cloud data is spliced ​​with the point cloud data of the reference data set. For the points in the overlapping area, the averaging method or the nearest neighbor method can be used for processing to eliminate data redundancy and inconsistency. Finally, a global point cloud data containing complete geometric information of the surface of the mobile phone camera module is generated.

[0074] Step S250: Perform projection transformation processing on the global point cloud data and the two-dimensional image data of each observation position, project the three-dimensional coordinates of the point cloud to the pixel coordinate system of the corresponding two-dimensional image through the camera parameters, establish a mapping relationship between the point cloud coordinates and the image pixel coordinates, and form a fusion data group with aligned spatial coordinates.

[0075] Projective transformation is the process of converting point cloud coordinates in 3D space into the pixel coordinate system of a 2D image. This is achieved using parameters such as the camera's intrinsic and extrinsic parameters. By establishing a mapping relationship between point cloud coordinates and image pixel coordinates, the geometric information in the point cloud data can be associated with the texture and color information in the 2D image data, forming a fused data set with aligned spatial coordinates.

[0076] In the scenario of automatic positioning and film application of mobile phone cameras, projection transformation processing is performed on the global point cloud data and the two-dimensional image data of each observation position. First, the intrinsic parameters (such as focal length, principal point coordinates, etc.) and extrinsic parameters (such as rotation matrix, translation vector, etc.) of the camera are obtained. Then, based on these parameters, the three-dimensional coordinates of each point in the global point cloud data are projected into the pixel coordinate system of the corresponding two-dimensional image. The pinhole camera model can be used for projection transformation. The projection formula of this model is: , where (u,v) is the pixel coordinate, (X,Y,Z) is the three-dimensional coordinate of the point cloud, f is the focal length, (c x ,c y ) are the principal point coordinates. This projection transformation establishes a mapping relationship between the point cloud coordinates and the image pixel coordinates. After processing all the point cloud data and 2D image data in this way, a fused data set with spatially aligned coordinates is formed. This fused data set contains the complete geometric information, texture, and color information of the mobile phone camera module surface, and they are spatially aligned.

[0077] Step S300: extracting a workpiece feature identification set from the fused data set, where the workpiece feature identification set includes key feature point position information and continuous contour line feature descriptions on the workpiece surface.

[0078] A workpiece feature set is a set of information used to describe the surface features of a workpiece. It includes the location information of key feature points and the feature description of continuous contour lines. This information is crucial for accurately identifying and locating the workpiece. Key feature points are distinctive points on the workpiece surface, such as corners and endpoints. Continuous contour lines are the boundary lines of the workpiece surface, describing the workpiece's outline.

[0079] In the scenario of automatically positioning and applying film to a mobile phone camera, a set of workpiece feature identifiers is extracted from a fused data set. This fused data set contains spatially aligned point cloud data and 2D image data of the mobile phone camera module surface. By processing and analyzing this data, the location information of key feature points and the feature description of continuous contour lines can be extracted. For example, key feature points can be identified for locations such as the lens edge and logo of a mobile phone camera, and continuous contour lines can be extracted for the overall shape of the camera module.

[0080] As an implementation manner, step S300 can be specifically implemented as the following steps S310 to S370: Step S310: performing noise filtering on the global point cloud data in the fused data set to eliminate discrete noise points in the point cloud data.

[0081] Noise filtering is the process of denoising global point cloud data. Discrete noise points are isolated points in point cloud data caused by measurement errors or environmental interference. These points will affect subsequent feature extraction and analysis results, so they need to be eliminated.

[0082] In the scenario of automatic positioning and film application for mobile phone cameras, noise filtering is performed on the global point cloud data in the fused data set. Methods such as statistical filtering or radius filtering can be used. Taking statistical filtering as an example, the average distance of each point's neighborhood is first calculated. The point is then judged as a noise point based on the average distance and standard deviation. If the average distance of a point's neighborhood is significantly greater than the global average distance, the point is considered a noise point and removed from the point cloud data. Statistical filtering can effectively eliminate isolated outliers in the mobile phone camera point cloud data, making the point cloud data smoother and more accurate.

[0083] Step S320: Identify a feature-significant region in the smoothed point cloud data using a feature detection algorithm, and use the center point of the region as a candidate key feature point.

[0084] Feature detection algorithms are used to find regions with significant features within point cloud data. Salient regions are areas within the point cloud data with unique geometric characteristics, such as corners and edges. Candidate key feature points are selected from these regions and serve as the targets for subsequent screening and verification.

[0085] In the scenario of automatically positioning and applying a film to a mobile phone camera, a feature detection algorithm is used to identify salient regions within the smoothed point cloud data. This can be done using algorithms such as the ISS (Intrinsic Shape Signature) algorithm or the Harris3D algorithm. Taking the ISS algorithm as an example, local surface features, such as curvature and normal, are first calculated for each point in the point cloud data. Regions with significant features are then filtered based on these features. For each salient region, the coordinates of its center point are calculated and used as a candidate key feature point. For locations such as the edge of a mobile phone camera lens and logos, multiple salient regions are identified, and the center point of each region becomes a candidate key feature point.

[0086] Step S330: Perform multi-view repeatability verification on the candidate key feature points, retain stable key feature points that can be detected under multiple observation perspectives, and form a key feature point position information set.

[0087] Multi-view repeatability verification is a process of further screening and verifying candidate key feature points. By comparing key feature points detected from different observation perspectives, it retains points that can be detected from multiple perspectives to ensure their stability and reliability. The key feature point location information set consists of the location information of stable key feature points that remain after multi-view repeatability verification. In the scenario of automatic positioning and film application for mobile phone cameras, multi-view repeatability verification is performed on candidate key feature points. Because point cloud data is collected from multiple observation positions, each candidate key feature point may have different detection results from different observation perspectives. By comparing the detection results from different perspectives, the frequency of occurrence of each candidate key feature point is counted. Points with a high frequency of occurrence are considered stable key feature points. The location information of all stable key feature points is recorded to form a key feature point location information set.

[0088] As an implementation manner, step S330 may be specifically implemented as the following steps S331 to S334: Step S331: Count the occurrence frequencies of each candidate key feature point under different observation angles to generate feature point detection frequency parameters.

[0089] The feature point detection frequency parameter describes how frequently a candidate key feature point is detected from different viewing angles. It is calculated by counting the frequency of each candidate key feature point's occurrence across multiple viewing angles. This parameter can help determine the stability of a candidate key feature point; a higher frequency indicates a more stable point.

[0090] Step S332: Calculate the similarity between the feature descriptors of each candidate key feature point at each observation perspective. The similarity is calculated by the mean cosine similarity of the descriptor vectors.

[0091] A feature descriptor is a vector used to describe the characteristics of a feature point. It contains the local geometric information of the feature point. By calculating the similarity between feature descriptors under different observation angles, the stability of the feature point under different viewing angles can be determined.

[0092] In a scenario where a mobile phone camera automatically locates and applies a protective film, the similarity between the feature descriptors of each candidate key feature point at each observation angle is calculated. First, a feature descriptor is calculated for each candidate key feature point at each observation angle, for example, using the FPFH (Fast Point Feature Histogram) descriptor. Then, for each candidate key feature point, the cosine similarity of its feature descriptor vectors at different observation angles is calculated. Finally, the cosine similarities at all different observation angles are averaged to obtain the mean similarity for that candidate key feature point.

[0093] Step S333: constructing a feature point stability evaluation index, which is a weighted product of a detection frequency parameter and a descriptor similarity mean. The weight parameter is dynamically adjusted according to the feature significance of the region where the feature point is located.

[0094] The feature point stability evaluation index comprehensively considers the detection frequency parameter and the mean descriptor similarity to assess the stability of candidate key feature points. The weighted product multiplies the detection frequency parameter and the mean descriptor similarity by a weighted sum. The weight parameter is dynamically adjusted based on the feature significance of the feature point's region to more accurately assess the stability of feature points in different regions.

[0095] In the scenario of automatic positioning and film application for mobile phone cameras, a feature point stability evaluation index is constructed. First, a weight parameter is determined based on the feature significance of the region where the feature point is located. For regions with high feature significance, such as the logo area of ​​a mobile phone camera, a higher weight is given to the detection frequency parameter; for regions with low feature significance, such as the lens surface, a higher weight is given to the mean descriptor similarity. Then, the detection frequency parameter and the mean descriptor similarity are weighted and multiplied according to the weight to obtain the feature point stability evaluation index.

[0096] Step S334: screening candidate key feature points with stability evaluation index values ​​higher than a preset threshold as stable key feature points. Stable key feature points have high detection repeatability and feature description consistency under multiple viewing angles.

[0097] The preset threshold is a pre-defined criterion used to screen candidate key feature points with high stability. Stable key feature points are candidate key feature points with stability evaluation index values ​​above the preset threshold. They have high detection repeatability and feature description consistency under multiple viewing angles and can accurately represent the key structure of the workpiece.

[0098] Step S340: performing boundary detection processing on the two-dimensional image data in the fused data set to extract the boundary contour line between the workpiece and the background in the image.

[0099] Boundary detection is the process of finding the boundary between the workpiece and the background in a two-dimensional image. The boundary contour line refers to the outer boundary of the workpiece in the image and contains the workpiece's shape information. In the scenario of automatic positioning and film application by a mobile phone camera, boundary detection is performed on the two-dimensional image data in the fused data set. Algorithms such as the Canny edge detection algorithm and the Sobel edge detection algorithm can be used. Taking the Canny edge detection algorithm as an example, the two-dimensional image is first Gaussian smoothed to reduce the impact of noise. Then, the image's gradient amplitude and direction are calculated. Next, non-maximum suppression is performed to retain only the points with the local maximum gradient amplitude. Finally, double threshold processing is performed to determine the true edge points and possible edge points using two thresholds and connect them into a continuous boundary contour line.

[0100] As an implementation manner, step S340 can be specifically implemented as the following steps S341 to S345: Step S341: performing spatial mapping processing on the two-dimensional image data and the corresponding point cloud data to generate a fused image with depth information, wherein each pixel in the fused image is associated with its corresponding three-dimensional depth value.

[0101] Spatial mapping is the process of spatially associating 2D image data with corresponding point cloud data. This process adds depth information from the point cloud data to the 2D image, generating a fused image with depth information. Each pixel in the fused image is associated with a 3D depth value, providing richer information for subsequent boundary detection and analysis. In the scenario of automatic positioning and film application using a mobile phone camera, spatial mapping is performed between the 2D image data and the corresponding point cloud data. Using the previously established mapping relationship between point cloud coordinates and image pixel coordinates, the depth information in the point cloud data is mapped to pixels in the 2D image. For example, for a pixel (u, v) in the 2D image, the mapping relationship is used to find its corresponding 3D coordinates (X, Y, Z) in the point cloud data, and the Z value is used as the depth value for that pixel. The depth values ​​of all pixels are then added to the 2D image to generate a fused image with depth information. For images captured by a mobile phone camera, this process ensures that each pixel has corresponding depth information, facilitating more accurate boundary detection.

[0102] Step S342: Perform multimodal feature enhancement processing on the fused image, and simultaneously calculate the brightness gradient of the grayscale image and the depth gradient of the depth image to generate a joint gradient response map. The boundary area between the workpiece and the background in the joint gradient response map shows a significant gradient change.

[0103] Multimodal feature enhancement comprehensively considers the brightness information of grayscale images and the depth information of depth images, enhancing image features by calculating their gradients. The brightness gradient reflects changes in brightness in grayscale images, while the depth gradient reflects changes in depth in depth images. The joint gradient response map is an image that combines the brightness and depth gradient information. At the boundary between the artifact and the background, both brightness and depth vary significantly, resulting in significant gradient changes in these areas of the joint gradient response map.

[0104] In the scenario of automatic positioning and film application of mobile phone cameras, multimodal feature enhancement is performed on the fused image. First, the fused image is separated into a grayscale image and a depth image. Then, the brightness gradient of the grayscale image and the depth gradient of the depth image are calculated respectively. The Sobel operator can be used to calculate the gradient. For the grayscale image, its gradient components Gx and Gy in the x and y directions are calculated, and then the gradient amplitude is calculated. For the depth image, the gradient components Dx and Dy in the x and y directions are also calculated, and then the depth gradient amplitude is calculated Finally, the brightness gradient amplitude and the depth gradient amplitude are weighted and summed to generate a joint gradient response map.

[0105] Step S343: extracting a boundary candidate region based on the joint gradient response map, and identifying a set of pixel points whose gradient response values ​​exceed the local mean as boundary candidate points through an adaptive threshold segmentation operation.

[0106] Boundary candidate region extraction is the process of finding possible boundary regions in the joint gradient response map. Adaptive threshold segmentation is a segmentation method that automatically adjusts the threshold based on local image features. This allows for more accurate identification of boundary candidate points. Boundary candidate points are pixels in the joint gradient response map where the gradient response value exceeds the local mean. These points are likely to be the boundary between the artifact and the background.

[0107] In the scenario of automatically positioning and applying a film to a mobile phone camera, boundary candidate regions are extracted based on the joint gradient response map. First, the joint gradient response map is divided into multiple local regions, for example, the image is divided into small 8×8 blocks. Then, the mean of the gradient response values ​​of each local region is calculated. For each pixel, its gradient response value is compared with the mean of the local region. If the gradient response value of the pixel exceeds the local mean, it is considered a boundary candidate point. This adaptive threshold segmentation operation can more accurately identify candidate points at the boundary of the mobile phone camera, avoiding the misjudgment that may result from using a fixed threshold.

[0108] Step S344: Perform depth continuity verification on the boundary candidate points, retain the pixels in the neighborhood whose depth value changes are less than a preset range, and form a boundary candidate area with consistent depth.

[0109] Depth continuity verification is a process of further screening boundary candidate points. It determines whether the point is a true boundary point by checking the change in the depth value of the pixels in the neighborhood. A boundary candidate area with consistent depth refers to an area composed of pixels retained after depth continuity verification. The depth value of these areas has less change and is more likely to be the boundary of the workpiece. In the scene where the mobile phone camera automatically locates and applies a film, the boundary candidate points are subjected to depth continuity verification. For each boundary candidate point, the depth value of the pixels in its neighborhood is checked. A preset range can be set, for example, the depth value change does not exceed 0.1 mm. If the depth value change of the pixels in the neighborhood of a boundary candidate point is less than 0.1 mm, the point is considered to have passed the depth continuity verification and is retained. For pixels with large changes in depth values ​​in the neighborhood, they are deleted from the boundary candidate points. Through this processing, a boundary candidate area with consistent depth is formed, which improves the accuracy of boundary extraction.

[0110] Step S345: Perform closure check on the boundary candidate areas with consistent depth, connect continuous boundary candidate points through contour tracking algorithm, and generate a closed workpiece boundary contour line. The contour line satisfies the dual characteristics of image grayscale boundary and depth discontinuous boundary.

[0111] Closure checking ensures that candidate boundary areas form closed contours, while contour tracking connects consecutive candidate boundary points to form a complete contour line. A closed workpiece boundary contour line is a continuous, enclosed boundary line obtained through closure checking and contour tracking. It satisfies both the image grayscale boundary and the depth discontinuity boundary, providing a more accurate representation of the workpiece's shape.

[0112] In the scenario of automatically positioning and applying a film to a mobile phone camera, the closedness of the boundary candidate areas with consistent depth is checked. The boundary candidate areas can be refined using algorithms such as the Zhang-Suen refinement algorithm or the Guo-Hall refinement algorithm to make the boundaries clearer. Then, a contour tracking algorithm, such as the boundary tracking algorithm, is used to start from a boundary candidate point and sequentially connect adjacent boundary candidate points according to certain rules until a closed contour line is formed. During the connection process, it is necessary to ensure that the contour line satisfies the dual characteristics of the image grayscale boundary and the depth discontinuity boundary. For the boundary of the mobile phone camera, this processing can obtain an accurate closed boundary contour line that can accurately depict the shape of the camera.

[0113] Step S350: converting the image boundary contour line into the three-dimensional space coordinate system of the global point cloud data through a projection mapping relationship, and generating a three-dimensional contour line aligned with the point cloud data.

[0114] A projection mapping relationship is a previously established mapping relationship between point cloud coordinates and image pixel coordinates. This relationship allows the boundary contour line in a 2D image to be transformed into the global point cloud coordinate system in 3D space, generating a 3D contour line aligned with the point cloud data. A 3D contour line can more accurately describe the position and shape of a workpiece's exterior in 3D space. In the scenario of automatic positioning and film application for a mobile phone camera, the image boundary contour line is transformed into the 3D coordinate system of the global point cloud data using a projection mapping relationship. For each pixel point on the image boundary contour line, its corresponding 3D coordinate in the global point cloud data is found using the projection mapping relationship. By concatenating the corresponding 3D coordinates of all pixels, a 3D contour line aligned with the point cloud data is generated. For example, for a pixel point (u, v) on the boundary contour line of a mobile phone camera image, its corresponding 3D coordinate (X, Y, Z) is found using the projection mapping relationship and added to the point set of the 3D contour line. Ultimately, a 3D contour line is obtained that accurately describes the shape of the mobile phone camera and is spatially aligned with the global point cloud data.

[0115] Step S360: performing continuity analysis on the three-dimensional contour lines, extracting contour line segments that are continuous and have consistent features, and forming a continuous contour line feature description set.

[0116] Continuity analysis is the process of inspecting and analyzing 3D contours to determine their continuity and feature consistency. Continuous and consistent contour segments are those within a 3D contour that share the same geometric features. These segments can represent the external features of different parts of a workpiece. A continuous contour feature description set is a collection of feature descriptions of these continuous and consistent contour segments, providing more detailed information for subsequent feature matching and positioning. In the scenario of automatic camera positioning and film application for mobile phones, continuity analysis is performed on 3D contours. First, the 3D contour is divided into multiple segments, and each segment is checked for continuity and feature consistency. Continuity and feature consistency are determined by calculating the distance and angular variation between adjacent points. If the distance between adjacent points is less than a preset threshold and the angular variation is also less than a preset threshold, the segment is considered continuous and feature consistent. All continuous and consistent contour segments are extracted to form a continuous contour feature description set. The three-dimensional contour line of a mobile phone camera can be divided into different continuous and consistent contour line segments such as the lens edge and peripheral structure, and the length, curvature and other features of each segment are described to form a continuous contour line feature description set.

[0117] Step S370: merging the key feature point position information set and the continuous contour line feature description set to form a workpiece feature identification set.

[0118] The purpose of merging the key feature point position information set with the continuous contour line feature description set is to integrate the information of the key feature points and continuous contour lines on the workpiece surface to form a more comprehensive and accurate workpiece feature identification set.

[0119] In the scenario of automatic positioning and film application using a mobile phone camera, the key feature point location information set obtained in step S330 is merged with the continuous contour line feature description set obtained in step S360. The key feature point location information and the continuous contour line feature description information are stored in the same data structure to form a workpiece feature identification set. For example, the 3D coordinates of the key feature points and the continuous contour line feature description information, such as length and curvature, can be stored in a list or dictionary to facilitate subsequent feature matching and positioning operations.

[0120] Step S400: performing feature matching association processing on the workpiece feature identification set and the pre-stored standard workpiece feature template to generate a feature matching correspondence set, which includes the position correspondence of key feature points and the shape matching relationship of contour lines.

[0121] Feature matching association processing compares and matches a set of workpiece feature identifiers with pre-stored standard workpiece feature templates. This process finds the corresponding relationships between the workpiece features and the standard template features. The feature matching correspondence set records the positional correspondences of key feature points and the shape matching relationships of contour lines.

[0122] As an implementation manner, step S400 can be specifically implemented as the following steps S410 to S460: Step S410: extracting a standard key feature point set and a standard contour line set from a pre-stored standard workpiece feature template.

[0123] Pre-stored standard workpiece feature templates are pre-stored in the system and represent ideal workpiece feature information. A standard key feature point set is a collection of representative points in the template that reflect the important structure of the workpiece. The position and characteristics of these points are relatively stable across different production batches and application scenarios. A standard contour line set is a collection of continuous curves in the template that describe the workpiece's outer contour boundaries.

[0124] Step S420: Calculate the spatial distance between the key feature points in the workpiece feature identification set and each point in the standard key feature point set to establish an initial matching pair.

[0125] Spatial distance is the distance between two points in three-dimensional space. By calculating the spatial distance between key feature points in the workpiece feature identification set and each point in the standard key feature point set, we can preliminarily determine their correspondence. Initial matching pairs are composed of key feature points with close distances, and these pairs will serve as the basis for further verification and screening.

[0126] In the scenario of automatic positioning and film application using a mobile phone camera, for each key feature point in the workpiece feature identification set, the spatial distance between it and each point in the standard key feature point set is calculated. Spatial distance can be calculated using the Euclidean distance formula. For a key feature point on the mobile phone camera, the distance between it and all key feature points in the standard template is calculated. The point with the smallest distance is used to form an initial matching pair with the key feature point.

[0127] Step S430: Verify the spatial relationship consistency of the initial matching pairs, select matching pairs that meet the consistency of relative position relationship as valid key feature point matching pairs, and form a position correspondence relationship of the key feature points.

[0128] Spatial consistency verification ensures that the relative spatial positions of the key feature points in the initial matching pairs are consistent with those in the standard template. Valid key feature point matching pairs are verified to meet the relative positional relationship requirements and more accurately reflect the correspondence between the workpiece and the standard template. The key feature point positional correspondence is a collection of these valid key feature point matching pairs.

[0129] In the scenario of automatic positioning and film application of mobile phone cameras, the spatial relationship consistency of the initial matching pairs is verified by comparing the relative position relationship of the neighborhood feature points of the key feature points in the initial matching pairs.

[0130] As an implementation manner, step S430 may be specifically implemented as the following steps S431 to S434: Step S431: extracting a set of neighborhood feature points of the key feature points in the initial matching pair of the reference data set, and constructing a local topological structure including the relative positions of the key feature points and the neighborhood points.

[0131] A neighborhood feature point set is a collection of other feature points within a certain range around a key feature point. The relative position of these points can reflect the local characteristics of the key feature point. The local topology is a structure composed of the relative positional relationships between key feature points and neighborhood feature points. It can be represented as a graph or matrix.

[0132] In the scenario of automatic positioning and film application of mobile phone cameras, for the key feature points in the initial matching pair of the benchmark data set, its neighborhood feature point set is extracted. The neighborhood range can be determined by setting a neighborhood radius. For example, all feature points within a radius of 1 mm with the key feature point as the center are regarded as neighborhood feature points. If there is a key feature point P in the initial matching pair of the benchmark data set, its coordinates are (x p ,y p ,z p ), by traversing all the feature points in the workpiece feature identification set, calculate the distance between each point and P, and if the distance is less than 1 mm, add the point to the neighborhood feature point set. Then, construct a local topological structure containing the relative positions of the key feature point P and the neighborhood points. It can be represented by a matrix, where each row of the matrix represents a neighborhood feature point, and the columns represent the relative distances between the neighborhood feature point and the key feature point P in the X, Y, and Z directions. For example, for the neighborhood feature point Q, its coordinates are (x q ,y q ,z q ), then the corresponding row element in the matrix is ​​(x q -x p ,y q -y p ,z q -z p ).

[0133] Step S432: extracting a set of neighborhood feature points corresponding to key feature points in the initial matching pairs of other observation positions, and constructing a local topological structure of the same scale.

[0134] For the corresponding key feature points in the initial matching pairs of other observation positions, it is also necessary to extract their neighborhood feature point sets and construct a local topological structure, and to ensure that the local topological structure scale is the same as that of the key feature points in the initial matching pairs of the benchmark data set, so as to facilitate subsequent comparison and verification. In the scenario of automatic positioning and film application of mobile phone cameras, for the corresponding key feature points in the initial matching pairs of other observation positions, the same method as step S431 is used to extract the neighborhood feature point set. For example, for the key feature point R in the initial matching pairs of other observation positions, 1 mm is also used as the neighborhood radius to traverse the feature points under the observation position and find the points with a distance of less than 1 mm from R as the neighborhood feature point set. Then, a local topological structure of the same scale is constructed. If the neighborhood feature point set of the key feature point in the benchmark data set has n points, then when constructing the local topological structure of the key feature points corresponding to the other observation positions, try to ensure that the neighborhood feature point set has n points. If there are less than n points in the neighborhood, they can be supplemented by interpolation or approximation methods; if there are too many points, they can be filtered to select the n points closest to the key feature point. The local topology is also represented by a matrix for subsequent comparison.

[0135] Step S433: Calculate the directional consistency and distance deviation value of the corresponding relative position vectors in the two sets of local topological structures. The directional consistency is measured by the cosine value of the vector angle, and the distance deviation value is measured by the absolute value of the vector length difference.

[0136] Directional consistency refers to the degree of directional similarity between the corresponding relative position vectors in two sets of local topological structures. Measuring this using the cosine of the angle between the two vectors provides a visual indicator of whether the two vectors are oriented in the same direction. Distance deviation refers to the difference in length between the corresponding relative position vectors in two sets of local topological structures. Measuring this using the absolute value of the difference in vector lengths provides a visual indicator of whether the two vectors are similar in length.

[0137] Step S434: Set the direction consistency threshold and the distance deviation threshold, and retain the initial matching pairs that meet the requirements of direction consistency higher than the threshold and distance deviation lower than the threshold as valid key feature point matching pairs. The valid matching pairs meet the requirements of global position correspondence and local structure consistency.

[0138] The directional consistency threshold and distance deviation threshold are pre-set criteria used to select initial matching pairs that meet the spatial relationship consistency requirements. Valid key feature point matching pairs are those that meet both directional consistency above the threshold and distance deviation below the threshold. They correspond in both global position and local structure, accurately reflecting the key feature point correspondence between the workpiece and the standard template.

[0139] Step S440: performing shape similarity measurement on the continuous contour lines in the workpiece feature identification set and the contour lines in the standard contour line set, and calculating feature difference values ​​between the two at equidistant sampling points.

[0140] Shape similarity measurement is a method used to evaluate the degree of similarity in shape between continuous contour lines in a workpiece feature identification set and contour lines in a standard contour line set. Equidistant sampling points are points selected at regular intervals on the contour line. By comparing the features at these equidistant sampling points, the feature difference between the two can be calculated. The smaller the feature difference value, the more similar the shapes of the two contour lines are. In the scenario of automatic positioning and film application by a mobile phone camera, a shape similarity measurement is performed on the continuous contour lines in the workpiece feature identification set and the contour lines in the standard contour line set. First, the contour lines are sampled at equal intervals. Then, the feature difference between the two sets of equidistant sampling points is calculated, which can be calculated using Euclidean distance. The Euclidean distances of all pairs of equidistant sampling points are summed to obtain the feature difference value.

[0141] Step S450: selecting the contour line pair with the smallest feature difference value as a valid contour line matching pair to form a shape matching relationship of the contour lines.

[0142] Valid contour matching pairs are those with the smallest feature difference values. They are the most similar in shape and can accurately reflect the contour correspondence between the workpiece and the standard template. The contour shape matching relationship is the set composed of these valid contour matching pairs.

[0143] In the scenario of automatic camera positioning and film application, a shape similarity measurement is performed on all continuous contours in a workpiece feature set and all contours in a standard contour set, resulting in a series of feature difference values. These feature difference values ​​are compared, and the contour pair with the smallest feature difference value is selected as a valid contour matching pair. All valid contour matching pairs are organized into a set to form a contour shape matching relationship.

[0144] Step S460: merging the valid key feature point matching pairs and the valid contour line matching pairs to form a feature matching correspondence set.

[0145] Merging valid key feature point matching pairs and valid contour matching pairs is to integrate the key feature point correspondences and contour line correspondences between the workpiece and the standard template, forming a more comprehensive and accurate feature matching correspondence set. This set contains the corresponding information between the workpiece and the standard template in terms of key features and contours.

[0146] For example, the valid key feature point matching pair set obtained in step S434 and the valid contour line matching pair set obtained in step S450 are merged. They can be stored in a unified data structure, for example, using a list, where the elements in the list are valid key feature point matching pairs and valid contour line matching pairs, respectively.

[0147] Step S500: Calculating the three-dimensional positioning coordinate information of the target workpiece in the scene based on the feature matching correspondence set. The three-dimensional positioning coordinate information includes the coordinates of the center of mass of the workpiece and spatial posture angle parameters.

[0148] Calculating the 3D positioning coordinates of the target workpiece in the scene based on a set of feature matching correspondences utilizes the feature correspondences between the workpiece and a standard template to determine the specific position and posture of the target workpiece in the current scene. The workpiece's center of mass coordinates, representing its center of gravity in 3D space, reflect its overall position. Spatial attitude angle parameters describe the workpiece's rotational state in 3D space, including rotation angles around the X, Y, and Z axes, and accurately represent the workpiece's posture.

[0149] As an implementation manner, step S500 can be specifically implemented as the following steps S510 to S570: Step S510: selecting multiple groups of valid key feature point matching pairs from the position correspondence of the key feature points.

[0150] Selecting multiple valid key feature point matching pairs from the positional correspondences of the key feature points is intended to provide sufficient information to calculate the positional transformation parameters of the target workpiece. Multiple valid key feature point matching pairs can more accurately reflect the positional correspondence between the workpiece and the standard template, thereby improving the accuracy of the calculation results.

[0151] In the scenario where the mobile phone camera automatically positions and applies a film, multiple groups of valid key feature point matching pairs are selected from the position correspondence of the key feature points obtained in step S434. For example, a matching pair between the center point of the lens and the center point of the lens in the standard template, a matching pair between the vertex of the lens peripheral marker and the corresponding vertex in the standard template, etc. are selected. If there are 10 groups of valid key feature point matching pairs in the position correspondence of the key feature points, 5 representative matching pairs are selected from them, such as matching pairs located at different positions of the lens, to ensure that the positional relationship between the mobile phone camera and the standard template can be fully reflected.

[0152] Step S520: Calculate the position transformation parameters of the target workpiece relative to the standard workpiece by using the scene coordinates and the standard coordinates of the valid key feature point matching pairs through a coordinate transformation algorithm.

[0153] A coordinate transformation algorithm is used to calculate the positional transformation of a target workpiece relative to a reference workpiece. By matching the scene coordinates and reference coordinates of valid key feature points, the target workpiece's spatial transformation parameters, such as translation, rotation, and scaling, can be determined. These positional transformation parameters can be represented by a transformation matrix, which converts the reference workpiece's coordinates into the target workpiece's coordinates.

[0154] In the scene of automatic positioning and film application of mobile phone cameras, coordinate transformation algorithms such as ICP (Iterative Closest Point) algorithm or SVD (Singular Value Decomposition) algorithm are used to calculate the position transformation parameters of the target workpiece relative to the standard workpiece using the scene coordinates and standard coordinates of the selected valid key feature point matching pairs. Taking the SVD algorithm as an example, for the 5 selected groups of valid key feature point matching pairs, their scene coordinates and standard coordinates are respectively combined into two point sets P and Q. First, the centroid C of the point sets P and Q is calculated. p and C q Then, remove the centroid of point sets P and Q to obtain new point sets P' and Q'. Next, calculate the covariance matrix of P' and Q' , where p i 'and q i ' are points in P' and Q' respectively. Perform singular value decomposition on the covariance matrix H and get Finally, calculate the rotation matrix R=VU T , translation vector t=C q -RC pThe rotation matrix R and the translation vector t constitute the position transformation parameters of the target workpiece relative to the standard workpiece.

[0155] Step S530: Based on the position transformation parameters, the center-of-mass coordinates of the standard workpiece are transformed into the scene coordinate system to obtain the center-of-mass coordinates of the target workpiece.

[0156] Transforming the standard workpiece's center of mass coordinates to the scene coordinate system based on the position transformation parameters is accomplished by applying the position transformation parameters to the standard workpiece's center of mass coordinates. The target workpiece's center of mass coordinates is the center of gravity in the scene coordinate system and accurately reflects the target workpiece's overall position in the current scene.

[0157] In the scenario of automatic positioning and film application of mobile phone cameras, the centroid coordinates C of the standard mobile phone camera are known. s , and the position transformation parameters (rotation matrix R and translation vector t) calculated in step S520. s Perform the transformation according to the transformation formula C t =RC s +t, where Ct is the center of mass coordinate of the target workpiece in the scene coordinate system. For example, the center of mass coordinate of a standard mobile phone camera is C s =(x s ,y s ,z s ), the rotation matrix , translation vector t=(t x ,t y ,t z ), then the center of mass coordinates of the target workpiece .

[0158] Step S540: Decomposing the position transformation parameters by using an attitude calculation algorithm, and extracting the rotation angle of the target workpiece around each coordinate axis as a spatial attitude angle parameter.

[0159] The posture calculation algorithm decomposes the rotation information in the position transformation parameters to obtain the target workpiece's rotation angles around each coordinate axis. The spatial posture angle parameter is an important parameter that describes the target workpiece's rotation state in three-dimensional space and can accurately represent the workpiece's posture.

[0160] As an implementation manner, step S540 may be specifically implemented as the following steps S541 to S547: Step S541: Separate the rotation component and the translation component from the position transformation parameter, where the rotation component is used to represent the spatial posture transformation relationship of the workpiece.

[0161] Position transformation parameters usually include rotational components and translational components. The rotational component describes the rotational state of the workpiece in three-dimensional space and can be represented by a rotation matrix. The translational component describes the translational state of the workpiece in three-dimensional space and can be represented by a translation vector. The purpose of separating the rotational component and the translational component from the position transformation parameters is to process the rotational component separately to extract the spatial posture angle parameters of the workpiece. For the position transformation parameters (rotation matrix R and translation vector t) calculated in step S520, the rotation matrix R is directly used as the rotation component, and the translation vector t is used as the translation component. The rotation matrix R can represent the rotation state of the mobile phone camera relative to the standard template. By further processing the rotation matrix R, the rotation angle of the mobile phone camera around each coordinate axis can be obtained.

[0162] Step S542: Convert the shape matching relationship of the contour line into a posture constraint condition, where the constraint condition includes the relative position consistency and direction alignment requirements of the matching contour line in the three-dimensional space.

[0163] Converting contour shape matching relationships into pose constraints is designed to leverage the contour shape information to further constrain the workpiece's pose. The requirement for relative position consistency of matching contours in 3D space states that the relative positions of the continuous contours in the workpiece feature set and the matching contours in the standard contour set should be consistent. The requirement for directional alignment states that the tangent directions of the two matching contours should be as consistent as possible.

[0164] In the scene of automatic positioning and film application of mobile phone cameras, for the valid contour line matching pairs obtained in step S450, they are converted into posture constraints. For example, for the circular contour line matching pairs of mobile phone cameras, it is required that in three-dimensional space, the center positions of the two circular contour lines are relatively consistent, and the tangent directions are as similar as possible at corresponding points. The posture constraints can be determined by calculating the relative positions and tangent directions of equidistant sampling points on the matching contour lines. For each pair of equidistant sampling points on the matching contour lines, the relative position deviation and tangent direction angle between them are calculated. If the relative position deviation exceeds a certain threshold or the tangent direction angle exceeds a certain range, it is considered that the posture constraints are not met.

[0165] Step S543: Construct a multi-feature joint optimization model, which uses the position error of key feature point matching and the shape error of contour line matching as a dual-objective optimization function.

[0166] The multi-feature joint optimization model comprehensively considers the positional error of key feature point matching and the shape error of contour matching to improve the accuracy of workpiece pose calculation. The core of this model is the dual-objective optimization function, which uses the positional error of key feature point matching and the shape error of contour matching as two objectives. By optimizing the values ​​of these two objective functions, a more accurate workpiece pose can be obtained.

[0167] In the scenario of automatic positioning and film application by mobile phone camera, a multi-feature joint optimization model is constructed. The position error of key feature point matching can be measured by calculating the Euclidean distance between the scene coordinates and the transformed standard coordinates of the effective key feature point matching pair. The shape error of contour line matching can be represented by the feature difference value calculated in step S440. The dual-objective optimization function can be expressed as ,in is the position error of key feature point matching, is the shape error of contour matching, w1 and w2 are weight coefficients, which are adjusted according to the importance of key feature points and contours in positioning.

[0168] Step S544: Adjust the parameters of the rotation component through an iterative optimization algorithm to minimize the total error value of the multi-feature joint optimization function and obtain the optimized rotation component.

[0169] The iterative optimization algorithm continuously adjusts the parameters of the rotational component to find a method that minimizes the total error of the multi-feature joint optimization function. Through multiple iterations, the parameters of the rotational component are gradually optimized to minimize the position error of key feature point matching and the shape error of contour matching, thereby achieving a more accurate workpiece pose.

[0170] In the scenario of automatically positioning and applying a film to a mobile phone camera, an iterative optimization algorithm, such as the gradient descent algorithm or the Levenberg-Marquardt algorithm, is used to adjust the parameters of the rotation component separated in step S541. Taking the gradient descent algorithm as an example, the parameters of the rotation component are first initialized, and then the gradient of the multi-feature joint optimization function under the current parameters is calculated. Based on the direction of the gradient, the parameters of the rotation component are updated with a certain step size. This process is repeated until the total error value of the multi-feature joint optimization function converges to a smaller value.

[0171] Step S545: performing orthogonality constraint processing on the optimized rotation components to ensure that the rotation components satisfy the mathematical properties of the orthogonal matrix.

[0172] Orthogonality constraints are further adjusted to ensure that the optimized rotation components meet the mathematical properties of orthogonal matrices. An orthogonal matrix is ​​one whose transpose is equal to its inverse. In three-dimensional rotations, the rotation matrix must be orthogonal to ensure the correctness of the rotation operation.

[0173] In the scenario of automatic positioning and film application of the mobile phone camera, the optimized rotation component obtained in step S544 is subjected to orthogonality constraint processing. The QR decomposition method can be used to decompose the rotation component into the product of an orthogonal matrix Q and an upper triangular matrix R, and then the orthogonal matrix Q is used as the rotation component that satisfies the orthogonality constraint. For example, for the optimized rotation component R opt , perform QR decomposition to get R opt =QR, and use the orthogonal matrix Q as the new rotation component, which ensures that the rotation component satisfies the mathematical properties of the orthogonal matrix, thereby ensuring the accuracy of the mobile phone camera posture calculation.

[0174] Step S546: Decompose the orthogonalized rotation components into basic rotation operations around the X-axis, Y-axis, and Z-axis, and calculate the rotation angle corresponding to each basic rotation operation as a spatial posture angle parameter.

[0175] Decomposing the orthogonalized rotation components into basic rotation operations about the X, Y, and Z axes is done to decompose 3D rotation into three basic rotation operations, allowing calculation of the rotation angles about each coordinate axis. The rotation angles corresponding to each basic rotation operation are the spatial attitude angle parameters, which accurately describe the workpiece's attitude in 3D space.

[0176] In the scenario of automatic positioning and film application of the mobile phone camera, the orthogonalized rotation component Q obtained in step S545 is decomposed into basic rotation operations around the X axis, Y axis, and Z axis. The Euler angle decomposition method can be used to express the rotation matrix Q as a rotation around the X axis. , rotate around the Y axis , rotate around the Z axis The product of the three basic rotation matrices, that is By solving this equation, we can get the rotation angle around the X axis, Y axis, and Z axis. .

[0177] Step S547: Perform multi-view consistency verification on the extracted spatial attitude angle parameters, compare the attitude angle differences calculated under different observation positions, and retain the angle parameters with a difference value less than a preset threshold as the final result.

[0178] Multi-view consistency verification ensures the consistency of the extracted spatial pose angle parameters across different observation positions. The pose angles calculated from different observation positions are compared. If the difference is less than a preset threshold, the angle parameter is considered reliable and retained as the final result. The preset threshold is a pre-defined standard used to determine whether the pose angle difference is within an acceptable range.

[0179] In the scenario of automatic positioning and film application of the mobile phone camera, the spatial attitude angle parameters extracted in step S546 are verified for consistency from multiple perspectives. Since the attitude angle of the mobile phone camera can be calculated at different observation positions, the differences in attitude angles calculated at these different observation positions are compared. For example, the rotation angle around the X axis calculated at the observation position directly above the lens is , the rotation angle around the X axis calculated from the camera's obliquely upward observation position is , calculate the difference value The preset threshold is set to 2°. Since the difference is less than the preset threshold, one of the two angle parameters is retained as the final result. The same verification process is performed on the rotation angles around the Y and Z axes, and finally a reliable spatial posture angle parameter is obtained.

[0180] Step S550: performing error verification on the shape matching relationship of the contour line, and calculating the position deviation of the matching contour line in the scene coordinate system.

[0181] Error verification of the contour shape matching relationship is performed to evaluate the accuracy of the matching contour in the scene coordinate system. Position deviation refers to the difference between the actual position of the matching contour in the scene coordinate system and the ideal position. By calculating the position deviation, we can determine the accuracy of the currently calculated workpiece posture and position.

[0182] In the mobile phone camera automatic positioning film application scenario, for the valid contour line matching pairs obtained in step S450, their positional deviation in the scene coordinate system is calculated. For each pair of equidistant sampling points on the matching contour line, the difference between their actual distance and ideal distance in the scene coordinate system is calculated. For example, for a circular contour line matching pair of the mobile phone camera, the distance deviation of the corresponding equidistant sampling points on the matching contour line in the scene coordinate system is calculated. The distance deviations of all equidistant sampling points are summed and averaged to obtain the positional deviation of the matching contour line in the scene coordinate system.

[0183] Step S560: If the position deviation meets the preset requirements, the currently calculated center of mass coordinates and spatial attitude angle parameters are retained as the final three-dimensional positioning coordinate information.

[0184] If the position deviation meets the preset requirements, the currently calculated workpiece posture and position are accurate. Therefore, the currently calculated center of mass coordinates and spatial posture angle parameters are retained as the final 3D positioning coordinate information. The preset requirement is a pre-set standard used to determine whether the position deviation is within an acceptable range.

[0185] Step S570: If the position deviation does not meet the preset requirement, reselect valid key feature point matching pairs and repeat the coordinate transformation calculation process until the error requirement is met.

[0186] If the position deviation does not meet the preset requirements, it means that the currently calculated workpiece posture and position are inaccurate and need to be recalculated. Reselect valid key feature point matching pairs and repeat the coordinate transformation calculation process. Through continuous adjustment and optimization, until the position deviation of the matching contour line in the scene coordinate system meets the preset requirements.

[0187] In the scenario of automatic positioning and film application by a mobile phone camera, if the position deviation of the matching contour line calculated in step S550 in the scene coordinate system is greater than 0.2 mm, it does not meet the preset requirements. Then, a group of valid key feature point matching pairs are reselected from the position correspondence of the key feature points obtained in step S434. For example, 5 groups of valid key feature point matching pairs were selected before, and another 5 groups are now reselected. Then, the coordinate transformation calculation process of steps S520-S560 is repeated, including calculating the position transformation parameters, decomposing the rotation components, calculating the spatial attitude angle parameters, verifying the position deviation, etc. This process is repeated until the position deviation of the matching contour line in the scene coordinate system meets the preset requirements, and accurate final three-dimensional positioning coordinate information is obtained. In this way, the positioning of the precision workpiece (mobile phone camera) based on three-dimensional vision is completed, providing accurate position and attitude information for subsequent operations such as film application.

[0188] It is understandable that the various algorithms involved in the above-mentioned introductions of the embodiments of the present invention, such as the Euclidean distance algorithm, the cosine distance algorithm, the posture solution algorithm, the SVD algorithm, etc., can all be learned from the relevant content in the prior art. In order to save space, they will not be expanded too much in the embodiments of the present invention. In addition, when implementing the scheme of the present invention, those skilled in the art can supplement the details according to the common knowledge in the field. For example, according to the common knowledge in the field, normalization can be used to eliminate dimensional conflicts before feature fusion, interpolation can be used to eliminate dimensional differences, and thresholds can be reasonably set based on historical data, experience or business scenario requirements. The model can be trained based on a general model training method, and the number of layers in the model structure can be set based on actual needs, the activation function can be selected, etc. The present invention will no longer provide redundant introductions to the overly detailed implementation process.

[0189] Figure 2 A hardware entity diagram of a computer system provided by an embodiment of the present invention is as follows Figure 2 As shown, the hardware entity of the computer system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and when the processor 1001 executes the program, the steps in the method of any of the above embodiments are implemented.

Claims

1. A precision workpiece positioning method based on three-dimensional vision, characterized in that: The method comprises: Acquire a three-dimensional visual data set of a scene containing a target workpiece, wherein the three-dimensional visual data set of the scene consists of point cloud data of the workpiece surface collected synchronously from multiple perspectives and two-dimensional image data of corresponding perspectives; Performing multimodal data fusion processing on the three-dimensional visual data set of the scene to generate a fused data set with aligned spatial coordinates, the fused data set including a mapping relationship between point cloud coordinates and two-dimensional image pixel coordinates; Extracting a workpiece feature identification set from the fused data set, wherein the workpiece feature identification set includes key feature point position information and continuous contour line feature description of the workpiece surface; Performing feature matching association processing on the workpiece feature identification set and a pre-stored standard workpiece feature template to generate a feature matching correspondence set, wherein the feature matching correspondence set includes a position correspondence of key feature points and a shape matching relationship of contour lines; The three-dimensional positioning coordinate information of the target workpiece in the scene is calculated based on the feature matching correspondence set, and the three-dimensional positioning coordinate information includes the coordinates of the center of mass of the workpiece and space posture angle parameters.

2. The three-dimensional vision-based precision workpiece positioning method according to claim 1, characterized in that: The step of obtaining a three-dimensional visual data set of a scene containing a target workpiece includes: Control the 3D vision acquisition equipment to collect data at multiple preset observation positions around the workpiece, and set fixed equipment posture parameters at each observation position; At each observation position, the laser radar sensor obtains point cloud data of the workpiece surface, wherein the point cloud data includes the three-dimensional spatial coordinate information of each point; At the same observation position, the RGB camera is used to synchronously collect two-dimensional image data of the corresponding viewing angle, wherein the two-dimensional image data contains texture and color information of the workpiece surface; Record the device pose parameters at each observation position as a spatial reference benchmark, which contains the position and orientation information of the device relative to the scene's fixed coordinate system; The point cloud data and two-dimensional image data collected at each observation position are stored according to the corresponding spatial reference benchmark to form a scene three-dimensional visual data set consisting of multiple groups of point cloud-image data pairs.

3. The three-dimensional vision-based precision workpiece positioning method according to claim 2, characterized in that: The performing multimodal data fusion processing on the scene three-dimensional visual data set to generate a fused data group with spatial coordinate alignment includes: Select any set of point cloud-image data pairs in the scene 3D visual data set as a reference data set, and extract feature points of the 2D image of the reference data set; Perform feature matching on the two-dimensional image data of other observation positions and the two-dimensional image data of the reference data set through a feature matching algorithm to obtain the transformation relationship between the images of different observation positions; Based on the transformation relationship and the spatial reference benchmark of each observation position, calculating the coordinate transformation parameters of the point cloud data of other observation positions relative to the point cloud data of the benchmark data group; The point cloud data of other observation positions are converted into the spatial coordinate system of the reference data set using the coordinate conversion parameters, and are spliced ​​with the point cloud data of the reference data set to generate global point cloud data; The global point cloud data and the two-dimensional image data of each observation position are projected and transformed. The three-dimensional coordinates of the point cloud are projected to the pixel coordinate system of the corresponding two-dimensional image through the camera parameters. The mapping relationship between the point cloud coordinates and the image pixel coordinates is established to form a fusion data group with aligned spatial coordinates.

4. The three-dimensional vision-based precision workpiece positioning method according to claim 3, characterized in that: The step of extracting a workpiece feature identification set from the fused data group includes: Perform noise filtering on the global point cloud data in the fused data group to eliminate discrete noise points in the point cloud data; Identify the feature-significant area in the smoothed point cloud data using a feature detection algorithm, and use the center point of the area as a candidate key feature point; Perform multi-view repeatability verification on candidate key feature points, retain stable key feature points that can be detected under multiple observation perspectives, and form a set of key feature point location information; Perform boundary detection on the two-dimensional image data in the fused data set to extract the boundary contour line between the workpiece and the background in the image; The image boundary contour line is converted to the three-dimensional space coordinate system of the global point cloud data through a projection mapping relationship to generate a three-dimensional contour line aligned with the point cloud data; Perform continuity analysis on the three-dimensional contour lines, extract continuous contour line segments with consistent features, and form a continuous contour line feature description set; The key feature point position information set is combined with the continuous contour line feature description set to form a workpiece feature identification set.

5. The three-dimensional vision-based precision workpiece positioning method according to claim 4, characterized in that: The step of performing feature matching association processing on the workpiece feature identification set and the pre-stored standard workpiece feature template to generate a feature matching correspondence relationship set includes: Extracting a standard key feature point set and a standard contour line set from a pre-stored standard workpiece feature template; Calculate the spatial distance between the key feature points in the workpiece feature identification set and each point in the standard key feature point set to establish an initial matching pair; Verify the spatial relationship consistency of the initial matching pairs, select the matching pairs that meet the consistency of relative position relationship as the valid key feature point matching pairs, and form the position correspondence relationship of the key feature points; Perform shape similarity measurement on the continuous contour lines in the workpiece feature identification set and the contour lines in the standard contour line set, and calculate the feature difference values ​​between the two at equidistant sampling points; Select the contour line pair with the smallest feature difference value as the valid contour line matching pair to form the shape matching relationship of the contour line; The valid key feature point matching pairs and valid contour line matching pairs are merged to form a feature matching correspondence set.

6. The three-dimensional vision-based precision workpiece positioning method according to claim 5, characterized in that: The calculating of the three-dimensional positioning coordinate information of the target workpiece in the scene based on the feature matching correspondence relationship set includes: Select multiple groups of valid key feature point matching pairs from the position correspondence of the key feature points; Through the coordinate transformation algorithm, the scene coordinates and standard coordinates of the effective key feature point matching pair are used to calculate the position transformation parameters of the target workpiece relative to the standard workpiece; Based on the position transformation parameters, the center-of-mass coordinates of the standard workpiece are transformed into the scene coordinate system to obtain the center-of-mass coordinates of the target workpiece; The position transformation parameters are decomposed and processed by the posture solution algorithm, and the rotation angle of the target workpiece around each coordinate axis is extracted as the spatial posture angle parameter; Perform error verification on the shape matching relationship of the contour line and calculate the position deviation of the matching contour line in the scene coordinate system; If the position deviation meets the preset requirements, the currently calculated center of mass coordinates and spatial attitude angle parameters are retained as the final three-dimensional positioning coordinate information; If the position deviation does not meet the preset requirements, the valid key feature point matching pairs are reselected and the coordinate transformation calculation process is repeated until the error requirements are met.

7. The three-dimensional vision-based precision workpiece positioning method according to claim 2, characterized in that: The three-dimensional vision acquisition device is controlled to collect data at a plurality of preset observation positions around the workpiece, and fixed device posture parameters are set at each observation position, including: Pre-scanning the workpiece surface to generate preliminary point cloud data including surface feature distribution, wherein the surface feature distribution includes texture complexity and curvature change rate information; constructing an observation position planning model based on the surface feature distribution information, wherein the observation position planning model takes coverage integrity of feature-rich areas and redundancy control of feature-sparse areas as optimization goals; Dynamically determining a target observation position set around the workpiece through the observation position planning model, wherein the density of observation points in the feature-rich area of ​​the target observation position set is higher than that in the feature-sparse area; At each target observation position, the position calibration module is used to adjust the lens orientation of the 3D vision acquisition device so that the lens optical axis maintains a preset angle relationship with the normal vector of the corresponding area on the workpiece surface. Record the device space coordinates and lens orientation parameters of each target observation position as fixed device pose parameters, where the device pose parameters include the device origin coordinates and the optical axis direction vector; Based on the device posture parameters, the three-dimensional vision acquisition device is controlled to synchronously acquire point cloud data and two-dimensional image data of each observation position to form a scene three-dimensional vision data set covering the distribution of surface features of the workpiece.

8. The three-dimensional vision-based precision workpiece positioning method according to claim 3, characterized in that: The method of performing feature matching on the two-dimensional image data of other observation positions and the two-dimensional image data of the reference data set by using a feature matching algorithm to obtain a transformation relationship between images at different observation positions includes: Extract feature points from the two-dimensional image data of the reference data set, and simultaneously obtain the three-dimensional coordinates of each feature point in the corresponding point cloud data as feature space anchor points; Perform the same feature point extraction and feature space anchor point acquisition operations on the two-dimensional image data of other observation positions to generate a feature point-space anchor point association set corresponding to the observation position; Calculate the similarity between the image feature descriptors of the feature points of the benchmark data set and the feature points of other observation positions to generate an initial similarity matching matrix; Based on the three-dimensional coordinate information of the feature space anchor point, the spatial Euclidean distance between the feature point of the benchmark data set and the feature point of other observation positions is calculated to generate a spatial distance matching matrix; Performing weighted fusion processing on the initial similarity matching matrix and the spatial distance matching matrix to generate a comprehensive matching score matrix, wherein the weight parameters of the weighted fusion are dynamically adjusted according to the texture complexity of the area where the feature points are located; Screening feature point pairs with comprehensive matching scores higher than a preset threshold as candidate matching pairs, wherein the candidate matching pairs simultaneously meet the requirements of image feature similarity and spatial proximity; Perform topological consistency verification on candidate matching pairs to ensure that the relative position relationship between the feature points of the reference data set is consistent with the relative position relationship between the feature points of other observation positions; The candidate matching pairs that have passed the topological consistency verification are retained as valid matching pairs, and the transformation relationship parameters between images at different observation positions are solved based on the coordinate information of the valid matching pairs.

9. The three-dimensional vision-based precision workpiece positioning method according to claim 4, characterized in that: The performing boundary detection processing on the two-dimensional image data in the fused data group to extract the boundary contour line between the workpiece and the background in the image includes: Performing spatial mapping processing on the two-dimensional image data and the corresponding point cloud data to generate a fused image with depth information, wherein each pixel in the fused image is associated with its corresponding three-dimensional depth value; performing multimodal feature enhancement processing on the fused image, and simultaneously calculating the brightness gradient of the grayscale image and the depth gradient of the depth image to generate a joint gradient response map, wherein the boundary area between the artifact and the background in the joint gradient response map exhibits a significant gradient change; Extracting a candidate boundary region based on the joint gradient response map, and identifying a set of pixels whose gradient response values ​​exceed the local mean as candidate boundary points through an adaptive threshold segmentation operation; Performing depth continuity verification on the candidate boundary points, retaining pixels in the neighborhood whose depth value changes are less than a preset range, and forming a candidate boundary area with consistent depth; The boundary candidate areas with consistent depth are checked for closure, and continuous boundary candidate points are connected by a contour tracking algorithm to generate a closed workpiece boundary contour line, which satisfies the dual characteristics of image grayscale boundary and depth discontinuous boundary.

10. A computer system comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images