Adaptive machining positioning method and system based on visual feature matching

Abstract

This invention provides an adaptive machining positioning method and system based on visual feature matching, relating to the field of computer vision technology. First, an initial visual image of the machining object and a preset machining reference image are acquired. First and second visual feature sets are extracted and initial feature associations are established. Then, hierarchical feature matching processing is performed to obtain feature matching mapping relationships, and initial machining pose parameters are calculated. During machining, dynamic visual images are continuously acquired, a third visual feature set is extracted, and dynamically matched with the second visual feature set to obtain real-time feature matching mapping relationships. Real-time machining pose parameters are then calculated, compared with the initial parameters, and adjustment commands are generated and sent to the control system. This achieves adaptive positioning during machining, improving machining accuracy and stability.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to an adaptive processing and localization method and system based on visual feature matching. Background Technology

[0002] In the field of precision manufacturing, the accuracy of machining and positioning is crucial for the processing and assembly of components with complex shapes and high precision requirements. Taking the manufacturing of components with minute structures and fine textures in some high-end electronic products as an example, the processing needs to be completed with extremely high precision. Any slight positioning deviation may cause the component to fail to assemble properly or its performance to degrade, thereby affecting the quality and reliability of the entire product.

[0003] Traditional machining positioning methods mainly rely on manual measurement and fixed fixture positioning. Manual measurement is not only inefficient but also easily affected by the accuracy of measuring tools and the skill level of operators, leading to significant errors in the measurement results. While fixed fixture positioning improves the stability of positioning to some extent, it lacks flexibility and cannot adapt to the machining needs of parts with different shapes and sizes. Furthermore, for some parts with complex curved surfaces, fixed fixtures struggle to achieve precise positioning.

[0004] Some existing technologies have introduced visual inspection technology for processing positioning, but these methods usually only acquire visual images of parts and compare them as a whole with preset template images. They do not fully consider the hierarchy and correlation of visual features, which makes it easy to make matching errors in complex environments (such as changes in lighting, occlusion, etc.), and cannot guarantee the accuracy and stability of processing positioning, making it difficult to meet the needs of high-end precision manufacturing. Summary of the Invention

[0005] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide an adaptive processing localization method based on visual feature matching, the method comprising: Acquire the initial visual image of the object to be processed and the preset processing reference image, extract the first visual feature set from the initial visual image, extract the second visual feature set from the processing reference image, and establish the initial feature association relationship between the first visual feature set and the second visual feature set; Based on the initial feature association relationship, hierarchical feature matching processing is performed on the first visual feature set and the second visual feature set to obtain the feature matching mapping relationship between the initial visual image of the processing object and the processing reference image; Based on the feature matching mapping relationship, calculate the initial machining pose parameters of the machining object in the coordinate system of the machining equipment; During the processing, dynamic visual images of the processing area are continuously acquired, a third visual feature set is extracted from the dynamic visual images, and the third visual feature set is dynamically matched with the second visual feature set to obtain a real-time feature matching mapping relationship. The real-time machining pose parameters of the machining object in the coordinate system of the machining equipment are calculated based on the real-time feature matching mapping relationship. The real-time machining pose parameters are compared with the initial machining pose parameters to generate a machining pose adjustment command, which is then sent to the control system of the machining equipment.

[0006] In another aspect, embodiments of the present invention also provide an adaptive processing and positioning system based on visual feature matching, including a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0007] Based on the above aspects, the embodiments of the present invention acquire an initial visual image of the object to be processed and a preset processing reference image, and extract the first and second visual feature sets respectively to establish an initial feature association relationship. This fully considers the inherent connection between visual features in different images. Based on the initial feature association relationship, hierarchical feature matching processing is performed, which can deeply explore the hierarchical structure of visual features and effectively improve the accuracy and robustness of feature matching. Even in complex environments, the feature matching mapping relationship between the initial visual image of the object to be processed and the processing reference image can be accurately obtained. The initial processing pose parameters are calculated according to the feature matching mapping relationship, providing accurate initial positioning information for the processing equipment. During the processing, dynamic visual images of the processing area are continuously acquired and a third visual feature set is extracted. Dynamic matching processing is performed with the second visual feature set, which can capture changes in the pose of the object to be processed in real time and obtain a real-time feature matching mapping relationship. By calculating the real-time processing pose parameters and comparing them with the initial processing pose parameters, a processing pose adjustment command is generated and sent to the control system of the processing equipment. This realizes the adaptive adjustment of the processing process, ensuring that the object to be processed always maintains an accurate pose throughout the processing process, greatly improving the accuracy and stability of processing, reducing the scrap rate, and improving the quality and efficiency of high-end precision manufacturing. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the execution flow of the adaptive processing and positioning method based on visual feature matching provided in an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram of exemplary hardware and software components of the adaptive processing positioning system based on visual feature matching provided in an embodiment of the present invention. Detailed Implementation

[0010] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating an adaptive processing and localization method based on visual feature matching provided in one embodiment of the present invention. The adaptive processing and localization method based on visual feature matching will be described in detail below.

[0011] Step S110: Obtain the initial visual image of the object to be processed and the preset processing reference image, extract the first visual feature set from the initial visual image, extract the second visual feature set from the processing reference image, and establish the initial feature association relationship between the first visual feature set and the second visual feature set.

[0012] In this embodiment, the object to be processed can be a housing component of an electronic device. When performing processing and positioning operations on this housing component, it is necessary to first acquire its initial visual image and a preset processing reference image. The initial visual image is acquired through a vision acquisition device installed in the working area of ​​the processing equipment. This vision acquisition device uses an industrial camera, and the lens focal length and shooting angle have been adjusted to clearly present the surface structure and edge contours of the housing component. The processing reference image is stored in the database of the processing control system. This processing reference image is generated based on the design parameters of the housing component and contains all visual feature information of the housing component in its standard state.

[0013] After extracting the first visual feature set from the initial visual image and the second visual feature set from the processing reference image, the initial feature association relationship is established by analyzing the key feature points in the two feature sets. This association relationship can initially reflect the correspondence of feature points in the two feature sets.

[0014] Step S111: Acquire the initial visual image of the object to be processed through the vision acquisition device, and retrieve the preset processing reference image stored in the processing control system.

[0015] Before acquiring the initial visual image, the vision acquisition device can set the shooting parameters. This includes adjusting the exposure parameters to avoid loss of image details due to reflections on the surface of the housing component; and setting the imaging resolution to ensure that the device can capture the subtle textures and pore structures on the surface of the housing component. During the acquisition process, the vision acquisition device is fixed on the support of the processing equipment, and the shooting direction is perpendicular to the plane where the housing component is placed to obtain a front view image.

[0016] The acquired initial visual image is transmitted to the image processing module in digital format. Simultaneously, the machining control system retrieves the corresponding machining reference image from the database based on the current machining task's identifier. This reference image maintains the same image format and resolution as the initial visual image to ensure smooth subsequent feature extraction and matching processing.

[0017] Step S112: Perform image enhancement processing on the initial visual image to obtain an enhanced initial image, and perform feature extraction processing on the enhanced initial image to identify the microstructure region and contour edge region in the enhanced initial image. Extract the texture distribution features, geometric shape features and spatial arrangement features in the microstructure region as microstructure features, and extract the edge line segment features, corner features and curvature change features in the contour edge region as contour edge features. Combine the microstructure features and contour edge features to form the first visual feature set.

[0018] When performing image enhancement processing on the initial visual image, denoising is performed first. A Gaussian filtering algorithm is used to smooth the image, filtering out high-frequency noise and making the surface texture of the outer shell components clearer. Next, contrast enhancement is performed by adjusting the grayscale range of the image to increase the grayscale difference between the edges of the outer shell components and the background, which facilitates subsequent region recognition.

[0019] After the initial image is enhanced, the feature extraction stage begins. An edge detection algorithm scans the enhanced initial image to identify microstructure regions and contour edge regions. For electronic device housing components, microstructure regions include surface heat dissipation hole arrays, fine textures in interface grooves, etc.; contour edge regions include the outer contour lines of the housing, the edge lines of internal mounting slots, etc.

[0020] In the microstructure region, when extracting texture distribution features, multiple sets of feature data reflecting texture direction and density are generated by analyzing the grayscale variation patterns of pixels within the region. When extracting geometric shape features, shape analysis is performed on structures such as heat dissipation holes and interface grooves to obtain feature data describing their contour morphology. When extracting spatial arrangement features, the relative positional relationships between multiple microstructures are calculated to form feature data characterizing their arrangement pattern.

[0021] In the contour edge region, when extracting edge line segment features, continuous edge pixels are fitted into line segments to obtain feature data such as the length and direction of the line segments. When extracting corner features, angle change points on the edge line are identified to obtain feature data such as the angle value and position of the corner. When extracting curvature change features, the curvature variation of the edge line is analyzed to generate feature data reflecting the curvature distribution.

[0022] The extracted microstructural features and contour edge features are integrated to form a first visual feature set. Each feature in this first visual feature set exists in the form of multi-dimensional data and contains key visual information of the shell component in the initial visual image.

[0023] Step S113: Perform image enhancement processing on the processing reference image to obtain an enhanced reference image, and perform feature extraction processing on the enhanced reference image to identify the microstructure region and contour edge region in the enhanced reference image. Extract the texture distribution features, geometric shape features and spatial arrangement features in the microstructure region as microstructure features, and extract the edge line segment features, corner features and curvature change features in the contour edge region as contour edge features. Combine the microstructure features and contour edge features to form a second visual feature set.

[0024] The image enhancement processing of the reference image adopts the same methods and parameters as the initial visual image. First, denoising is performed to remove noise generated during image storage and transmission; then, contrast enhancement is performed to make the features in the reference image more prominent.

[0025] After the enhanced baseline image is generated, it is processed according to the same feature extraction procedure as the enhanced initial image. After identifying the microstructure region and the contour edge region, the corresponding microstructure features and contour edge features are extracted respectively. The extraction criteria of the above features are consistent with the first visual feature set to ensure that the two feature sets are comparable.

[0026] The extracted features are integrated to form a second visual feature set, which contains key visual feature data of the shell components in a standard state, and serves as a reference standard for subsequent feature matching.

[0027] Step S114: Extract the corner features from the first visual feature set as the first key feature point, extract the corner features from the second visual feature set as the second key feature point, calculate the feature similarity between the first key feature point and the second key feature point, and establish an initial feature association relationship between the first visual feature set and the second visual feature set based on the feature similarity. The initial feature association relationship is used to characterize the correspondence between the first key feature point and the second key feature point.

[0028] Corner features are selected from the first set of visual features as the first key feature points. These corner features are typically located at the corners of outer casing components, the corners of mounting slots, etc., and have high recognizability. Similarly, corner features are selected from the second set of visual features as the second key feature points.

[0029] When calculating the feature similarity between the first and second key feature points, feature parameters such as their angle values ​​and the distribution of surrounding edge segments are compared. The feature similarity is obtained by combining the matching degree of multiple parameters. When the feature similarity reaches a set threshold, the two key feature points are considered to have a corresponding relationship.

[0030] Based on these correspondences, initial feature associations are established. These initial feature associations are stored in list form, recording the corresponding sequence number and similarity value between the first and second key feature points.

[0031] Step S120: Perform hierarchical feature matching processing on the first visual feature set and the second visual feature set based on the initial feature association relationship to obtain the feature matching mapping relationship between the initial visual image of the processing object and the processing reference image.

[0032] Based on the initial feature association, hierarchical feature matching processing is performed on the first and second visual feature sets. First, coarse matching of contour edge features is performed, followed by fine matching of microstructure features. Then, the two types of matching results are verified and filtered. Finally, a feature matching pair set is obtained, and a feature matching mapping relationship is generated based on this set. This process can gradually improve the accuracy of feature matching and ensure that the obtained feature matching mapping relationship can accurately reflect the correspondence between the initial visual image and the processing reference image.

[0033] Step S121: Based on the correspondence between the first key feature point and the second key feature point in the initial feature association relationship, perform coarse matching processing on the contour edge features in the first visual feature set and the contour edge features in the second visual feature set. Perform direction consistency verification on the edge line segment features in the contour edge features to determine whether the extension direction of the edge line segment matches. Perform angle matching verification on the corner features to determine whether the included angle of the corner is within the preset range. Perform trend matching verification on the curvature change features to determine whether the curvature change trend is consistent. Combine the verified contour edge features to form the contour matching result.

[0034] Taking the correspondence between the first key feature point and the second key feature point in the initial feature association relationship as a reference, the contour edge features associated with the first key feature point in the first visual feature set are compared with the contour edge features associated with the corresponding second key feature point in the second visual feature set to carry out coarse matching processing.

[0035] During the coarse matching process, edge segment features, corner features, and curvature change features are validated separately. After passing each validation, the contour edge features that meet the conditions are combined to form the contour matching result. This contour matching result initially reflects the correspondence of contour edge features in the two types of visual feature sets.

[0036] Step S1211: Extract edge line segment features from the contour edge features of the first visual feature set, determine the start coordinates and end coordinates of each edge line segment, and calculate the direction vector of the edge line segment based on the start coordinates and end coordinates.

[0037] Edge segment features are selected from the contour edge features of the first visual feature set. Each edge segment feature includes its start and end coordinates in the image. The direction vector of the edge segment is obtained by calculating the difference between the start and end coordinates. This direction vector is represented in the form of horizontal and vertical components, which can reflect the extension direction of the edge segment.

[0038] For example, for a certain edge line segment of the outer contour of a shell component, after the starting coordinates and ending coordinates are determined, the coordinate difference between the two in the horizontal and vertical directions is calculated to form the direction vector of the edge line segment.

[0039] Step S1212: Extract edge line segment features from the contour edge features of the second visual feature set, determine the start coordinates and end coordinates of each edge line segment, and calculate the direction vector of the edge line segment based on the start coordinates and end coordinates.

[0040] Using the same method as in step S1211, edge segment features are extracted from the contour edge features of the second visual feature set. The start and end coordinates of each edge segment are determined, and its direction vector is calculated. Ensure that the calculation process and parameter definitions are consistent with step S1211 for subsequent direction vector comparison.

[0041] Step S1213: For the edge line segment features around the first and second key feature points corresponding in the initial feature association relationship, calculate the angle between the edge line segment direction vector in the first visual feature set and the edge line segment direction vector in the second visual feature set.

[0042] In the initial feature association relationships, the corresponding first and second key feature points are identified. Using these key feature points as the center, a certain area is delineated, and the edge line segment features within this area are extracted.

[0043] The angle between the direction vectors of the edge segments extracted from the first visual feature set and the corresponding direction vectors of the edge segments in the second visual feature set is calculated. During the calculation, the angle between the direction vectors is derived using the vector dot product formula; this angle reflects the degree of difference between the directions of the two edge segments.

[0044] Step S1214: Determine whether the calculated included angle is less than the preset direction threshold. If it is less than the preset direction threshold, determine that the extension direction of the edge line segment matches. If it is greater than or equal to the preset direction threshold, determine that the extension direction of the edge line segment does not match.

[0045] The calculated angle is compared with a preset direction threshold. When the angle is less than the direction threshold, it indicates that the extension directions of the two edge segments are similar, and the direction is determined to be matched. When the angle is greater than or equal to the direction threshold, it indicates that the extension directions of the two edge segments are significantly different, and the direction is determined to be mismatched.

[0046] Step S1215: Perform directional consistency verification on all corresponding edge line segment features, count the number of edge line segment features that pass the verification, and determine that the directional consistency verification of the contour edge features has passed when the proportion of the number of verified features to the total number exceeds the preset proportion threshold.

[0047] Perform orientation consistency verification on each corresponding edge segment feature in the initial feature association relationship, and record the number of verifications that pass. Calculate the ratio of the number of verifications that pass to the total number of verifications. When this ratio exceeds a preset threshold, the orientation consistency verification of the contour edge feature is considered successful.

[0048] Step S1216: Perform angle matching verification on the corner features to determine whether the included angle of the corner is within the preset range.

[0049] Extract the angle values ​​of the corresponding corner features from the first visual feature set and the second visual feature set, and calculate the difference between the two. Determine whether the difference is within the preset angle deviation range. If it is within the angle deviation range, the angle matching verification of the corner feature is deemed successful; if it exceeds the angle deviation range, it is deemed unsuccessful.

[0050] Step S1217: Perform trend matching verification on the curvature change characteristics to determine whether the curvature change trends are consistent.

[0051] Extract the corresponding curvature change feature data from the first and second visual feature sets, and analyze their changing trends. By comparing the similarity of the order and magnitude of the increase and decrease in curvature values, determine whether the curvature change trends are consistent. If the changing trends of the two are basically consistent, the trend matching verification of the curvature change features is considered successful; otherwise, it is considered unsuccessful.

[0052] Step S122: Using the contour matching result as a constraint, perform fine matching processing on the microstructure features in the first visual feature set and the microstructure features in the second visual feature set. Based on the contour matching result, determine the matching region of the microstructure features. Perform region similarity comparison on the texture distribution features in the microstructure features to determine whether the texture distribution patterns are consistent. Perform shape similarity calculation on the geometric shape features to determine whether the contours of the geometric shapes match. Perform positional relationship verification on the spatial arrangement features to determine whether the relative positions between microstructures conform to the preset rules. Combine the verified microstructure features to form the microstructure matching result.

[0053] The contour matching results are used as constraints to limit the matching range of microstructural features. Within this range, fine-tuning is performed on the microstructural features in the first and second visual feature sets. Texture distribution features, geometric shape features, and spatial arrangement features are validated separately. The validated microstructural features are combined to form the microstructural matching result. This microstructural matching result can more accurately reflect the correspondence of microstructural features in the two sets of visual features.

[0054] Step S1221: Based on the contour matching result, determine the image region where the microstructure features are located, delineate the image region as the first texture region in the first visual feature set, and delineate the corresponding region as the second texture region in the second visual feature set.

[0055] Based on the positional information of the contour edge features in the contour matching result, the image region containing the microstructural features is determined. In the first visual feature set, this image region is designated as the first texture region; in the second visual feature set, the second texture region is designated according to the corresponding contour position. It is ensured that the relative positions of the two texture regions in the image are consistent with the contour matching result.

[0056] Step S1222: Divide the first texture region into texture units, divide the first texture region into multiple texture units of the same size, and extract the texture direction, texture density and texture periodicity features of each texture unit.

[0057] The first texture region is divided into grids of fixed size to obtain multiple texture units of the same size. For each texture unit, the texture direction is determined by analyzing the grayscale variation direction of the pixels; the texture density is obtained by counting the number of texture elements per unit area; and the periodicity feature of the texture is extracted by analyzing the recurrence pattern of the texture pattern. These features exist in the form of multi-dimensional data and together constitute the feature description of the texture unit.

[0058] Step S1223: Divide the second texture region into texture units, divide the second texture region into multiple texture units of the same size, and extract the texture direction, texture density and texture periodicity features of each texture unit.

[0059] The second texture region is processed using the same segmentation method and feature extraction criteria as in step S1222. It is divided into texture units of the same number and size as the first texture region, and the texture direction, texture density, and texture periodicity features of each texture unit are extracted to ensure that the extracted features are comparable to those of the first texture region.

[0060] Step S1224: Perform feature comparison between each texture unit in the first texture region and the corresponding texture unit in the second texture region, and calculate the similarity of texture direction, texture density, and texture periodicity.

[0061] For the texture units corresponding to the positions in the first and second texture regions, their texture direction, texture density, and texture periodicity features are compared. Three similarity indices are obtained by calculating the similarity between the feature data. For example, for texture direction, the difference in the orientation angles between the two is calculated; the smaller the difference, the higher the similarity. For texture density, the relative deviation of the density values ​​between the two is calculated; the smaller the deviation, the higher the similarity. For texture periodicity, the similarity between the period length and the pattern within the period is compared; the higher the similarity, the higher the similarity.

[0062] Step S1225: The three similarity indicators are comprehensively evaluated. If the comprehensive evaluation result exceeds the preset texture similarity threshold, the distribution pattern of the texture unit is determined to be matched; otherwise, it is determined to be mismatched.

[0063] The similarity indices of texture direction, texture density, and texture periodicity are weighted and combined to obtain a comprehensive evaluation result. This result is compared with a preset texture similarity threshold. If the similarity exceeds the threshold, the distribution pattern of the texture unit is determined to match; otherwise, it is determined not to match.

[0064] Step S1226: Statistically analyze the matching results of all texture units. When the proportion of the number of matched texture units to the total number of texture units exceeds the preset region matching threshold, the region similarity comparison of the texture distribution features is determined to be successful.

[0065] The number of matching texture units in the first and second texture regions is counted, and their proportion to the total number of texture units is calculated. When this proportion exceeds a preset region matching threshold, the region similarity comparison of the texture distribution features is considered successful.

[0066] Step S1227: Perform shape similarity calculation on the geometric shape features to determine whether the outlines of the geometric shapes match.

[0067] Geometric shape feature data of corresponding microstructures are extracted from the first and second visual feature sets. This data includes the coordinate information of multiple points on the contour. The shape similarity is obtained by calculating the degree of overlap between the contours formed by the two sets of coordinate information. When the shape similarity exceeds a preset shape threshold, the geometric contours are determined to match; otherwise, they are determined to not match.

[0068] Step S1228: Verify the positional relationship of the spatial arrangement features to determine whether the relative positions between microstructures conform to the preset rules.

[0069] Extract the spatial coordinates of multiple microstructures from the first and second visual feature sets, and calculate the distance and angular relationships between the microstructures. Compare these relationships with preset rules, such as the distance ratio between adjacent microstructures and their arrangement angles. If the deviation between the actual relationships and the preset rules is within the allowable range, the positional relationship verification of the spatial arrangement features is considered successful; otherwise, it is considered unsuccessful.

[0070] Step S123: Perform feature association verification on the contour matching results and microstructure matching results, and check whether the spatial positional relationship between the contour edge features and the microstructure features is consistent, so that the matched contour edge features can accurately define the distribution area of ​​the microstructure features.

[0071] The contour matching results and microstructure matching results are correlated to analyze the spatial relationship between contour edge features and microstructure features. It is checked whether the microstructure features are located within the area defined by the corresponding contour edge features, and whether their relative positions conform to the structural design of the shell component. Through the above verification, it is ensured that the contour edge features can accurately define the distribution area of ​​the microstructure features.

[0072] Step S124: Based on the correlation verification results, filter the contour matching results and microstructure matching results, remove matching features with inconsistent spatial relationships, and retain the matching features that have passed the verification.

[0073] Based on the feature association verification results, features in the contour matching and microstructure matching results are filtered. Matching features with inconsistent spatial relationships are removed from the results, retaining only those that passed verification. After this filtering, the accuracy of the matching results is further improved.

[0074] Step S125: Integrate the retained contour matching results and microstructure matching results to generate a set of feature matching pairs.

[0075] The feature correspondences in the filtered contour matching results and microstructure matching results are integrated to form a set of feature matching pairs. Each feature matching pair contains one feature from the first visual feature set and one corresponding feature from the second visual feature set. These feature matching pairs cover the correspondence between contour edge features and microstructure features of the shell component. For example, a line segment feature on the shell edge and a corresponding line segment feature in the processing reference image form one matching pair; a feature of a texture area on the shell surface and a corresponding texture area feature in the processing reference image form another matching pair. During the integration process, it is necessary to ensure that the feature types of each matching pair are consistent, that is, contour edge features correspond to contour edge features, and microstructure features correspond to microstructure features, to avoid incorrect matching between different types of features. Simultaneously, the integrated set of feature matching pairs is numbered for subsequent retrieval and processing.

[0076] Step S126: Construct a homography transformation model between the initial visual image and the processing reference image based on the feature matching pair set. Generate a feature matching mapping relationship through the homography transformation model. The feature matching mapping relationship is used to characterize the spatial coordinate transformation relationship between each pixel in the initial visual image and the corresponding pixel in the processing reference image.

[0077] By utilizing the coordinate information of feature points in the feature matching pair set, a homography transformation model is constructed between the initial visual image and the processing reference image. The homography transformation model can describe the projection transformation relationship between two planes. For the planar image of the outer shell component, this homography transformation model can accurately reflect the coordinate transformation law between the initial visual image and the processing reference image.

[0078] When constructing the homography transformation model, a sufficient number of feature matching pairs are selected from the feature matching pair set. The coordinates of the feature points in the initial visual image and the coordinates of the corresponding feature points in the processing reference image are extracted from each matching pair. The homography matrix is ​​then solved using this coordinate information. This homography matrix contains the parameters for achieving coordinate transformation between the two images. During the solution process, it is necessary to ensure that the selected feature matching pairs are evenly distributed and cover all key areas of the outer shell component to improve the accuracy of the homography matrix.

[0079] Once the homography matrix is ​​obtained, the homography transformation model can be determined. Using this model, the coordinates of any pixel in the initial visual image and its corresponding pixel in the processing reference image can be calculated, thus generating a feature matching mapping relationship. For example, a pixel at a corner of the shell in the initial visual image can accurately find its corresponding pixel in the processing reference image through the feature matching mapping relationship.

[0080] Step S130: Calculate the initial machining pose parameters of the machining object in the coordinate system of the machining equipment according to the feature matching mapping relationship.

[0081] After obtaining the feature matching mapping relationship, the initial machining pose parameters of the electronic device housing component in the machining equipment coordinate system can be calculated based on this. The initial machining pose parameters reflect the position and orientation of the housing component relative to the machining equipment at the start of machining and are an important reference for subsequent machining operations.

[0082] Step S131: Extract the homography matrix between the initial visual image and the processing reference image from the feature matching mapping relationship. The homography matrix is ​​used to characterize the projection transformation relationship between the two planes.

[0083] The feature matching mapping relationship includes the homography matrix between the initial visual image and the processing reference image, which is obtained through the previous steps. When extracting this homography matrix, its completeness and validity must be verified to ensure that the parameters in the matrix accurately describe the projection transformation between the two images. The element values ​​of the homography matrix reflect the transformation information between the images in terms of translation, rotation, scaling, etc., and are key data for calculating pose parameters.

[0084] Step S132: Decompose the homography matrix using camera calibration parameters to obtain a rotation matrix and a translation vector. The camera calibration parameters include the camera intrinsic parameter matrix and distortion coefficients.

[0085] Camera calibration parameters are acquired in advance during the installation and commissioning phase of the vision acquisition device, including the camera intrinsic parameter matrix and distortion coefficients. The camera intrinsic parameter matrix is ​​related to parameters such as the camera's focal length and principal point, while the distortion coefficients are used to correct image distortion errors.

[0086] The homography matrix is ​​decomposed using camera calibration parameters. During decomposition, the homography matrix is ​​transformed into a matrix form reflecting 3D spatial transformations by incorporating the camera intrinsic parameter matrix, thereby separating the rotation matrix and translation vector. The rotation matrix describes the rotational attitude of the outer shell component relative to the camera coordinate system, while the translation vector describes the positional offset of the outer shell component in the camera coordinate system. The decomposition process must adhere to strict mathematical transformation rules to ensure the accuracy of the obtained rotation matrix and translation vector.

[0087] Step S133: Convert the rotation matrix into Euler angle representation to obtain the rotation angle parameters of the processed object. The rotation angle parameters include the rotation angle around the X-axis, the rotation angle around the Y-axis, and the rotation angle around the Z-axis.

[0088] While rotation matrices can accurately describe the rotational attitude of shell components, Euler angles are more intuitive and easier to understand for actual machining operations. Converting a rotation matrix to Euler angles requires calculations following a specific rotational order (e.g., first around the X-axis, then the Y-axis, and finally the Z-axis).

[0089] During the transformation process, a series of trigonometric function operations are used to solve for the rotation angles around the X, Y, and Z axes from the elements of the rotation matrix. These rotation angle parameters can clearly reflect the tilt and rotation state of the outer shell component in space. For example, the rotation angle around the X axis reflects the degree of tilt of the outer shell component in the front-back direction, the rotation angle around the Y axis reflects the degree of tilt in the left-right direction, and the rotation angle around the Z axis reflects the degree of torsion of the outer shell component.

[0090] Step S134: Convert the translation vector into coordinate values ​​in the coordinate system of the processing equipment to obtain the translation position parameters of the processing object. The translation position parameters include the translation distance in the X-axis direction, the translation distance in the Y-axis direction, and the translation distance in the Z-axis direction.

[0091] The translation vector is obtained in the camera coordinate system, while the machining operation is performed in the machining equipment coordinate system. Therefore, it is necessary to convert the translation vector into coordinate values ​​in the machining equipment coordinate system. During the conversion process, the transformation relationship between the camera coordinate system and the machining equipment coordinate system must be clearly defined, and this transformation relationship is determined through prior coordinate calibration.

[0092] Based on the transformation matrix between the two coordinate systems, the translation vector is transformed to obtain the translation distance of the outer shell component in the X-axis, Y-axis and Z-axis directions in the coordinate system of the processing equipment. The above translation distance parameters reflect the specific position of the outer shell component on the worktable of the processing equipment and are an important basis for determining the processing start point and path.

[0093] Step S135: Combine the rotation angle parameters and translation position parameters to form the initial machining pose parameters of the machining object in the coordinate system of the machining equipment. The initial machining pose parameters are used to determine the position and orientation of the machining object at the start of machining.

[0094] The rotation angle parameters (rotation angles around the X, Y, and Z axes) and translation position parameters (translation distances in the X, Y, and Z directions) obtained earlier are combined to form the initial machining pose parameters of the shell component.

[0095] The initial machining pose parameters are a comprehensive set of parameters that fully describe the spatial state of the housing component relative to the machining equipment at the start of machining. The machining equipment's control system adjusts the initial position and orientation of the machining tool based on these parameters to ensure that the machining operation is performed accurately according to the preset program. For example, based on the initial machining pose parameters, the machining equipment's spindle will adjust to a suitable height and angle, aligning with the area of ​​the housing component to be machined.

[0096] Step S140: During the processing, continuously acquire dynamic visual images of the processing area, extract a third visual feature set from the dynamic visual images, and perform dynamic matching processing between the third visual feature set and the second visual feature set to obtain a real-time feature matching mapping relationship.

[0097] During the manufacturing process of electronic device housing components, the position and orientation of the components may undergo subtle changes due to factors such as processing forces and equipment vibrations. To monitor these changes in real time and make adjustments, it is necessary to continuously acquire dynamic visual images of the processing area and perform corresponding feature extraction and matching processing.

[0098] Step S141: During the processing operation of the processing equipment, dynamic visual images of the processing area are continuously acquired by the visual acquisition device at preset time intervals. The dynamic visual images reflect the real-time state of the surface of the processing object during the processing.

[0099] The visual acquisition device captures images of the processing area at preset time intervals to obtain dynamic visual images. The time interval needs to be determined based on the dynamic characteristics of the processing process, ensuring that the position and orientation changes of the outer casing components can be captured in a timely manner, while avoiding increasing the data processing burden due to excessive acquisition frequency.

[0100] Dynamic visual images can clearly reflect the real-time state of the surface of a housing component during processing, including machining marks, surface deformation, and positional displacement. For example, during the drilling process of a housing component, dynamic visual images can show whether the drilling position is accurate and whether the housing component experiences slight displacement due to force. The acquired dynamic visual images are transmitted in real time to the image processing unit for further processing.

[0101] Step S142: Preprocess the dynamic visual image to remove background interference outside the processing area in the dynamic visual image, retain the effective area of ​​the surface of the processing object, and obtain the preprocessed dynamic visual image.

[0102] Dynamic visual images may contain background elements such as the worktable, fixtures, and cutting tools of processing equipment, which can affect the accuracy of subsequent feature extraction and matching. Therefore, it is necessary to preprocess dynamic visual images to remove background interference.

[0103] The preprocessing process includes image segmentation and region extraction. Image segmentation algorithms are used to distinguish regions belonging to the outer shell from the background in a dynamic visual image. Differences in color, grayscale, and texture between the outer shell and the background can be used as segmentation criteria. For example, if the outer shell surface is made of metal and has specific reflective properties, it can be distinguished from the non-metallic background.

[0104] After removing background interference, the effective area of ​​the shell component surface is retained to obtain a preprocessed dynamic visual image. The effective area contains real-time feature information of the shell component during the manufacturing process.

[0105] Step S143: Perform feature extraction processing on the preprocessed dynamic visual image, identify the microstructure region and contour edge region, extract the texture distribution features, geometric shape features and spatial arrangement features in the microstructure region as microstructure features, extract the edge line segment features, corner features and curvature change features in the contour edge region as contour edge features, and combine the microstructure features and contour edge features to form a third visual feature set.

[0106] The method for feature extraction from the preprocessed dynamic visual image is similar to that for the initial visual image, but it needs to consider the changes in the surface features of the shell component during processing. For example, during processing, some microstructures of the shell component may be damaged, and the contour edges may also be altered due to cutting.

[0107] After identifying the microstructure regions and contour edge regions, corresponding features are extracted. For microstructure features, attention should be paid to the newly formed texture, geometry, and spatial arrangement after processing, such as the texture of the hole wall after drilling, the shape of the hole, and its positional distribution. For contour edge features, real-time information such as the length of edge segments, the angle of corners, and changes in curvature during processing should be captured. The extracted microstructure features and contour edge features are combined to form a third visual feature set.

[0108] Step S144: Perform dynamic matching processing between the third visual feature set and the second visual feature set, and adjust the matching strategy based on real-time state changes during the processing.

[0109] The third visual feature set reflects the real-time features of the shell component during processing, while the second visual feature set represents the features of the processing reference image. Dynamic matching of these two sets is then performed to determine the real-time position and orientation changes of the shell component.

[0110] During dynamic matching, the matching strategy needs to be adjusted according to the real-time changes in the processing status. For example, when new features (such as drilling or grooving) appear in the processing area, the matching strategy should focus on matching these new features with the corresponding features in the processing reference image; when the outer shell component undergoes significant deformation, the similarity threshold for matching should be appropriately relaxed to avoid matching failure due to excessive feature changes.

[0111] Step S145: During the dynamic matching process, the possible location of features at the current time is predicted based on the matching result of the previous time step to narrow down the matching search area, and the contour edge features in the third visual feature set are quickly matched with the contour edge features in the second visual feature set to determine the correspondence of the reference contour.

[0112] To improve the efficiency of dynamic matching, the possible location of features at the current moment is predicted based on the matching result of the previous moment, thereby narrowing the matching search area. The matching result of the previous moment contains the location information of the feature points, and combined with the motion state of the processing equipment, the movement range of the feature points at the current moment can be roughly predicted.

[0113] Step S1451: Obtain the real-time feature matching pair set obtained from the previous dynamic matching process, extract the coordinate information of each feature pair, and obtain the feature point coordinates in the third visual feature set and the corresponding feature point coordinates in the second visual feature set at the previous moment.

[0114] Retrieve the previous real-time feature matching pair set from the storage unit. This set contains the correspondence between features in the third visual feature set and the second visual feature set. Extract the coordinates of the feature points in the third visual feature set and the coordinates of the corresponding feature points in the second visual feature set from each feature pair. This coordinate information is the basic data for predicting the feature position at the current moment.

[0115] For example, the real-time feature matching set of the previous moment contains the correspondence of a corner feature of the shell. The coordinates of the corner in the third visual feature set and the coordinates in the second visual feature set are extracted for subsequent position prediction.

[0116] Step S1452: Analyze the time interval from the previous moment to the current moment, and combine it with the motion parameters of the processing equipment to predict the displacement and rotation angle of the processing object during the processing process. The motion parameters include the feed speed and rotation speed of the processing equipment.

[0117] The time interval from the previous moment to the current moment is calculated; this time interval is a preset image acquisition time interval. Simultaneously, motion parameters of the processing equipment, such as feed rate and rotational speed, are obtained from the control system of the processing equipment.

[0118] Based on time intervals and motion parameters, kinematic calculations are used to predict the displacement and rotation angle of the housing component during that time period. For example, if the feed rate of the processing equipment remains constant in a certain direction, the displacement of the housing component in that direction can be roughly calculated by combining the time interval; the rotation angle can be calculated based on the rotational speed and the time interval.

[0119] Step S1453: Based on the predicted displacement and rotation angle, calculate the possible position coordinates of each feature point in the third visual feature set at the previous moment in the dynamic visual image at the current moment, and obtain the predicted position of the feature point.

[0120] Based on the predicted displacement and rotation angle, the coordinates of each feature point in the third visual feature set at the previous moment are transformed to obtain the possible position coordinates of these feature points in the dynamic visual image at the current moment, i.e., the predicted position of the feature points.

[0121] The coordinate transformation process follows the mathematical rules of translation and rotation. The coordinates of the feature point at the previous moment are translated according to the predicted displacement, and then rotated according to the predicted rotation angle to obtain the predicted position. For example, if the coordinates of a feature point at the previous moment are (x1, y1), the predicted displacement is (dx, dy), and the rotation angle is θ, then the predicted position coordinates at the current moment are calculated through the corresponding translation and rotation.

[0122] Step S1454: Using the predicted position as the center, set a search area of ​​a preset size as the matching search area for the feature points at the current moment. The size of the matching search area is determined according to the movement speed of the processing object and the image acquisition frame rate.

[0123] Centered on the predicted location of the feature points, a pre-defined area is designated as the matching search region. The size of the search region is not fixed but adjusted according to the movement speed of the outer shell components and the image acquisition frame rate. If the outer shell components move quickly and the image acquisition frame rate is low, the search region should be appropriately increased to prevent feature points from moving out of the search region; conversely, the search region can be appropriately decreased to improve matching efficiency.

[0124] For example, when the outer casing component moves at a relatively high speed in a certain direction, the search area centered on the predicted position will expand accordingly in that direction to ensure that the actual position of the feature point at the current moment is included.

[0125] Step S1455: For each feature point in the third visual feature set, perform matching processing only with feature points in the second visual feature set within the corresponding matching search area; wherein, when performing feature matching within the matching search area, prioritize matching the feature point closest to the predicted position.

[0126] In the dynamic matching process, for each feature point in the third visual feature set, matching is no longer performed across the entire image; instead, it is only matched against feature points in the second visual feature set within the corresponding matching search region. This method significantly narrows the search range and improves the matching speed.

[0127] Within the matching search area, the similarity between feature points in the third visual feature set and feature points in the second visual feature set is calculated. Feature points that are closest to the predicted location and have high similarity are preferentially selected as matching targets. For example, when there are multiple possible matching feature points in the search area, the feature point closest to the predicted location is considered first. If its similarity meets the requirements, it is determined as a matching point; if it does not meet the requirements, other nearby feature points are checked in turn.

[0128] Using the above method, the correspondence between the contour edge features in the third visual feature set and the contour edge features in the second visual feature set can be quickly determined, i.e., the reference contour correspondence.

[0129] Step S146: Perform matching processing on the microstructure features based on the reference contour correspondence, and integrate the contour matching results and microstructure matching results obtained from the matching processing to generate a real-time feature matching pair set.

[0130] After determining the correspondence with the reference contour, the microstructural features are matched based on this relationship. Matching the contour edge features provides approximate positional constraints for matching the microstructural features, making the matching of microstructural features more targeted.

[0131] The matching method for microstructural features is similar to the previous fine-matching process, including the comparison of texture distribution features, geometric shape features, and spatial arrangement features. The contour matching results and microstructural matching results are integrated, inconsistent matching pairs are removed, and accurate matching relationships are retained to generate a real-time feature matching pair set. This real-time feature matching pair set reflects the correspondence between the features of the shell component and the processing reference features at the current moment.

[0132] Step S147: Construct a real-time homography transformation model between the dynamic visual image and the processing reference image based on the real-time feature matching pair set, and generate a real-time feature matching mapping relationship through the real-time homography transformation model.

[0133] By utilizing the feature point coordinate information in the real-time feature matching pair set, a real-time homography transformation model between the dynamic visual image and the processing reference image is constructed. The construction method is similar to that of the initial homography transformation model, solving for the real-time homography matrix by selecting appropriate feature matching pairs.

[0134] The real-time homography transformation model reflects the projection transformation relationship between the dynamic visual image and the processing reference image at the current moment. This model can generate a real-time feature matching mapping relationship, that is, the spatial coordinate transformation relationship between each pixel in the dynamic visual image and the corresponding pixel in the processing reference image. This real-time feature matching mapping relationship provides a basis for calculating the real-time processing pose parameters of the shell component.

[0135] Step S150: Calculate the real-time machining pose parameters of the machining object in the coordinate system of the machining equipment according to the real-time feature matching mapping relationship, compare the real-time machining pose parameters with the initial machining pose parameters, generate a machining pose adjustment command, and send the machining pose adjustment command to the control system of the machining equipment.

[0136] The real-time machining pose parameters of the shell component are calculated by real-time feature matching mapping relationship, and compared with the initial machining pose parameters. Adjustment instructions are generated based on the differences to ensure machining accuracy.

[0137] Step S151: Calculate the real-time machining pose parameters of the machining object in the coordinate system of the machining equipment based on the real-time feature matching mapping relationship.

[0138] Step S1511: Extract the real-time homography matrix between the dynamic visual image and the processing reference image from the real-time feature matching mapping relationship.

[0139] The real-time feature matching mapping relationship includes a real-time homography matrix between the dynamic visual image and the processing reference image. This real-time homography matrix is ​​obtained by solving the set through real-time feature matching. When extracting this matrix, it is necessary to check whether it can accurately reflect the projection transformation between the two images at the current moment.

[0140] Step S1512: Decompose the real-time homography matrix using camera calibration parameters to obtain the real-time rotation matrix and the real-time translation vector.

[0141] Camera calibration parameters include the camera intrinsic parameter matrix and distortion coefficients. These parameters are determined through a calibration process after the vision acquisition device is installed and are used to describe the camera's optical characteristics and imaging properties. When decomposing the real-time homography matrix, the camera intrinsic parameter matrix is ​​used to preprocess the real-time homography matrix to eliminate the influence of camera optical characteristics on the projection transformation.

[0142] During the decomposition process, the real-time homography matrix is ​​broken down into a real-time rotation matrix and a real-time translation vector through matrix operations. The real-time rotation matrix represents the rotational relationship of the dynamic visual image relative to the processing reference image in three-dimensional space, while the real-time translation vector represents the translational relationship between the two. After decomposition, the real-time rotation matrix needs to be orthogonalized to ensure that it satisfies the inherent properties of a rotation matrix. If the verification fails, the extraction process of the real-time homography matrix is ​​re-examined to eliminate error interference.

[0143] Step S1513: Convert the real-time rotation matrix into Euler angle representation to obtain the real-time rotation angle parameters of the workpiece. The real-time rotation angle parameters include the real-time rotation angle around the X-axis, the real-time rotation angle around the Y-axis, and the real-time rotation angle around the Z-axis.

[0144] The real-time rotation matrix is ​​a three-dimensional matrix. To more intuitively describe the rotation state of the machined object, it needs to be converted into Euler angles. During the conversion process, calculations are performed according to a preset rotation order (e.g., first around the X-axis, then around the Y-axis, and finally around the Z-axis).

[0145] By utilizing the operational relationships between matrix elements, the real-time rotation angles around the X-axis, Y-axis, and Z-axis are calculated separately. These angle parameters clearly reflect the rotation of the electronic device's housing components around each coordinate axis during manufacturing. After the conversion, the consistency between the three rotation angles must be checked to avoid contradictory descriptions of rotation states.

[0146] Step S1514: Convert the real-time translation vector into coordinate values ​​in the coordinate system of the processing equipment to obtain the real-time translation position parameters of the processing object. The real-time translation position parameters include the real-time translation distance in the X-axis direction, the real-time translation distance in the Y-axis direction, and the real-time translation distance in the Z-axis direction.

[0147] The real-time translation vector is obtained in the image coordinate system and needs to be transformed to the machining equipment coordinate system to meet the motion control requirements of the machining equipment. During the transformation process, the transformation relationship between the camera coordinate system and the machining equipment coordinate system is utilized. This transformation relationship is determined through pre-calibration and includes rotation and translation parameters.

[0148] The real-time translation vector is sequentially transformed from the camera coordinate system to the machining equipment coordinate system to obtain the real-time translation distances along the X, Y, and Z axes in the machining equipment coordinate system. These translation distance parameters directly reflect the amount of translation of the electronic device housing component relative to its initial position in each direction during machining, and are key data for position adjustment. After the transformation, the rationality of the transformation result needs to be verified to ensure that it is within the movement range of the machining equipment.

[0149] Step S1515: Combine the real-time rotation angle parameters and the real-time translation position parameters to form the real-time machining pose parameters of the machining object in the coordinate system of the machining equipment. The real-time machining pose parameters are used to reflect the actual position and posture of the machining object at the current moment.

[0150] The obtained real-time rotation angle parameters (real-time rotation angles around the X-axis, Y-axis, and Z-axis) and real-time translation position parameters (real-time translation distances in the X-axis, Y-axis, and Z-axis directions) are combined in a preset order to form real-time machining pose parameters.

[0151] This parameter set comprehensively describes the actual position and orientation of the electronic device's housing component relative to the machining equipment's coordinate system at the current moment, serving as the basis for subsequent comparison with the initial machining pose parameters. During the assembly process, the parameter format must be standardized to ensure that it can be correctly recognized and processed by the machining equipment's control system.

[0152] Step S152: Compare the real-time machining pose parameters with the initial machining pose parameters, generate a machining pose adjustment command, and send the machining pose adjustment command to the control system of the machining equipment.

[0153] After obtaining the real-time machining pose parameters, they need to be compared with the initial machining pose parameters to determine the pose deviation of the machined object. Based on the deviation, corresponding machining pose adjustment commands are generated and sent to the control system of the machining equipment to realize real-time adjustment of the pose of the machined object.

[0154] Step S1521: Compare the real-time rotation angle parameter in the real-time machining pose parameters with the rotation angle parameter in the initial machining pose parameters, and calculate the rotation angle deviation value. The rotation angle deviation value includes the rotation angle deviation around the X-axis, the rotation angle deviation around the Y-axis, and the rotation angle deviation around the Z-axis.

[0155] The angle values ​​around the X-axis, Y-axis, and Z-axis are extracted from both the real-time and initial rotation angle parameters, and the corresponding angle values ​​are compared pairwise. The rotation angle deviation value around each coordinate axis is calculated by the difference between the angle values.

[0156] For example, by subtracting the angle value around the X-axis from the angle value around the X-axis in the real-time rotation angle parameters, the rotation angle deviation value around the X-axis is obtained. Similarly, the rotation angle deviation values ​​around the Y-axis and Z-axis are calculated. The above deviation values ​​intuitively reflect the degree of deviation of the machined object from its initial state in the rotation direction.

[0157] Step S1522: Compare the real-time translation position parameters in the real-time machining pose parameters with the translation position parameters in the initial machining pose parameters, and calculate the translation position deviation value. The translation position deviation value includes the translation deviation in the X-axis direction, the translation deviation in the Y-axis direction, and the translation deviation in the Z-axis direction.

[0158] Similarly, the translation distance values ​​in the X, Y, and Z axes of the real-time and initial translation position parameters are extracted and compared pairwise. The translation position deviation values ​​in each direction are calculated by the difference between the translation distance values.

[0159] That is, the translation distance in the X-axis direction of the real-time translation position parameters is subtracted from the translation distance in the X-axis direction of the initial translation position parameters to obtain the translation deviation value in the X-axis direction. The translation deviation values ​​in the Y-axis and Z-axis directions are obtained by analogy. The above deviation values ​​reflect the deviation of the workpiece from its initial state in terms of position.

[0160] Step S1523: Combine the rotation angle deviation value and the translation position deviation value to form a pose deviation value, which is used to characterize the difference between the current pose of the machining object and the initial set pose.

[0161] The calculated rotation angle deviations around the X-axis, Y-axis, and Z-axis, and the translation position deviations in the X-axis, Y-axis, and Z-axis directions are combined in a certain order to form the pose deviation values.

[0162] This pose deviation value comprehensively reflects the difference between the current pose and the initial set pose of the electronic device's housing components, serving as the basis for determining whether pose adjustment is necessary. When combining these values, the coordinate axes and directions corresponding to each deviation value must be clearly defined to avoid confusion.

[0163] Step S1524: Determine whether the pose deviation value exceeds the preset allowable deviation range. If it does not exceed the allowable deviation range, no machining pose adjustment command is generated. If it exceeds the allowable deviation range, calculate the adjustment amount of the machining equipment based on the pose deviation value. The adjustment amount includes position adjustment amount and posture adjustment amount. The position adjustment amount is determined based on the translation position deviation value, and the posture adjustment amount is determined based on the rotation angle deviation value.

[0164] The preset allowable deviation range is determined based on the machining accuracy requirements of the electronic device housing components. This preset allowable deviation range includes the maximum permissible values ​​for each rotation angle deviation and translation position deviation. Each deviation in the pose deviation value is compared with the allowable deviation range.

[0165] If all deviation values ​​are within the allowable deviation range, it indicates that the pose of the workpiece meets the machining requirements, requiring no adjustment and no machining pose adjustment command is generated. If any deviation value exceeds the allowable deviation range, the adjustment amount of the machining equipment needs to be calculated. The position adjustment amount is determined based on the translational position deviation value, and is usually equal in magnitude but opposite in direction to the translational position deviation value. The attitude adjustment amount is determined based on the rotational angle deviation value, and is also equal in magnitude but opposite in direction to the rotational angle deviation value, to counteract the effects of the deviation.

[0166] Step S1525: Generate a machining pose adjustment command based on the adjustment amount. The machining pose adjustment command includes motion control parameters for each axis of the machining equipment.

[0167] Based on the calculated position and attitude adjustments, a machining pose adjustment command is generated. This machining pose adjustment command needs to include motion control parameters for each axis of the machining equipment (such as the X-axis, Y-axis, Z-axis, and rotary axes). These motion control parameters include motion direction, motion speed, motion distance, or rotation angle.

[0168] The direction of motion is determined by the sign of the adjustment amount, the speed of motion is set according to the processing requirements and the magnitude of the adjustment amount, and the distance of motion or rotation angle is determined by the position adjustment amount and the attitude adjustment amount. When generating instructions, it is necessary to ensure that the instruction format conforms to the requirements of the processing equipment control system so that the control system can accurately parse and execute it.

[0169] Step S1526: Send the machining pose adjustment command to the control system of the machining equipment, so that the control system adjusts the movement of the machining equipment according to the machining pose adjustment command, so that the pose of the machining object is restored to the set range.

[0170] The generated machining pose adjustment commands are sent to the control system of the machining equipment via a data transmission interface. After receiving the commands, the control system parses them, extracts the motion control parameters, and controls the various axes of the machining equipment to perform corresponding movements based on these parameters.

[0171] For example, the X-axis drive mechanism is controlled to move the machining platform or tool a corresponding distance in the X-axis direction based on the position adjustment amount in the X-axis direction, and the rotation mechanism is controlled to rotate the workpiece around the Y-axis by a corresponding angle based on the attitude adjustment amount around the Y-axis. Through the above movements, the position and posture of the electronic device housing component are gradually restored to the set range, ensuring the accuracy and stability of the machining process.

[0172] Figure 2 The illustration shows exemplary hardware and software components of an adaptive processing positioning system 100 based on visual feature matching, which can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the adaptive processing positioning system 100 based on visual feature matching and to perform the functions in this application.

[0173] For example, the visual feature matching-based adaptive machining positioning system 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the visual feature matching-based adaptive machining positioning system 100 may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The visual feature matching-based adaptive machining positioning system 100 also includes an I / O interface 150 between the computer and other input / output devices.

[0174] Furthermore, this embodiment of the invention also provides a readable storage medium, wherein computer-executable instructions are preset in the readable storage medium, and when the processor executes the computer-executable instructions, the above-mentioned adaptive processing and positioning method based on visual feature matching is implemented.

[0175] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. An adaptive processing localization method based on visual feature matching, characterized in that, The method includes: Acquire the initial visual image of the object to be processed and the preset processing reference image, extract the first visual feature set from the initial visual image, extract the second visual feature set from the processing reference image, and establish the initial feature association relationship between the first visual feature set and the second visual feature set; Based on the initial feature association relationship, hierarchical feature matching processing is performed on the first visual feature set and the second visual feature set to obtain the feature matching mapping relationship between the initial visual image of the processing object and the processing reference image; Based on the feature matching mapping relationship, calculate the initial machining pose parameters of the machining object in the coordinate system of the machining equipment; During the processing, dynamic visual images of the processing area are continuously acquired, a third visual feature set is extracted from the dynamic visual images, and the third visual feature set is dynamically matched with the second visual feature set to obtain a real-time feature matching mapping relationship. The real-time machining pose parameters of the machining object in the coordinate system of the machining equipment are calculated based on the real-time feature matching mapping relationship. The real-time machining pose parameters are compared with the initial machining pose parameters to generate a machining pose adjustment command, which is then sent to the control system of the machining equipment.

2. The adaptive processing localization method based on visual feature matching according to claim 1, characterized in that, The process of acquiring an initial visual image of the object to be processed and a preset processing reference image, extracting a first visual feature set from the initial visual image, extracting a second visual feature set from the processing reference image, and establishing an initial feature association between the first visual feature set and the second visual feature set includes: The initial visual image of the object to be processed is acquired by a vision acquisition device, and the preset processing reference image stored in the processing control system is retrieved. An image enhancement process is performed on the initial visual image to obtain an enhanced initial image. Feature extraction processing is then performed on the enhanced initial image to identify microstructure regions and contour edge regions in the enhanced initial image. Texture distribution features, geometric shape features, and spatial arrangement features within the microstructure regions are extracted as microstructure features. Edge line segment features, corner features, and curvature change features within the contour edge regions are extracted as contour edge features. The microstructure features and contour edge features are combined to form a first visual feature set. The processing reference image is subjected to image enhancement processing to obtain an enhanced reference image. Feature extraction processing is then performed on the enhanced reference image to identify microstructure regions and contour edge regions in the enhanced reference image. Texture distribution features, geometric shape features, and spatial arrangement features within the microstructure regions are extracted as microstructure features. Edge line segment features, corner features, and curvature change features within the contour edge regions are extracted as contour edge features. The microstructure features and contour edge features are combined to form a second visual feature set. Corner features from the first visual feature set are extracted as first key feature points, and corner features from the second visual feature set are extracted as second key feature points. The feature similarity between the first key feature point and the second key feature point is calculated. An initial feature association relationship between the first visual feature set and the second visual feature set is established based on the feature similarity. The initial feature association relationship is used to characterize the correspondence between the first key feature point and the second key feature point.

3. The adaptive processing localization method based on visual feature matching according to claim 1, characterized in that, The hierarchical feature matching process performed on the first visual feature set and the second visual feature set based on the initial feature association relationship to obtain the feature matching mapping relationship between the initial visual image of the processing object and the processing reference image includes: Based on the correspondence between the first key feature point and the second key feature point in the initial feature association relationship, coarse matching processing is performed on the contour edge features in the first visual feature set and the contour edge features in the second visual feature set. The direction consistency of the edge line segment features in the contour edge features is verified to determine whether the extension direction of the edge line segment matches. The angle matching verification is performed on the corner features to determine whether the included angle of the corner is within a preset range. The trend matching verification is performed on the curvature change features to determine whether the curvature change trend is consistent. The verified contour edge features are combined to form the contour matching result. Using the contour matching result as a constraint, the microstructure features in the first visual feature set and the microstructure features in the second visual feature set are subjected to fine matching processing. Based on the contour matching result, the matching region of the microstructure features is determined. The texture distribution features in the microstructure features are compared for regional similarity to determine whether the texture distribution pattern is consistent. The geometric shape features are calculated for shape similarity to determine whether the contours of the geometric shapes match. The spatial arrangement features are verified for positional relationship to determine whether the relative positions between microstructures conform to the preset rules. The verified microstructure features are combined to form the microstructure matching result. Feature association verification is performed on the contour matching results and microstructure matching results to check whether the spatial positional relationship between the contour edge features and the microstructure features is consistent, so that the matched contour edge features can accurately define the distribution area of ​​the microstructure features. Based on the correlation verification results, the contour matching results and microstructure matching results are filtered to remove matching features with inconsistent spatial relationships and retain the matching features that have passed the verification. The retained contour matching results and microstructure matching results are integrated to generate a set of feature matching pairs; Based on the feature matching pair set, a homography transformation model is constructed between the initial visual image and the processing reference image. The feature matching mapping relationship is generated through the homography transformation model. The feature matching mapping relationship is used to characterize the spatial coordinate transformation relationship between each pixel in the initial visual image and the corresponding pixel in the processing reference image.

4. The adaptive processing localization method based on visual feature matching according to claim 3, characterized in that, The step of verifying the directional consistency of edge line segment features in the contour edge features, and determining whether the extension directions of the edge line segments match, includes: Extract edge line segment features from the contour edge features of the first visual feature set, determine the start coordinates and end coordinates of each edge line segment, and calculate the direction vector of the edge line segment based on the start coordinates and end coordinates; Extract edge line segment features from the contour edge features of the second visual feature set, determine the start coordinates and end coordinates of each edge line segment, and calculate the direction vector of the edge line segment based on the start coordinates and end coordinates; For the edge line segment features around the first and second key feature points corresponding to the initial feature association relationship, the angle between the edge line segment direction vector in the first visual feature set and the edge line segment direction vector in the second visual feature set is calculated; Determine whether the calculated included angle is less than a preset direction threshold. If it is less than the preset direction threshold, determine that the extension direction of the edge line segment matches. If it is greater than or equal to the preset direction threshold, determine that the extension direction of the edge line segment does not match. Perform directional consistency verification on all corresponding edge line segment features, count the number of edge line segment features that pass the verification, and determine that the directional consistency verification of the contour edge features has passed when the proportion of the number of verified features to the total number exceeds a preset proportion threshold.

5. The adaptive processing localization method based on visual feature matching according to claim 3, characterized in that, The step of performing region similarity comparison on the texture distribution features in the microstructure features to determine whether the texture distribution patterns are consistent includes: Based on the contour matching results, the image region where the microstructure features are located is determined. This image region is defined as the first texture region in the first visual feature set, and the corresponding region is defined as the second texture region in the second visual feature set. The first texture region is divided into texture units, and the texture direction, texture density and texture periodicity features of each texture unit are extracted. The second texture region is divided into texture units, which are then divided into multiple texture units of the same size. The texture direction, texture density, and texture periodicity features of each texture unit are extracted. Each texture unit in the first texture region is compared with the corresponding texture unit in the second texture region to calculate the similarity of texture direction, texture density, and texture periodicity. The three similarity indicators are comprehensively evaluated. If the comprehensive evaluation result exceeds the preset texture similarity threshold, the distribution pattern of the texture unit is determined to match; otherwise, it is determined not to match. The matching results of all texture units are statistically analyzed. When the proportion of the number of matched texture units to the total number of texture units exceeds the preset region matching threshold, the region similarity comparison of the texture distribution features is deemed to have passed.

6. The adaptive processing localization method based on visual feature matching according to claim 1, characterized in that, The step of calculating the initial machining pose parameters of the machining object in the coordinate system of the machining equipment based on the feature matching mapping relationship includes: The homography matrix between the initial visual image and the processing reference image is extracted from the feature matching mapping relationship. The homography matrix is ​​used to characterize the projection transformation relationship between the two planes. The homography matrix is ​​decomposed using camera calibration parameters to obtain a rotation matrix and a translation vector. The camera calibration parameters include the camera intrinsic parameter matrix and distortion coefficients. The rotation matrix is ​​converted into Euler angles to obtain the rotation angle parameters of the workpiece. The rotation angle parameters include the rotation angle around the X-axis, the rotation angle around the Y-axis, and the rotation angle around the Z-axis. The translation vector is converted into coordinate values ​​in the coordinate system of the processing equipment to obtain the translation position parameters of the processing object. The translation position parameters include the translation distance in the X-axis direction, the translation distance in the Y-axis direction, and the translation distance in the Z-axis direction. The rotation angle parameters and translation position parameters are combined to form the initial machining pose parameters of the workpiece in the coordinate system of the machining equipment. The initial machining pose parameters are used to determine the position and orientation of the workpiece at the start of machining.

7. The adaptive processing localization method based on visual feature matching according to claim 1, characterized in that, During the processing, dynamic visual images of the processing area are continuously acquired, a third visual feature set is extracted from the dynamic visual images, and the third visual feature set is dynamically matched with a second visual feature set to obtain a real-time feature matching mapping relationship, including: During the processing operation of the processing equipment, dynamic visual images of the processing area are continuously acquired by the visual acquisition device at preset time intervals. The dynamic visual images reflect the real-time state of the surface of the object being processed during the processing. The dynamic visual image is preprocessed to remove background interference outside the processing area and retain the effective area of ​​the surface of the processing object, thus obtaining the preprocessed dynamic visual image. The preprocessed dynamic visual image is subjected to feature extraction processing to identify microstructure regions and contour edge regions. Texture distribution features, geometric shape features and spatial arrangement features in the microstructure region are extracted as microstructure features, and edge line segment features, corner features and curvature change features in the contour edge region are extracted as contour edge features. The microstructure features and contour edge features are combined to form a third visual feature set. The third visual feature set and the second visual feature set are dynamically matched, and the matching strategy is adjusted based on the real-time state changes during the processing. During the dynamic matching process, the possible location of features at the current time is predicted based on the matching result of the previous time step to narrow down the matching search area. The contour edge features in the third visual feature set are quickly matched with the contour edge features in the second visual feature set to determine the correspondence of the reference contour. Based on the reference contour correspondence, the microstructure features are matched, and the contour matching results and microstructure matching results obtained from the matching process are integrated to generate a set of real-time feature matching pairs. Based on the set of real-time feature matching pairs, a real-time homography transformation model is constructed between the dynamic visual image and the processing reference image, and a real-time feature matching mapping relationship is generated through the real-time homography transformation model.

8. The adaptive processing localization method based on visual feature matching according to claim 7, characterized in that, The method of predicting the possible location of features at the current time based on the matching result of the previous time step to narrow down the matching search area includes: Obtain the set of real-time feature matching pairs obtained from the previous dynamic matching process, extract the coordinate information of each feature pair, and obtain the coordinates of the feature points in the third visual feature set and the corresponding feature points in the second visual feature set in the previous time step. By analyzing the time interval from the previous moment to the current moment and combining it with the motion parameters of the processing equipment, the displacement and rotation angle of the object being processed are predicted during the processing. The motion parameters include the feed rate and rotation speed of the processing equipment. Based on the predicted displacement and rotation angle, the possible position coordinates of each feature point in the third visual feature set at the previous moment in the dynamic visual image at the current moment are calculated to obtain the predicted position of the feature point. Centered on the predicted position, a search area of ​​a preset size is set as the matching search area for the feature points at the current moment. The size of the matching search area is determined according to the movement speed of the processing object and the image acquisition frame rate. For each feature point in the third visual feature set, matching is performed only with feature points in the second visual feature set within the corresponding matching search area; wherein, when performing feature matching within the matching search area, the feature point closest to the predicted position is matched first.

9. The adaptive processing localization method based on visual feature matching according to claim 1, characterized in that, The step of calculating the real-time machining pose parameters of the machining object in the coordinate system of the machining equipment based on the real-time feature matching mapping relationship includes: Extract the real-time homography matrix between the dynamic visual image and the processing reference image from the real-time feature matching mapping relationship; The real-time homography matrix is ​​decomposed using camera calibration parameters to obtain the real-time rotation matrix and the real-time translation vector. The real-time rotation matrix is ​​converted into Euler angles to obtain the real-time rotation angle parameters of the workpiece. The real-time rotation angle parameters include the real-time rotation angle around the X-axis, the real-time rotation angle around the Y-axis, and the real-time rotation angle around the Z-axis. The real-time translation vector is converted into coordinate values ​​in the coordinate system of the processing equipment to obtain the real-time translation position parameters of the processing object. The real-time translation position parameters include the real-time translation distance in the X-axis direction, the real-time translation distance in the Y-axis direction, and the real-time translation distance in the Z-axis direction. The real-time rotation angle parameters and real-time translation position parameters are combined to form the real-time machining pose parameters of the machining object in the coordinate system of the machining equipment. The real-time machining pose parameters are used to reflect the actual position and posture of the machining object at the current moment. And, the step of comparing the real-time machining pose parameters with the initial machining pose parameters, generating a machining pose adjustment command, and sending the machining pose adjustment command to the control system of the machining equipment includes: The real-time rotation angle parameter in the real-time machining pose parameters is compared with the rotation angle parameter in the initial machining pose parameters to calculate the rotation angle deviation value, which includes the rotation angle deviation around the X-axis, the rotation angle deviation around the Y-axis, and the rotation angle deviation around the Z-axis. The real-time translation position parameters in the real-time machining pose parameters are compared with the translation position parameters in the initial machining pose parameters, and the translation position deviation value is calculated. The translation position deviation value includes the translation deviation in the X-axis direction, the translation deviation in the Y-axis direction, and the translation deviation in the Z-axis direction. The rotation angle deviation value and the translation position deviation value are combined to form the pose deviation value, which is used to characterize the difference between the current pose of the machining object and the initial set pose. Determine whether the pose deviation value exceeds the preset allowable deviation range. If it does not exceed the allowable deviation range, no machining pose adjustment command is generated. If it exceeds the allowable deviation range, calculate the adjustment amount of the machining equipment based on the pose deviation value. The adjustment amount includes position adjustment amount and posture adjustment amount. The position adjustment amount is determined based on the translation position deviation value, and the posture adjustment amount is determined based on the rotation angle deviation value. Based on the adjustment amount, a machining posture adjustment command is generated, which includes motion control parameters for each axis of the machining equipment. The machining posture adjustment command is sent to the control system of the machining equipment so that the control system adjusts the movement of the machining equipment according to the machining posture adjustment command, so that the posture of the machining object is restored to the set range.

10. An adaptive processing positioning system based on visual feature matching, characterized in that, The device includes a processor and a memory, the memory being connected to the processor. The memory is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the memory to implement the adaptive processing and localization method based on visual feature matching as described in any one of claims 1-9.

Images