A Machine Vision-Based Assembly Verification Method for Drilling and Cutting Composite Tools

By employing multi-view machine vision and multi-scale feature fusion methods, the accuracy and consistency issues of traditional drilling and cutting composite tool assembly status detection have been resolved, enabling automated, reliable identification and comprehensive verification of tool assembly components.

CN123118550APending Publication Date: 2026-11-17NORTH CHINA UNIVERSITY OF TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611348463.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-02
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Traditional drilling and cutting composite tool assembly status detection relies on manual observation or single-view images, which is easily affected by metal surface reflection, component obstruction, and changes in tool placement posture. This results in low accuracy in identifying component assembly anomalies, poor consistency of identification results under different viewpoints, difficulty in taking into account both overall assembly structure information and local detail features, and difficulty in quantifying actual assembly deviations and evaluating the combined effect of multiple assembly deviations on the tool rotation working state.

Method used

A tool assembly verification method based on multi-view machine vision is adopted. A tool assembly verification coordinate system is established. A recognition method combining multi-scale assembly feature deep fusion and global-local assembly feature adaptive enhancement is used. Through multi-view image acquisition and image preprocessing, an assembly state recognition model is constructed to locate the assembly parts and determine their state, and then perform comprehensive verification.

Benefits of technology

It improves the accuracy and stability of assembly component area identification and assembly status judgment, enhances the adaptability to partial occlusion, metal reflection and single-view feature loss, and realizes reliable joint identification and comprehensive verification of assembly component area positioning and assembly status of drilling and cutting composite tool.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118550A_ABST
    Figure CN123118550A_ABST
Patent Text Reader

Abstract

This invention discloses a machine vision-based method for assembling and verifying drilling and cutting composite tools. The method includes acquiring multi-view assembly images of the tool, establishing a tool assembly verification coordinate system, identifying the assembly status of tool components, correcting the identification results of the tool component assembly status, and comprehensively verifying the tool assembly. This invention relates to the field of image data processing technology, specifically to a machine vision-based method for assembling and verifying drilling and cutting composite tools. This solution innovatively proposes a method based on multi-view machine vision, a tool assembly verification coordinate system, and tool assembly structural constraints, improving the stability of multi-view assembly status identification results; it proposes a method based on deep fusion of multi-scale assembly features and adaptive enhancement of global and local assembly features, improving the accuracy of tool component assembly status identification; and it uses a comprehensive verification method based on the actual pose of the assembled components, assembly deviation parameters, and the actual working envelope, improving the rationality and accuracy of the overall assembly quality verification of the drilling and cutting composite tool.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, specifically to a machine vision-based method for assembling and verifying drilling and cutting composite tools. Background Technology

[0002] The drilling and cutting composite tool assembly verification method refers to an intelligent quality verification method that utilizes machine vision technology for automated assembly scenarios of precision CNC tools. By acquiring high-definition images of each assembly component of the drilling and cutting composite tool, it performs fully automatic and accurate verification and qualification judgment on the dimensional accuracy, geometric tolerances, and functional fit of the tool assembly. This transforms the drilling and cutting composite tool assembly verification from the inefficient mode of traditional manual contact measurement and sampling verification to intelligent online verification, effectively improving the detection accuracy, operation efficiency, and result consistency of tool assembly verification, reducing the subjective error and missed detection risk of manual inspection, and ensuring the cutting performance and service life of the drilling and cutting composite tool.

[0003] However, traditional drilling and cutting composite tool assembly status detection mainly relies on manual observation or single-view images, which is easily affected by metal surface reflection, component occlusion, and changes in tool placement posture, resulting in low accuracy in component assembly anomaly identification and poor consistency of identification results under different viewpoints. Existing models for tool component assembly status identification cannot take into account both overall assembly structure information and local detailed features, which easily leads to the loss of features of small-sized assembly components, inaccurate component area positioning, and insufficient stability in identifying subtle assembly statuses. Traditional drilling and cutting composite tool assembly verification mainly relies on the assembly status of a single component or a single dimensional parameter for qualification judgment, making it difficult to quantify actual assembly deviations and evaluate the impact of multiple assembly deviations on the tool's rotational working state, resulting in inaccurate positioning of assembly anomalies and insufficient correlation between detection results and the actual working state of the tool. Summary of the Invention

[0004] To address the above issues and overcome the shortcomings of existing technologies, this invention provides a machine vision-based method for verifying the assembly of drilling and cutting composite tools. Traditional drilling and cutting composite tool assembly status detection relies primarily on manual observation or single-view images, which are easily affected by metal surface reflections, component occlusion, and changes in tool placement posture, leading to low accuracy in identifying component assembly anomalies and poor consistency of identification results across different viewpoints. This solution innovatively proposes an assembly status recognition and correction method based on multi-view machine vision, a tool assembly verification coordinate system, and tool assembly structural constraints. This improves the accuracy of component area identification and assembly status judgment in complex metal structure environments, enhances adaptability to local occlusion, metal reflections, and missing features from a single viewpoint, and improves the stability and reliability of multi-view assembly status recognition results. Furthermore, addressing the technical problems of existing tool component assembly status recognition models failing to simultaneously consider overall assembly structure information and local detail features, easily leading to feature loss of small-sized assembly components, inaccurate component area positioning, and insufficient stability in identifying subtle assembly statuses, this invention… This solution proposes a tool component assembly state recognition method based on deep fusion of multi-scale assembly features and adaptive enhancement of global and local assembly features. This enhances the model's ability to discriminate key details such as cutting edges, fastening structures, and component orientation, as well as its adaptability to reflections, occlusions, and changes in viewing angle. It improves the accuracy, stability, and robustness of assembly component area recognition and assembly state judgment, achieving reliable joint recognition of assembly component area positioning and assembly state for drilling-cutting composite tools. Addressing the technical problem that traditional drilling-cutting composite tool assembly verification mainly relies on the assembly state of individual components or single dimensional parameters for qualification judgment, leading to inaccurate positioning of assembly anomalies and insufficient correlation between detection results and the actual working state of the tool, this solution innovatively proposes a comprehensive verification method based on the actual pose of the assembly component, assembly deviation parameters, and the actual working envelope. This enhances the correlation evaluation capability between assembly geometric deviations and the actual working range of the tool, improving the rationality and accuracy of the overall assembly quality judgment of drilling-cutting composite tools. It achieves a comprehensive verification method that moves from assembly state recognition to quantification of assembly deviations and joint evaluation of actual working state.

[0005] The technical solution adopted by this invention is as follows: This invention provides a machine vision-based method for assembling and verifying drilling and cutting composite tools, which includes the following steps:

[0006] Step S1: Obtain multi-view assembly images of the cutting tool;

[0007] Step S2: Establish the tool assembly verification coordinate system;

[0008] Step S3: Tool component assembly status identification;

[0009] Step S4: Correction of tool component assembly status identification results;

[0010] Step S5: Comprehensive verification of tool assembly.

[0011] Further, in step S1, the acquisition of multi-view assembly images of the cutting tool specifically involves fixing the drilling and cutting composite tool on a controllable rotating mechanism, controlling the drilling and cutting composite tool to rotate sequentially around the central axis of the tool body according to a preset rotation angle, and acquiring images at each preset rotation angle using an industrial camera, while recording the rotation angle information corresponding to each acquired image to obtain the original data of the multi-view assembly images of the cutting tool, and performing image preprocessing operations on the original data of the multi-view assembly images of the cutting tool to obtain optimized data of the multi-view assembly images of the cutting tool.

[0012] The raw data of the multi-view assembly image of the tool includes multi-view assembly image data of the reference tool and multi-view assembly image data of the target tool.

[0013] The image preprocessing operations specifically include image denoising and image enhancement.

[0014] Further, in step S2, the establishment of the tool assembly verification coordinate system specifically involves identifying the tool body reference features of the target drilling and cutting composite tool based on the multi-view assembly image data of the target tool in the multi-view assembly image optimization data, determining the tool body contour, tool body central axis, reference end face, and circumferential reference structure, and determining the circumferential reference direction based on the position of the circumferential reference structure relative to the tool body central axis to obtain the tool coordinate reference features. Finally, the tool assembly verification coordinate system is established based on the tool coordinate reference features, wherein the intersection of the tool body central axis and the reference end face is taken as the origin, the direction along the tool body central axis is taken as the axial coordinate axis, the projection direction of the circumferential reference direction in the plane perpendicular to the tool body central axis is taken as the first radial coordinate axis, and the second radial coordinate axis is determined according to the right-hand coordinate system rule to obtain the tool assembly verification coordinate system corresponding to the target drilling and cutting composite tool.

[0015] Furthermore, in step S3, the identification of the assembly state of the tool component specifically includes the following steps:

[0016] Step S31: Construct a tool component assembly state recognition model, which specifically includes the following steps:

[0017] Step S311: Initial feature extraction of the assembly structure, specifically, the tool multi-view assembly image data is mapped by convolution operation, and batch normalization and nonlinear activation are performed in sequence to obtain the initial feature map of the tool assembly structure.

[0018] Step S312: Multi-scale assembly feature deep fusion, specifically including the following steps:

[0019] Step S3121: Medium and high scale assembly feature extraction, specifically, the initial feature map of the tool assembly structure is downsampled and the downsampled features are input into the corresponding residual SwinTransformer block to obtain medium and high scale assembly features;

[0020] Step S3122: Low-scale assembly feature extraction, specifically, downsampling the mid-to-high-scale assembly features and inputting the downsampled features into the corresponding residual SwinTransformer block to obtain low-scale assembly features;

[0021] Step S3123: Low-complexity assembly feature fusion, specifically, using convolution operation to perform feature mapping on low-scale assembly features to obtain corresponding deep mapping features, then concatenating the deep mapping features and low-scale assembly features along the channel dimension, and using linear mapping units to perform channel compression on the concatenated features to obtain low-complexity assembly fusion features.

[0022] Step S3124: Deep assembly relationship feature extraction, specifically, inputting the low-complexity assembly fusion feature into the corresponding residual SwinTransformer block to obtain deep assembly relationship features;

[0023] Step S3125: Medium- and high-scale assembly feature recovery, specifically, upsampling the deep assembly relationship features, then introducing medium- and high-scale assembly features through long skip connections, concatenating the upsampled deep features and medium- and high-scale assembly features along the channel dimension, and performing channel compression through a linear mapping unit. Finally, the residual SwinTransformer block is input for cross-scale feature association and fusion to obtain the medium- and high-scale recovered assembly features.

[0024] Step S3126: Original scale cross-layer feature recovery, specifically, upsampling is performed on the recovered assembly features at medium and high scales, and then the upsampled features are concatenated with the initial feature map of the tool assembly structure along the channel dimension through long skip connections. Channel compression is performed through linear mapping units, and convolution operation is used to perform local smoothing and feature dimension adjustment on the fused features to obtain a multi-scale assembly fusion depth feature map.

[0025] Step S313: Adaptive enhancement of global and local assembly features, specifically including the following steps:

[0026] Step S3131: Global assembly semantic feature extraction, specifically, dividing the multi-scale assembly fusion deep feature map into average values. Each pooling neighborhood region is averaged and pooled to obtain the global assembly semantic features for that region. These regions are then combined according to their spatial positions in the original feature map to obtain the global assembly semantic feature map. The formula used is as follows:

[0027] ;

[0028] In the formula, This represents the global assembly semantic feature obtained by average pooling the r-th pooling neighborhood region. Indicates the current pooling neighborhood region. Indicates the pooling neighborhood region The number of features contained within, where i represents the index of the feature point within the pooling neighborhood. Represents global assembly semantic features;

[0029] Step S3132: Local key assembly feature soft pooling enhancement, specifically, performing soft pooling processing on each pooling neighborhood region, calculating the corresponding soft pooling weight based on the response intensity of each feature point in the pooling neighborhood region, performing weighted aggregation based on the soft pooling weights corresponding to each feature point to obtain the local key assembly feature corresponding to the r-th pooling neighborhood region, and combining the local key assembly features corresponding to different pooling neighborhood regions according to their spatial location to obtain the local key assembly feature map;

[0030] Step S3133: Adaptive fusion of assembly features, specifically by setting learnable balance parameters and mapping these parameters using the Sigmoid function to obtain a value range of [value range missing]. The adaptive fusion coefficients of the interval are used to weight and fuse the global assembly semantic feature map and the local key assembly feature map to obtain the assembly discrimination enhancement feature map.

[0031] Step S314: Joint discrimination output of assembled components, specifically including the following steps:

[0032] Step S3141: Assembly component region discrimination, specifically, the assembly discrimination enhancement feature map is processed by a region candidate network to generate candidate regions, and each candidate assembly component region is obtained. The region features corresponding to each candidate region are extracted by the region of interest alignment operation. The region features are used to predict the component category by the region category through a fully connected layer and a Softmax classifier. At the same time, the region boundary adjustment parameters corresponding to each candidate assembly component region are determined by the fully connected layer, and the boundaries of each candidate assembly component region are corrected to obtain the component region and component category of each assembly component.

[0033] Step S3142: Assembly state discrimination, specifically, based on the regions of each assembly component, the region features of each component region are extracted from the assembly discrimination enhancement feature map using the region of interest alignment operation, and the region features are classified through a fully connected layer and a Softmax classifier to obtain the predicted probability of each assembly component belonging to different preset assembly states, and the assembly state with the highest predicted probability is selected as the initial assembly state of the corresponding assembly component.

[0034] Step S32: Supervised training of the model, specifically, using the reference tool multi-view assembly image data in the tool multi-view assembly image optimization data as the training data of the model, performing iterative training of the recognition model, and obtaining the trained tool component assembly state recognition model.

[0035] Step S33: Target tool component area and assembly state recognition, specifically, inputting the target tool multi-view assembly image data from the optimized multi-view assembly image data of the tool into the trained tool component assembly state recognition model to obtain the component category, component area and initial assembly state corresponding to each assembly component;

[0036] Step S34: Obtain the tool component assembly status recognition result. Specifically, according to the component category corresponding to each assembly component, the component category, component area and initial assembly status are combined accordingly to obtain the recognition result of each assembly component of the target tool.

[0037] Further, in step S4, the correction of the tool component assembly state recognition result specifically involves: based on the preset rotation angle corresponding to the multi-view assembly image of each target tool and the tool assembly verification coordinate system, spatially corresponding the assembly components identified under different rotation angles is performed, establishing the association relationship of the recognition results of the same assembly component under different rotation angles; based on the preset tool assembly structure constraint relationship, constraint verification is performed on the corresponding assembly component recognition results under each rotation angle; when the component area or initial assembly state under the target rotation angle does not satisfy the tool assembly structure constraint relationship, and there are associated recognition results that satisfy the tool assembly structure constraint relationship under other rotation angles, the associated recognition results are used as the correction basis to correct the component area and initial assembly state under the target rotation angle, obtaining the corrected component area and corrected assembly state corresponding to each assembly component; and according to the preset rotation angle corresponding to the multi-view assembly image of each target tool, the corrected component area and corrected assembly state of each assembly component under different rotation angles are associated and integrated to obtain the tool component assembly state correction result.

[0038] Furthermore, in step S5, the comprehensive verification of the tool assembly specifically includes the following steps:

[0039] Step S51: Determine the actual pose of the assembly component. Specifically, based on the correction component area of ​​each assembly component in the tool component assembly state correction result, and combined with the preset rotation angle and tool assembly verification coordinate system corresponding to the multi-view assembly image of each target tool, establish the spatial correspondence between the correction component areas of the same assembly component under different rotation views, perform coordinate transformation and spatial pose calculation on the correction component areas under each rotation view, determine the actual position parameters and actual direction parameters of each assembly component in the tool assembly verification coordinate system, and obtain the actual assembly pose corresponding to each assembly component.

[0040] Step S52: Assembly deviation calculation, specifically, to obtain the standard position parameters and standard direction parameters of each assembly component of the target drilling and cutting composite tool in the tool assembly verification coordinate system, compare the actual position parameters and actual direction parameters of each assembly component with the corresponding standard position parameters and standard direction parameters, and calculate the axial position deviation, radial position deviation, direction angle deviation and circumferential phase deviation of each assembly component to obtain the assembly deviation parameters corresponding to each assembly component.

[0041] Step S53: Determine the actual working envelope of the tool. Specifically, based on the actual assembly posture of each assembly component and the central axis of the tool body, construct the actual working envelope corresponding to the target drilling and cutting composite tool, and compare the actual working envelope with the preset standard working envelope in space to obtain the tool working envelope deviation.

[0042] Step S54: Comprehensive verification, specifically, based on the tool component assembly status correction results, the assembly deviation parameters corresponding to each assembly component, and the tool working envelope deviation, compare them with the corresponding preset verification conditions, and make a comprehensive judgment based on the comparison results to determine the qualified assembly status, abnormal assembly components, and corresponding assembly abnormality types of the target drilling and cutting composite tool, and obtain the tool assembly comprehensive verification result, thereby realizing the joint verification of the assembly status, assembly position deviation, and rotation working envelope of the target drilling and cutting composite tool.

[0043] The beneficial effects achieved by the present invention using the above solution are as follows:

[0044] (1) In view of the technical problem that the traditional drilling and cutting composite tool assembly status detection mainly relies on manual observation or single-view images, which is easily affected by metal surface reflection, component occlusion and tool placement posture changes, resulting in low accuracy of component assembly abnormality identification and poor consistency of identification results under different views, this solution innovatively proposes an assembly status identification and correction method based on multi-view machine vision, tool assembly verification coordinate system and tool assembly structure constraints. By collecting assembly images of drilling and cutting composite tools at different rotation angles, a tool assembly verification coordinate system based on the tool body structure is established. Combined with the tool assembly structure constraint relationship, the component area and initial assembly status under different rotation angles are correlated, verified and corrected. This improves the accuracy of assembly component area identification and assembly status judgment in complex metal structure environment, enhances the adaptability to local occlusion, metal reflection and single-view feature loss, improves the stability and reliability of multi-view assembly status identification results, and realizes the automated and reliable identification and correction of the assembly status of each assembly component of drilling and cutting composite tool.

[0045] (2) To address the technical problems of existing tool component assembly state recognition models, which are difficult to take into account both overall assembly structure information and local detail features, and are prone to feature loss of small-sized assembly components, inaccurate component area positioning, and insufficient stability of fine assembly state recognition under conditions of metal surface reflection, local occlusion, and different rotation angles, this solution proposes a tool component assembly state recognition method based on multi-scale assembly feature deep fusion and global and local assembly feature adaptive enhancement. Through hierarchical downsampling, residual SwinTransformer feature modeling, cross-scale long jump connection, and upsampling recovery, multi-scale fusion of the overall tool assembly relationship and local structural details is achieved. Combined with average pooling, soft pooling, and learnable fusion weights, the contribution of global assembly semantic features and local key assembly features is adaptively balanced, which improves the feature representation and area positioning capabilities of assembly components at different scales, enhances the model's ability to distinguish key details such as cutting edges, fastening structures, and component orientation, and improves its adaptability to reflection, occlusion, and angle changes. This improves the accuracy, stability, and robustness of assembly component area recognition and assembly state judgment, and realizes reliable joint recognition of assembly component area positioning and assembly state of drilling and cutting composite tool assembly.

[0046] (3) In view of the technical problems that traditional drilling and cutting composite tool assembly verification mainly relies on the assembly status of a single component or a single dimensional parameter for qualification judgment, it is difficult to quantify the actual assembly deviation and evaluate the impact of multiple assembly deviations on the tool rotation working state, resulting in inaccurate positioning of assembly abnormalities and insufficient correlation between the test results and the actual working state of the tool, this solution innovatively proposes a comprehensive verification method based on the actual pose of the assembly components, assembly deviation parameters and actual working envelope. By determining the actual assembly pose of each assembly component in a unified tool assembly verification coordinate system, calculating the axis assembly deviation parameters, and constructing the actual working envelope of the tool rotation state based on the actual assembly pose of the drilling and cutting components, the actual working envelope is compared with the preset standard working envelope, which enhances the correlation evaluation ability between assembly geometric deviation and the actual working range of the tool, improves the rationality and accuracy of the overall assembly quality judgment of drilling and cutting composite tools, and realizes the comprehensive verification from assembly status identification to assembly deviation quantification and joint evaluation of actual working state. Attached Figure Description

[0047] Figure 1 A flowchart illustrating a machine vision-based drilling and cutting composite tool assembly and verification method provided by the present invention.

[0048] Figure 2 This is a flowchart illustrating step S3;

[0049] Figure 3 This is a flowchart illustrating step S31;

[0050] Figure 4 This is a flowchart illustrating step S312;

[0051] Figure 5 This is a flowchart illustrating step S5;

[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0053] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0054] In the description of this invention, it should be understood that the terms upper, lower, front, back, left, right, top, bottom, inner, and outer, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the system or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0055] Example 1, see Figure 1 The technical solution adopted by this invention is as follows: This invention provides a machine vision-based method for assembling and verifying drilling and cutting composite tools, which includes the following steps:

[0056] Step S1: Obtain multi-view assembly images of the tool, specifically by image acquisition and image preprocessing operations to obtain optimized data of multi-view assembly images of the tool;

[0057] Step S2: Establish the tool assembly verification coordinate system. Specifically, based on the multi-view assembly image of the target tool, identify the tool body central axis, reference end face and circumferential reference structure. Take the intersection of the tool body central axis and the reference end face as the coordinate origin, and determine the axial coordinate axis and radial coordinate axis according to the tool body central axis and circumferential reference structure to obtain the tool assembly verification coordinate system.

[0058] Step S3: Tool component assembly state recognition. Specifically, this involves constructing a tool component assembly state recognition model that includes initial feature extraction of assembly structure, deep fusion of multi-scale assembly features, adaptive enhancement of global and local assembly features, and joint discrimination of assembly components. Then, supervised training of the model and recognition of the target tool component region and assembly state are performed to obtain the component category, component region, and initial assembly state of each assembly component. Finally, the corresponding combinations are performed to obtain the recognition results of each assembly component of the target tool.

[0059] The multi-scale assembly feature deep fusion is achieved through hierarchical downsampling, residual SwinTransformer feature modeling, cross-scale long skip connections, and upsampling operations to obtain a multi-scale assembly fusion deep feature map.

[0060] The adaptive enhancement of global and local assembly features obtains global assembly semantic features and local key assembly features through average pooling and soft pooling, respectively, and then uses learnable fusion weights to adaptively weight and fuse the global assembly semantic features and local key assembly features to obtain assembly discrimination enhancement features.

[0061] Step S4: Correction of tool component assembly status recognition results. Specifically, based on the tool assembly verification coordinate system, establish the correlation between the recognition results of the same assembly component under different rotation angles, and combine the tool assembly structure constraint relationship to perform constraint verification on the recognition results of the corresponding assembly components under each rotation angle to obtain the tool component assembly status correction results.

[0062] Step S5: Comprehensive verification of tool assembly. Specifically, based on the tool component assembly state correction results and the tool assembly verification coordinate system, the actual pose of the assembly components is determined, the assembly deviation is calculated, and the actual working envelope of the tool is obtained in sequence. The actual assembly pose, assembly deviation parameters, and tool working envelope deviation of each assembly component are obtained respectively. Finally, a comprehensive verification is performed based on the tool component assembly state correction results, the assembly deviation parameters of each assembly component, and the tool working envelope deviation to obtain the comprehensive verification result of the tool assembly.

[0063] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, the acquisition of multi-view assembly images of the tool is used to obtain the assembly structure information of the drilling and cutting composite tool at different circumferential positions, providing image data for the identification and positioning of each assembly component of the tool. Specifically, the drilling and cutting composite tool is fixed on a controllable rotating mechanism, and the drilling and cutting composite tool is controlled to rotate sequentially around the central axis of the tool body according to a preset rotation angle. An industrial camera is used to acquire images at each preset rotation angle, and the rotation angle information corresponding to each acquired image is recorded to obtain the original data of the multi-view assembly images of the tool. Image preprocessing is then performed on the original data of the multi-view assembly images of the tool to obtain optimized data of the multi-view assembly images of the tool.

[0064] The raw data of the multi-view assembly image of the tool includes multi-view assembly image data of the reference tool and multi-view assembly image data of the target tool.

[0065] The reference tool multi-view assembly image data and the target tool multi-view assembly image data are used to characterize the image features, standard position features, and standard direction features of the tool body, drilling component, cutting component, and fastening component under different rotation angles in the standard assembly state. They all include image data collected by the drilling and cutting composite tool at multiple preset rotation angles, and are grouped according to the corresponding preset rotation angles. At the same time, the correlation between each image data and the corresponding rotation angle is established so that the images collected by the reference tool and the target tool at the same preset rotation angle have a viewpoint correspondence.

[0066] The preset rotation angle is determined by the central axis of the drill-cutting composite tool as the rotation axis, and multiple acquisition angles are determined according to preset angle intervals within a preset rotation cycle. Preferably, the preset rotation angle includes 0°, 45°, 90°, 135°, 180°, 225°, 270° and 315°.

[0067] The reference tool multi-view assembly image data also includes reference annotation results corresponding to each reference tool multi-view assembly image;

[0068] The reference annotation results include the component area annotation results and assembly status annotation results for each assembled component;

[0069] The component region annotation results are used to identify the target regions of the tool body, drilling components, cutting components and fastening components in the corresponding multi-view assembly image of the reference tool, and to establish the association between each target region and the corresponding component category;

[0070] The assembly status labeling result is used to identify the preset assembly status of each assembly component in the corresponding target area. The preset assembly status includes at least one of normal, missing, misassembled, and reversed assembly, and establishes the association between each assembly component, target area and corresponding assembly status.

[0071] The image preprocessing operation is used to reduce the interference of random noise, uneven lighting and metal surface reflection on the features of the assembled parts in the multi-view assembly image of the tool, and improve the regional recognizability of the tool body, drilling parts, cutting parts and fastening parts. Specifically, it includes image denoising processing and image enhancement processing. The multi-view assembly images of the tool after image denoising and image enhancement processing are combined to obtain optimized data of multi-view assembly images of the tool.

[0072] The image denoising process specifically involves using a median filtering algorithm to sort the neighboring pixels of the multi-view assembly image of the tool, and replacing the gray value of the current pixel with the median of the gray values ​​of the neighboring pixels in order to remove impulse noise and random noise generated during image acquisition, while preserving the edge and contour features of each assembly component, thus obtaining a denoised multi-view assembly image of the tool.

[0073] The image enhancement process specifically includes contrast enhancement, brightness adjustment, and edge enhancement;

[0074] The contrast enhancement is used to enhance the local grayscale differences between different areas of the tool and reduce the impact of strong local reflections or uneven brightness on the identification of component areas. Specifically, it adopts a limited contrast adaptive histogram equalization algorithm to divide the denoised tool multi-view assembly image into multiple local areas, and performs grayscale histogram redistribution and contrast limitation on each local area respectively.

[0075] The brightness adjustment adopts a gamma correction algorithm, which specifically performs nonlinear mapping on pixel gray values ​​according to the overall brightness distribution of the image, so as to enhance the features of assembly parts in low brightness areas and suppress local overbright areas, thereby improving the brightness consistency of tool images under different rotation angles.

[0076] The edge enhancement employs an unsharpened mask algorithm, which extracts the difference information between the original image and its smoothed image, and then superimposes the difference information onto the brightness-adjusted image according to a preset enhancement coefficient to enhance the edge and contour features of the tool body, drilling component, cutting component, and fastening component.

[0077] Example 3, see Figure 1 This embodiment is based on the above embodiment. In step S2, the establishment of the tool assembly verification coordinate system is used to establish a unified assembly reference datum based on the structurally stable tool body in the target drilling and cutting composite tool. This datum uniformly represents the positional relationship of the tool body and each assembly component under different rotation angles, reducing the impact of tool placement posture and image angle changes on subsequent assembly state recognition and assembly deviation calculation. Specifically, based on the multi-view assembly image data of the target tool in the multi-view assembly image optimization data, the tool body reference feature is identified for the target drilling and cutting composite tool. The tool body contour, tool body central axis, reference end face, and circumferential reference structure are determined. The circumferential reference direction is determined according to the position of the circumferential reference structure relative to the tool body central axis, and the tool coordinate reference feature is obtained. Finally, the tool assembly verification coordinate system is established according to the tool coordinate reference feature. The intersection of the tool body central axis and the reference end face is taken as the origin of the coordinate system. The direction along the tool body central axis is taken as the axial coordinate axis. The projection direction of the circumferential reference direction in the plane perpendicular to the tool body central axis is taken as the first radial coordinate axis. The second radial coordinate axis is determined according to the right-hand coordinate system rule, and the tool assembly verification coordinate system corresponding to the target drilling and cutting composite tool is obtained.

[0078] The circumferential reference structure is an inherent structure on the cutter body that can characterize a fixed circumferential position and is different from the rotationally symmetric structure of the cutter body, including at least one of a positioning groove, a positioning hole, a positioning notch, a reference surface, and a mounting groove with a fixed circumferential position;

[0079] The tool body reference feature recognition is used to extract inherent structural features of the tool body that do not change with the assembly state of the assembly component to be verified from the multi-view assembly image data of the target tool, so as to determine the axial reference, axial position reference, and circumferential direction reference required to establish the tool assembly verification coordinate system. Specifically, edge feature extraction and contour filtering are performed on the main body area of ​​the tool body in the multi-view assembly image data of the target tool to obtain the tool body contour corresponding to the target drilling and cutting composite tool. Based on the two axial contour edges extending along the length direction of the tool body contour, straight line fitting is performed. The axial extension direction of the tool body is determined according to the fitting results of the two axial contour edges. The central axis of the tool body is determined according to the middle position between the two axial contour edges. End edge features are extracted in the preset axial end area of ​​the tool body contour, and end edges that satisfy the preset position relationship with the central axis of the tool body are filtered. Straight line or contour fitting is performed on the filtered end edges to determine the reference end face of the tool body. The circumferential reference direction of the target drilling and cutting composite tool is determined according to the relative position relationship between the center position of the circumferential reference structure and the central axis of the tool body. Thus, the tool coordinate reference features including the central axis of the tool body, the reference end face, and the circumferential reference direction are obtained.

[0080] The central axis of the tool body is used to characterize the axial extension direction of the target drilling and cutting composite tool, and serves as the axial reference of the tool assembly verification coordinate system.

[0081] The reference end face is used to determine the axial reference position of the target drilling and cutting composite tool along the central axis of the tool body;

[0082] The circumferential reference structure is used to determine the circumferential zero position of the target drilling and cutting composite tool around the central axis of the tool body;

[0083] The circumferential reference direction is used to characterize the radial reference direction determined by the circumferential zero position.

[0084] By performing the above operations, this solution addresses the technical problem that traditional drilling and cutting composite tool assembly status detection mainly relies on manual observation or single-view images, which is easily affected by metal surface reflection, component occlusion, and changes in tool placement posture, resulting in low accuracy in identifying component assembly anomalies and poor consistency of identification results under different viewpoints. This innovatively proposes an assembly status identification and correction method based on multi-view machine vision, a tool assembly verification coordinate system, and tool assembly structure constraints. By acquiring assembly images of the drilling and cutting composite tool at different rotation angles, a tool assembly verification coordinate system based on the tool body structure is established. Combined with the tool assembly structure constraints, the component areas and initial assembly states under different rotation viewpoints are correlated, verified, and corrected. This improves the accuracy of component area identification and assembly status judgment in complex metal structure environments, enhances adaptability to local occlusion, metal reflection, and missing features from a single viewpoint, and improves the stability and reliability of multi-view assembly status identification results. This achieves automated and reliable identification and correction of the assembly status of each component of the drilling and cutting composite tool.

[0085] Example 4, see Figure 1 , Figure 2 , Figure 3 and Figure 4 This embodiment is based on the above embodiment. In step S3, the tool component assembly state identification is used to identify the component areas corresponding to each assembly component in the target drilling and cutting composite tool, and to determine the initial assembly state of each assembly component; specifically, it includes the following steps:

[0086] Step S31: Construct a tool component assembly state recognition model to establish a joint recognition model for component region localization and assembly state of multi-view assembly images for drilling and cutting composite tools; specifically including the following steps:

[0087] Step S311: Initial feature extraction of the assembly structure, used to perform initial feature mapping on the local edge, contour and texture information in the multi-view assembly image of the tool. While maintaining the original spatial positional relationship of each assembly component, the image pixel information is converted into shallow structural features of a preset dimension, providing basic feature data for subsequent multi-scale assembly feature deep fusion. Specifically, local neighborhood feature mapping is performed on the multi-view assembly image data of the tool through convolution operation, and batch normalization and nonlinear activation processing are performed in sequence to enhance the edge, contour and local texture response of the tool body, drilling component, cutting component and fastening component, so as to obtain the initial feature map of the tool assembly structure.

[0088] The spatial height and width of the initial feature map of the tool assembly structure are completely consistent with the input multi-view assembly image data of the tool, and the number of channels is a preset feature dimension, which is used to carry the shallow structural features and spatial position information of each assembly component of the tool.

[0089] The formula used is as follows:

[0090] ;

[0091] In the formula, This represents the multi-view assembly image of the tool corresponding to the k-th preset rotation angle. This represents the kernel parameters corresponding to shallow convolution operations. This indicates the corresponding bias parameter. This represents the convolution operation. This indicates batch normalization processing. Represents a non-linear activation function. This represents the initial feature diagram of the tool assembly structure.

[0092] Step S312: Multi-scale assembly feature deep fusion is used to perform hierarchical scale transformation, deep assembly relationship feature extraction, and cross-scale feature recovery on the initial feature map of the tool assembly structure. This allows the model to expand the feature perception range and reduce the computational cost of deep features while retaining fine-grained information such as the edges of cutting components, the contours of fastening components, and local mounting structures. This achieves the collaborative representation of the overall tool assembly structure features and local assembly detail features, resulting in a multi-scale assembly fusion deep feature map. Specifically, it includes the following steps:

[0093] Step S3121: Medium-to-high scale assembly feature extraction, used to expand the feature perception range while reducing the spatial resolution of the initial feature map of the tool assembly structure, extracting medium-to-high resolution assembly features corresponding to the tool body contour, drilling component edges, cutting component contours, and local structures of fastening components; specifically, the initial feature map of the tool assembly structure is downsampled, reducing the spatial height and width of the feature map to half of the original spatial size, and the downsampled features are input into the corresponding residual SwinTransformer block. Feature modeling of the correlation between local assembly structures is performed through window self-attention and shifted window feature interaction to obtain medium-to-high scale assembly features; the formula used is as follows:

[0094] ;

[0095] ;

[0096] ;

[0097] In the formula, This represents the mid-to-high-scale assembly features. Its spatial height and width are each half of the initial feature map of the tool assembly structure. It mainly carries the contours, edges, and mid-to-high-scale local structural information of each assembly component. H represents the spatial height of the initial feature map of the tool assembly structure, W represents the spatial width of the initial feature map of the tool assembly structure, and C represents the preset number of feature channels corresponding to the initial feature map of the tool assembly structure and the final multi-scale assembly fusion depth feature map. This represents the number of channels corresponding to mid-to-high-scale assembly features. This represents the downsampling operation applied to the initial feature map of the tool assembly structure. This represents the residual SwinTransformer block operation used to extract mid-to-high-scale assembly-related features;

[0098] Step S3122: Low-scale assembly feature extraction, used to further expand the feature perception range, extracts the overall assembly layout and spatial relationship information between the tool body, drilling components, cutting components, and fastening components while reducing the computational load of deep feature operations; specifically, it downsamples the mid-to-high-scale assembly features, reducing the spatial height and width of the feature map to one-quarter of the initial feature map of the tool assembly structure, and inputs the downsampled features into the corresponding residual SwinTransformer block. Through feature association modeling under a larger perception range, the overall structural relationship between different assembly components is extracted to obtain low-scale assembly features; the formula used is as follows:

[0099] ;

[0100] ;

[0101] In the formula, This represents low-scale assembly features, with its height and width each being one-quarter of the initial feature diagram of the tool assembly structure. It is mainly used to reflect the large-scale spatial layout and overall structural relationships between different assembly components. This represents the number of channels corresponding to the low-scale assembly features. This represents the downsampling operation applied to mid- to high-scale assembly features. This represents the residual SwinTransformer operation used to extract low-scale overall assembly layout features;

[0102] Step S3123: Low-complexity assembly feature fusion, used to further extract deep semantic features while preserving the original structural information of low-scale assembly features, and to suppress the increase in computation caused by feature fusion through channel compression, thus balancing the expressive power of deep features and the computational efficiency of the model; specifically, convolutional operations are used to perform feature mapping on low-scale assembly features to obtain corresponding deep mapping features, then the deep mapping features and low-scale assembly features are concatenated along the channel dimension, and the concatenated features are channel compressed using linear mapping units to obtain low-complexity assembly fusion features; the formula used is as follows:

[0103] ;

[0104] In the formula, This indicates low-complexity assembly and fusion features. This represents a 1×1 convolution operation that performs deep feature mapping on low-scale assembly features. This indicates a feature concatenation operation performed along the feature channel dimension. This represents a linear mapping operation that performs channel compression and dimension adjustment on low-scale splicing features;

[0105] Step S3124: Deep assembly relationship feature extraction, used to further establish long-distance spatial relationships between different assembly components under low spatial resolution conditions, improving the model's ability to represent the overall assembly relationship of drilling, cutting, and fastening components; specifically, the low-complexity assembly fusion features are input into the corresponding residual SwinTransformer block, and deep association modeling of assembly features in different spatial regions is performed through window self-attention and shifted window feature interaction to obtain deep assembly relationship features; the formula used is as follows:

[0106] ;

[0107] In the formula, It represents the characteristics of deep assembly relationships, used to characterize the deep spatial connections and overall assembly structure relationships between the tool body, drilling components, cutting components, and fastening components. This represents the residual SwinTransformer operation used for deep assembly relationship modeling of low-complexity assembly fusion features.

[0108] Step S3125: Mid-to-high scale assembly feature recovery, used to restore deep assembly semantic features to an intermediate spatial scale and fuse the first-scale assembly features retained during the encoding stage to compensate for the loss of cutting component edges, fastening component contours, and local installation structure information caused during continuous downsampling. Specifically, the deep assembly relationship features are upsampled to restore their spatial height and width to half of the initial feature map of the tool assembly structure. Then, mid-to-high scale assembly features are introduced through long skip connections. The upsampled deep features and mid-to-high scale assembly features are concatenated along the channel dimension and channel compression is performed through a linear mapping unit. Finally, the residual SwinTransformer block is input for cross-scale feature association and fusion to obtain the mid-to-high scale recovered assembly features. The formula used is as follows:

[0109] ;

[0110] In the formula, This indicates the reconstructed assembly features at medium to high scales. This represents the upsampling operation applied to features of deep assembly relationships. This represents a linear mapping operation that performs channel compression and dimension alignment on the splicing result of deep assembly relationship features and mid-to-high-scale assembly features. This represents the residual SwinTransformer operation used to fuse the recovered deep features with the mid-to-high-scale local assembly features;

[0111] Step S3126: Original-scale cross-layer feature recovery, used to further restore the assembly features recovered at medium and high scales to the original feature space scale, and to fuse shallow assembly structure features to compensate for fine-grained information such as small-sized fasteners, cutting edges, and installation area boundaries, forming the final multi-scale assembly fusion depth features; specifically, upsampling is performed on the assembly features recovered at medium and high scales to restore the feature map space height and width to the initial features of the tool assembly structure. Figure 1 Then, the upsampled features are concatenated with the initial feature map of the tool assembly structure along the channel dimension through long skip connections. Channel compression is performed through linear mapping units, and convolution operations are used to locally smooth the fused features and adjust the feature dimensions to obtain a multi-scale assembly fusion depth feature map. The formula used is as follows:

[0112] ;

[0113] ;

[0114] In the formula, This represents a multi-scale assembly fusion depth feature map. This represents a 3×3 convolution operation that performs local smoothing, feature remapping, and output dimension adjustment on the final cross-scale fused features. This represents a linear mapping operation that performs channel compression and output dimension adjustment on the stitching result of the mid-to-high-scale restored assembly features and the initial feature map of the tool assembly structure. This represents the upsampling operation applied to the recovery of assembly features at mid-to-high scales;

[0115] Step S313: Adaptive enhancement of global and local assembly features, used to collaboratively enhance the global assembly semantic information and local key structural information of the multi-scale assembly fusion depth feature map. This ensures that the enhanced features can represent the overall assembly relationship between the drilling components, cutting components, fastening components, and the tool body, while also highlighting fine-grained discriminative features such as cutting edges, local structures of fastening components, and reverse assembly directions. It reduces feature response differences caused by metal surface reflection, local occlusion, and different rotational viewpoints, providing highly discriminative assembly enhancement features for subsequent joint discrimination of component regions and assembly states. Specifically, this includes the following steps:

[0116] Step S3131: Global assembly semantic feature extraction, used to extract relatively stable overall structural responses from the multi-scale assembly fusion depth features, to obtain global features that can reflect the overall assembly layout of the tool body, drilling components, cutting components, and fastening components; specifically, the multi-scale assembly fusion depth feature map is divided into average values... Each pooling neighborhood region is averaged and pooled to obtain the global assembly semantic features for that region. These regions are then combined according to their spatial positions in the original feature map to obtain the global assembly semantic feature map. The formula used is as follows:

[0117] ;

[0118] In the formula, This represents the global assembly semantic feature obtained by average pooling the r-th pooling neighborhood region. Indicates the current pooling neighborhood region. Indicates the pooling neighborhood region The number of features contained within, where i represents the index of the feature point within the pooling neighborhood. Represents global assembly semantic features;

[0119] Step S3132: Soft pooling enhancement of local key assembly features is used to highlight high-response local structures in multi-scale assembly fusion depth features, improve the expressive power of fine-grained assembly features such as cutting edge edges, local contours of fastened parts, mounting boundaries, and part orientation, while avoiding the loss of other effective local information caused by max pooling retaining only a single maximum response. Specifically, soft pooling is performed on each pooling neighborhood region, and the corresponding soft pooling weight is calculated based on the response intensity of each feature point in the pooling neighborhood region. Weighted aggregation is then performed based on the soft pooling weights corresponding to each feature point to obtain the local key assembly features corresponding to the r-th pooling neighborhood region. The local key assembly features corresponding to different pooling neighborhood regions are combined according to their spatial location to obtain the local key assembly feature map. The formula used is as follows:

[0120] ;

[0121] ;

[0122] In the formula, This represents the soft pooling weight corresponding to the i-th feature point within r pooling neighborhoods. This represents the activation response value corresponding to the i-th feature point. This represents the activation response value corresponding to the j-th feature point. This represents the local key assembly feature corresponding to the r-th pooling neighborhood region;

[0123] Step S3133: Adaptive fusion of assembly features, used to dynamically determine the contribution of overall assembly structural features and local assembly detail features based on model training results, enabling the feature enhancement process to adapt to feature differences under different types of assembly parts and different rotational viewpoints; specifically, setting a learnable balance parameter, mapping the learnable balance parameter through the Sigmoid function, and obtaining a value range of... The adaptive fusion coefficients of the interval are used to weight and fuse the global assembly semantic feature map and the local key assembly feature map to obtain the assembly discrimination enhancement feature map.

[0124] Step S314: Joint discrimination output of assembly components, used to jointly discriminate the component category, component area, and initial assembly state of each assembly component in the target drilling and cutting composite tool, specifically including the following steps:

[0125] Step S3141: Assembly component region discrimination, used to determine the component categories and spatial regions corresponding to drilling components, cutting components and fastening components in the target drilling and cutting composite tool; specifically, candidate regions are generated from the assembly discrimination enhancement feature map through a region candidate network to obtain each candidate assembly component region, and the region features corresponding to each candidate region are extracted by the region of interest alignment operation, and the component category is predicted by the region features through a fully connected layer and a Softmax classifier. At the same time, the region boundary adjustment parameters corresponding to each candidate assembly component region are determined by the fully connected layer, and the boundaries of each candidate assembly component region are corrected to obtain the component region and component category of each assembly component;

[0126] Step S3142: Assembly state discrimination, used to determine the initial assembly state corresponding to each valid assembly component; specifically, based on the region of each assembly component, the region features of each component region are extracted from the assembly discrimination enhancement feature map using the region of interest alignment operation, and the region features are classified through a fully connected layer and a Softmax classifier to obtain the predicted probability of each assembly component belonging to different preset assembly states, and the assembly state with the highest predicted probability is selected as the initial assembly state of the corresponding assembly component.

[0127] Step S32: Supervised training of the model, specifically, using the reference tool multi-view assembly image data in the tool multi-view assembly image optimization data as the training data of the model, performing iterative training of the recognition model, and obtaining the trained tool component assembly state recognition model.

[0128] The iterative training of the recognition model specifically involves using a joint loss function to optimize the gap between the model's prediction results and the true label distribution, and using a backpropagation algorithm to jointly optimize the recognition model parameters. The loss function value is continuously monitored and optimized until it converges, at which point the model training is stopped, and the trained tool component assembly state recognition model is obtained.

[0129] The formula used is as follows:

[0130] ;

[0131] In the formula, This represents the joint loss function of the tool component assembly state recognition model. The candidate region target loss is used to constrain the ability of the region candidate network to distinguish between the assembly component region and the background region. A binary cross-entropy loss function is employed. This represents the component category loss, used to constrain the difference between the predicted category results and the corresponding true component categories for drilling, cutting, and fastening components. A multi-class cross-entropy loss function is employed. The assembly state classification loss is used to constrain the difference between the predicted initial assembly state of each assembly component and the corresponding actual assembly state. A multi-class cross-entropy loss function is employed. The component region regression loss is used to constrain the positional deviation between the predicted component region and the actual labeled region, and the SmoothL1 loss function is adopted.

[0132] Step S33: Target tool component area and assembly state recognition, which is used to locate the region and determine the state of the assembly components of the target drilling and cutting composite tool under different rotation angles using the trained tool component assembly state recognition model; specifically, the target tool multi-view assembly image data in the tool multi-view assembly image optimization data is input into the trained tool component assembly state recognition model to obtain the component category, component area and initial assembly state of each assembly component.

[0133] Step S34: Obtain the tool component assembly status recognition result. Specifically, according to the component category corresponding to each assembly component, the component category, component area and initial assembly status are combined accordingly to obtain the recognition result of each assembly component of the target tool.

[0134] By performing the above operations, this solution addresses the technical problems of existing tool component assembly state recognition models, which struggle to simultaneously consider overall assembly structure information and local detail features. Furthermore, these models are prone to feature loss in small-sized assembly components, inaccurate component region positioning, and insufficient stability in identifying subtle assembly states under conditions such as metal surface reflection, partial occlusion, and different rotational viewing angles. This solution proposes a tool component assembly state recognition method based on multi-scale assembly feature deep fusion and adaptive enhancement of global and local assembly features. Through hierarchical downsampling, residual SwinTransformer feature modeling, cross-scale long-skip connections, and upsampling recovery, multi-scale fusion of the overall tool assembly relationship and local structural details is achieved. Combined with average pooling, soft pooling, and learnable fusion weights, the contribution of global assembly semantic features and local key assembly features is adaptively balanced, improving the feature representation and region positioning capabilities of assembly components at different scales. This enhances the model's ability to discriminate key details such as cutting edges, fastening structures, and component orientation, as well as its adaptability to reflection, occlusion, and viewing angle changes. Ultimately, this improves the accuracy, stability, and robustness of assembly component region recognition and assembly state discrimination, achieving reliable joint recognition of assembly component region positioning and assembly state for drilling and cutting composite tools.

[0135] Example 5, see Figure 1This embodiment is based on the above embodiment. In step S4, the tool component assembly state recognition result correction is used to perform consistency verification and correction on the component area and initial assembly state by utilizing the recognition information of the same assembly component under different rotation angles and the structural constraint relationship between the tool assembly components. This reduces misidentification caused by metal surface reflection, local occlusion, and missing features from a single viewpoint, thereby improving the accuracy and stability of the tool component assembly state recognition result. Specifically, based on the preset rotation angle corresponding to the multi-view assembly image of each target tool and the tool assembly verification coordinate system, spatial correspondence is performed on the assembly components identified under different rotation angles. The association relationship of the recognition results of the same assembly component under different rotation angles is established. The tool assembly structure constraint relationship is set, and the identification results of the corresponding assembly parts under each rotation view are constrained and verified. When the part area or initial assembly state under the target rotation view does not meet the tool assembly structure constraint relationship, and there are associated identification results that meet the tool assembly structure constraint relationship under other rotation views, the associated identification results are used as the correction basis to correct the part area and initial assembly state under the target rotation view, and the corrected part area and corrected assembly state corresponding to each assembly part are obtained. According to the preset rotation angle corresponding to the multi-view assembly image of each target tool, the corrected part area and corrected assembly state of each assembly part under different rotation views are associated and integrated to obtain the tool part assembly state correction result.

[0136] The tool assembly structure constraint relationships include the axial position constraint relationship between the drilling component and the tool body, used to verify the assembly relationship of the drilling component relative to the axial reference of the tool body; the position matching constraint relationship between the cutting component and the corresponding installation area, used to verify the spatial correspondence between the cutting component and the preset installation area; and the assembly association constraint relationship between the fastening component and the corresponding fastened component, used to verify the connection position relationship between the fastening component and the corresponding assembly component.

[0137] Example 6, see Figure 1 and Figure 5 This embodiment is based on the above embodiment. In step S5, the tool assembly comprehensive verification is used to determine the actual assembly pose of each assembly component, calculate the assembly deviation of the actual assembly pose relative to the preset standard assembly pose, and make a comprehensive judgment in combination with the actual working envelope of the target drilling and cutting composite tool under rotation working state to obtain the tool assembly comprehensive verification result; specifically, it includes the following steps:

[0138] Step S51: Determine the actual pose of the assembly components, which is used to determine the actual assembly position and actual assembly direction of each assembly component in the target drilling and cutting composite tool relative to the tool body; specifically, based on the correction component area of ​​each assembly component in the tool component assembly state correction result, and combined with the preset rotation angle and tool assembly verification coordinate system corresponding to the multi-view assembly image of each target tool, establish the spatial correspondence between the correction component areas of the same assembly component under different rotation views, perform coordinate transformation and spatial pose calculation on the correction component areas under each rotation view, determine the actual position parameters and actual direction parameters of each assembly component in the tool assembly verification coordinate system, and obtain the actual assembly pose corresponding to each assembly component;

[0139] Step S52: Assembly deviation calculation, used to quantify the difference between the actual assembly pose and the standard assembly pose of each assembly component, and to determine the positional and directional offsets of each assembly component relative to the preset assembly datum; specifically, to obtain the standard position parameters and standard direction parameters of each assembly component of the target drilling and cutting composite tool in the tool assembly verification coordinate system, to compare the actual position parameters and actual direction parameters of each assembly component with the corresponding standard position parameters and standard direction parameters, and to calculate the axial position deviation, radial position deviation, directional angle deviation and circumferential phase deviation of each assembly component respectively, so as to obtain the assembly deviation parameters corresponding to each assembly component;

[0140] The axial position deviation is used to indicate the degree of offset of the assembled component relative to the standard installation position along the central axis of the tool body;

[0141] The radial position deviation is used to indicate the degree of offset between the radial mounting position of the assembly component and the standard radial position relative to the central axis of the tool body.

[0142] The orientation angle deviation is used to represent the angular difference between the actual installation direction of the assembled component and the standard installation direction;

[0143] The circumferential phase deviation is used to represent the angular difference between the actual circumferential installation position of the assembly component around the central axis of the tool body and the standard circumferential installation position.

[0144] Step S53: Determine the actual working envelope of the tool, which is used to characterize the spatial geometric range of action formed by the rotation of the target drilling and cutting composite tool around the central axis of the tool body in the actual assembly state, and evaluate the influence of the actual assembly deviation of each assembly component on the spatial geometric range of action; specifically, based on the actual assembly pose of each assembly component and the central axis of the tool body, construct the actual working envelope corresponding to the target drilling and cutting composite tool, and compare the actual working envelope with the preset standard working envelope in space to obtain the tool working envelope deviation;

[0145] The construction of the actual working envelope is specifically as follows: based on the actual position and direction parameters of the drilling and cutting components, with the central axis of the tool body as the rotation axis, each working component is rotated and mapped according to the working rotation direction of the target drilling and cutting composite tool to obtain the spatial sweep area formed by each working component in one complete rotation cycle. The spatial sweep areas corresponding to the drilling and cutting components are merged, and the outer boundary of the merged spatial sweep area is used as the actual working envelope of the target drilling and cutting composite tool.

[0146] The determination of the tool working envelope deviation specifically involves unifying the actual working envelope and the standard working envelope into the tool assembly and verification coordinate system, determining the maximum range of action of the actual working envelope and the preset standard working envelope along the radial direction of the tool body and the range of action along the axial direction of the tool body, determining the radial envelope deviation based on the difference between the corresponding radial ranges, determining the axial envelope deviation based on the difference between the corresponding axial ranges, and using the radial envelope deviation and the axial envelope deviation as the tool working envelope deviation of the target drilling and cutting composite tool.

[0147] Step S54: Comprehensive verification, which combines the corrected assembly status of each assembly component, assembly deviation parameters, and tool working envelope deviation to comprehensively judge the assembly quality of the target drilling and cutting composite tool and obtain the tool assembly comprehensive verification result; specifically, based on the tool component assembly status correction result, the assembly deviation parameters corresponding to each assembly component, and the tool working envelope deviation, it is compared with the corresponding preset verification conditions, and a comprehensive judgment is made according to the comparison results to determine the qualified assembly status, abnormal assembly components, and corresponding assembly abnormality types of the target drilling and cutting composite tool, and obtain the tool assembly comprehensive verification result, thereby realizing the joint verification of the assembly status, assembly position deviation, and rotation working envelope of the target drilling and cutting composite tool;

[0148] The comprehensive comparison is used to determine whether the corrected assembly state of each assembly component meets the preset normal assembly requirements, whether the deviation parameters of each assembly component are within the corresponding allowable deviation range, and whether the tool working envelope deviation is within the preset allowable envelope range. Specifically, when the corrected assembly state of each assembly component meets the preset normal assembly requirements, all assembly deviation parameters are within the corresponding allowable deviation range, and the tool working envelope deviation is within the preset allowable envelope range, the target drilling and cutting composite tool is determined to be qualified for assembly. Otherwise, based on the assembly components that do not meet the preset verification conditions and the corresponding assembly anomaly type, the assembly anomaly type is determined, and the tool assembly comprehensive verification result is obtained. The preset parameters are determined in advance based on the design drawings, three-dimensional design model, assembly process requirements, and assembly quality acceptance requirements of the target drilling and cutting composite tool.

[0149] The standard assembly pose is determined based on the designed installation position and direction of each assembly component, and is uniformly converted to the tool assembly verification coordinate system;

[0150] The preset standard working envelope is constructed by rotating and sweeping around the central axis of the tool body, based on the standard assembly posture and standard structural dimensions of the drilling and cutting components.

[0151] The allowable range of the assembly deviation parameters is determined based on the design tolerances and assembly process tolerances of each assembly component;

[0152] The preset envelope allowable range is determined based on the design working dimensions and corresponding allowable tolerances of the target drilling and cutting composite tool;

[0153] The assembly anomaly types may include at least one of the following: missing parts, incorrect parts, reversed parts, out-of-tolerance axial position, out-of-tolerance radial position, out-of-tolerance directional angle, out-of-tolerance circumferential phase, out-of-tolerance radial working envelope, and out-of-tolerance axial working envelope.

[0154] By performing the above operations, this solution addresses the technical problem that traditional drilling and cutting composite tool assembly verification mainly relies on the assembly status of individual components or single dimensional parameters for qualification judgment. This makes it difficult to quantify actual assembly deviations and evaluate the combined effect of multiple assembly deviations on the tool's rotational working state, leading to inaccurate positioning of assembly anomalies and insufficient correlation between test results and the actual working state of the tool. This innovatively proposes a comprehensive verification method based on the actual pose of the assembly components, assembly deviation parameters, and the actual working envelope. By determining the actual assembly pose of each assembly component in a unified tool assembly verification coordinate system, calculating the axis assembly deviation parameters, and constructing the actual working envelope of the tool under rotational state based on the actual assembly poses of the drilling and cutting components, the actual working envelope is compared with a preset standard working envelope. This enhances the correlation evaluation capability between assembly geometric deviations and the actual working range of the tool, improves the rationality and accuracy of the overall assembly quality judgment of the drilling and cutting composite tool, and realizes a comprehensive verification from assembly status identification to the quantification of assembly deviations and joint evaluation of the actual working state.

[0155] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0156] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

[0157] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A machine vision-based method for assembling and verifying drilling and cutting composite tools, characterized in that: The method includes the following steps: Step S1: Obtain multi-view assembly images of the tool, specifically by image acquisition and image preprocessing operations to obtain optimized data of multi-view assembly images of the tool; Step S2: Establish the tool assembly verification coordinate system. Specifically, based on the multi-view assembly image of the target tool, identify the tool body central axis, reference end face and circumferential reference structure. Take the intersection of the tool body central axis and the reference end face as the coordinate origin, and determine the axial coordinate axis and radial coordinate axis according to the tool body central axis and circumferential reference structure to obtain the tool assembly verification coordinate system. Step S3: Tool component assembly state recognition. Specifically, this involves constructing a tool component assembly state recognition model that includes initial feature extraction of assembly structure, deep fusion of multi-scale assembly features, adaptive enhancement of global and local assembly features, and joint discrimination of assembly components. Then, supervised training of the model and recognition of the target tool component region and assembly state are performed to obtain the component category, component region, and initial assembly state of each assembly component. Finally, the corresponding combinations are performed to obtain the recognition results of each assembly component of the target tool. Step S4: Correction of tool component assembly status recognition results. Specifically, based on the tool assembly verification coordinate system, establish the correlation between the recognition results of the same assembly component under different rotation angles, and combine the tool assembly structure constraint relationship to perform constraint verification on the recognition results of the corresponding assembly components under each rotation angle to obtain the tool component assembly status correction results. Step S5: Comprehensive verification of tool assembly. Specifically, based on the tool component assembly state correction results and the tool assembly verification coordinate system, the actual pose of the assembly components is determined, the assembly deviation is calculated, and the actual working envelope of the tool is obtained in sequence. The actual assembly pose, assembly deviation parameters, and tool working envelope deviation of each assembly component are obtained respectively. Finally, a comprehensive verification is performed to obtain the comprehensive verification result of the tool assembly.

2. The machine vision-based drilling and cutting composite tool assembly and verification method according to claim 1, characterized in that: In step S2, the establishment of the tool assembly verification coordinate system specifically involves identifying the tool body reference features of the target drilling and cutting composite tool based on the multi-view assembly image data of the target tool in the multi-view assembly image optimization data. This identifies the tool body contour, tool body central axis, reference end face, and circumferential reference structure. The circumferential reference direction is determined based on the position of the circumferential reference structure relative to the tool body central axis, thus obtaining the tool coordinate reference features. Finally, the tool assembly verification coordinate system is established based on the tool coordinate reference features. The intersection of the tool body central axis and the reference end face is taken as the origin, the direction along the tool body central axis is taken as the axial coordinate axis, and the projection direction of the circumferential reference direction in the plane perpendicular to the tool body central axis is taken as the first radial coordinate axis. The second radial coordinate axis is determined according to the right-hand coordinate system rule, thus obtaining the tool assembly verification coordinate system corresponding to the target drilling and cutting composite tool.

3. The machine vision-based drilling and cutting composite tool assembly and verification method according to claim 2, characterized in that: In step S3, the identification of the assembly status of the tool component specifically includes the following steps: Step S31: Construct a tool component assembly state recognition model, which specifically includes the following steps: Step S311: Initial feature extraction of the assembly structure, specifically, the tool multi-view assembly image data is mapped by convolution operation, and batch normalization and nonlinear activation are performed in sequence to obtain the initial feature map of the tool assembly structure. Step S312: Deep fusion of multi-scale assembly features; Step S313: Adaptive enhancement of global and local assembly features; Step S314: Joint discrimination output of assembled components, specifically including the following steps: Step S3141: Assembly component region discrimination, specifically, the assembly discrimination enhancement feature map is processed by a region candidate network to generate candidate regions, and each candidate assembly component region is obtained. The region features corresponding to each candidate region are extracted by the region of interest alignment operation. The region features are used to predict the component category by the region category through a fully connected layer and a Softmax classifier. At the same time, the region boundary adjustment parameters corresponding to each candidate assembly component region are determined by the fully connected layer, and the boundaries of each candidate assembly component region are corrected to obtain the component region and component category of each assembly component. Step S3142: Assembly state discrimination, specifically, based on the regions of each assembly component, the region features of each component region are extracted from the assembly discrimination enhancement feature map using the region of interest alignment operation, and the region features are classified through a fully connected layer and a Softmax classifier to obtain the predicted probability of each assembly component belonging to different preset assembly states, and the assembly state with the highest predicted probability is selected as the initial assembly state of the corresponding assembly component. Step S32: Supervised training of the model, specifically, using the reference tool multi-view assembly image data in the tool multi-view assembly image optimization data as the training data of the model, performing iterative training of the recognition model, and obtaining the trained tool component assembly state recognition model. Step S33: Target tool component area and assembly state recognition, specifically, inputting the target tool multi-view assembly image data from the optimized multi-view assembly image data of the tool into the trained tool component assembly state recognition model to obtain the component category, component area and initial assembly state corresponding to each assembly component; Step S34: Obtain the tool component assembly status recognition result. Specifically, according to the component category corresponding to each assembly component, the component category, component area and initial assembly status are combined accordingly to obtain the recognition result of each assembly component of the target tool.

4. The machine vision-based drilling and cutting composite tool assembly and verification method according to claim 3, characterized in that: The deep fusion of multi-scale assembly features specifically includes the following steps: Step S3121: Medium and high scale assembly feature extraction, specifically, the initial feature map of the tool assembly structure is downsampled and the downsampled features are input into the corresponding residual SwinTransformer block to obtain medium and high scale assembly features; Step S3122: Low-scale assembly feature extraction, specifically, downsampling the mid-to-high-scale assembly features and inputting the downsampled features into the corresponding residual SwinTransformer block to obtain low-scale assembly features; Step S3123: Low-complexity assembly feature fusion, specifically, using convolution operation to perform feature mapping on low-scale assembly features to obtain corresponding deep mapping features, then concatenating the deep mapping features and low-scale assembly features along the channel dimension, and using linear mapping units to perform channel compression on the concatenated features to obtain low-complexity assembly fusion features. Step S3124: Deep assembly relationship feature extraction, specifically, inputting the low-complexity assembly fusion features into the corresponding residual SwinTransformer block to obtain deep assembly relationship features; Step S3125: Medium- and high-scale assembly feature recovery, specifically, upsampling the deep assembly relationship features, then introducing medium- and high-scale assembly features through long skip connections, concatenating the upsampled deep features and medium- and high-scale assembly features along the channel dimension, and performing channel compression through a linear mapping unit. Finally, the residual SwinTransformer block is input for cross-scale feature association and fusion to obtain the medium- and high-scale recovered assembly features. Step S3126: Original scale cross-layer feature recovery, specifically, upsampling is performed on the recovered assembly features at medium and high scales, and then the upsampled features are concatenated with the initial feature map of the tool assembly structure along the channel dimension through long skip connections. Channel compression is performed through linear mapping units, and convolution operation is used to perform local smoothing and feature dimension adjustment on the fused features to obtain a multi-scale assembly fusion depth feature map.

5. The machine vision-based drilling and cutting composite tool assembly and verification method according to claim 3, characterized in that: The adaptive enhancement of global and local assembly features specifically includes the following steps: Step S3131: Global assembly semantic feature extraction, specifically, dividing the multi-scale assembly fusion deep feature map into average values. Each pooling neighborhood region is averaged and pooled to obtain the global assembly semantic features for that region. These regions are then combined according to their spatial positions in the original feature map to obtain the global assembly semantic feature map. The formula used is as follows: ; In the formula, This represents the global assembly semantic feature obtained by average pooling the r-th pooling neighborhood region. Indicates the current pooling neighborhood region. Indicates the pooling neighborhood region The number of features contained within, where i represents the index of the feature point within the pooling neighborhood. Represents global assembly semantic features; Step S3132: Local key assembly feature soft pooling enhancement, specifically, performing soft pooling processing on each pooling neighborhood region, calculating the corresponding soft pooling weight based on the response intensity of each feature point in the pooling neighborhood region, performing weighted aggregation based on the soft pooling weights corresponding to each feature point to obtain the local key assembly feature corresponding to the r-th pooling neighborhood region, and combining the local key assembly features corresponding to different pooling neighborhood regions according to their spatial location to obtain the local key assembly feature map; Step S3133: Adaptive fusion of assembly features, specifically by setting learnable balance parameters and mapping these parameters using the Sigmoid function to obtain a value range of [value range missing]. The adaptive fusion coefficients of the interval are used to weight and fuse the global assembly semantic feature map and the local key assembly feature map to obtain the assembly discrimination enhancement feature map.

6. The machine vision-based drilling and cutting composite tool assembly and verification method according to claim 1, characterized in that: In step S4, the tool component assembly state recognition result correction specifically involves spatially corresponding the assembly components identified under different rotation angles based on the preset rotation angles corresponding to the multi-view assembly images of each target tool and the tool assembly verification coordinate system. This establishes an association relationship between the recognition results of the same assembly component under different rotation angles. Based on the preset tool assembly structure constraint relationship, the recognition results of the corresponding assembly components under each rotation angle are constrained and verified. When the component area or initial assembly state under the target rotation angle does not satisfy the tool assembly structure constraint relationship, and there are associated recognition results that satisfy the tool assembly structure constraint relationship under other rotation angles, the associated recognition results are used as the correction basis to correct the component area and initial assembly state under the target rotation angle, resulting in the corrected component area and corrected assembly state corresponding to each assembly component. Finally, according to the preset rotation angles corresponding to the multi-view assembly images of each target tool, the corrected component areas and corrected assembly states of each assembly component under different rotation angles are associated and integrated to obtain the tool component assembly state correction result.

7. The machine vision-based drilling and cutting composite tool assembly and verification method according to claim 1, characterized in that: In step S5, the comprehensive verification of the tool assembly specifically includes the following steps: Step S51: Determine the actual pose of the assembly component. Specifically, based on the correction component area of ​​each assembly component in the tool component assembly state correction result, and combined with the preset rotation angle and tool assembly verification coordinate system corresponding to the multi-view assembly image of each target tool, establish the spatial correspondence between the correction component areas of the same assembly component under different rotation views, perform coordinate transformation and spatial pose calculation on the correction component areas under each rotation view, determine the actual position parameters and actual direction parameters of each assembly component in the tool assembly verification coordinate system, and obtain the actual assembly pose corresponding to each assembly component. Step S52: Assembly deviation calculation, specifically, to obtain the standard position parameters and standard direction parameters of each assembly component of the target drilling and cutting composite tool in the tool assembly verification coordinate system, compare the actual position parameters and actual direction parameters of each assembly component with the corresponding standard position parameters and standard direction parameters, and calculate the axial position deviation, radial position deviation, direction angle deviation and circumferential phase deviation of each assembly component to obtain the assembly deviation parameters corresponding to each assembly component. Step S53: Determine the actual working envelope of the tool. Specifically, based on the actual assembly posture of each assembly component and the central axis of the tool body, construct the actual working envelope corresponding to the target drilling and cutting composite tool, and compare the actual working envelope with the preset standard working envelope in space to obtain the tool working envelope deviation. Step S54: Comprehensive verification, specifically, based on the tool component assembly status correction results, the assembly deviation parameters corresponding to each assembly component, and the tool working envelope deviation, compare them with the corresponding preset verification conditions, and make a comprehensive judgment based on the comparison results to determine the qualified assembly status, abnormal assembly components, and corresponding assembly abnormality types of the target drilling and cutting composite tool, and obtain the tool assembly comprehensive verification result, thereby realizing the joint verification of the assembly status, assembly position deviation, and rotation working envelope of the target drilling and cutting composite tool.

8. The method for assembling and verifying a drilling and cutting composite tool based on machine vision according to claim 1, characterized in that: In step S1, the acquisition of multi-view assembly images of the cutting tool specifically involves fixing the drilling and cutting composite tool on a controllable rotating mechanism, controlling the drilling and cutting composite tool to rotate sequentially around the central axis of the tool body at a preset rotation angle, and acquiring images at each preset rotation angle using an industrial camera. Simultaneously, the rotation angle information corresponding to each acquired image is recorded to obtain the original data of the multi-view assembly images of the cutting tool. Image preprocessing is then performed on the original data of the multi-view assembly images of the cutting tool to obtain optimized data of the multi-view assembly images of the cutting tool. The raw data of the multi-view assembly image of the tool includes multi-view assembly image data of the reference tool and multi-view assembly image data of the target tool. The image preprocessing operations specifically include image denoising and image enhancement.