Double-path mixed cutting assembly connecting piece target detection method and system

Abstract

The invention relates to a double-path mixed cutting assembly connecting piece target detection method and system, and belongs to the technical field of machine vision detection and artificial intelligence. The method comprises the steps that a to-be-detected assembly image in an industrial environment is acquired; the assembly image is input into a target detection model, the model adopts a Mamba network to extract long-range dependence features, and the position, the category and the confidence coefficient of a connecting piece are output through a YOLOv5 detection head; reconstructing training data based on a dual-path self-adaptive hybrid cutting strategy: cutting and expanding a target domain high-confidence connector area to a source domain, and performing reverse cutting and mixing to avoid label shielding; aligning distribution of a source domain and a target domain through a feature discriminator, and optimizing the model in combination with a joint loss function; and finally, generating a detection report according to confidence coefficient screening detection results. According to the method, the problems of insufficient small target feature extraction and cross-domain data imbalance are solved, and the assembly scene connector detection precision is improved.

Description

Technical Field

[0001] This application relates to the technical fields of machine vision detection and artificial intelligence, and more specifically, to a method and system for target detection of assembly connectors with dual-path hybrid cropping. Background Art

[0002] Automated assembly has become the mainstream of industrial development. However, for complex assemblies (such as engines, automotive braking systems, etc.), some key processes still require manual operation. Complex assemblies have characteristics such as a large number and variety of connectors, and problems such as missing or misassembling of connectors caused by human factors often occur. There are the following technical bottlenecks in the method for target detection of assembly connectors: Limitations of local features: Detection methods based on CNN are limited by the receptive field of the convolution kernel and are difficult to capture the global context information of small targets (such as bolts and gaskets), resulting in a high missed detection rate in complex assembly scenarios.

[0003] Annotation dependence and insufficient generalization: Supervised learning requires a large amount of manual annotation and is difficult to adapt to small-batch production of multiple varieties; while unsupervised domain adaptation methods in the migration process: Are sensitive to feature changes of small targets, and background noise easily interferes with feature alignment; Traditional hybrid strategies (such as CutMix) cause mutual occlusion of source domain / target domain labels and damage the integrity of labels.

[0004] Imbalance between computational efficiency and accuracy: Although Transformer can model long-range dependencies, it has a large number of parameters and is difficult to meet the requirements of real-time detection.

[0005] The prior art, such as the Chinese patent application with the publication number CN118968019A, discloses a method for small target detection based on Mamba feature fusion. This method proposes a Mamba feature fusion architecture (Transformer-Mamba hybrid encoder) and captures global context through the SS2D module. However, this method does not solve the problem of label occlusion during cross-domain data mixing; and its STIoU loss function is designed for general small targets and does not optimize the deformation robustness of rigid connectors in the assembly scenario. Summary of the Invention

[0006] To solve the above technical problems, the present invention proposes a method and system for target detection of assembly connectors with dual-path hybrid cropping.

[0007] The technical solution of the present invention is as follows: The present invention proposes a method for target detection of assembly connectors with dual-path hybrid cropping, including the following steps: Obtain an image of the assembly to be detected in the industrial environment; Input the assembly image to be detected into the target detection model; the target detection model includes a feature extraction module and a feature discriminator module. The feature extraction module uses the Mamba network to extract the long-range dependence features of the input data and outputs the detection results through the YOLOv5 detection head, including the position, category, and confidence of the connecting parts; train the target detection model through a dual-path adaptive hybrid cropping strategy; Among them, the dual-path adaptive hybrid cropping strategy reconstructs the training images input to the target detection model based on the image data in the source domain data and the target domain data; the source domain data is the 3D model image of the assembly simulating the real environment and the automatic annotation labels; the target domain data is the real-shot assembly image and the connecting part labels are manually annotated; Screen the detection results according to the confidence output by the trained target detection model to generate a detection report for the assembly connecting parts.

[0008] Preferably, the process of training the target detection model through the dual-path adaptive hybrid cropping strategy is as follows: Input the source domain dataset and the target domain dataset; Perform comprehensive data augmentation on the source domain and target domain data respectively to generate enhanced images; Input the enhanced source domain and target domain images into the feature extraction module with shared parameters to extract the source domain features and the target domain features respectively; input the source domain features and the target domain features into the feature discriminator module, and align the two-domain feature distributions through adversarial training, and at the same time calculate the supervised loss of the source domain features; Generate mixed training data through the dual-path adaptive hybrid cropping strategy, including: Crop and expand the connecting part regions with confidence higher than the threshold in the target domain image and mix them into the source domain image to obtain the first mixed image; Crop and expand the labeled connecting part regions of the source domain image and mix them into the target domain image to obtain the second mixed image; After inputting the first mixed image and the second mixed image into the feature extraction module to extract features, send them to the feature discriminator module for feature distribution alignment, and calculate the supervised loss of the first mixed image; Iteratively optimize the target detection model, update the parameters jointly with the supervised loss and the discriminator loss until the target detection model converges.

[0009] Preferably, the comprehensive data augmentation includes: rotation, scaling, brightness and contrast adjustment, affine transformation, elastic transformation, sharpening, and blurring.

[0010] Preferably, the stop conditions for cropping area expansion include: Expand to the edge of the source domain image or the target domain image; Expand to other connecting part labels in the source domain image or the target domain image; The cropped source domain image or target domain image and the existing connector annotation box obscure each other.

[0011] Preferably, the supervised loss includes classification loss, localization loss, and confidence loss. The classification loss is calculated by cross-entropy, the localization loss is calculated by the intersection over union of the predicted box and the ground truth box, and the confidence loss is a binary classification loss.

[0012] Preferably, the discriminator loss includes: the feature distribution alignment loss between the source domain and the target domain, and the feature consistency loss between the first mixed image and the second mixed image.

[0013] Preferably, the Mamba network uses a state space model (SSM) for multi-level feature extraction. The extracted multi-level features are fused by a path aggregation feature pyramid network (PAFPN). The fused features are upsampled by a YOLOv5 detection head to output detection results.

[0014] On the other hand, the present invention also provides an assembly connector target detection system with dual-path hybrid cropping, including: A data acquisition module that acquires an assembly image to be detected in an industrial environment; A detection and recognition module that inputs the assembly image to be detected into a target detection model. The target detection model includes a feature extraction module and a feature discriminator module. The feature extraction module uses a Mamba network to extract long-range dependence features of the input data and outputs detection results including the position, category, and confidence of the connector through a YOLOv5 detection head. The target detection model is trained by a dual-path adaptive hybrid cropping strategy; Among them, the dual-path adaptive hybrid cropping strategy reconstructs the training images input to the target detection model based on the image data in the source domain data and the target domain data. The source domain data is the three-dimensional model image of the assembly simulating the real environment and the automatically annotated labels. The target domain data is the real-shot assembly image with manually annotated connector labels; A result output module that screens the detection results according to the confidence output by the trained target detection model and generates a detection report of the assembly connector.

[0015] The present invention has the following beneficial effects: (1) An adaptive data hybrid cropping strategy is proposed, enabling the cropped images to be adaptively mixed into the target images, ensuring the label integrity of the mixed images, enhancing the data feature extraction ability, and solving the problem of mutual occlusion of source data and target data labels caused by the traditional cropping and mixing strategy cutmix.

[0016] (2) A dual-path cropping and mixing strategy for source data and target data is proposed. This strategy aims to crop and mix the two types of data with each other and carry out training under the joint constraint of soft and hard losses, so as to enhance the feature extraction ability of the domain adaptation framework for physical data, solve the problem of uneven quantities of the two types of data, and improve the accuracy of target detection data.

[0017] (3) The domain adaptation framework of the present invention uses Mamba+YOLOv5 as the feature extraction module, and integrates a data augmentation method that combines set transformation, embossing, and enhanced parameter diversification, reducing the visual distribution difference between target data and source data and improving the accuracy of target data detection.

[0018] (4) The present invention adopts data augmentation techniques such as elastic transformation and affine augmentation to bridge the domain difference between virtual rendered images and real-shot assembly images, improving the adaptability to industrial scenarios; experiments show that on the engine assembly dataset, mAP50 reaches 80.4%, which is 4.3% higher than the optimal comparison method ConfMix. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the algorithm flow of the domain adaptation assembly connector target detection based on the adaptive hybrid cropping strategy of the present invention; Figure 2 It is a schematic diagram of the dual-path adaptive hybrid cropping strategy process of the present invention; Figure 3 It is a structural diagram of the feature extraction module of the assembly connector of the present invention; Figure 4 It is a schematic diagram of the dataset production process of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0021] It should be understood that the step numbers used in the text are only for convenient description and do not limit the execution order of the steps.

[0022] It should be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless otherwise clearly specified in the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0023] The terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0024] The term "and / or" refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0025] Example 1: To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions of the present invention in conjunction with specific embodiments of the present application and with reference to the accompanying drawings.

[0026] To solve the problems of the prior art, the present invention provides a method for target detection of an assembly connector with dual-path hybrid cropping, including the following steps: Obtain an assembly image to be detected in an industrial environment; Input the assembly image to be detected into a target detection model; the target detection model is trained by a dual-path adaptive hybrid cropping strategy; the target detection model includes a feature extraction module and a feature discriminator module, wherein the feature extraction module uses a Mamba network to extract long-range dependence features of the input data and outputs detection results through a YOLOv5 detection head, including the position, category, and confidence of the connector; Screen the detection results according to the confidence to generate a detection report of the assembly connector; Among them, the dual-path adaptive hybrid cropping strategy reconstructs the training images input to the target detection model based on the image data in the source domain data and the target domain data; the source domain data is the three-dimensional model image of the assembly simulating the real environment and the automatic annotation labels; the target domain data is the actual photographed assembly image and the manually annotated connector labels.

[0027] This embodiment takes the identification of the engine assembly connector as an example, such as Figure 1As shown in the figure, the training input data of the object detection model consists of source domain data, target domain data, and mixed data. First, the source domain data is sent to the feature extraction module (Mamba + YOLOv5) after comprehensive data augmentation processing, and supervised loss calculation is performed using source domain object testing to output the training result, obtaining the source test output. After that, the target domain data is sent to the feature extraction module (Mamba + YOLOv5) after comprehensive data augmentation to learn the data features. At the same time, the discriminator loss (gradient reversal layer + discriminator module) is used to ensure that the target data can extract features similar to the source data, obtaining the target test output. Finally, the source domain object detection labels (labels) and the target test output generate the first mixed image, the second mixed image, and the mixed label through the dual-path adaptive hybrid cropping strategy, and the two mixed images are sent to the feature extraction module (Mamba + YOLOv5) again. The test result (mixed test output 1) generated by the first mixed image and the pseudo-label (mixed label) are used for supervised loss calculation, and the discriminator loss (gradient reversal layer + discriminator module) ensures that the images generated by the two mixing methods can extract similar features.

[0028] As a preferred implementation manner of this embodiment, the process of training the object detection model through the dual-path adaptive hybrid cropping strategy is as follows: Input the source domain data set and the target domain data set; as Figure 4 shown, the construction process of the source domain data set is as follows: According to the engine size in the real scenario, establish the corresponding three-dimensional engine model; Establish the world coordinate system to ensure that its coordinate origin coincides with the geometric center of the three-dimensional engine model; Render the engine model and label the color of the connecting parts, and output the three-dimensional engine model simulating the real environment and the three-dimensional engine model after labeling the connecting parts; Centered on the coordinate origin, rotate the rendered engine model and the connecting part-labeled engine model respectively, output the rendered video and the labeled video, and then further process the connecting part-labeled video to convert it into the object detection label video; Intercept the two types of videos frame by frame to generate the synthetic engine assembly image and the label image.

[0029] The target domain data set mimics the production steps of the source domain data set. First, rotate and shoot the engine at the specified shooting angle to output the rotation videos at each angle; then intercept the video frame by frame to output the physical engine assembly image; finally, manually label the connecting parts in the engine image using the object detection label annotation software.

[0030] In this embodiment, the target domain dataset contains a total of 3,222 physical engine assembly images, among which 946 are in the validation set, 276 are in the test set, and 2,000 are in the training set. The source domain dataset contains a total of 21,340 assembly images and labels. The resolution is 1024 1024.

[0031] Perform comprehensive data augmentation on the source domain and target domain data respectively to generate augmented images; As a preferred implementation manner of this embodiment, the comprehensive data augmentation includes: rotation, scaling, brightness and contrast adjustment, affine transformation, elastic transformation, sharpening, and blurring processing. The present invention adds image processing techniques such as affine transformation, elastic transformation, sharpening, and blurring on the basis of the original ordinary data augmentation (rotation, scaling, brightness and contrast adjustment, etc.) to better bridge the visual differences between the source data and the target data.

[0032] Input the augmented source domain and target domain images into the feature extraction module with shared parameters to extract source domain features and target domain features respectively; input the source domain features and target domain features into the feature discriminator module, and align the feature distributions of the two domains through adversarial training, and at the same time calculate the supervised loss of the source domain features; Generate mixed training data through a dual-path adaptive hybrid cropping strategy, including: Crop and expand the connector regions in the target domain images with confidence higher than the threshold and mix them into the source domain images to obtain the first mixed image; Crop and expand the labeled connector regions of the source domain images and mix them into the target domain images to obtain the second mixed image; After inputting the first mixed image and the second mixed image into the feature extraction module to extract features, send them into the feature discriminator module for feature distribution alignment, and calculate the supervised loss of the first mixed image; Iteratively optimize the object detection model, jointly update the parameters with the supervised loss and the discriminator loss until the object detection model converges. Specifically: The two mixed images are extracted with features again through the feature extraction module, and then the features of the two mixed images are jointly sent into the feature discriminator module to ensure that they have similar feature distributions. The features extracted from the first mixed image are used to calculate the supervised loss with their corresponding labels (mixed labels); continuously iterate and execute the above process until the set number of training iterations is reached and then stop training, and use the parameter model after training as the detection model.

[0033] The parameters applied in the training stage of the present invention are as follows: The feature extraction module uses the SGD optimizer with an initial learning rate of 0.001. The feature discriminator module uses the Adam optimizer with an initial learning rate of 0.000025. The batch size used in the entire training process is 2, and a total of 150 iterations of training are completed.

[0034] The dual-path adaptive hybrid cropping strategy of the present invention aims to increase the diversity of source domain data and target domain data, solve the imbalance problem between the two types of data, enhance the extraction of shared features between the two types of data, and improve the object detection ability of target domain data. The specific process of the adaptive hybrid cropping strategy is as Figure 2 shown. Taking the mixing of target domain data into source domain data as an example, first, the target domain data is input into the feature extraction module. After generating the test results (target test output), the category of each pixel point in the test results is obtained according to the confidence threshold; then, the position of the cropping label is determined based on this category and cropped; finally, the cropped label is mixed into the source domain image to synthesize a new mixed image (the first mixed image), and the mixed label is updated according to the position of the cropped image in the source domain image.

[0035] In addition, the present invention also designs an adaptive cropped image expansion strategy to effectively capture the positional relationship between adjacent parts, that is, starting from the original position of the cropped image, this image is expanded pixel by pixel until the condition for stopping the expansion is met.

[0036] As a preferred implementation manner of this embodiment, the conditions for stopping the expansion of the cropping area include: Expanding to the edge of the source domain image or the target domain image; Expanding to other connector labels in the source domain image or the target domain image; The cropped source domain image or target domain image and the existing connector annotation boxes overlap each other.

[0037] The dual-path learning strategy not only needs to mix the target domain data into the source domain data, but also needs to mix the source domain data into the target domain data. Its method is roughly similar to the target domain image cropping strategy. First, select the connector label from the source domain data and find the position of the corresponding connector in the assembly image according to this label; then crop and expand this connector image; finally, mix this connector image into the target domain image (the second mixed image), and update the mixed label according to the position of the cropped image in the mixed image.

[0038] As a preferred implementation manner of this embodiment, the supervised loss includes classification loss, localization loss, and confidence loss. The classification loss is calculated by cross-entropy, the localization loss is calculated by the intersection over union of the predicted box and the ground truth box, and the confidence loss is a binary classification loss.

[0039] Furthermore, according to the characteristics of the input values and output values of the object detection method of the present invention, the loss function is defined as shown in Formula 1. This loss function ( ) consists of the supervised loss ( and the discriminator loss and specifically is: ; (1) The supervised loss mainly includes the object detection loss of the source domain data ( ) and the supervised loss when the target domain image cropping and mixing are applied to the source domain image ( ). The supervised loss or has the formula as follows: ; (2) In the formula, is the classification loss, is the localization loss, is the confidence loss. The classification loss is calculated through the cross-entropy loss function and is used to measure the performance of the model in object classification; the localization loss is calculated by computing the intersection over union ( ) between the predicted bounding box and the ground truth bounding box to ensure that the model can accurately locate the position of the connector in the image; the confidence loss is a binary classification loss aiming to measure the confidence prediction of the model for the existence of an object. The formulas for the three types of losses are: ; (3) ; (4) ; (5) In the formula, is the number of connectors in the image, is the number of categories, and respectively represent the ground truth label and probability of the th object in the classification loss, and respectively represent the predicted bounding box and label, and respectively represent the ground truth label and probability of the th bounding box in the confidence loss.

[0040] As a preferred implementation manner of this embodiment, the discriminator loss includes: the feature distribution alignment loss between the source domain and the target domain and the feature consistency loss between the first mixed image and the second mixed image.

[0041] Furthermore, the two discriminator loss functions are respectively the feature discriminator loss ( ) between the source domain data and the target domain data and the feature discriminator loss ) between the dual-path cropping and mixing strategies. Among them: ; (6) ; (7) ; (8) ; (9) In the formula, and respectively represent the generator adversarial loss and discriminator loss between the dual-path clipping and mixing strategies, is the feature discriminator module between the two types of mixed data, and represent the output features after the two types of mixed data are subjected to feature extraction; and respectively represent the generator adversarial loss and discriminator loss between the source domain data and the target domain data, is the feature discriminator module between the two types of data, and represent the output features after the two types of data pass through the feature extraction module; is the input data sampled from the target domain; is the input data sampled from the source domain.

[0042] As a preferred implementation manner of this embodiment, the Mamba network uses the state space model SSM for multi-level feature extraction, and the extracted multi-level features are fused through the path aggregation feature pyramid network PAFPN, and the fused features are upsampled by the YOLOv5 detection head to output the detection result.

[0043] Furthermore, as Figure 3 shown, the feature extraction module of the present invention designs and improves the Mamba network, which is composed of Mamba and YOLOv5. After the source domain data and the target domain data are subjected to data augmentation, they are input into the Mamba+YOLOv5 feature extraction module. The training process of this module is as follows: First, the input data passes through the downsampling SSM feature extraction module to extract multi-level feature information (P3, P4, and P5); then, the path aggregation feature pyramid module PAFPN is used to fuse the feature information of P3-P5. PAFPN is an efficient feature pyramid aggregation network, which can effectively integrate image features of different scales and enhance the model's detection ability for different-sized connectors in the image; finally, the fused feature map is input into the upsampling module (YOLOv5) to output the test result.

[0044] Furthermore, the hidden state dimension of the downsampling SSM feature extraction module is set to the average aspect ratio of the assembly connectors (such as 16:1) to optimize the feature extraction of small targets such as long bolts.

[0045] Performance test: Use the trained object detection model to test the images in the target domain test set and output the test results to evaluate the effectiveness of the method proposed by the present invention.

[0046] To verify the proposed dual-path hybrid cropping method for detecting assembly connectors of the present invention, four existing methods: Mamba YOLO, PTDA, ADDA, and ConfMix are compared with the method of the present invention. The engine assembly connector dataset made by the present invention is used for all datasets, and the evaluation metrics are the overall average accuracy (mAP50 / 95) and precision (P). The detection accuracy of physical assembly connectors is shown in Table 1.

[0047] Table 1 Detection accuracy of physical assembly connectors

[0048] As can be seen from Table 1, compared with the supervised method (Mamba YOLO), the proposed domain adaptive object detection method of the present invention has obvious advantages in all three evaluation metrics, with improvements of 42.5%, 25.5%, and 47.0% respectively; compared with the ConfMix method, although the method of the present invention is slightly lacking in the mAP95 metric, it has better performance in mAP50 and precision, and the three evaluation metrics are 80.4%, 47.3%, and 77.2% respectively. Compared with the other two domain adaptive methods (PTDA, ADDA), the proposed method of the present invention also has certain advantages in the three evaluation metrics.

[0049] Example 2: This embodiment provides a dual-path hybrid cropping system for detecting assembly connectors, including: A data acquisition module that acquires the assembly image to be detected in the industrial environment; A detection and recognition module that inputs the assembly image to be detected into the object detection model; the object detection model includes a feature extraction module and a feature discriminator module. The feature extraction module uses the Mamba network to extract the long-range dependence features of the input data and outputs the detection results through the YOLOv5 detection head, including the position, category, and confidence of the connector; the object detection model is trained through a dual-path adaptive hybrid cropping strategy. Among them, the dual-path adaptive hybrid cropping strategy reconstructs the training images input to the object detection model based on the image data in the source domain data and the target domain data; the source domain data is the three-dimensional model image of the assembly simulating the real environment and the automatic annotation labels; the target domain data is the real-shot assembly image, and the connector labels are manually annotated. A result output module that screens the detection results according to the confidence output by the trained object detection model and generates a detection report of the assembly connector.

[0050] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent the cases of A existing alone, A and B existing simultaneously, and B existing alone. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0051] Those of ordinary skill in the art can realize that the various units and algorithm steps described in the embodiments disclosed herein can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0052] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0053] In several embodiments provided by the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0054] The above are only the embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. An object detection method for an assembly connector with dual-path hybrid cutting, characterized in that, It includes the following steps: Obtain the assembly image to be detected in the industrial environment; Input the assembly image to be detected into the target detection model; the target detection model includes a feature extraction module and a feature discriminator module. The feature extraction module uses the Mamba network to extract the long-range dependence features of the input data and outputs the detection results through the YOLOv5 detection head, including the position, category, and confidence of the connecting parts; Train the target detection model through a dual-path adaptive hybrid cropping strategy; Among them, the dual-path adaptive hybrid cropping strategy reconstructs the training images input to the target detection model based on the image data in the source domain data and the target domain data; the source domain data is the three-dimensional model image of the assembly simulating the real environment and the automatic annotation labels; the target domain data is the real-shot assembly image, and the connecting part labels are manually annotated; Screen the detection results according to the confidence output by the trained target detection model to generate a detection report of the assembly connecting parts.

2. The method for detecting the target of the assembly connector with double-path hybrid cutting according to claim 1, characterized in that: The process of training the target detection model through the dual-path adaptive hybrid cropping strategy is as follows: Input the source domain dataset and the target domain dataset; Perform comprehensive data augmentation on the source domain and target domain data respectively to generate enhanced images; Input the enhanced source domain and target domain images into the feature extraction module with shared parameters to extract the source domain features and target domain features respectively; Input the source domain features and the target domain features into the feature discriminator module, and align the feature distributions of the two domains through adversarial training, and at the same time calculate the supervised loss of the source domain features; Generate hybrid training data through the dual-path adaptive hybrid cropping strategy, including: Crop and expand the connecting part regions in the target domain image with confidence higher than the threshold and mix them into the source domain image to obtain the first hybrid image; Crop and expand the labeled connecting part regions of the source domain image and mix them into the target domain image to obtain the second hybrid image; After inputting the first hybrid image and the second hybrid image into the feature extraction module to extract features, send them to the feature discriminator module for feature distribution alignment, and calculate the supervised loss of the first hybrid image; Iteratively optimize the target detection model, and update the parameters by combining the supervised loss and the discriminator loss until the target detection model converges.

3. The method for detecting the target of the assembly connecting piece with double-path hybrid cutting according to claim 2, wherein: The comprehensive data augmentation includes: rotation, scaling, brightness and contrast adjustment, affine transformation, elastic transformation, sharpening, and blurring.

4. The method for detecting the target of the assembly connector with double-path hybrid cutting according to claim 2, wherein: The stop conditions for cropping region expansion include: Expanding to the edge of the source domain image or the target domain image; Expanding to other connecting part labels in the source domain image or the target domain image; The cropped source domain image or target domain image and the existing connecting part annotation boxes overlap each other.

5. The method for detecting the target of the assembly connecting piece with double-path hybrid cutting according to claim 2, characterized in that: The supervised loss includes classification loss, localization loss, and confidence loss. The classification loss is calculated by cross-entropy, the localization loss is calculated by the intersection over union of the predicted box and the true box, and the confidence loss is a binary classification loss.

6. The method for detecting the target of the assembly connector with double-path hybrid cutting according to claim 2, characterized in that: The discriminator loss includes: the feature distribution alignment loss between the source domain and the target domain and the feature consistency loss between the first hybrid image and the second hybrid image.

7. The method for detecting the target of the assembly connecting piece with double-path hybrid cutting according to claim 1, characterized in that: The Mamba network uses a state space model (SSM) for multi-level feature extraction. The multi-level features extracted are fused by a path aggregation feature pyramid network (PAFPN), and the fused features are upsampled by a YOLOv5 detection head to output detection results.

8. An assembly connector target detection system for double-path hybrid cutting, characterized in that, It includes: A data acquisition module that acquires assembly images to be detected in an industrial environment; A detection and recognition module that inputs the assembly images to be detected into a target detection model. The target detection model includes a feature extraction module and a feature discriminator module. The feature extraction module uses the Mamba network to extract long-range dependence features of the input data and outputs detection results through a YOLOv5 detection head, including the position, category, and confidence of the connectors; Training the target detection model through a dual-path adaptive hybrid cropping strategy; Among them, the dual-path adaptive hybrid cropping strategy reconstructs the training images input to the target detection model based on the image data in the source domain data and the target domain data. The source domain data is the 3D model images of the assembly simulating the real environment and the automatically annotated labels. The target domain data is the actual photographed assembly images with manually annotated connector labels; A result output module that screens the detection results according to the confidence levels output by the trained target detection model and generates a detection report for the assembly connectors.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the dual-path hybrid cropping method for detecting assembly connectors according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the dual-path hybrid cropping method for detecting assembly connectors according to any one of claims 1 to 7.

Images