A deep learning-based process-aware cable joint visual recognition method

Abstract

The present application relates to the field of computer vision and automatic assembly technology, and particularly relates to a process perception type cable joint visual identification method based on deep learning, which comprises collecting cable joints on an assembly station through an industrial camera; a network model based on an attention mechanism and process modulation is constructed; the network model comprises a backbone feature extraction layer, a feature enhancement layer, a process feature modulation layer and an anchor-free detection layer in sequence; a loss function used for training is constructed, the network model is trained using a cable joint image dataset, and a trained network model is obtained; a cable joint image to be identified is outputted through the trained network model to obtain a detection result of the cable joint; through the combination of hierarchical feature extraction, multi-scale attention fusion and process feature modulation mechanism, the present application significantly improves the robustness and precision of industrial visual detection under complex illumination, background interference and occlusion conditions.

Description

Technical Field

[0001] This invention relates to the field of computer vision and automated assembly technology, specifically to a deep learning-based process-aware cable connector visual recognition method. Background Technology

[0002] With the continuous improvement of automation in mechanical manufacturing, cables, as an important carrier for equipment interconnection and signal transmission, directly affect the stability and safety of the entire production system through their inspection and assembly quality. In the cable assembly process, the appearance, shape, and assembly accuracy of cable connectors are among the most critical inspection items. Traditional manual inspection methods are inefficient and prone to errors, making them unsuitable for the high precision and consistency requirements of mechanical assembly. In recent years, with the development of convolutional neural networks and computer vision, deep learning-based automated inspection methods have gradually replaced traditional manual visual inspection and traditional image algorithms, demonstrating advantages such as high inspection accuracy, strong adaptability, and good scalability.

[0003] The prior art discloses a fast object detection method based on convolutional neural networks, see [link to previous text]. Figure 9 As shown, this method constructs a FastDet network as the object detection framework. Channel splitting, cross-stage concatenation, and residual connections are introduced into the backbone network to improve feature extraction capabilities. An improved cross-type FPN is used in the neck network, and an ECA attention module is integrated to enhance attention to key information. The detection head structure borrows the decoupling idea of ​​YOLOX, separating classification and regression tasks. An improved SimOTA matching strategy is used to ensure good model stability during training. Furthermore, this method introduces the SIOU geometric metric into the loss function, improving the accuracy of bounding box regression and demonstrating good generalization performance across multiple visual tasks.

[0004] However, the aforementioned detection methods are primarily geared towards target recognition in general scenarios, and still have certain limitations in identifying fine-grained targets such as industrial cables. While their network structures are lightweight and efficient, ordinary attention mechanisms struggle to capture key geometric details when dealing with targets like cable joints—which have fine textures, large scale variations, and high appearance similarity. Furthermore, convolutional neural networks are not specifically designed for industrial process data or particular assembly scenarios, making it difficult to handle interference from cluttered cable backgrounds, drastic lighting changes, and complex workstation environments. In addition, the feature fusion strategy of this method still has room for improvement in its performance on multi-scale small targets, leading to potential missed detections or location misalignments in dense cable joint scenarios.

[0005] Therefore, a new method is needed to achieve high-precision and robust cable connector identification in complex industrial scenarios. This method should fully integrate multi-scale attention mechanisms, take into account both spatial and channel feature responses, and introduce process-related information to improve the specificity of feature expression, thereby achieving accurate identification and stable detection of different types of cable connectors. Summary of the Invention

[0006] To address the problems existing in the background technology, traditional image recognition algorithms based on thresholding or edge detection often suffer from unstable recognition in complex situations such as uneven lighting and cluttered backgrounds. While some existing deep learning methods have achieved rapid detection of general objects, they still have shortcomings in tasks such as fine-grained morphological differentiation and multi-scale cable joint localization. This invention provides a deep learning-based process-aware visual recognition method for cable joints. This method can perform depth-optimized detection based on cable joint features, and while ensuring high accuracy, it also has real-time performance and deployability.

[0007] The first objective of this invention is to provide a deep learning-based process-aware visual recognition method for cable connectors, comprising:

[0008] The cable connector images at the assembly station are captured using an industrial camera. At the same time, a corresponding structured text annotation is configured for each captured image to obtain a cable connector image dataset.

[0009] A network model based on attention mechanism and process modulation is constructed; the network model includes, in sequence, a backbone feature extraction layer, a feature enhancement layer, a process feature modulation layer, and an anchorless detection layer;

[0010] Construct the loss function for training and train the network model using a cable connector image dataset to obtain the trained network model;

[0011] The image of the cable connector to be identified is processed by a trained network model to output the detection results of the cable connector.

[0012] The backbone feature extraction layer uses an improved convolutional neural network as the backbone network, coupled with a multi-scale attention module, to extract multi-scale feature representations.

[0013] The feature enhancement layer employs a feature pyramid network equipped with a simplified self-attention module to adaptively enhance the saliency of cable connector edges and local details in both spatial and channel dimensions.

[0014] The process feature modulation layer uses a process feature linear modulation module to embed the production line process and process information into the visual representation and modulate it layer by layer according to the channel, so that the network's visual reasoning is consistent with the expected assembly process constraints.

[0015] The frameless detection layer is a frameless detection head constructed using a frameless single-stage target detection algorithm. It outputs the detection results of cable connectors by predicting class probabilities, bounding box coordinates, and centrality scores.

[0016] In one embodiment, the backbone feature extraction layer uses ConvNeXt V2 as the backbone network to extract features from the input cable connector image. The ConvNeXt V2 backbone network consists of five consecutive downsampling feature extraction stages. A multi-scale attention module is introduced between adjacent feature extraction stages of ConvNeXt V2 to enhance the output features of the previous stage and use the enhanced features as the input of the next stage.

[0017] In one embodiment, the backbone feature extraction layer uses three sets of features from different semantic levels of ConvNeXt V2 as the final output of the backbone network.

[0018] In one embodiment, the feature enhancement layer is based on a feature pyramid network, and a simplified self-attention module is introduced at the feature interaction position of each layer of the pyramid to adaptively enhance the features of each layer before transmission and fusion, thereby obtaining multi-scale features as the input basis for subsequent feature representation construction and judgment modules.

[0019] In one embodiment, the process feature linear modulation module includes a dual-branch input structure of visual feature branch and process feature branch, as well as an independent parameter generation branch;

[0020] The independent parameter generation branch is used to generate modulation parameters from the process description information in the production process; the visual feature branch is used to form a visual feature embedding sequence from the multi-scale features output by the feature enhancement layer; and the process feature branch is used to generate a process embedding sequence corresponding to the visual feature embedding sequence.

[0021] The process feature linear modulation module concatenates the visual feature embedding sequence with the process embedding sequence and generates joint features through linear mapping.

[0022] Based on the modulation parameters, a channel-wise linear modulation operation is performed on the joint features to obtain the modulated feature sequence. The modulated feature sequence is then processed by a lightweight feature modeling module to obtain process-aware features. The lightweight feature modeling module includes a self-attention layer and a feedforward network.

[0023] In one embodiment, the process feature linear modulation module further includes: remapping the process-aware features back to the feature map space according to their corresponding scales, restoring them to a multi-scale feature representation, and obtaining process-aware multi-scale features.

[0024] In one embodiment, the anchorless detection head is equipped with three independent prediction branches: a classification branch, a regression branch, and a centrality branch. Each branch is composed of multiple convolution operators connected in series and outputs the corresponding prediction result at the end. The classification branch is used to predict the cable connector type corresponding to the current position; the regression branch is used to predict the distance from the current position to the target boundary; and the centrality branch is used to measure whether the current position is close to the geometric center of the target.

[0025] In one embodiment, the loss function used for training is constructed by creating a joint loss function consisting of classification loss, boundary regression loss, and centrality loss.

[0026] In one embodiment, during training, based on the multi-scale feature map output by the feature pyramid network, the spatial locations on each scale feature layer are divided into samples. When any feature location is located inside the actual cable connector annotation box, the location is regarded as a positive sample, and the other locations are regarded as negative samples. The positive sample locations are used to participate in the loss calculation of category prediction, boundary regression and centrality prediction.

[0027] A second objective of this invention is to provide a system for a deep learning-based process-aware visual recognition method for cable connectors, comprising:

[0028] The data acquisition module is used to acquire cable connectors on the assembly station using an industrial camera. At the same time, it configures corresponding structured text annotations for each acquired image to obtain a cable connector image dataset.

[0029] A network construction module is used to build a network model based on attention mechanisms and process modulation. The network model sequentially includes a backbone feature extraction layer, a feature enhancement layer, a process feature modulation layer, and an anchorless detection layer. The backbone feature extraction layer uses an improved convolutional neural network as the backbone network, coupled with a multi-scale attention module, to extract multi-scale feature representations. The feature enhancement layer uses a feature pyramid network equipped with a simplified self-attention module to adaptively enhance the saliency of cable connector edges and local details in spatial and channel dimensions. The process feature modulation layer uses a process feature linear modulation module to embed production line processes and flow information into the visual representation and modulate it layer by layer according to channel, ensuring that the network's visual inference is consistent with the expected assembly process constraints. The anchorless detection layer is an anchorless detection head built using a single-stage target detection algorithm, which outputs the detection results of the cable connector by predicting the class probability, bounding box coordinates, and centrality score.

[0030] The network training module is used to construct the loss function for training and train the network model using a cable connector image dataset to obtain the trained network model.

[0031] The recognition module is used to output the detection results of the cable connectors by passing the images of the cable connectors to be recognized through a trained network model.

[0032] The present invention has at least the following beneficial effects:

[0033] This invention provides a deep learning-based visual recognition method for process-aware cable joints. The method enhances the perception of underlying geometric features through a ConvNeXt V2 backbone network, enabling the system to capture complete edge morphology and structural transition areas even when faced with cable joints exhibiting fine surface textures and high structural similarity. Simultaneously, an efficient multi-scale attention (EMA) module is incorporated to adaptively enhance and recalibrate features at different scales and channels, further improving the network's ability to represent key local structures. Because the backbone network employs modified convolutional units and feature normalization strategies in each layer, the system maintains a stable response to cable joint targets under different shooting positions and lighting conditions, thereby reducing recognition offsets caused by changes in shooting angle or environment. Furthermore, this backbone structure maintains high representational capability while ensuring inference efficiency, enabling fast and stable feature extraction on low-to-medium computing power platforms, which is beneficial for long-term operation and deployment in industrial production environments.

[0034] This invention employs a Feature Pyramid Network (FPN) and a SimAM parameterless attention module in the multi-scale feature fusion stage, effectively improving the problem of "over-concentration or over-dispersion of attention" in traditional models under complex backgrounds. The FPN achieves the fusion of features at different depths through a top-down and laterally connected cross-layer information integration mechanism, balancing high-resolution spatial details with deep semantic information, thus achieving a balance between texture details and semantic feature expression. The SimAM module adaptively adjusts key pixel regions within the fused feature map, automatically enhancing the response of key structures, such as the end face structure of cable connectors, crimped areas, and wire edges, without additional annotation. Even under conditions of noise, oil contamination, and reflection interference, this mechanism maintains stable feature activation and high-precision localization capabilities, significantly improving the performance of fine-grained cable connector detection tasks and outperforming traditional convolutional feature fusion methods.

[0035] The PFNM process feature modulation mechanism proposed in this invention establishes a closer link between visual features and production rules. Industrial cable connector assemblies have relatively fixed structural patterns, and PFNM can embed these patterns into the visual channel in the form of linear weights, so that features are constrained to "conform to the expected process structure" from the formation stage. This approach effectively reduces false detections and missed detections caused by unknown operating conditions, especially in production lines with long process flows and many types of cable connectors. For example, when different models of cable connectors have insignificant differences in appearance, the process bias signal introduced by PFNM can help the system correctly determine whether the structural shape conforms to the process information of that batch, thereby reducing interference between similar structures and making the detection closer to the actual assembly logic.

[0036] This invention employs the FCOS (Focused Capture on Frames) anchor-free structure, significantly reducing the difficulty of training and deploying the detection model. FCOS completes classification, regression, and centrality prediction at the pixel level, eliminating the need for manually setting anchor box sizes and aspect ratios. This avoids the numerous manually preset parameters required in traditional anchor box methods, making the model more flexible when migrating across platforms. When faced with different camera resolutions, different field of view ranges, or changes in workstation distances, this invention can quickly adapt to new detection environments without recalibrating the anchor box patterns. This feature is particularly suitable for modern assembly lines with multi-workstation, multi-device collaboration, reducing maintenance costs and improving assembly efficiency.

[0037] This invention strengthens the collaboration between modules in its overall structure, giving the system greater robustness. Regarding detection speed, both the backbone network and the attention module employ lightweight designs, enabling the entire model to maintain high accuracy while maintaining fast inference speed. In terms of generalization ability, a process modulation mechanism enhances structural consistency, allowing the model to maintain stable performance in unfamiliar scenarios. Regarding assembly feasibility, this invention supports deployment on common industrial cameras and GPU platforms, requiring no additional hardware and facilitating rapid integration into existing assembly systems.

[0038] In long-term assembly scenarios, this invention demonstrates strong adaptability to issues such as lighting, contamination, and occlusion. The synergistic effect of the attention module and PFNM prevents the model from over-relying on a single visual signal, instead enabling comprehensive judgment based on structural morphology, process logic, and contextual information, thus reducing detection fluctuations caused by short-term environmental variations. The anchor-free detection method reduces the model's sensitivity to scale inconsistencies, allowing the system to consistently output detection results even when faced with cable connectors of various sizes and angles.

[0039] Compared to traditional convolution-based target detection methods, this invention not only significantly improves recognition capabilities and positioning accuracy but also demonstrates higher stability, flexibility, and maintainability in practical industrial applications. Its feature extraction, attention control, process information fusion, and anchor-free detection strategy form a close complementary relationship, enabling the detection system to maintain reliable performance in changing environments. This provides a mature, stable, and widely applicable technical solution for automated cable connector identification. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the overall structural framework of the present invention;

[0041] Figure 2 This is a structural diagram of the EMA (Efficient Multi-scale Attention) multi-scale attention module in this invention;

[0042] Figure 3 This is a diagram of the ConvNeXt V2 backbone network structure with the EMA module added in this invention;

[0043] Figure 4 This is a diagram of the Feature Pyramid (FPN) structure with the SimAM module added in this invention;

[0044] Figure 5 This is a structural diagram of the SimAM module in this invention;

[0045] Figure 6 This is a schematic diagram of the PFNM (Process Feature Numerical Modulation) linear modulation module in this invention;

[0046] Figure 7 This is a structural diagram of the FCOS frameless detection head used in this invention;

[0047] Figure 8 This is a visual schematic diagram of the cable connector detection method of the present invention;

[0048] Figure 9 This is a flowchart illustrating a fast target detection method based on convolutional neural networks in the background technology. Detailed Implementation

[0049] To illustrate the technical means and effects adopted by the present invention in order to achieve the intended purpose, the following detailed description is provided in conjunction with embodiments.

[0050] This invention addresses the challenges of automated assembly lines in modern industry, where cable connection quality directly impacts equipment stability and safe operation. With increasingly rapid production pace, manual inspection of cable connectors is insufficient to meet efficiency and accuracy requirements. Traditional image algorithms based on threshold or edge detection often struggle with complex situations such as uneven lighting and cluttered backgrounds. While some existing deep learning methods achieve rapid detection of general objects, they fall short in tasks such as fine-grained morphological differentiation and multi-scale cable connector localization. Therefore, a detection method capable of deep optimization based on cable connector features is needed, ensuring high accuracy while also offering real-time performance and deployability.

[0051] The purpose of this invention is to provide a deep learning-based process-aware cable connector visual recognition method. By introducing the ConvNeXt V2 backbone network with stronger local modeling capabilities, and coordinating it with an efficient multi-scale attention (EMA) module, the network can extract geometrically meaningful structural cues at a shallow level. Furthermore, with the help of the SimAM module, the Feature Pyramid Network (FPN) can automatically focus on key regions during the fusion process. At the same time, a Process Feature Linear Modulation (PFNM) module is designed to inject process information from the production process into the visual model, making the feature representation closer to the real operation scenario. Finally, combined with the FCOS anchorless frame detection structure, the detection process can achieve accurate localization without a large number of prior frames, simplifying the deployment and parameter tuning difficulty.

[0052] To achieve the above objectives, a deep learning-based process-aware cable connector visual recognition method includes:

[0053] S1. Use an industrial camera to capture images of cable connectors at the assembly station. Simultaneously, configure corresponding structured text annotations for each captured image to obtain a cable connector image dataset.

[0054] This invention constructs a customized dataset for visual recognition and process analysis of cable connectors. The dataset originates from actual industrial assembly environments, acquired by photographing cable connectors at assembly stations using industrial cameras. The collected images have high resolution, meeting the needs for analyzing the appearance details of cable connectors. In this embodiment, a total of 8000 cable connector images were collected, covering samples of various models and different assembly states.

[0055] Simultaneously with image acquisition, each image is assigned a corresponding structured text annotation. The text annotation describes process-related attributes that are typically difficult to obtain directly from visual images. This includes cable connector type information, internal structural features, and key assembly parameters related to assembly quality. Specifically, cable connector type information describes interface specifications and coding methods; internal structural features describe structural attributes such as conductor quantity and shielding type; and assembly parameters reflect features closely related to process quality, such as crimping height and shielding contact status.

[0056] For example, for a sample image of a cable connector, the corresponding text annotation could be described as "M12-A coded female connector, four-core shielded cable, currently in the crimping process stage, target crimping height range is 2.2–2.4 mm, shielding layer adopts 360-degree ring crimping method, ambient temperature 28℃, humidity 65%". Through this method, the process annotation not only clarifies the electrical standards and structural form of the cable connector, but also records dynamic information such as the current assembly stage, target process range, and environmental conditions, thereby providing the visual model with process constraint information closely related to the assembly state.

[0057] After image acquisition and text annotation, the dataset was divided into training, validation, and test sets according to a preset ratio. The training set contains 5600 image samples for training the parameters of the deep learning model; the validation set contains 1200 image samples for parameter tuning and performance evaluation during model training; and the test set contains 1200 image samples for independent validation of the trained model. Each subset includes corresponding structured text annotations to ensure consistency in the distribution of process attributes across different datasets.

[0058] By associating cable connector images with corresponding process text information, the constructed dataset can simultaneously reflect the appearance features and assembly process characteristics of cable connectors, providing multi-source information support for subsequent deep learning-based methods. Thus, even when the appearances are similar, different cable connectors can still be distinguished by combining process attributes, and the assembly status can be determined.

[0059] S2. Construct a network model based on attention mechanism and process modulation; the network model consists of a backbone feature extraction layer, a feature enhancement layer, a process feature modulation layer, and an anchorless detection layer; see [link to documentation]. Figure 1 As shown, the network model includes, in sequence, a backbone feature extraction network, a feature fusion network, a process feature modulation network, a detection head, and an output.

[0060] This invention constructs a complete network model: backbone—multi-scale attention—feature fusion—process modulation—anchorless detection head. The overall network uses ConvNeXt V2 as the backbone, combined with EMA, SimAM, and PFNM, as well as an anchorless detection head based on FCOS, forming an end-to-end pipeline from image to detection output. The specific processing procedures of each layer in the network model are as follows:

[0061] 2.1 The backbone feature extraction layer adopts an improved convolutional neural network as the backbone network, coupled with a multi-scale attention module, to extract multi-scale feature representations;

[0062] The backbone feature extraction layer uses ConvNeXt V2 as the backbone network to extract features from the input cable connector image. The ConvNeXt V2 backbone network consists of five consecutive downsampling feature extraction stages. A multi-scale attention module is introduced between adjacent feature extraction stages of ConvNeXt V2 to enhance the output features of the previous stage and use the enhanced features as the input of the next stage.

[0063] The backbone feature extraction layer uses three sets of features from different semantic levels of ConvNeXt V2 as the final output of the backbone network.

[0064] In this invention, ConvNeXt V2 is used as the backbone network to extract features from the input cable connector image. ConvNeXt V2 is a hierarchical feature extraction structure based on convolutional neural networks. It introduces depthwise separable convolution, inverted bottleneck structure, and learnable scale modulation parameters on the basis of traditional convolutional networks, thereby reducing computational complexity while ensuring feature representation ability. It is suitable for modeling targets with small size and many texture details in cable connectors.

[0065] The ConvNeXt V2 backbone network consists of five feature extraction stages. Each stage sequentially downsamples the input feature map to progressively reduce spatial resolution while simultaneously increasing channel dimensions. Each stage contains several convolutional units to extract semantic features at different scales. An efficient multi-scale attention (EMA) module is introduced between adjacent feature extraction stages in ConvNeXt V2 to enhance the output features of the previous stage and use the enhanced features as input for the next stage. This guides the network to focus more on key spatial locations and channel responses during the progressive downsampling and semantic abstraction process.

[0066] See Figure 2 As shown, the processing procedure of the EMA module includes:

[0067] Let the feature map output by the previous network stage be...

[0068]

[0069] in, This represents the feature map input to the EMA module; Indicates the number of channels in the feature map; and These represent the height and width of the feature map in the spatial dimension, respectively.

[0070] First, the input feature map A pointwise convolutional transformation is performed to obtain a unified feature representation. The calculation method is as follows:

[0071]

[0072] in, This represents the feature map after pointwise convolution; This represents the kernel parameters corresponding to pointwise convolution; This represents the convolution operation.

[0073] Subsequently, feature maps The data is fed into parallel spatial feature processing paths. In one of these paths, spatial information is aggregated using global average pooling, calculated as follows:

[0074]

[0075] in, Represents the global space context vector; Representation of feature map In the The first channel, the first line, number The value at the column position; and These represent spatial dimension indices.

[0076] The above calculations yielded the following results. Used to characterize the average response of each channel in the global space.

[0077] In another spatial feature processing path, the feature map... Depthwise convolution is applied to extract local spatial relationships; the calculation method is as follows:

[0078]

[0079] in, Represents local spatial features after depthwise convolution; This represents the kernel parameters corresponding to depthwise convolution; This represents a depthwise convolution operation.

[0080] The outputs of the two spatial paths described above are linearly mapped and fused to generate spatial attention weights, calculated as follows:

[0081]

[0082] in, Indicates spatial attention weights; and Indicates the corresponding linear transformation parameters; This represents the Sigmoid activation function.

[0083] Meanwhile, the EMA module also includes a channel feature processing path for modulating the importance of different channels. This path also obtains channel statistics through global average pooling, calculated as follows:

[0084]

[0085] in, This represents the channel statistics vector, with dimensions of... .

[0086] The channel statistical vectors are then input into a nonlinear transformation structure composed of multiple linear mappings to generate channel attention weights, which are calculated as follows:

[0087]

[0088] in, Indicates channel attention weights; and They represent the linear transformation parameters, respectively. Represents a non-linear activation function; This represents the Sigmoid activation function.

[0089] Obtain spatial attention weights With channel attention weights Then, they are applied to the original input feature maps respectively. The calculation method is as follows:

[0090]

[0091]

[0092] in, This represents the feature map after spatial attention modulation. This represents the feature map after channel attention modulation; This indicates element-wise multiplication.

[0093] Subsequently, the two types of features mentioned above are concatenated along the channel dimension, and then... Convolutional processes are used to fuse features, resulting in a comprehensive feature representation, which is calculated as follows:

[0094]

[0095] in, This represents the fused feature map; This indicates a channel-level concatenation operation; express Convolution operation.

[0096] Based on this, a multi-scale feature extraction operation is applied to the fused features to enhance the expressive power of features at different spatial scales. The calculation method is as follows:

[0097]

[0098] in, This indicates the final output characteristics of the EMA module; This represents a multi-scale feature extraction operator composed of different convolution kernel sizes.

[0099] See Figure 3 As shown, combining the backbone network structure and the inter-stage attention enhancement mechanism, this embodiment uses three sets of features from different semantic levels of ConvNeXt V2 as the final output of the backbone network, and defines them as follows: This is used for subsequent multi-scale feature fusion and target detection, and its specific sources are as follows:

[0100] This represents the direct output features of the third feature extraction stage of ConvNeXt V2. These features have high spatial resolution and can effectively preserve the local geometry and edge details of the cable connector. Considering that introducing attention modulation too early might weaken the original structural representation of lower-level features, this embodiment does not introduce EMA processing at this layer to ensure the integrity of the basic spatial information.

[0101] This indicates that the output features from stage 3, after being enhanced by the EMA module, are input into the fourth feature extraction stage and output by that stage as mid-level semantic features. By introducing attention modulation before entering deeper semantic abstraction, the responsiveness of mid-scale features to key regions is effectively improved.

[0102] This indicates that the output features from stage 4, after being enhanced by the EMA module, are input into stage 5, where they are output as high-level semantic features. These features possess the strongest semantic expressive power and are used to characterize the overall structure and category discrimination information of the cable connector.

[0103] Thus, the ConvNeXt V2 backbone network, while ensuring the semantic progression of multi-layer features, achieves attention-guided inter-layer feature enhancement, providing a clear structure and semantically complementary multi-scale input for the subsequent feature pyramid network.

[0104] 2.2 The feature enhancement layer employs a feature pyramid network equipped with a simplified self-attention module to adaptively enhance the saliency of cable connector edges and local details in both spatial and channel dimensions;

[0105] The feature enhancement layer is based on a feature pyramid network, and a simplified self-attention module is introduced at the feature interaction position of each layer of the pyramid. This module is used to adaptively enhance the features of each layer before transmission and fusion, and obtain multi-scale features, which serve as the input basis for the subsequent feature representation construction and judgment module.

[0106] In this invention, after extracting multi-layer features from the input image, a feature fusion network is constructed between the backbone network output and the detection head to fully integrate spatial and semantic information from features at different scales. This feature fusion network is implemented based on a Feature Pyramid Network (FPN), and a simplified self-attention module (SimAM) is introduced at the feature interaction locations of each pyramid layer to improve the consistency and discriminative ability of multi-scale features.

[0107] Based on the ConvNeXt V2 backbone network structure and its output feature definition described in Section 2.1, this embodiment defines three sets of features at different semantic levels. As input to the feature fusion network. Among them, This is the direct output feature of the third feature extraction stage. and These are the mid-level and high-level semantic features obtained after being enhanced by the EMA module and entering the deeper feature extraction stage, respectively.

[0108] See Figure 4 As shown, to unify the channel dimensions and facilitate subsequent fusion processing, the features of each layer are first horizontally mapped:

[0109]

[0110] In the formula, This represents the pyramid feature after channel alignment. Indicates the first corresponding to the layer Convolution weights; This represents the convolution operation.

[0111] Subsequently, a bidirectional feature transfer structure, employing both top-down and bottom-up approaches, is used to achieve a step-by-step fusion of high-level semantic information and low-level detailed information. During feature transfer, spatial dimensions are aligned through upsampling or downsampling operations, and feature superposition operations are performed.

[0112]

[0113] In the formula, Indicates an upsampling operation; This represents the low-level features after being fused with high-level features.

[0114] To avoid information redundancy or response imbalance caused by the simple superposition of features at different scales in the traditional feature fusion process, this embodiment introduces the SimAM module at the feature fusion node of each pyramid level to perform adaptive weight modulation on the features before and after fusion.

[0115] See Figure 5 As shown, the SimAM module uses each neuron in the feature map as a basic processing unit, and measures the importance of the neuron by calculating its energy value in a local region. Let the input feature map be:

[0116]

[0117] In the formula, Indicates the number of channels; , These represent the height and width of the feature map, respectively.

[0118] For any neuron in the feature map Its energy function is defined as:

[0119]

[0120] In the formula, Indicates the first The values ​​of each neuron; This represents the mean of the corresponding channel in the spatial dimension; This represents the variance of the corresponding channel in the spatial dimension; This is the regularization coefficient, used to suppress the impact of noise on energy calculations.

[0121] Based on the above energy values, calculate the attention weights corresponding to the neurons:

[0122]

[0123] In the formula, Represents neurons Attention weights; It is a very small constant used to avoid zero denominators and improve numerical stability.

[0124] Finally, attention weights are applied to the original feature map to obtain the enhanced feature representation:

[0125]

[0126] In the formula, This represents the feature map after SimAM modulation; Indicates by all The weight matrix is ​​composed of; This indicates element-wise multiplication.

[0127] See Figure 4 As shown, in this embodiment, the SimAM module is positioned at key nodes between each level of the feature pyramid to adaptively enhance the features of each level before transmission and fusion. Its processing can be summarized as follows:

[0128]

[0129] in, Indicates the characteristics of the original pyramids; This represents the features enhanced by SimAM.

[0130] The enhanced features then participate in feature fusion operations in the uplink and downlink paths, thereby forming a more coordinated response relationship across different scales. The multi-scale features obtained after the above fusion and enhancement processes... This serves as the input basis for subsequent feature representation construction and judgment modules.

[0131] This embodiment achieves effective fusion and saliency enhancement of multi-scale features without significantly increasing model parameters and computational load. This enables the feature pyramid to maintain stable and discriminative feature representation when facing cable joint targets with large size differences and high structural similarity, providing reliable input for subsequent detection and judgment.

[0132] 2.3 The process feature modulation layer adopts a process feature linear modulation module to embed the production line process and process information into the visual representation and modulate it layer by layer according to the channel, so that the network's visual reasoning is consistent with the expected assembly process constraints.

[0133] To improve the ability of the detection results to judge the consistency of cable manufacturing process, this embodiment introduces a process feature linear modulation (PFNM) module after the feature fusion network. This module is used to explicitly integrate process semantic information into the visual feature expression process, thereby achieving adaptive adjustment of the cable joint feature response under different process conditions.

[0134] The process feature linear modulation module includes a dual-branch input structure with visual feature branch and process feature branch, as well as an independent parameter generation branch;

[0135] Among them, the visual feature branch is used to form a visual feature embedding sequence from the multi-scale features output by the feature enhancement layer; the process feature branch is used to generate a process embedding sequence and participate in the construction of the visual-process joint feature as an explicit semantic feature to achieve information-level fusion; the independent parameter generation branch is used to generate modulation parameters to perform channel-by-channel linear modulation on the joint feature, and to perform conditional control and adaptive adjustment of the joint representation from the feature distribution level.

[0136] The process feature linear modulation module concatenates the visual feature embedding sequence with the process embedding sequence and generates joint features through linear mapping.

[0137] Based on the modulation parameters, a channel-wise linear modulation operation is performed on the joint features to obtain the modulated feature sequence. The modulated feature sequence is then processed by a lightweight feature modeling module to obtain process-aware features. The lightweight feature modeling module includes a self-attention layer and a feedforward network.

[0138] The process feature linear modulation module further includes: remapping the process-aware features back to the feature map space according to their corresponding scales, restoring them to a multi-scale feature representation, and obtaining the process-aware multi-scale features.

[0139] See Figure 6 As shown, the PFNM module adopts a dual-branch input structure, including a visual feature branch and a process feature branch, which maintain a one-to-one correspondence in the temporal or spatial dimensions. The visual feature branch receives multi-scale features output from the feature pyramid network and enhanced by the SimAM module. Since features at different scales differ in spatial resolution and channel dimension, this embodiment first flattens the features at each scale to eliminate spatial inconsistencies caused by scale differences. Then, channel alignment is used to uniformly adjust the number of feature channels at different scales to a preset dimension. Furthermore, a linear mapping method is employed to map the processed features at each scale to the same feature embedding space, thereby forming a visual feature embedding sequence, represented as:

[0140]

[0141] The process feature branch is used to introduce process information related to the cable connector manufacturing process. Its input is a process embedding sequence that corresponds one-to-one with the aforementioned visual features. Specifically, the process embedding sequence consists of pre-acquired process feature data, which includes, but is not limited to, assembly process type, process parameter range, quality status label, or prior process description information. After discrete encoding or numerical normalization, the aforementioned process feature data is converted into a fixed-dimensional process feature embedding vector through a linear mapping method, thereby forming a process embedding sequence, represented as:

[0142]

[0143] In the formula, Indicates the first Each visual feature embedding vector; Indicates the relationship with the first Embedded vectors of process features corresponding to each visual feature; Indicates the number of feature units.

[0144] To enable joint modeling of visual and process information within the same feature space, this embodiment concatenates the corresponding visual embeddings and process embeddings, and generates a joint feature representation through linear mapping:

[0145]

[0146] In the formula, Indicates the first Visual-process joint embedding; Represents the linear projection weight matrix; Indicates the bias term; This indicates a vector concatenation operation.

[0147] Through the above mapping, the appearance features and the corresponding process attributes can participate in feature calculation simultaneously during the subsequent modulation process.

[0148] In this embodiment, the PFNM module has a dedicated parameter generation branch, used to generate modulation parameters via a mapping network based on key process description information from the cable connector process file. The process description information includes: process identification parameters. Used to distinguish different assembly process types; process coding parameters Used to characterize the current assembly step; process pressure parameters Used to describe crimping or locking conditions; environmental parameters It is used to describe environmental conditions such as temperature and humidity.

[0149] The above process parameters are embedded and then input into a multilayer perceptron (MLP), which outputs scaling and offset coefficients respectively.

[0150]

[0151]

[0152] In the formula, Indicates the first The channel scaling factor corresponding to each feature unit; Indicates the first Channel offset corresponding to each feature unit; , This represents the mapping function used to generate modulation parameters.

[0153] After obtaining the modulation parameters, the PFNM module performs a channel-by-channel linear modulation operation on the joint embedding features, which is calculated as follows:

[0154]

[0155] In the formula, This represents the feature embedding after process modulation; This indicates element-wise multiplication.

[0156] This linear modulation process enables the network to maintain its original visual structure information while adaptively adjusting the response intensity of each channel feature according to different process states, thereby enhancing the feature expression related to the current production conditions.

[0157] The modulated feature sequence:

[0158]

[0159] The input is then fed into a lightweight feature modeling module for further processing. This module includes a self-attention layer and a feedforward network, and its computation process is as follows:

[0160]

[0161] In the formula, This represents the self-attention operation, used to model the relationships between feature units; This represents a feedforward network used for feature nonlinear mapping. This represents the process-aware feature representation of the final output.

[0162] After completing feature modeling, embed the process-aware features. Remapped back to the feature map space according to its corresponding scale, restoring it to a multi-scale feature representation, which serves as the input for the subsequent detection head, denoted as:

[0163]

[0164] In the formula, Indicates the first The feature map after linear modulation of the process features maintains the same spatial resolution as the corresponding feature pyramid network output. Compared to the original feature map, Process semantic constraints are explicitly introduced into the channel response to provide process-aware multi-scale feature inputs for subsequent anchorless detection heads.

[0165] 2.4 The frameless detection layer is a frameless detection head constructed using a frameless single-stage target detection algorithm. It outputs the detection results of cable joints by predicting the class probability, bounding box coordinates, and centrality score.

[0166] The anchorless detection head has three independent prediction branches: a classification branch, a regression branch, and a centrality branch. Each branch consists of multiple convolutional operators connected in series and outputs the corresponding prediction result at the end. The classification branch is used to predict the cable connector type corresponding to the current position; the regression branch is used to predict the distance from the current position to the target boundary; and the centrality branch is used to measure whether the current position is close to the geometric center of the target.

[0167] In this invention, after feature extraction, feature fusion, and process feature modulation are completed, this embodiment uses an anchorless detection head to make a final prediction of the cable joint target.

[0168] The anchorless detection head is built based on the FCOS concept, see [link / reference]. Figure 7 As shown, by directly predicting the target category, position offset, and centrality information at each spatial location of the multi-scale feature map, the matching ambiguity and parameter redundancy problems caused by the preset anchor box are avoided, thereby improving the stability and generalization ability of the detection process.

[0169] The anchorless detection head is positioned in the process-aware multi-scale feature layer output by the PFNM module. Above this, predictions are made for each feature layer separately. For any scale feature layer... Its spatial resolution is The detection head sets up three independent prediction branches on this feature layer, namely:

[0170] 1) Classification branch, used to predict the cable connector type corresponding to the current location;

[0171] 2) Regression branch, used to predict the distance from the current location to the target boundary;

[0172] 3) Centrality branch, used to measure whether the current position is close to the geometric center of the target.

[0173] Each prediction branch consists of multiple concatenated convolution operators, and outputs the corresponding prediction result at the end.

[0174] Classification branch at the feature layer Output category probability map:

[0175]

[0176] in, Indicates the number of cable connector types; , They represent the first The height and width of the layer feature map; Indicates the first Pixel positions of layer feature maps Belongs to the first Predicted probability of cable connectors.

[0177] The regression branch is used to predict the distance from each pixel in the feature map to its corresponding target boundary, and its output format is as follows:

[0178]

[0179] in, Represents the pixel coordinates in the feature map; This indicates the distance from the pixel to the left boundary of the target. This indicates the distance from the pixel to the upper boundary of the target. This indicates the distance from the pixel to the right boundary of the target. This indicates the distance from the pixel to the bottom boundary of the target.

[0180] The bounding box position of the corresponding target can be uniquely determined using the four distance parameters mentioned above.

[0181] To suppress low-quality predicted points far from the target center, this embodiment introduces a centrality branch in the detection head to evaluate the proximity of the current position to the target's geometric center. The calculation method is as follows:

[0182]

[0183] In the formula, Indicated in the feature layer Upper pixel position Centrality predictions; , These represent the minimum and maximum values, respectively; the centrality value range is... arrive The larger the value, the closer the location is to the center of the target area.

[0184] By employing the aforementioned anchorless detection head structure, the network can directly determine the category and regress the location of cable joint targets on multi-scale feature maps, avoiding missed or false detections caused by improper anchor frame size selection. Simultaneously, a centrality prediction mechanism is introduced to effectively suppress unstable responses in edge regions, improving the positioning accuracy and detection reliability of small-sized, partially occluded, or complex background cable joint targets.

[0185] The aforementioned anchorless detection head works in conjunction with the aforementioned feature extraction network, feature fusion network, and process feature modulation module to form a complete high-precision identification process for cable connectors.

[0186] S3. Construct the loss function for training and train the network model using the cable connector image dataset to obtain the trained network model;

[0187] The loss function used for training is constructed by creating a joint loss function consisting of classification loss, boundary regression loss, and centrality loss.

[0188] During training, based on the multi-scale feature map output by the feature pyramid network, the spatial locations on each scale feature layer are divided into samples. When any feature location is inside the actual cable connector annotation box, the location is regarded as a positive sample, and the rest are regarded as negative samples. The positive sample locations are used to participate in the loss calculation of category prediction, boundary regression and centrality prediction.

[0189] In this invention, a joint loss function matching the cable connector identification task is constructed to coordinately constrain the functional branches of the convolutional neural network, so that the network parameters gradually converge during backpropagation, thereby obtaining an identification network model that meets the accuracy requirements of industrial applications.

[0190] During training, based on the multi-scale feature maps output by the feature pyramid network, spatial locations at each scale feature layer are partitioned into samples. When a feature location lies within the actual cable connector bounding box, that location is considered a positive sample; other locations are considered negative samples. Positive sample locations are used in the loss calculations for class prediction, boundary regression, and centrality prediction. The total number of samples is... This quantity is used to normalize the overall loss to avoid adverse effects on the training process due to differences in the number of samples from different images or batches.

[0191] For the three types of output results of the anchorless detection head, this embodiment constructs a joint loss function consisting of classification loss, boundary regression loss, and centrality loss, the overall form of which is expressed as:

[0192]

[0193] The meanings of each symbol are explained below:

[0194] : Represents the total loss function during the training phase;

[0195] : Indicates the number of positions of positive samples in the current training batch;

[0196] : Represents the spatial coordinates on the feature map;

[0197] : Represents the classification loss function used to constrain the category prediction results;

[0198] : indicates the first The category prediction result output by the classification branch on the layer feature map;

[0199] : Represents the actual category label at the corresponding location;

[0200] : Represents the boundary regression loss function based on the intersection-union ratio;

[0201] : Represents the boundary distance parameter for regression branch prediction;

[0202] : Represents the boundary distance parameter corresponding to the actual annotation box;

[0203] : Indicates the centrality branch at position The prediction results at the location;

[0204] : Represents the true centrality value corresponding to this position;

[0205] , : Represents the weighting coefficient used to balance the contributions of different loss terms.

[0206] The classification loss is used to constrain the network's ability to distinguish cable connector types. This embodiment employs an improved focus loss form, assigning different weights to easily classified and difficult-to-classify samples, thereby alleviating the problem of imbalanced positive and negative sample numbers. Its calculation method can be expressed as:

[0207]

[0208] in, : Represents the model's predicted probability of the true class; : Represents the class weight coefficient, used to balance samples from different classes; : Indicates a regulation factor used to reduce the impact of easily classified samples on the total loss.

[0209] Boundary regression loss is used to constrain the geometric consistency between the predicted bounding box and the ground truth bounding box. This embodiment uses a regression loss form based on intersection-over-union (IoU), which guides network optimization by directly measuring the degree of overlap between the predicted and ground truth boxes, and is defined as:

[0210]

[0211] in, : Indicates the intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box; : Represents the predicted bounding box parameters; : Represents the actual bounding box parameters.

[0212] At the same time, true centrality is introduced. The loss term is weighted to reduce the adverse impact of the location far from the target center on the regression results.

[0213] The centrality loss is used to constrain the centrality prediction results output by the network, ensuring that they reflect the proximity of feature locations to the geometric center of the target. This embodiment uses a binary cross-entropy loss, calculated as follows:

[0214]

[0215] in, This represents the centrality value predicted by the model; This represents the centrality value calculated based on the actual annotations.

[0216] During training, the network parameters are optimized end-to-end using the aforementioned joint loss function. The backpropagation algorithm is employed to calculate gradients, and the network weights are iteratively updated using an optimizer. Parameters for each module, including the backbone network, feature fusion network, process feature linear modulation module, and anchorless detection head, are jointly updated during the same training process. This gradually aligns visual feature representation with process semantic information, thereby improving the accuracy and stability of cable connector recognition in complex industrial scenarios.

[0217] S4. Output the detection results of the cable connectors by passing the trained network model through the image of the cable connectors to be identified.

[0218] After completing network training and obtaining stable converged model parameters, the neural network is used to perform forward inference on the cable image to be detected, so as to realize the target localization, category recognition and quality judgment output of the cable connector.

[0219] 4.1 Preprocessing before inference. During the inference phase, the input cable assembly image undergoes the same preprocessing operations as in the training phase to ensure consistent input data distribution. Specifically, this includes: scaling the original image to ensure its shorter side meets the network input requirements; then normalizing the pixel values ​​and constructing the network input tensor according to a preset channel order. In industrial applications requiring real-time performance, the input resolution can be appropriately reduced or half-precision numerical representation can be used for inference calculations without significantly reducing recognition accuracy.

[0220] 4.2 Forward Inference and Candidate Box Recovery. The preprocessed image is input into the trained convolutional neural network, passing sequentially through a feature extraction network, a feature fusion network, and an anchorless detection head to obtain prediction results at each scale feature layer. For any scale feature layer, at each spatial location, the network outputs three types of prediction information: category prediction value, boundary regression parameters, and centrality prediction value.

[0221] The boundary regression results are represented as the distance from a pixel to the target boundary, and the corresponding candidate bounding boxes can be recovered according to the following relationship:

[0222]

[0223] In the formula, Indicates the first One candidate detection box; This represents the pixel coordinates of the feature map corresponding to the candidate box; , , , These represent the predicted distances from the pixel to the left, top, right, and bottom boundaries of the target, respectively.

[0224] 4.3 Overall Confidence Score Calculation. To comprehensively measure the reliability of candidate targets in category determination and spatial localization, an overall confidence score is calculated for each candidate box. The calculation method is as follows:

[0225]

[0226] In the formula, Indicates the first The overall confidence score of each candidate box; This represents the predicted probability value of the category corresponding to the candidate box; This represents the predicted centrality value corresponding to the candidate box, which reflects the degree of proximity between the location and the geometric center of the target.

[0227] By introducing a centrality factor, low-quality predictions that are far from the target center naturally receive lower scores, thereby improving the stability of the overall inference results.

[0228] 4.4 Post-processing screening and process constraint correction. All candidate boxes are sorted according to their overall confidence scores. The results are sorted from highest to lowest, and then non-maximum suppression is performed to remove redundant detection results with highly overlapping spatial locations. When the intersection-union ratio (IUU) of any two candidate boxes exceeds a preset threshold... At that time, only candidate boxes with higher confidence scores are retained.

[0229] After completing the geometric screening, the detection results are further corrected by combining process information: when the process vector input by the process feature linear modulation module indicates that the probability of a certain type of cable connector is low in the current station or process state, the candidate box score of the corresponding category is given a suppression weight, thereby reducing the risk of false detection caused by process inconsistency.

[0230] 4.5 Results Output and Deployment. The final test results after post-processing and filtering are output in a structured format. Each test result includes at least: target bounding box coordinates, cable connector category identifier, category confidence, centrality score, and compliance judgment mark based on process rules.

[0231] When deployed in industrial settings, the trained model can be exported in a format suitable for inference acceleration, and inference efficiency can be improved by combining optimization techniques such as quantization and tensor fusion. Process feature vectors can be input in real time by the manufacturing execution system or field control system via a communication interface, and a default process configuration can be activated in case of communication failure to ensure continuous system operation.

[0232] Through the reasoning and result output process in step four above, stable detection and accurate judgment of cable connector targets can be achieved in automated assembly scenarios, and the visual recognition results can be effectively integrated with the actual process status, thereby improving the availability and reliability of the detection results in real production scenarios.

[0233] The above detailed embodiments provide a comprehensive implementation path for the present invention, including network structure, module mathematical description, training and inference processes, and engineering considerations. Any substitutions, adjustments, or optimizations made to the above steps, parameters, or module forms without departing from the essential technical approach of the present invention should be considered equivalent implementations falling within the protection scope of the present invention.

[0234] After completing the cable joint detection process, the acquired cable images are processed according to the method of this invention to obtain detection results with significant stability. The processed detection images show that the method of this invention can output stable confidence information even under interference conditions such as partial occlusion, similar surface textures of the cable joints, or repetitive background patterns. The predicted bounding boxes fit the actual target boundaries more closely, while effectively suppressing false detections. See also... Figure 8As shown, the detection boxes in key structural regions such as end face structures, metal textures, and sheath edges are accurate and continuous, indicating that the process feature linear modulation (PFNM) and feature fusion strategy can enhance the network's ability to recognize micro-structural features and improve the robustness and precision of detection.

[0235] This invention provides a system for a deep learning-based process-aware visual recognition method for cable connectors, comprising:

[0236] The data acquisition module is used to acquire cable connectors on the assembly station using an industrial camera. At the same time, it configures corresponding structured text annotations for each acquired image to obtain a cable connector image dataset.

[0237] A network construction module is used to build a network model based on attention mechanisms and process modulation. The network model consists of a backbone feature extraction layer, a feature enhancement layer, a process feature modulation layer, and an anchorless detection layer. The backbone feature extraction layer uses an improved convolutional neural network as the backbone network, coupled with a multi-scale attention module, to extract multi-scale feature representations. The feature enhancement layer uses a feature pyramid network equipped with a simplified self-attention module to adaptively enhance the saliency of cable connector edges and local details in spatial and channel dimensions. The process feature modulation layer uses a process feature linear modulation module to embed production line processes and flow information into the visual representation and modulate it layer by layer according to channel, ensuring that the network's visual inference is consistent with the expected assembly process constraints. The anchorless detection layer is an anchorless detection head built using a single-stage target detection algorithm, which outputs the detection results of the cable connector by predicting the class probability, bounding box coordinates, and centrality score.

[0238] The network training module is used to construct the loss function for training and train the network model using a cable connector image dataset to obtain the trained network model.

[0239] The recognition module is used to output the detection results of the cable connectors by passing the images of the cable connectors to be recognized through a trained network model.

[0240] In summary, this invention provides a deep learning-based visual recognition method for process-aware cable connectors. This method uses an improved ConvNeXt V2 network as its backbone structure, combined with an efficient multi-scale attention (EMA) module, to construct a rich and context-sensitive feature representation, thereby enhancing the network's ability to perceive subtle textures and complex backgrounds. In the feature fusion stage, a Feature Pyramid Network (FPN) with an added SimAM attention mechanism is employed. Through joint spatial and channel modeling, adaptive fusion of multi-scale information is achieved, effectively preserving the overall geometric shape and edge details of the cable connector. Subsequently, a newly designed Process Feature Linear Modulation (PFNM) module is introduced. This module injects semantic information related to the assembly process into the feature map through a linear modulation strategy, thereby achieving adaptive reshaping of the feature distribution during visual inference. This ensures that the detection results are consistent with the actual assembly process, improving the model's process awareness and localization accuracy. Finally, an anchorless prediction head based on an improved fully convolutional single-stage (FCOS) detection framework outputs the cable connector's category, bounding box coordinates, and centrality score, achieving efficient end-to-end detection and accurate localization. This invention significantly improves the robustness and accuracy of industrial visual inspection under complex lighting, background interference, and occlusion conditions by combining hierarchical feature extraction, multi-scale attention fusion, and process feature modulation mechanisms. The method is simple in structure and highly computationally efficient, and can be widely applied to high-precision visual recognition of cable connectors, pose detection, precise assembly control, and visual recognition of other precision components. It has good feasibility for automated assembly and significant application and promotion value.

[0241] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep learning-based process-aware cable connector visual recognition method, characterized in that, include: The cable connector images at the assembly station are captured using an industrial camera. At the same time, a corresponding structured text annotation is configured for each captured image to obtain a cable connector image dataset. A network model based on attention mechanism and process modulation is constructed; the network model includes, in sequence, a backbone feature extraction layer, a feature enhancement layer, a process feature modulation layer, and an anchorless detection layer; Construct the loss function for training and train the network model using a cable connector image dataset to obtain the trained network model; The image of the cable connector to be identified is processed by a trained network model to output the detection results of the cable connector. The backbone feature extraction layer uses an improved convolutional neural network as the backbone network, coupled with a multi-scale attention module, to extract multi-scale feature representations. The feature enhancement layer employs a feature pyramid network equipped with a simplified self-attention module to adaptively enhance the saliency of cable connector edges and local details in both spatial and channel dimensions. The process feature modulation layer uses a process feature linear modulation module to embed the production line process and process information into the visual representation and modulate it layer by layer according to the channel, so that the network's visual reasoning is consistent with the expected assembly process constraints. The frameless detection layer is a frameless detection head constructed using a frameless single-stage target detection algorithm. It outputs the detection results of cable joints by predicting class probabilities, bounding box coordinates, and centrality scores. The process feature linear modulation module includes a dual-branch input structure with visual feature branch and process feature branch, as well as an independent parameter generation branch; The independent parameter generation branch is used to generate modulation parameters from the process description information in the production process; the visual feature branch is used to form a visual feature embedding sequence from the multi-scale features output by the feature enhancement layer; and the process feature branch is used to generate a process embedding sequence corresponding to the visual feature embedding sequence. The process feature linear modulation module concatenates the visual feature embedding sequence with the process embedding sequence and generates joint features through linear mapping. Based on the modulation parameters, a channel-wise linear modulation operation is performed on the joint features to obtain the modulated feature sequence. The modulated feature sequence is then processed by a lightweight feature modeling module to obtain process-aware features. The lightweight feature modeling module includes a self-attention layer and a feedforward network.

2. The deep learning-based process-aware cable connector visual recognition method according to claim 1, characterized in that, The backbone feature extraction layer uses ConvNeXt V2 as the backbone network to extract features from the input cable connector image. The ConvNeXt V2 backbone network consists of five consecutive downsampling feature extraction stages. A multi-scale attention module is introduced between adjacent feature extraction stages of ConvNeXt V2 to enhance the output features of the previous stage and use the enhanced features as the input of the next stage.

3. The deep learning-based process-aware cable connector visual recognition method according to claim 2, characterized in that, The backbone feature extraction layer uses three sets of features from different semantic levels of ConvNeXt V2 as the final output of the backbone network.

4. The deep learning-based process-aware cable connector visual recognition method according to claim 1, characterized in that, The feature enhancement layer is based on a feature pyramid network, and a simplified self-attention module is introduced at the feature interaction position of each layer of the pyramid. This module is used to adaptively enhance the features of each layer before transmission and fusion, and obtain multi-scale features, which serve as the input basis for the subsequent feature representation construction and judgment module.

5. The deep learning-based process-aware cable connector visual recognition method according to claim 1, characterized in that, The process feature linear modulation module further includes: remapping the process-aware features back to the feature map space according to their corresponding scales, restoring them to a multi-scale feature representation, and obtaining the process-aware multi-scale features.

6. The deep learning-based process-aware cable connector visual recognition method according to claim 1, characterized in that, The anchorless detection head has three independent prediction branches: a classification branch, a regression branch, and a centrality branch. Each branch consists of multiple convolutional operators connected in series and outputs the corresponding prediction result at the end. The classification branch is used to predict the cable connector type corresponding to the current position; the regression branch is used to predict the distance from the current position to the target boundary; and the centrality branch is used to measure whether the current position is close to the geometric center of the target.

7. The deep learning-based process-aware cable connector visual recognition method according to claim 1, characterized in that, The loss function used for training is constructed by creating a joint loss function consisting of classification loss, boundary regression loss, and centrality loss.

8. The deep learning-based process-aware cable connector visual recognition method according to claim 7, characterized in that, During training, based on the multi-scale feature map output by the feature pyramid network, the spatial locations on each scale feature layer are divided into samples. When any feature location is inside the actual cable connector annotation box, the location is regarded as a positive sample, and the rest are regarded as negative samples. The positive sample locations are used to participate in the loss calculation of category prediction, boundary regression and centrality prediction.

9. A system for the deep learning-based process-aware cable connector visual recognition method as described in claim 1, characterized in that, include: The data acquisition module is used to acquire cable connectors on the assembly station using an industrial camera. At the same time, it configures corresponding structured text annotations for each acquired image to obtain a cable connector image dataset. A network construction module is used to build a network model based on attention mechanisms and process modulation. The network model sequentially includes a backbone feature extraction layer, a feature enhancement layer, a process feature modulation layer, and an anchorless detection layer. The backbone feature extraction layer uses an improved convolutional neural network as the backbone network, coupled with a multi-scale attention module, to extract multi-scale feature representations. The feature enhancement layer uses a feature pyramid network equipped with a simplified self-attention module to adaptively enhance the saliency of cable connector edges and local details in spatial and channel dimensions. The process feature modulation layer uses a process feature linear modulation module to embed production line processes and flow information into the visual representation and modulate it layer by layer according to channel, ensuring that the network's visual inference is consistent with the expected assembly process constraints. The anchorless detection layer is an anchorless detection head built using a single-stage target detection algorithm, which outputs the detection results of the cable connector by predicting the class probability, bounding box coordinates, and centrality score. The network training module is used to construct the loss function for training and train the network model using a cable connector image dataset to obtain the trained network model. The recognition module is used to output the detection results of the cable connectors by passing the images of the cable connectors to be recognized through a trained network model.

Images