Cable accessory installation process management system and method based on deep neural network

Abstract

This invention discloses a cable accessory installation process control system and method based on deep neural networks. The system includes a cable accessory installation video data preprocessing module, a YOLOv8-based DNN object detection module, and a finite state machine comprehensive information analysis module. The video data preprocessing module converts the video into video frames, which are then input into the YOLOv8-based DNN object detection module to extract feature maps and output bounding boxes, category labels, and confidence scores. The finite state machine comprehensive information analysis module incorporates a work log and a time module to calculate process processing time and workflow to determine process qualification. This invention, using the YOLOv8-based DNN object detection module and the finite state machine comprehensive information analysis module, can effectively control the cable accessory installation process, improving the efficiency and quality of cable accessory installation management.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cable accessory detection, in particular to a cable accessory installation process control system and method based on a deep neural network. BACKGROUND

[0002] With the continuous expansion of modern cable installation scale and the increasing complexity of system structure, we are facing more and more quality problems of cable equipment. Among them, the quality problem of cable accessories accounts for a large proportion, especially in the installation and operation stage of accessories. Therefore, the quality control of the installation process and operation stage of cable accessories becomes crucial. Due to the complex cable accessory installation process, we are facing the challenge of being difficult to effectively control; in view of this problem, the present application proposes a cable accessory installation process control system and method based on a deep neural network.

[0003] The target detection algorithm based on a deep neural network is mainly divided into two categories: one-stage target detection algorithm and two-stage target detection algorithm. The one-stage target detection algorithm is usually fast, but the accuracy is relatively low, which is suitable for scenes with high real-time requirements, and the one-stage target detection algorithm usually has a simpler design and implementation method, which is convenient for transplantation and application. Common one-stage target detection algorithms include YOLO, SSD, etc. The two-stage target detection algorithm usually can get more accurate target detection results, but the speed is slower, which is suitable for scenes requiring high precision. Common two-stage target detection algorithms include Faster R-CNN, R-FCN, etc. The present application uses the YOLO algorithm in the one-stage target detection algorithm. SUMMARY

[0004] The purpose of the present application is to provide a cable accessory installation process control system and method based on a deep neural network to control the cable accessory installation process, evaluate the quality of the on-site construction process, reduce the impact of human factors on the quality of accessory installation, and improve the efficiency and quality of cable accessory installation control.

[0005] The technical scheme of the present application: a cable accessory installation process management system based on a deep neural network, comprising a cable accessory installation video data preprocessing module, a DNN target detection module based on YOLOV8, and a finite state machine comprehensive information analysis module; the process of cable accessory installation is recorded in the form of video, the cable accessory installation video data preprocessing module reads the video file, obtains the file name and path information, slices the video into pictures, performs image enhancement on the pictures, and eliminates pictures with unqualified definition; the processed pictures are input into the trained DNN target detection module based on YOLOV8 for detection, feature maps are extracted, and a bounding box, a class label, and a confidence score are output, thereby realizing the identification of the operating tools and the state of the cable accessory required for the cable accessory installation process; the state of the operating tools and the cable accessory identified by the DNN target detection module based on YOLOV8 is input into the finite state machine (FSM) comprehensive information analysis module, the FSM comprehensive information analysis module subdivides the cable accessory installation process into multiple procedures, and the FSM comprehensive information analysis module determines the procedure according to the state of the operating tools and the cable accessory identified by the DNN target detection module based on YOLOV8; for each procedure, the start and end states of the procedure are taken as state nodes in the FSM, the state of the operating tools and the cable accessory required by each procedure is used to trigger the state transition of the FSM, and a work log and a time module are added to the FSM, when the start state of the procedure is detected, the time is started to be recorded and the operating tools appearing are recorded; when the end state of the procedure is detected, the time is ended to be recorded and the operating tools appearing are recorded, the processing time of the procedure is calculated, the obtained processing time of the procedure is compared with the standard processing time of the procedure, and whether the processing time of the procedure meets the standard is determined; according to the state of the operating tools and the cable accessory appearing at the start of the procedure and the state of the operating tools and the cable accessory appearing at the end of the procedure, whether the operation flow of the procedure meets the standard is determined.

[0006] Further, the method for image enhancement on the pictures is a logarithmic transformation enhancement method: the pictures are converted into numpy arrays, and the pixel values are logarithmically transformed, normalized, and converted into 8-bit images; the way to eliminate pictures with unqualified definition is to calculate the gradient of the image using the Sobel operator and obtain the definition score by calculating the mean and standard deviation of the gradient, and the pictures with unqualified definition are discarded.

[0007] Further, the DNN target detection module based on YOLOV8 is composed of a feature extraction network, a feature fusion network, and a prediction network, the feature extraction network is used to extract features, the feature fusion network is used to fuse the features of the shallower layer and the features of the deeper layer in a skip connection manner, and the prediction network is used to output a bounding box, a class label, and a confidence score.

[0008] Furthermore, the feature extraction network includes 5 feature extraction stages, the feature fusion network includes 4 feature fusion stages, and the prediction network includes 3 decoupled head modules; the first feature extraction stage is the first convolutional module, the second feature extraction stage is composed of the second convolutional module and the first C2f module in sequence, the third feature extraction stage is composed of the third convolutional module and the second C2f module in sequence, the fourth feature extraction stage is composed of the fourth convolutional module and the third C2f module in sequence, and the fifth feature extraction stage is composed of the fifth convolutional module, the fourth C2f module and the SPPF module in sequence.

[0009] Furthermore, the images processed by the cable accessory installation video data preprocessing module undergo feature extraction in the first feature extraction stage. These feature maps are then input into the second feature extraction stage, the third, and the fourth stages. Finally, the feature maps extracted in the fifth and fourth stages are input into the fifth feature extraction stage. The feature maps extracted in the fifth stage are then upsampled and fused with those extracted in the fourth stage. The fused feature maps are then processed by the fifth C2f module, and subsequently upsampled and fused with those extracted in the third stage. The fused feature maps are processed by the sixth C2f module to obtain the first fused feature map. The first fused feature map is then processed by the sixth convolution module and fused with the feature map output by the fifth C2f module. The fused feature map is then processed by the seventh C2f module to obtain the second fused feature map. The second fused feature map is then processed by the seventh convolution module and fused with the feature map extracted in the fifth feature extraction stage. The fused feature map is then processed by the eighth C2f module to obtain the third fused feature map. The first fused feature map, the second fused feature map, and the third fused feature map are then input into the first decoupling head module, the second decoupling head module, and the third decoupling head module, respectively, for prediction.

[0010] Furthermore, the C2f module consists of a separation layer, a bottleneck layer, a connection layer, and multiple convolutional layers. For the input features, the features are first extracted through a convolutional layer, then input to the bottleneck layer through the separation layer to reduce the number of parameters, and then extracted through another convolutional layer before being connected to the next convolutional layer through the connection layer.

[0011] Furthermore, the first decoupling head module, the second decoupling head module, and the third decoupling head module have the same structure. The first decoupling head module includes two branches. The first branch includes, in sequence, an eighth convolutional module, a two-dimensional convolutional module, and a Bbox loss module. The second branch includes, in sequence, a ninth convolutional module, a two-dimensional convolutional module, and a Cls loss module. The eighth convolutional module and the ninth convolutional module are both two consecutive convolutional layers.

[0012] This invention also provides a method for controlling the installation process of cable accessories based on deep neural networks, the steps of which are as follows:

[0013] Step S1: Video preprocessing: Slice the video into a series of JPG format images, perform image enhancement on the images, evaluate the image sharpness, use the Sobel operator to calculate the gradient of the image and obtain the sharpness score by calculating the mean and standard deviation of the gradient, and discard images that do not meet the sharpness requirements.

[0014] Step S2: Establish and train a YOLOv8-based DNN object detection module: The YOLOv8-based DNN object detection module consists of a feature extraction network, a feature fusion network, and a prediction network. The feature extraction network extracts features, and the feature fusion network fuses shallower and deeper feature maps using skip connections. The prediction network outputs bounding boxes, class labels, and confidence scores. Clear videos of cable accessory installation are captured at the factory, and blurry videos of cable accessory installations from multiple locations are collected. Images of the tools required for cable accessory installation are obtained through web scraping. The collected videos and images are processed using a cable accessory installation video data preprocessing module. The processed data is divided into training, cross-validation, and test datasets in a 6:2:2 ratio. The training dataset is used to train the YOLOv8-based DNN object detection module, and the cross-validation and test datasets are used to validate and test the module.

[0015] Step S3: Subdivide the cable accessory installation process into multiple procedures. For each procedure, use its start and end states as state nodes in a finite state machine. When the start state of the procedure is detected, start recording the time. When the end state of the procedure is detected, stop recording the time and calculate the procedure processing time. Compare the obtained procedure processing time with the standard procedure processing time to determine whether the procedure processing time meets the standard.

[0016] Furthermore, the detection steps of the DNN object detection module based on YOLOv8 are as follows:

[0017] Step S21: Image preprocessing; Input an image, and after feature extraction, generate three tensors with different shapes and sizes;

[0018] Step S22: Generate and concatenate prediction information: After convolving the three tensors generated in step S21, two types of tensors are generated. The first tensor is used to predict the position and size information of the detection box, and the second tensor is used to predict the category information of the detection box. The two tensors are concatenated into a new tensor.

[0019] Step S23: Generate anchor box coordinates and scaling ratio: The new tensor generated in step S22 is fed into the make_anchors function to generate anchor box coordinates and scaling ratio for each pixel of the feature map;

[0020] Step S24: Segment the feature map tensor; Use the view function to change the shape of the new tensor generated in step S22, concatenate the tensor with the three tensors generated in step S21, and then segment it according to the second dimension, dividing it into tensor box and tensor cls.

[0021] Step S25: Generate detection boxes and category information: The tensor box is reshaped by the view function, and then the second and third dimensions are swapped by the transpose function. Multi-class probability normalization is performed along the channel direction. Finally, a new tensor is generated by convolution and the view function. The new tensor and the anchor box coordinates are fed into the dist2bbox function to generate the position and size information of the predicted box. The tensor cls is processed by the sigmoid function to generate the category information of the predicted box.

[0022] Furthermore, the cable accessory installation process is divided into several steps using a finite state machine (FSM) integrated information analysis module. The start and end operations of each step are used as nodes for state transitions within the FSM integrated information analysis module. The state transitions are triggered by the required tools and the state of the cable accessories for each step. A work log and time module are added to the FSM to record the processing time and the tools used in each step. The processing time of each step is compared with the standard processing time to determine if the process time meets the standard. The operation flow of each step is also determined based on the tools used at the start and end of the step. The cable accessory installation steps are as follows:

[0023] Process S30: Initial state;

[0024] Procedure S31: Begin stripping the outer sheath, the cable state changes from the initial state to the procedure S31 state, end the asphalt cleaning operation and record the duration of procedure S31;

[0025] Step S32: Begin scraping the conductive layer downwards from the outer sheath break, the cable state changes from step S31 to step S32, end the scraping of the conductive layer and record the duration of step S32.

[0026] Step S33: Begin welding aluminum strips onto the metal sheath. The cable state changes from step S32 to step S33. End the welding operation and record the duration of step S33.

[0027] Step S34: Begin removing the metal sheath, the cable status changes from step S33 to step S34, end the cleaning of the metal sheath break and record the duration of step S34.

[0028] Step S35: Begin cutting off the water-blocking layer of the cable; the cable status changes from the state in step S34 to the state in step S35; end the cable cutting operation and record the duration of step S35;

[0029] Step S36: Begin stripping the wire core insulation, the cable state changes from step S35 to step S36, end the stripping of wire core insulation and record the duration of step S36.

[0030] Step S37: Begin scraping the cable insulation shield with a glass plate. The cable condition changes from the state in step S36 to the state in step S37. End the operation of making the cut cone surface and record the duration of step S37.

[0031] Step S38: Begin the insulation layer polishing operation. The cable state changes from the state of step S37 to the state of step S38. End the insulation polishing operation and record the duration of step S38.

[0032] The beneficial effects of this invention are as follows: This invention uses a YOLOv8-based DNN object detection module and a finite state machine integrated information analysis module to control the installation process of cable accessories, improving the efficiency and quality of cable accessory installation control; This invention uses a logarithmic transformation enhancement method for image enhancement, which can enhance the brightness and contrast of images, making the difference between clear and blurry images more obvious. Discarding blurry images ensures that only clearer images are used in subsequent processing, improving the accuracy and effect of processing; The C2f module has a richer gradient flow and adjusts the number of channels for different scale models, enabling the model to better learn image features, improving detection accuracy and robustness; Using the SPPF module, three different sized feature maps can be output in different parts to detect small, medium, and large-scale objects, allowing the model to better adapt to objects of different sizes and improving the model's robustness; A top-down feature fusion network is used to fuse low-level feature maps with high-level feature maps, thereby generating feature maps with richer semantic information. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the overall structure of the present invention.

[0034] Figure 2 This is a diagram showing the overall architecture of the YOLOv8-based DNN object detection module of the present invention.

[0035] Figure 3This is a schematic diagram of the C2f module structure of the present invention. Detailed Implementation

[0036] The invention will now be explained with reference to the accompanying drawings.

[0037] refer to Figure 1 This system, based on a deep neural network, controls the installation process of cable accessories. It includes a video data preprocessing module, a YOLOv8-based DNN object detection module, and a finite state machine (FSM) integrated information analysis module. The system records the cable accessory installation process as video. The video data preprocessing module reads the video file, obtains the filename and path information, slices the video into images, performs image enhancement, and removes images with insufficient clarity. The processed images are then input into the trained YOLOv8-based DNN object detection module for detection, extracting feature maps and outputting bounding boxes, category labels, and confidence scores. This enables the identification of the operating tools and cable accessories required for the installation process. The identified operating tools and cable accessory states from the YOLOv8-based DNN object detection module are input into the FSM integrated information analysis module, which then analyzes the cable... The accessory installation process is subdivided into multiple steps. The finite state machine integrated information analysis module determines the current step based on the state of the operating tools and cable components identified by the YOLOv8-based DNN target detection module. For each step, the start and end states are used as state nodes in the finite state machine. The state transition of the finite state machine is triggered by the state of the operating tools and cable accessories required for each step. A work log and time module are added to the finite state machine. When the start state of the step is detected, the time and the operating tools appearing are recorded. When the end state of the step is detected, the time recording stops and the operating tools appearing are recorded. The step processing time is calculated and compared with the standard step processing time to determine whether the step processing time meets the standard. Based on the state of the operating tools and cable accessories appearing at the start and end of the step, the process operation flow is determined to be compliant.

[0038] The image enhancement method is the logarithmic transformation enhancement method: convert the image into a NumPy array, and perform logarithmic transformation, normalization, and conversion of pixel values ​​into an 8-bit image. The method for removing images that do not meet the sharpness requirements is to use the Sobel operator to calculate the gradient of the image and obtain a sharpness score by calculating the mean and standard deviation of the gradient, and then discard the images that do not meet the sharpness requirements.

[0039] refer to Figure 2The DNN object detection module based on YOLOv8 consists of a feature extraction network, a feature fusion network, and a prediction network. The feature extraction network extracts features, the feature fusion network uses skip connections to fuse shallower feature maps with deeper feature maps, and the prediction network outputs bounding boxes, class labels, and confidence scores.

[0040] refer to Figure 2 The feature extraction network includes 5 feature extraction stages, the feature fusion network includes 4 feature fusion stages, and the prediction network includes 3 decoupled head modules. The first feature extraction stage is the first convolutional module, the second feature extraction stage is composed of the second convolutional module and the first C2f module in sequence, the third feature extraction stage is composed of the third convolutional module and the second C2f module in sequence, the fourth feature extraction stage is composed of the fourth convolutional module and the third C2f module in sequence, and the fifth feature extraction stage is composed of the fifth convolutional module, the fourth C2f module and the SPPF module in sequence.

[0041] refer to Figure 2 After the video data preprocessing module for cable accessory installation processes the images, feature maps are extracted in the first feature extraction stage. These feature maps are then input into the second feature extraction stage, the third, and the fourth stages. Finally, the feature maps extracted in the fifth stage are upsampled and fused with those extracted in the fourth stage. The fused feature map is then processed by the fifth C2f module, upsampled, and fused with the feature map extracted in the third stage. The fused feature map is input into the sixth C2f module for processing to obtain the first fused feature map. The first fused feature map is processed by the sixth convolution module and then fused with the feature map output by the fifth C2f module. The fused feature map is input into the seventh C2f module for processing to obtain the second fused feature map. The second fused feature map is processed by the seventh convolution module and then fused with the feature map extracted in the fifth feature extraction stage. The fused feature map is input into the eighth C2f module for processing to obtain the third fused feature map. The first fused feature map, the second fused feature map, and the third fused feature map are input into the first decoupling head module, the second decoupling head module, and the third decoupling head module, respectively, for prediction.

[0042] refer to Figure 3 The C2f module consists of a separation layer, a bottleneck layer, a connection layer, and multiple convolutional layers. For input features, the features are first extracted through a convolutional layer, then input through a separation layer to the bottleneck layer to reduce the number of parameters, and then extracted through another convolutional layer before being connected to the next convolutional layer through a connection layer.

[0043] refer toFigure 2 The first decoupling head module, the second decoupling head module, and the third decoupling head module have the same structure. The first decoupling head module includes two branches. The first branch includes the eighth convolutional module, the two-dimensional convolutional module, and the Bbox loss module in sequence. The second branch includes the ninth convolutional module, the two-dimensional convolutional module, and the Cls loss module in sequence. The eighth convolutional module and the ninth convolutional module are both two consecutive convolutional layers.

[0044] A method for controlling the installation process of cable accessories based on deep neural networks, comprising the following steps:

[0045] Step S1: Video preprocessing: Slice the video into a series of JPG format images, perform image enhancement on the images, evaluate the image sharpness, use the Sobel operator to calculate the gradient of the image and obtain the sharpness score by calculating the mean and standard deviation of the gradient, and discard images that do not meet the sharpness requirements.

[0046] Step S2: Establish and train a YOLOv8-based DNN object detection module: The YOLOv8-based DNN object detection module consists of a feature extraction network, a feature fusion network, and a prediction network. The feature extraction network extracts features, and the feature fusion network fuses shallower and deeper feature maps using skip connections. The prediction network outputs bounding boxes, class labels, and confidence scores. Clear videos of cable accessory installation are captured at the factory, and blurry videos of cable accessory installations from multiple locations are collected. Images of the tools required for cable accessory installation are obtained through web scraping. The collected videos and images are processed using a cable accessory installation video data preprocessing module. The processed data is divided into training, cross-validation, and test datasets in a 6:2:2 ratio. The training dataset is used to train the YOLOv8-based DNN object detection module, and the cross-validation and test datasets are used to validate and test the module.

[0047] Step S3: The finite state machine integrated information analysis module subdivides the cable accessory installation process into multiple procedures. Based on the state of the operating tools and cable components identified by the DNN target detection module based on YOLOv8, the finite state machine integrated information analysis module determines the current procedure.

[0048] The detection steps of the YOLOv8-based DNN object detection module are as follows:

[0049] Step S21: Image preprocessing; Input an image, and after feature extraction, generate three tensors with different shapes and sizes;

[0050] Step S22: Generate and concatenate prediction information: After convolving the three tensors generated in step S21, two types of tensors are generated. The first tensor is used to predict the position and size information of the detection box, and the second tensor is used to predict the category information of the detection box. The two tensors are concatenated into a new tensor.

[0051] Step S23: Generate anchor box coordinates and scaling ratio: The new tensor generated in step S22 is fed into the make_anchors function to generate anchor box coordinates and scaling ratio for each pixel of the feature map;

[0052] Step S24: Segment the feature map tensor; Use the view function to change the shape of the new tensor generated in step S22, concatenate the tensor with the three tensors generated in step S21, and then segment it according to the second dimension, dividing it into tensor box and tensor cls.

[0053] Step S25: Generate detection boxes and category information: The tensor box is reshaped by the view function, and then the second and third dimensions are swapped by the transpose function. Multi-class probability normalization is performed along the channel direction. Finally, a new tensor is generated by convolution and the view function. The new tensor and the anchor box coordinates are fed into the dist2bbox function to generate the position and size information of the predicted box. The tensor cls is processed by the sigmoid function to generate the category information of the predicted box.

[0054] The cable accessory installation process is divided into several steps using a finite state machine (FSM) integrated information analysis module. The start and end operations of each step serve as nodes for state transitions within the FSM integrated information analysis module. The state transitions are triggered by the required tools and the state of the cable accessories for each step. A work log and time module are added to the FSM to record the processing time and the tools used in each step. The processing time of each step is compared with the standard processing time to determine if the process time meets the standard. The operation flow of each step is also determined based on the tools used at the start and end of the step. The cable accessory installation steps are as follows:

[0055] Process S30: Initial state;

[0056] Procedure S31: Begin stripping the outer sheath, the cable state changes from the initial state to the procedure S31 state, end the asphalt cleaning operation and record the duration of procedure S31;

[0057] Step S32: Begin scraping the conductive layer downwards from the outer sheath break, the cable state changes from step S31 to step S32, end the scraping of the conductive layer and record the duration of step S32.

[0058] Step S33: Begin welding aluminum strips onto the metal sheath. The cable state changes from step S32 to step S33. End the welding operation and record the duration of step S33.

[0059] Step S34: Begin removing the metal sheath, the cable status changes from step S33 to step S34, end the cleaning of the metal sheath break and record the duration of step S34.

[0060] Step S35: Begin cutting off the water-blocking layer of the cable; the cable status changes from the state in step S34 to the state in step S35; end the cable cutting operation and record the duration of step S35;

[0061] Step S36: Begin stripping the wire core insulation, the cable state changes from step S35 to step S36, end the stripping of wire core insulation and record the duration of step S36.

[0062] Step S37: Begin scraping the cable insulation shield with a glass plate. The cable condition changes from the state in step S36 to the state in step S37. End the operation of making the cut cone surface and record the duration of step S37.

[0063] Step S38: Begin the insulation layer polishing operation. The cable state changes from the state of step S37 to the state of step S38. End the insulation polishing operation and record the duration of step S38.

[0064] This invention also provides an embodiment for verifying the detection steps of a YOLOv8-based DNN object detection module. Taking a 640*640 image as an example, after feature extraction and preprocessing, three tensors with shapes [B,C,80,80], [B,C,40,40], and [B,C,20,20] are generated; where B is the amount of data for word training, and C is the depth of the feature map generated after convolution of the input image. The tensors in the feature map are convolved to generate two types of tensors. Taking tensors with input sizes W and H as an example, the first type of tensor is obtained by CV2, with a shape of [B,4*r]. The first tensor, [reg_max, W, H], is used to predict the position and size of the detection box; where reg_max controls the range of predicted coordinate offsets in bounding box regression. The second tensor, obtained through CV3, has a shape of [B, nc, W, H] and is used to predict the category information of the detection box; where nc is the number of predicted categories. The two tensors are concatenated into a tensor of shape [B, 4*reg_max + nc, W, H] and fed into the make_anchors function to generate anchor box coordinates and scaling for each pixel of the feature map. The view function is used to view the tensor [B, 4*reg_max + nc, W, H]. The tensors of the three feature maps are changed from [nc, W, H] to [B, 4*reg_max + nc, W*H]. Then, the tensors of the three feature maps are concatenated to generate a tensor of shape [B, 4*reg_max + nc, A] (A = W1*H1 + W2*H2 + W3*H3). This tensor is then segmented along the second dimension into a tensor of shape [B, 4*reg_max, A] named 'box' and a tensor of shape [B, nc, A] named 'cls'. The 'box' tensor is transformed into a tensor of shape [B, 4, C, A] (C = reg_max) using the view function. Finally, the tensor is transformed into a tensor of shape [B, 4, C, A] using the transpose function. The first and third dimensions are swapped to generate a tensor of shape [B,C,4,A]. This tensor is then normalized for multi-class probabilities along the channel direction, followed by convolution and a view function to finally generate a tensor of shape [B,4,A]. The value of the second dimension represents the position offset. Finally, the generated tensor and anchor box coordinates are fed into the dist2bbox function to generate the position and size information (x,y,w,h) of the predicted bounding box. The tensor cls of shape [B,nc,A] is passed through a sigmoid function to generate the predicted bounding box category information; where x and y are the center coordinates, and w and h are the width and height, respectively.

[0065] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A cable accessory installation process management system based on a deep neural network, comprising a cable accessory installation video data preprocessing module, a YOLOV8-based DNN target detection module, and a finite state machine comprehensive information analysis module. The process of installing the cable accessory is recorded in the form of a video, the cable accessory installation video data preprocessing module reads the video file, obtains the file name and path information, cuts the video into pictures, performs image enhancement on the pictures, and rejects pictures with unqualified definition; the processed pictures are input into the trained YOLOV8-based DNN target detection module for detection, feature maps are extracted, and a bounding box, a class label and a confidence score are output, thereby realizing recognition of the operating tools required for the cable accessory installation process and the state of the cable accessory; The state of the operating tools and the cable accessory recognized by the YOLOV8-based DNN target detection module is input into the finite state machine comprehensive information analysis module, the finite state machine comprehensive information analysis module subdivides the cable accessory installation process into multiple procedures, and judges the procedure according to the state of the operating tools and the cable accessory recognized by the YOLOV8-based DNN target detection module; for each procedure, the start and end states of the procedure are taken as state nodes in the finite state machine, the state transition of the finite state machine is triggered by the operating tools and the state of the cable accessory required for each procedure, and a work log and a time module are added to the finite state machine, when the start state of the procedure is detected, the time is started to be recorded and the operating tools appearing are recorded; when the end state of the procedure is detected, the time is ended to be recorded and the operating tools appearing are recorded, the processing time of the procedure is calculated, the obtained processing time of the procedure is compared with the standard processing time of the procedure, and whether the processing time of the procedure meets the standard is judged; according to the state of the operating tools and the cable accessory appearing at the start of the procedure and the state of the operating tools and the cable accessory appearing at the end of the procedure, whether the operation flow of the procedure meets the standard is judged.

2. The deep neural network-based cable accessory installation process governance system of claim 1, wherein, The method of image enhancement on the pictures is a logarithmic transformation enhancement method: the pictures are converted into numpy arrays, and the pixel values are logarithmically transformed, normalized and converted into 8-bit images; the method of rejecting pictures with unqualified definition is: using a Sobel operator to calculate the gradient of the image and obtaining a definition score by calculating the mean and standard deviation of the gradient, and discarding pictures with unqualified definition.

3. The deep neural network-based cable accessory installation process governance system of claim 1, wherein, The YOLOV8-based DNN target detection module is composed of a feature extraction network, a feature fusion network and a prediction network, features are extracted through the feature extraction network, the feature fusion network is used to fuse the feature maps of the shallower layer and the feature maps of the deeper layer in a skip connection manner, and the prediction network outputs a bounding box, a class label and a confidence score.

4. The deep neural network-based cable accessory installation process governance system of claim 3, wherein, The feature extraction network includes five feature extraction stages, the feature fusion network includes four feature fusion stages, and the prediction network includes three decoupled head modules; The first feature extraction stage is a first convolution module, the second feature extraction stage is composed of a second convolution module and a first C2f module in sequence, the third feature extraction stage is composed of a third convolution module and a second C2f module in sequence, the fourth feature extraction stage is composed of a fourth convolution module and a third C2f module in sequence, and the fifth feature extraction stage is composed of a fifth convolution module, a fourth C2f module and an SPPF module in sequence.

5. The deep neural network-based cable accessory installation process governance system of claim 4, wherein, The picture processed by the cable accessory installation video data preprocessing module is subjected to a first feature extraction stage to extract a feature map, the feature map extracted by the first feature extraction stage is input into a second feature extraction stage, the feature map extracted by the second feature extraction stage is input into a third feature extraction stage, the feature map extracted by the third feature extraction stage is input into a fourth feature extraction stage, the feature map extracted by the fourth feature extraction stage is input into a fifth feature extraction stage; The feature map extracted by the fifth feature extraction stage is subjected to feature fusion with the feature map extracted by the fourth feature extraction stage through upsampling, the fused feature map is input into a fifth C2f module for processing, and then subjected to feature fusion with the feature map extracted by the third feature extraction stage through upsampling, the fused feature map is input into a sixth C2f module to obtain a first fused feature map, the first fused feature map is subjected to feature fusion with the feature map output by the fifth C2f module through a sixth convolution module, the fused feature map is input into a seventh C2f module to obtain a second fused feature map, the second fused feature map is subjected to feature fusion with the feature map extracted by the fifth feature extraction stage through a seventh convolution module, and the fused feature map is input into an eighth C2f module to obtain a third fused feature map; the first fused feature map, the second fused feature map and the third fused feature map are respectively input into a first decoupling head module, a second decoupling head module and a third decoupling head module for prediction.

6. The deep neural network-based cable accessory installation process governance system of claim 4, wherein, The C2f module is composed of a separation layer, a bottleneck layer, a connection layer and a plurality of convolution layers, for input features, first, a convolution layer is used to extract features, then the separation layer is used to input the bottleneck layer to reduce the number of parameters, and then another convolution layer is used to extract features through the connection layer and the next convolution layer.

7. The deep neural network-based cable accessory installation process governance system of claim 5, wherein, The first decoupling head module, the second decoupling head module and the third decoupling head module have the same structure, the first decoupling head module includes two branches, the first branch includes an eighth convolution module, a two-dimensional convolution module and a Bbox loss module in sequence; the second branch includes a ninth convolution module, a two-dimensional convolution module and a Cls loss module in sequence; the eighth convolution module and the ninth convolution module are both two continuous convolution layers.

8. A deep neural network-based cable accessory installation process management method, characterized by, The steps are as follows: Step S1: video preprocessing: slice the video into a series of JPG format pictures, perform image enhancement on the pictures, evaluate the image definition, calculate the gradient of the image using the Sobel operator and obtain the definition score by calculating the mean and standard deviation of the gradient, and discard the pictures with unqualified definition; Step S2: Establish and train the DNN target detection module based on YOLOV8: The DNN target detection module based on YOLOV8 is composed of a feature extraction network, a feature fusion network and a prediction network, the features are extracted through the feature extraction network, the feature fusion network is used to fuse the features of the shallow layer and the deep layer in a skip connection manner, and the boundary box, the class label and the confidence score are output through the prediction network; clear cable accessory installation videos are shot in the factory, fuzzy videos of cable accessory installation in multiple places are collected, pictures of operation tools required for cable accessory installation are obtained through network crawling, and the collected videos and pictures are processed through the cable accessory installation video data preprocessing module; the processed data is divided into training data set, cross-validation data set and test data set in the ratio of 6:2:2 respectively; The DNN target detection module based on YOLOV8 is trained using the training data set, and is verified and tested using the cross-validation data set and the test data set; Step S3: Subdivide the cable accessory installation process into multiple procedures, for each procedure, take its start and end state as state nodes in the finite state machine, when the procedure start state is detected, start recording the time, when the procedure end state is detected, end recording the time and calculate the procedure processing time, compare the obtained procedure processing time with the standard procedure processing time to determine whether the procedure processing time is up to standard.

9. The deep neural network-based cable accessory installation process management method according to claim 8, characterized in that, The detection steps of the DNN target detection module based on YOLOV8 are as follows: Step S21: image preprocessing; input an image, the image generates three tensors with different shapes and sizes after feature extraction; Step S22: generate and splice prediction information: generate two kinds of tensors after convolution of the three tensors generated in step S21, the first kind of tensor is used to predict the position and size information of the detection box, and the second kind of tensor is used to predict the category information of the detection box; splice the two kinds of tensors into a new tensor; Step S23: generate anchor box coordinates and scaling ratio: send the new tensor spliced in step S22 into the make_anchors function to generate anchor box coordinates and scaling ratio for each pixel point of the feature map; Step S24: split the feature map tensor; use the view function to change the shape of the new tensor spliced in step S22, splice the tensor with the shape changed with the three tensors generated in step S21, and then split along the second dimension to get tensor box and tensor cls; Step S25: generate detection box and category information: change the shape of tensor box through the view function, then exchange the second dimension and the third dimension through the transpose function, perform multi-class probability normalization along the channel direction, and finally generate a new tensor through convolution and the view function, send the new tensor and the anchor box coordinates into the dist2bbox function to generate the position and size information of the prediction box; generate the prediction box category information after the tensor cls passes through the sigmoid function.

10. The deep neural network-based cable accessory installation process management method of claim 8, wherein, The cable accessory installation process is divided into several procedures by the finite state machine comprehensive information analysis module, the starting operation and the ending operation of each procedure are taken as the nodes of the state transition of the finite state machine comprehensive information analysis module, the state transition of the finite state machine is triggered by the operation tools required by each procedure and the state of the cable accessory, the work log and the time module are added to the finite state machine to record the processing time of each procedure and the operation tools appearing, whether the processing time of the procedure meets the standard is judged according to the comparison between the processing time of the procedure and the standard processing time of the procedure; whether the operation flow of the procedure meets the standard is judged according to the operation tools appearing at the beginning of the procedure and the operation tools appearing at the end of the procedure, and the cable accessory installation procedure is as follows: Procedure S30: initial state; Procedure S31: start stripping the outer sheath operation, the cable state is transformed from the initial state to the procedure S31 state, end the cleaning asphalt operation and record the duration of procedure S31; Procedure S32: start scraping the conductive layer down the outer sheath break, the cable state is transformed from the procedure S31 state to the procedure S32 state, end the scraping operation and record the duration of procedure S32; Procedure S33: start welding aluminum strip on the metal sheath, the cable state is transformed from the procedure S32 state to the procedure S33 state, end the welding operation and record the duration of procedure S33; Procedure S34: start removing the metal sheath operation, the cable state is transformed from the procedure S33 state to the procedure S34 state, end the cleaning operation at the metal sheath break and record the duration of procedure S34; Procedure S35: start cutting off the cable water-blocking layer operation; the cable state is transformed from the procedure S34 state to the procedure S35 state; end the cutting operation and record the duration of procedure S35; Procedure S36: start stripping the core insulation operation, the cable state is transformed from the procedure S35 state to the procedure S36 state, end the stripping operation and record the duration of procedure S36; Procedure S37: start scraping the cable insulation shield with a glass sheet, the cable state is transformed from the procedure S36 state to the procedure S37 state, end the break taper operation and record the duration of procedure S37; Procedure S38: start the insulation layer polishing operation, the cable state is transformed from the procedure S37 state to the procedure S38 state, end the insulation polishing operation and record the duration of procedure S38.

Images