A method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules

Abstract

The present invention discloses a method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules. The method comprises: 1) acquiring a two-dimensional image of the module surface and generating a dataset of various fastener types; 2) building and training a deep learning model; 3) performing coarse positioning and type identification on the fasteners, and cropping fastener region sub-images; 4) pre-processing each fastener sub-image and extracting fastener BLOB features; 5) solving the fastener's two-dimensional pose based on ellipse fitting and region intersection methods; 6) acquiring a three-dimensional point cloud of the module surface and calibrating the two- and three-dimensional coordinate systems; 7) cropping the fastener region point cloud using the two-dimensional coarse positioning information; 8) pre-processing and plane segmenting the point cloud; and 9) fitting the fastener top surface equation to solve for the top surface height and the inclination angle of the central axis. The advantage of the present invention is that it can accurately identify and locate the type, pose, and other characteristics of a large number of threaded fasteners of various types on the surface of retired lithium battery modules, even in complex background environments.

Description

Technical Field

[0001] The present invention relates to the field of visual target detection, and in particular to a method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules. Background Art

[0002] With the rapid development of intelligent manufacturing, machine vision technology is being used more and more widely in industrial automation inspection. In the automated disassembly of retired power lithium battery modules, the use of machine vision to quickly identify and locate a large number of various threaded fasteners of different sizes and shapes on the surface of lithium battery modules is gradually becoming a new development trend.

[0003] Threaded connections are a common way to group single battery cells. A single module requires numerous threaded fasteners to be disassembled. Current disassembly methods rely primarily on manual or semi-automatic methods, which are inefficient and costly. Machine vision, a core technology for robotic environmental perception, can simultaneously identify critical information such as the type, size, and posture of a large number of fasteners. However, due to the low degree of standardization in lithium battery modules, the structural dimensions of modules, as well as the size and type of fasteners on their surfaces, vary from manufacturer to manufacturer. Currently, there is no mature visual recognition technology capable of identifying and locating various fasteners on their surfaces without prior knowledge of the module's structural information. Existing fastener visual recognition and location methods are primarily categorized into two types: 2D image recognition and location, and 3D point cloud recognition and location. While 2D image recognition and location methods offer high speed and accuracy, they cannot obtain the 3D position information of the fasteners. 3D point cloud recognition and location methods can obtain 3D position information of fasteners, but they require large amounts of data, are slow, have low accuracy, and struggle to remove interference from the complex background surrounding the target.

[0004] Therefore, existing fastener identification and positioning methods have certain deficiencies and limitations, and cannot meet the industrial demand for rapid identification and positioning of a large number of multi-type threaded fasteners. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and propose a visual recognition and positioning method for threaded fasteners on the surface of retired lithium battery modules based on two-dimensional images and three-dimensional point clouds. The specific implementation is: using a two-dimensional camera to collect typical surface images of retired lithium battery modules, after constructing a sample set, labeling various types of fasteners in the image and dividing the samples proportionally to obtain a fastener dataset; building a SqueezeNet-SSD deep learning model based on the SqueezeNet backbone network and the SSD model, setting the training parameters and completing the model training and testing based on the dataset; using the trained model to identify the type of fasteners in the module image to be detected, and at the same time Record the coordinates of the rectangular area where each fastener is located, and crop the fastener sub-image based on the coordinates to complete the rough positioning of the fastener; pre-process the sub-image and extract the fastener BLOB feature area based on the large law, and further use the ellipse fitting and area intersection method to solve the precise two-dimensional center coordinates and head posture angles of various fasteners; use a three-dimensional camera to collect the surface point cloud of the lithium battery module and complete the calibration of the two-dimensional and three-dimensional coordinate systems; crop the fastener area point cloud based on the two-dimensional rough positioning coordinates and the calibration information; pre-process the cropped point cloud and realize the segmentation of the fastener top surface point cloud; perform plane fitting on the fastener top surface point cloud, obtain the top surface equation, and further solve the fastener top surface height and the center axis inclination angle. The present invention can meet the needs of efficient and accurate recognition and positioning of a large number of fasteners of various types under the complex background environment of the retired power lithium battery module surface, and compared with the existing fastener recognition and positioning algorithms, the target detection results of the present invention are not easily affected by the high reflective properties of metal fasteners and external light, and can still have high recognition efficiency and accuracy when detecting a large number of targets.

[0006] To achieve the above objectives, an embodiment of the present invention provides a method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules, comprising the following steps:

[0007] Step 1: Use a two-dimensional camera to collect surface images of typical retired lithium battery modules, ensuring that the number of sample images of the module surface is no less than 800, and the number of each type of fasteners included is no less than 5,000. At the same time, in order to achieve sample balance, the difference in the number of fasteners of each type should not exceed 100. Use operations such as brightness adjustment and affine transformation to enhance the initial sample images and obtain a batch of new sample images. Merge the new and old batches of sample images into a sample set, and uniformly scale the images in the sample set to a size of 1280*1280px. Use general deep learning tools to label the fasteners in each image to obtain a dataset of various common fasteners on the module surface;

[0008] Step 2: Build a SqueezeNet-SSD deep learning model based on the SqueezeNet backbone network and the SSD model, set hyperparameters, divide the fastener dataset constructed in step 1 proportionally, and then train and test the model;

[0009] Step 3: Use the model trained in Step 2 to identify the type of fasteners in the image of the module to be inspected. The coordinates of the corner points of the rectangular area where each fastener is located are recorded. The fastener sub-image is cropped based on these coordinates to complete the coarse positioning of the fasteners.

[0010] Step 4: Complete the preprocessing of each fastener sub-image in step 3 and extract the fastener BLOB feature area;

[0011] Step 5: Based on the BLOB feature area extracted in step 4, the ellipse fitting and region intersection method are used to solve the two-dimensional pose information such as the center coordinates of the fastener head and the groove posture angle;

[0012] Step 6: Use a 3D camera to collect the surface point cloud of the retired lithium battery module and calibrate the relative position between the 2D image coordinate system and the 3D point cloud coordinate system;

[0013] Step 7: crop the point cloud of the area where the fastener is located based on the two-dimensional coarse positioning coordinates of step 3 and the calibration information of step 6;

[0014] Step 8: Preprocess the fastener point cloud obtained by cropping in step 7, and use a plane segmentation algorithm to segment the point cloud, separating the fastener top surface point cloud from the rest of the point cloud;

[0015] Step 9: Perform plane fitting on the point cloud of the top surface of the fastener to obtain the top surface equation and top surface normal vector, further solve the top surface height of the fastener and the inclination angle of the central axis to complete the 3D pose recognition of the fastener;

[0016] Furthermore, step 2 includes:

[0017] Step 2-1: Build the SqueezeNet backbone network based on two convolutional layers, seven to eight Fire modules, three max pooling layers, one global average pooling layer, and a Softmax function. The Fire module structure consists of squeeze layers, which consist of a series of consecutive 1x1 convolutions, and expand layers, which consist of a series of consecutive 1x1 and 3x3 convolutions.

[0018] Step 2-2: Replace the original VGG16 backbone network in the SDD model with SqueezeNet to build the SqueezeNet-SSD deep learning model. The Softmax function in the SqueezeNet network can determine the type of fastener, and the additional convolutional layers in the SSD model can detect the location of fasteners of different sizes.

[0019] Step 2-3: Set the model parameters such as the number of iterations, batch size, initial learning rate, momentum, etc. After dividing the fastener dataset constructed in step 1 proportionally, train and test the model.

[0020] Furthermore, step 3 includes:

[0021] Step 3-1: Use the Softmax function in the trained model as a classifier, map the neuron outputs of each feature layer before the function to the interval (0, 1), and determine the type of fastener target through probability prediction. The Softmax function is shown in Equation (1).

[0022]

[0023] Where: z i is the output eigenvalue of the i-th node; C is the number of output nodes, which is the same as the number of fastener types.

[0024] Step 3-2: The SqueezeNet backbone network in the trained model is used to extract image features, and then the SSD model generates multiple prior frames of different scales in the additional convolution layer. The prior frame scale in the kth additional convolution layer is S k The calculation formula is as shown in formula (2).

[0025]

[0026] Where: S min Indicates the minimum proportion of the prior frame to the input image; S max Indicates the maximum proportion of the prior frame to the input image; m is the number of additional convolutional layers.

[0027] Calculate the intersection-over-union (IOU) between each prior frame, use the non-maximum suppression method to screen the prior frames, remove inappropriate prior frames, and retain the prior frame with the highest probability of the target as the prediction frame.

[0028] Step 3-3: Crop a single threaded fastener sub-image from the original image using the predicted box as the area, and record the position of the upper left corner of the predicted box (u i ,v i )、Length and width (w i ,h i ) and target type information to complete the rough positioning of the module surface fasteners.

[0029] Furthermore, step 4 is to perform image filtering, enhancement and other preprocessing operations on each fastener sub-image, and then use the Otsu method to solve the threshold T that maximizes the inter-class variance between the foreground and background to achieve adaptive segmentation of the fastener sub-image. The pixel value g(x,y) of the binary image after segmentation is:

[0030]

[0031] Where: f(x,y) is the pixel value of the pixel point with pixel coordinates (x,y); T is the optimal threshold.

[0032] After morphological processing of the segmented image, the region with a pixel value of 0 is used as the fastener BLOB feature region. If the number of BLOB regions extracted from a single fastener sub-image is greater than 1, the BLOB regions are merged for unified processing later.

[0033] Furthermore, step 5 includes:

[0034] Step 5-1: Obtain the exact center of the fastener based on the fastener BLOB ellipse fitting method. Assume that the general equation of the ellipse is:

[0035] ax 2 +bxy+cy 2 +dx+ey+f=0 (4)

[0036] Where: a, b, c, d, e, f are the coefficients of the ellipse equation.

[0037] The coefficients of the equation are solved using the least squares method and the extreme value principle, and the center coordinates and the major and minor axis values ​​of the ellipse are further calculated. The center of the ellipse is used as the precise center of the fastener head, and the major axis is the head circle diameter D. 紧 .

[0038] Step 5-2: Take the center as the vertex and the preset angle θ as the vertex angle, R 扇 Construct a sector for the radius. This sector rotates counterclockwise around the center of the fastener with an angle α as a step. Record the area S that intersects with the fastener BLOB during the rotation. i When S i When the minimum value is taken, the fastener head attitude angle is solved based on the angle ψ between the fan-shaped vertex angle bisector and the starting edge. The angle ψ can be calculated using formula (5):

[0039]

[0040] Where: ψ is the iWhen the minimum value is taken, the angle between the bisector of the fan-shaped vertex angle and the starting side; when calculating the posture angle of the hexagonal screw, β is taken as 60°; when calculating the posture angle of the cross screw, β is taken as 90°; when calculating the posture angle of the slotted screw, β is taken as 180°.

[0041] Furthermore, step 8 is: performing preprocessing operations such as voxel filtering and statistical filtering on the cropped fastener area point cloud, and then using the random sampling consensus algorithm (RANSAC) to achieve planar adaptive segmentation of the fastener area point cloud, and taking the part of the point cloud with the largest height value as the fastener top surface point cloud.

[0042] Furthermore, step 9 is: using the least squares method to perform plane fitting on the top surface point cloud to ensure that the average distance from all data points to the fitting plane is minimized, and finally obtaining a plane equation that satisfies formula (6).

[0043] Ax+By+Cz+D=0 (6)

[0044] Among them, A, B, C, and D are the plane parameters of the fitting, and the normal vector of the fitting plane is

[0045] The mean height of the point cloud projected on the fitting plane = is the height of the center of the top surface of the fastener. The inclination angle of the center axis of the fastener is the angle between the center axis of the fastener and the X, Y, and Z axes, denoted as α, β, and γ, which reflects the three-dimensional posture of the fastener. The calculation formulas are as follows: (7) to (9).

[0046]

[0047]

[0048]

[0049] Where: are the unit vectors in the positive directions of the X, Y, and Z axes respectively, is the normal vector of the fitted plane.

[0050] Compared with the prior art, the technical solution of the present invention has the following significant beneficial effects:

[0051] (1) Good versatility. The present invention can maintain high identification and positioning accuracy for various threaded fasteners on the surface of retired lithium battery modules of different models, and can effectively distinguish different fasteners with similar structures and sizes.

[0052] (2) Good real-time performance. The present invention can detect a large number of threaded fasteners on the surface of retired lithium battery modules quickly and efficiently, and can cooperate with robots to realize the automatic and rapid disassembly of fasteners.

[0053] (3) Simple processing and high detection accuracy. The present invention simplifies the processing process, processes a small amount of data, has low requirements for equipment configuration, saves costs and can ensure high accuracy.

[0054] (4) Stable and reliable. The present invention is not easily affected by environmental factors. In the case of complex background environment and lighting changes on the module surface, the method of the present invention can still accurately obtain information such as the type, size, and posture of the fasteners, thereby ensuring the continuity and reliability of visual detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart of the method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules of the present invention;

[0056] Figure 2 is a schematic diagram of an image and point cloud acquisition device;

[0057] Figure 3 It is the SqueezeNet backbone network structure;

[0058] Figure 4 This is the SqueezeNet-SSD network model structure diagram;

[0059] Figure 5 It is a flowchart of the two-dimensional rough positioning of threaded fasteners;

[0060] Figure 6 This is the BLOB segmentation result diagram of three types of fasteners;

[0061] Figure 7 This is a schematic diagram of the fan-shaped area of ​​the area intersection method;

[0062] Figure 8 It is a schematic diagram of the two-dimensional head posture angle recognition of three types of fasteners; DETAILED DESCRIPTION

[0063] The present invention will be further described with reference to the accompanying drawings and embodiments. Obviously, the embodiments described are only part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those skilled in the art should fall within the scope of protection of the embodiments of the present invention.

[0064] Figure 1 This is a process based on the visual identification and positioning method of threaded fasteners on the surface of retired lithium battery modules. It specifically includes the following steps:

[0065] S1: Use a two-dimensional camera to collect the surface image of the retired lithium battery module, such as Figure 2As shown, the two-dimensional image acquisition device includes a two-dimensional camera and an array light source. The retired lithium battery module is moved to the bottom of the two-dimensional camera and the light source via a conveyor belt, and the camera takes 1,000 images of the module surface. The number of hexagonal, cross, and slotted screws is counted as 6,784, 6,736, and 6,751, respectively. The captured image data is transmitted to a computer, and image enhancement is achieved by applying brightness adjustment, affine transformation, and contrast change to expand the fastener sample set and obtain a total of 2,000 sample images. Each image in the sample set is uniformly scaled to a size of 1,280*1,280px. The Deep Learning Tool is used to label various fasteners in the image to obtain a dataset of various common fasteners on the module surface.

[0066] S2: Construct the SqueezeNet backbone network based on two convolutional layers, eight Fire module structures, three maximum pooling layers, one global average pooling layer and the Softmax function, as shown in Figure 3 As shown. The original VGG16 backbone network in the SDD model is replaced with SqueezeNet to build the SqueezeNet-SSD deep learning model. The model structure is as follows Figure 4 As shown in the figure, the model hyperparameters were set as follows: number of iterations to 200, batch size to 8, initial learning rate to 0.1, and momentum to 0.9. The training, validation, and test sets were divided into a ratio of 7:2:1 for training and testing.

[0067] S3: Use the model trained in S2 to identify the type of fasteners in the image of the detection module and use the Softmax function in the trained model as a classifier to predict the type of fastener target. The Softmax function is as shown in formula (1), where z i is the output eigenvalue of the ith node, and C is the number of output nodes. In this embodiment, the fastener slot head types include hexagonal, cross, and slotted, so C=3 is set.

[0068]

[0069] Then the SqueezeNet backbone network extracts the image features and uses the SSD model to generate multiple prior frames of different scales in the additional convolution layer. The scale of the prior frame in the kth additional convolution layer is S k The calculation formula is as shown in formula (2).

[0070]

[0071] Where: S min Set to 0.2, indicating the minimum proportion of the prior frame to the input image; S maxIt is set to 0.9, indicating that the prior frame occupies the maximum proportion of the input image; m is set to 5, indicating the number of additional convolutional layers.

[0072] Calculate the intersection-over-union (IOU) between each prior frame, use the non-maximum suppression method to filter the prior frames, and retain the prior frame with the highest probability of the target as the prediction frame. Record the coordinates of the corner points of the rectangular area where each fastener is located (u i ,v i ), and based on the coordinates, the fastener sub-image is cropped to complete the rough positioning of the fastener. The process is as follows Figure 5 shown.

[0073] S4: After performing image filtering, enhancement, and other preprocessing operations on each fastener sub-image, the Otsu method is used to find the threshold T that maximizes the inter-class variance between the foreground and background, thus achieving adaptive segmentation of the fastener sub-image. The pixel value g(x,y) of the binary image after segmentation is:

[0074]

[0075] Where: f(x,y) is the pixel value of the pixel point with pixel coordinates (x,y); T is the optimal threshold.

[0076] After morphological processing is performed on the segmented image, the area with a pixel value of 0 is used as the fastener BLOB feature area. In this embodiment, the segmentation results of the three types of fastener BLOBs are as follows: Figure 6 As shown in the figure, the number of BLOB areas of the cross and slotted screws is 2, so the BLOB areas of the screws are merged into 1 area for unified processing later.

[0077] S5: Obtain the precise center of the fastener based on the fastener BLOB ellipse fitting method. Assume that the general equation of the ellipse is:

[0078] ax 2 +bxy+cy 2 +dx+ey+f=0 (4)

[0079] Where: a, b, c, d, e, f are the coefficients of the ellipse equation.

[0080] The coefficients of the equation are solved using the least squares method and the extreme value principle, and the center coordinates and the major and minor axis values ​​of the ellipse are further calculated. The center of the ellipse is used as the precise center of the fastener head, and the major axis is the head circle diameter D. 紧 .

[0081] With the center as the vertex, the preset angle θ (θ = 30°) is the vertex angle, R 扇 (R 扇 =0.6D 紧 ) is the radius to construct a sector area, such as Figure 7As shown. This area rotates counterclockwise around the center of the fastener with an angle α (α = 0.1°) as a step. The area S that intersects with the fastener BLOB during the rotation is recorded. i When S i When the minimum value is taken, the fastener head attitude angle is solved based on the angle ψ (ψ=i*α+θ / 2) between the fan-shaped vertex angle bisector and the starting edge. The angle ψ can be calculated using formula (5):

[0082]

[0083] Where: ψ is the i When the minimum value is taken, the angle between the bisector of the fan-shaped top angle and the starting side; when calculating the posture angle of the hexagonal screw, β is taken as 60°; when calculating the posture angle of the cross screw, β is taken as 90°; when calculating the posture angle of the slotted screw, β is taken as 180°

[0084] The schematic diagram of the three types of fastener posture angle recognition is as follows Figure 8 As shown in this embodiment, the three fastener head two-dimensional posture angles They are 32.7°, 14.4° and 162.5° respectively.

[0085] S6: The decommissioned lithium battery module is transported via a conveyor belt directly below a 3D camera, where it collects a point cloud of the module's surface. The relative positions of the 2D image coordinate system and the 3D point cloud coordinate system are calibrated based on the conveyor belt's travel distance and the proportional relationship between the 2D and 3D camera images and the point cloud coordinates.

[0086] S7: Crop the point cloud of the fastener area based on the 2D coarse positioning coordinates of S3 and the calibration information of S6.

[0087] S8: Perform voxel filtering, statistical filtering and other preprocessing operations on the cropped fastener area point cloud. Then, use the random sampling consensus algorithm (RANSAC) to achieve plane adaptive segmentation of the fastener area point cloud, and take the part of the point cloud with the largest height value as the top surface point cloud of the fastener.

[0088] S9: Use the least squares method to perform plane fitting on the top surface point cloud to ensure that the average distance from all data points to the fitting plane is minimized, and finally obtain the plane equation that satisfies formula (6).

[0089] Ax+By+Cz+D=0 (6)

[0090] Among them, A, B, C, and D are the plane parameters of the fitting, and the normal vector of the fitting plane is

[0091] The mean height of the point cloud projected on the fitting plane It is the center height of the top surface of the fastener.

[0092] The calculation formulas for the inclination angles α, β, and γ of the center axis of the fastener are as shown in equations (7) to (9) respectively.

[0093]

[0094]

[0095]

[0096] Where: are the unit vectors in the positive directions of the X, Y, and Z axes respectively, is the normal vector of the fitted plane.

[0097] In this embodiment, the equations of the planes fitted on the top surfaces of the three types of fasteners are: 0.122x+0.002y-1.369z+0.998=0, -0.026x+0.046y-1.427z+0.999=0, and 0.071x+0.016y-1.434z+0.962=0, respectively. The normal directions of the planes are: The top surface heights are 716.451mm, 697.328mm, and 701.288mm respectively, and the inclination angles of the central axis are α1=88.6°, β1=87.4°, γ1=2.9°; α1=87.9°, β1=88.4°, γ1=2.9°; α1=88.3°, β1=86.8°, γ1=1.2°.

[0098] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium is capable of storing a computer program that, when executed by the data processing unit, can execute the invention content of the method for visually identifying and locating threaded fasteners on the surface of a retired lithium battery module provided by the present invention and some or all of the steps in each embodiment. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0099] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a computer program, i.e., a software product. The computer program software product can be stored in a storage medium and includes several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, a MUU, or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0100] The present invention provides a method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules. While there are numerous methods and approaches for implementing this technical solution, the aforementioned are merely preferred embodiments of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications are also within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules, characterized in that: The method described is: Step 1: Obtain a 2D image of the module surface and create a dataset of various fasteners; Step 2: Build and train a deep learning model; Step 3: Complete the rough positioning and type identification of the fasteners and crop the fastener area sub-image; Step 4: Preprocess each fastener sub-image and extract the fastener BLOB features; Step 5: Solve the 2D pose of the fastener based on ellipse fitting and region intersection method; Step 6: Obtain the 3D point cloud of the module surface and complete the calibration of the 2D and 3D coordinate systems; Step 7: Use the 2D coarse positioning information to crop the fastener area point cloud; Step 8: Preprocess the point cloud and perform plane segmentation; Step 9: Fit the top surface equation of the fastener and solve the top surface height and the inclination angle of the center axis; In step 2, a SqueezeNet-SSD deep learning model is built based on the SqueezeNet backbone network and the SSD model, and hyperparameters are set; after the fastener dataset constructed in step 1 is divided proportionally, the model is trained and tested; the specific steps are as follows: Step 2-1: Construct a SqueezeNet backbone network based on two convolutional layers, seven or eight Fire module structures, three maximum pooling layers, one global average pooling layer, and a Softmax function. The Fire module structure consists of a squeeze layer and an expand layer. The former includes continuous 1*1 convolutions, and the latter includes continuous 1*1 convolutions and 3*3 convolutions. Step 2-2: Replace the original VGG16 backbone network in the SDD model with SqueezeNet to build the SqueezeNet-SSD deep learning model. The Softmax function in the SqueezeNet network can obtain the type of fastener, and the additional convolutional layers in the SSD model can detect the location of fasteners of different sizes. Step 2-3: Set the model parameter values ​​for the number of iterations, batch size, initial learning rate, and momentum; After dividing the fastener dataset constructed in step 1 proportionally, the model is trained and tested.

2. The method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules according to claim 1, characterized in that: In step 1, a two-dimensional camera is used to capture surface images of typical retired lithium battery modules, ensuring that the number of sample images of the module surface is no less than 800, and the number of each type of fasteners included is no less than 5,000; at the same time, to achieve sample balance, the difference in the number of each type of fasteners should not exceed 100; an operation method is used to enhance the initial sample images to obtain a batch of new sample images; the new and old batches of sample images are merged into a sample set, and the images in the sample set are uniformly scaled to a size of 1280*1280px; a general deep learning tool is used to label the fasteners in each image to obtain a dataset of various common fasteners on the module surface.

3. The method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules according to claim 1, characterized in that: In step 3, the model trained in step 2 is used to identify the type of fasteners in the image of the module to be inspected, and the coordinates of the corner points of the rectangular area where each fastener is located are recorded. The fastener sub-image is cropped based on the coordinates to complete the coarse positioning of the fasteners. The specific steps are as follows: Step 3-1: Using the Softmax function in the model trained in step 2 as a classifier, the outputs of the neurons in each feature layer before the function are mapped to the interval (0, 1), and the type of the fastener target is determined through probabilistic prediction; The Softmax function is as shown in formula (1); Where: z i is the output feature value of the i-th node; C is the number of output nodes, which is the same as the number of fastener types; Step 3-2: The SqueezeNet backbone network in the model trained in Step 2 is used to extract image features, and then the SSD model generates multiple prior frames of different scales in the additional convolution layer. The prior frame scale in the k-th additional convolution layer is S k The calculation formula is as shown in formula (2); Where: S min Indicates the minimum proportion of the prior frame to the input image; S max Indicates the maximum proportion of the prior frame to the input image; m is the number of additional convolutional layers; Calculate the intersection-over-union (IOU) ratio between each priori boxes and use the non-maximum suppression method to filter the priori boxes, remove inappropriate priori boxes, and retain the priori boxes that are most likely to contain fastener targets as the prediction boxes; Step 3-3: Crop a single threaded fastener sub-image from the original image using the predicted box as the area, and record the position of the upper left corner of the predicted box (u i ,v i )、Length and width (w i ,h i ) and target type information to complete the rough positioning of the module surface fasteners.

4. The method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules according to claim 1, characterized in that: In step 4, preprocessing of each fastener sub-image obtained in step 3 is completed, and the BLOB that best represents the connected domain features of the fastener head is extracted to facilitate subsequent fastener pose recognition. The specific process is as follows: after preprocessing each fastener sub-image obtained in step 3, the Otsu method is used to solve the threshold T that maximizes the variance between the foreground and background classes, thereby achieving adaptive segmentation of the fastener sub-image; The pixel value g(x,y) of the binary image after segmentation is: Where: f(x,y) is the pixel value of the pixel point with pixel coordinates (x,y); T is the optimal threshold; After morphological processing of the segmented image, the area with pixel value 0 is used as the fastener BLOB feature area; If the number of BLOB regions extracted from a single fastener sub-image is greater than 1, the BLOB regions are merged for unified processing later.

5. The method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules according to claim 1, characterized in that: In step 5, based on the fastener BLOB feature area extracted in step 4, ellipse fitting and region intersection method are used to solve the two-dimensional pose information such as the center coordinates of the fastener head and the groove posture angle; the specific steps are: Step 5-1: Obtain the exact center of the fastener based on the fastener BLOB ellipse fitting method. Assume that the general equation of the ellipse is: ax 2 +bxy+cy 2 +dx+ey+f=0 (4) Where: a, b, c, d, e, f are the coefficients of the ellipse equation, x and y are the three-dimensional coordinate position variables respectively; The coefficients of the equation are solved using the least squares method and the extreme value principle, and the center coordinates and the major and minor axis values ​​of the ellipse are further calculated. The center of the ellipse is used as the precise center of the fastener head, and the major axis is the head circle diameter D. 紧 ; Step 5-2: Take the center as the vertex and the preset angle θ as the vertex angle, R 扇 Construct a sector area for the radius; the area rotates counterclockwise around the center of the fastener with an angle α as the step size; record The intersection area S with the fastener BLOB during its rotation i ; When S i When the minimum value is taken, the fastener head attitude angle is solved based on the angle ψ between the fan-shaped vertex angle bisector and the starting edge at this time; the attitude angles of various types of screws The angle ψ can be calculated using formula (5): Where: ψ is the i When the minimum value is taken, the angle between the bisector of the fan-shaped vertex angle and the starting side; when calculating the posture angle of the hexagonal screw, β is taken as 60°; when calculating the posture angle of the cross screw, β is taken as 90°; when calculating the posture angle of the slotted screw, β is taken as 180°.

6. The method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules according to claim 1, characterized in that: In step 8, the point cloud of the fastener region obtained by cropping in step 7 is first preprocessed by voxel filtering and statistical filtering. Then, the random sampling consensus algorithm RANSAC is used to achieve planar adaptive segmentation of the point cloud of the fastener region, and the point cloud with the largest height value is taken as the point cloud of the top surface of the fastener.

7. The method for visually identifying and locating threaded fasteners on the surface of retired lithium battery modules according to claim 1, characterized in that: In step 9, plane fitting is performed on the point cloud of the top surface of the fastener to obtain the top surface equation and the top surface normal vector, and the height of the top surface of the fastener and the inclination angle of the central axis are further solved to complete the three-dimensional pose recognition of the fastener. The specific steps are: The least squares method is used to fit the top surface point cloud to ensure that the average distance from all data points to the fitting plane is minimized, and finally the plane equation that satisfies formula (6) is obtained; Ax+By+Cz+D=0 (6) Among them: A, B, C, D are the plane parameters of the fitting, and the normal vector of the fitting plane is x, y, and z are three-dimensional coordinate position variables; The mean height of the point cloud projected on the fitting plane is the height value of the center of the top surface of the fastener; the inclination angle of the center axis of the fastener is the angle between the center axis of the fastener and the X, Y, and Z axes, recorded as α, β, and γ, reflecting the three-dimensional posture of the fastener. The calculation formulas are as follows: (7) to (9); Where: are the unit vectors in the positive directions of the X, Y, and Z axes respectively, is the normal vector of the fitted plane.

Images