Visual positioning method for center point of plate spring based on multi-model collaboration

Abstract

The invention relates to a plate spring center point visual positioning method based on multi-model cooperation, and belongs to the technical field of plate spring center hole machining. Comprising the following steps: S01, acquiring a plurality of images under different exposure parameters, and fusing the multi-exposure images by using an IFCNN model to obtain an image with moderate exposure; s02, a YOLOV8 model is adopted to carry out target detection on the image, all the plate springs are accurately recognized, ROIs (Regions of Interest) of the plate springs are extracted, and each ROI only comprises a single plate spring; s03, performing fine segmentation by using U-Net, and extracting an accurate contour of a single plate spring; and S04, a binocular vision system is adopted, precise three-dimensional positioning of the center point of the plate spring is achieved through double-camera parallax calculation, and space coordinates (X, Y and Z) of the center point of the plate spring are obtained. According to the method, the space coordinates of the center point of the plate spring are accurately obtained in the plate spring punching process by combining the target detection and semantic segmentation methods, and automatic center punching of the plate spring plate is conveniently achieved.

Description

Technical Field

[0001] The present application belongs to the technical field of leaf spring center hole processing, and specifically relates to a leaf spring center point visual positioning method based on multi-model collaboration. Background Art

[0002] A leaf spring is an elastic suspension element composed of multiple or single elastic steel plates (usually spring steel). It is mainly used in suspension systems of cars, trucks, trains, trailers, etc. to support the vehicle body, cushion vibrations, and absorb impacts. In suspension systems of cars, trucks, railways, etc., leaf springs usually need to be fastened to the axle or suspension bracket, and the main function of the center hole is to install the central bolt to ensure that multiple springs can be precisely aligned, fixed, and evenly stressed. The central bolt is used to clamp multiple leaf springs together to prevent misalignment between the springs. This bolt also cooperates with the U-bolt and other suspension components to enable the leaf spring to be stably fixed to the axle, ensuring the reliability of the suspension structure.

[0003] Leaf springs are subject to bending and shear forces during use. Without a proper fixing method, the spring may misalign or deform, affecting suspension performance. With a center-drilled hole, the bolt fixing point becomes the center of symmetry for the entire leaf spring, ensuring more uniform bending deformation, avoiding localized stress concentration, and minimizing material fatigue damage.

[0004] In a multi-leaf spring, the leaves slide relative to each other under load to cushion the impact. The hole for the center bolt allows the leaves to slide in a controlled manner, reducing wear and preventing misalignment between the leaves.

[0005] The center hole position is usually standardized, which facilitates the rapid assembly of leaf springs of different specifications on the production line and improves manufacturing efficiency. During assembly or repair, the center hole can be used as an alignment reference to ensure that all leaf spring components are installed correctly.

[0006] Certain suspension systems (such as multi-link or pneumatic suspensions) require adjustment of the leaf spring's mounting position. The center hole can be used for additional mounting pins or support structures to enhance suspension adaptability. In specific applications (such as modified vehicles and specialty vehicles), the center hole can serve as an additional mounting point for adjusting suspension parameters.

[0007] Currently, most leaf spring manufacturers still use manual positioning for punching, but this method has certain limitations, such as insufficient precision, low efficiency, high labor costs, and poor consistency. Therefore, automated and intelligent leaf spring punching technology is gradually becoming the development direction of the industry.

[0008] Using vision systems for positioning in leaf spring production plants is complex, particularly due to the brightly lit workshop environment during the day and dimly lit at night. This leads to significant variations in image brightness, contrast, and shadows, resulting in significant differences in image quality captured by the vision system over different time periods. This uneven illumination can affect edge recognition, feature extraction, and precise positioning of targets, reducing detection accuracy and even leading to misidentification or missed detections, impacting the stability and reliability of the leaf spring drilling and assembly processes. Furthermore, leaf springs are made of metal and have a highly reflective surface. Highlights may appear in images in well-lit environments, further impacting target positioning accuracy. Summary of the Invention

[0009] The technical problem to be solved by this application is: to overcome the shortcomings of the existing technology and provide a visual positioning method for the center point of a leaf spring based on multi-model collaboration. This application combines target detection and semantic segmentation methods to accurately obtain the spatial coordinates of the center point of the leaf spring during the leaf spring punching process, thereby facilitating the automated center punching of the leaf spring plate.

[0010] The technical solution adopted by this application to solve the problems existing in the prior art is:

[0011] A leaf spring center point visual positioning method based on multi-model collaboration includes the following steps:

[0012] S01, by collecting multiple images under different exposure parameters, and then using the IFCNN model to fuse the multiple exposure images, an image with moderate exposure is obtained;

[0013] S02. Use the YOLOV8 model to perform target detection on the image, accurately identify all leaf springs, and extract their regions of interest (ROIs). Each ROI contains only a single leaf spring.

[0014] S03. Use U-Net to perform fine segmentation and extract the precise contour of a single leaf spring;

[0015] S04. Use a binocular vision system to achieve accurate three-dimensional positioning of the center point of the leaf spring through dual-camera parallax calculation, and obtain the spatial coordinates (X, Y, Z) of the center point of the leaf spring.

[0016] Preferably, in step S01, the brightness of the current lighting environment is determined by the following formula, the average pixel value of the image is calculated, and the degree of lighting sufficiency of the current working environment is determined. The formula is as follows:

[0017]

[0018] Among them, I avg Represents the average brightness of the image, I(x,y) is the grayscale value of the pixel, M, N are the width and height of the image, according to Iavg The value of is used to judge the current exposure situation and decide whether it is necessary to collect additional images with different exposure levels.

[0019] Preferably, the logic for determining whether additional images with different exposure levels need to be collected is as follows:

[0020] When I avg If the image brightness is within the normal range (i.e., the image brightness is moderate), target detection can be performed directly without collecting additional exposure images;

[0021] When I avg When deviating from the normal range:

[0022] If the current exposure is high (too bright), you need to capture two images with worse exposure (i.e., lower the exposure value);

[0023] If the current exposure is extremely low (too dark), you need to capture two images with better exposure (i.e., increase the exposure value);

[0024] For other situations (such as too dark or too bright), it is only necessary to obtain a better-exposed picture and a worse-exposed picture for subsequent image processing.

[0025] Preferably, in step S01, three images with different exposures are input into the IFCNN model to obtain an image I with moderate exposure. fusion .

[0026] Preferably, in step S02, the image I fusion When input into the YOLOV8 model, the shape of the top leaf spring is an n×6 tensor, where n represents the number of detected targets and each target contains the following 6 values:

[0027] x_center, the x coordinate of the center point of the target box; y_center, the y coordinate of the center point of the target box; width, the width of the target box; height, the height of the target box; confidence, the confidence of the target (0-1); class_id, the category index of the target;

[0028] The coordinates of the upper left corner and lower right corner of the bounding box (x min ,y min ) and (x max ,y max ) can be calculated using the following formula:

[0029]

[0030] The ROI area of ​​the leaf spring plate in the image can be taken out according to the acquired coordinates, and there is only one leaf spring plate in each ROI.

[0031] Preferably, in step S03, the leaf spring ROI region is fed into a trained U-Net model to obtain a segmentation map of the leaf spring.

[0032] Preferably, in the segmentation diagram of the leaf spring, the leaf spring plate area and other areas are marked with different RGB color codes.

[0033] Preferably, in step S04, the binocular vision system is calibrated using two steps: monocular calibration and binocular calibration.

[0034] Preferably, monocular correction mainly calculates the intrinsic parameters of the camera to dedistort the lens and infer the depth, while binocular correction calculates the extrinsic parameters of the left and right cameras. After knowing the extrinsic parameters, the left and right cameras can be rotated by a certain angle so that the same-name points of the left and right cameras are in the same plane and on the same horizontal line.

[0035] Preferably, due to the different viewing angles of the binocular cameras, the positions of the same object in the left and right images will be offset. This offset is called parallax (d):

[0036] d=u L -u R

[0037] Among them, u L is the x-coordinate of the center point of the leaf spring in the left image, u R is the x-coordinate of the center point of the leaf spring in the right image, and the depth Z is calculated by triangulation.

[0038]

[0039] Where Z is the depth distance of the spring center point in the camera, f is the camera focal length, B is the camera baseline, and d is the parallax value (the offset of the matched pixels). The X and Y coordinates of the spring center point are calculated as follows:

[0040]

[0041] Finally, the three-dimensional coordinates (X, Y, Z) of the center point of the leaf spring are obtained, which represents the real space position of the target in the left camera coordinate system.

[0042] Compared with the prior art, this application has the following beneficial effects:

[0043] (1) It can adapt to the complex environment of the workshop, eliminate the influence of light and external environment on image quality, and the extracted area of ​​interest only contains the single leaf spring plate, which facilitates the accuracy of the positioning of the punching area in the later stage.

[0044] (2) Through this method, the punching points of the leaf spring can be quickly and accurately positioned, which facilitates the promotion and development of automated and intelligent leaf spring punching technology, improves work efficiency, and reduces labor intensity and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present application is further described below with reference to the accompanying drawings and examples.

[0046] Figure 1 This is a schematic diagram of the camera pose in this application.

[0047] Figure 2 This is a schematic diagram of the shooting effect before binocular correction in this application.

[0048] Figure 3 This is a schematic diagram of the shooting effect after binocular correction in this application.

[0049] Figure 4 For the IFCNN model structure in this application,

[0050] Figure 5 This is the YOLOV11 model structure in this application. DETAILED DESCRIPTION

[0051] The visual positioning method of the leaf spring center point based on multi-model collaboration of the present application is further described in detail in conjunction with the accompanying drawings, but it does not limit the present application.

[0052] A visual positioning method for the center point of a leaf spring based on multi-model collaboration. Before calculating the center point of a leaf spring using this method, the system needs to be preprocessed. The preprocessing includes the following items.

[0053] 1. Build a binocular system

[0054] Set two cameras with the same parameters as follows Figure 1 The camera is installed in the position shown. Due to installation errors, it is impossible to achieve completely parallel optical axes of the left and right cameras. Therefore, calibration is required. Binocular vision system calibration involves two steps: monocular calibration and binocular calibration. Monocular calibration primarily calculates the camera's intrinsic parameters to dedistort the lens and estimate depth. Binocular calibration calculates the extrinsic parameters of the left and right cameras. Once these parameters are known, the left and right cameras can be rotated by a certain angle so that their synonyms are on the same plane and horizontal line.

[0055] The implementation of monocular correction includes the following five steps:

[0056] The first step is to collect chessboard images. You need multiple chessboard images from different angles (more than 10 images are recommended).

[0057] Understand the number of corner points on a chessboard.

[0058] The second step is to use cv::findChessboardCorners() to identify the corners of the chessboard.

[0059] The third step is to use cv::cornerSubPix() to improve the accuracy of corner detection.

[0060] The fourth step is to use the function cv::calibrateCamera() to calculate the camera parameters, including: camera intrinsic parameters (focal length fx, fy and principal point cx, cy); distortion parameters (radial and tangential distortion); rotation matrix R and translation vector T for each image.

[0061] The fifth step is to use cv::undistort() to remove image distortion.

[0062] The implementation of binocular correction includes the following three steps:

[0063] The first step is to calculate the stereo rectification matrix in OpenCV using the cv::stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, imageSize, R, T, R1, R2, P1, P2, Q) function. Where cameraMatrix1 and cameraMatrix2 are the intrinsic parameter matrices of the left and right cameras; distCoeffs1 and distCoeffs2 are the distortion parameters of the left and right cameras; imageSize is the image size; and R and T are the camera extrinsic parameters (rotation matrix and translation vector). The output of the stereoRectify function consists of three parts: R1 and R2 are the rectification rotation matrices of the left and right cameras; P1 and P2 are the new projection matrices; and Q is the reprojection matrix.

[0064] In the second step, the remapping table is calculated using the function cv::initUndistortRectifyMap(cameraMatrix1,distCoeffs1,R1,P1,imageSize,CV_16SC2,map1,map2). The function initUndistortRectifyMap outputs the mapping tables map1 and map2 for image correction.

[0065] The third step is to use the function cv::remap(image,rectifiedImage,map1,map2,cv::INTER_LINEAR) to perform remapping. Here, image is the input image and rectifiedImage is the rectified image.

[0066] 2. Building a Deep Learning Environment

[0067] Install Anaconda and ensure it is correctly added to the system environment variables. Open Anaconda Prompt and create a new virtual environment using the command conda create -n pytorch_env python=3.8. Once created, activate the environment using conda activate pytorch_env to enter the newly created environment. Install the GPU version of PyTorch according to the official installation command. After installation, run python to enter the Python interactive environment and enter importtorch; print(torch.__version__) to confirm that PyTorch is installed successfully. Check whether the GPU is correctly detected by running print(torch.cuda.is_available()). If it returns True, it means that PyTorch has been correctly configured for GPU support. Finally, install related dependent libraries, such as pip install numpy, matplotlib, and opencv-python, to support subsequent data processing and model training.

[0068] 3. Build a multi-exposure fusion model (IFCNN) and model training.

[0069] First, use Pytorch to build the IFCNN model in Pycharm. The model structure is as follows: Figure 4As shown. The input of the model includes two RGB images of size 3×224×224, representing images of different exposure levels. First, in the feature extraction stage, the two input images undergo two layers of convolution (CONV1 and CONV2) respectively. The two layers of convolution share weights (TiedWeights) to ensure that the extracted features are compared at the same scale. After these two layers of convolution, each input image is converted into a 64×224×224-dimensional feature map, capturing low-level and high-level image features such as edges, textures, and structural information. In the feature fusion stage, the two feature maps are fed into the fusion module (FUSE), which uses adaptive fusion strategies such as weighted fusion, attention mechanism, or maximum selection to combine the features of different images into a unified fused feature map. The fused feature map still maintains a size of 64×224×224 dimensions to ensure that complete information is retained. Next, in the Feature Reconstruction phase, the fused feature map undergoes two layers of convolution (CONV3 and CONV4) to gradually restore the original image format. CONV3 further refines and optimizes the fused features, while CONV4 maps the 64-channel feature map back to a 3-channel RGB image, maintaining the same output format as the input. Ultimately, the model outputs a fused image (3×224×224), which contains the key information from both input images while removing any redundancy.

[0070] Secondly, the IFCNN model is trained using the MEFB dataset. The input image pairs are fed into the IFCNN model for forward propagation and the loss function is calculated. MSE (mean square error) or SSIM (structural similarity) loss is usually used to measure the difference between the fused image and the reference image. Perceptual loss may also be combined to optimize visual quality. After calculating the loss, backpropagation is performed to update the network weights through gradient calculation. An optimizer (such as Adam or SGD) is used for gradient descent to reduce the loss value. After each epoch, the model performance is evaluated on the validation set. Indicators such as PSNR (peak signal-to-noise ratio), SSIM, and information entropy are calculated to check the fusion quality of the model, and hyperparameter adjustments are performed, such as adjusting the learning rate and optimizer parameters. After training is completed, the test set is used for final evaluation to ensure that the model has good generalization ability on different types of image fusion tasks, and the trained weight file is saved.

[0071] Considering that the model input is three pictures with different exposure levels during the trial phase, dynamic architecture adaptation and parameter compatibility design are required. First, the core structure of the original dual-channel encoder remains unchanged. By adding an expandable third input channel and using 1x1 convolution for feature dimension alignment, a zero-initialization strategy is used to ensure that the newly added parameters do not destroy the pre-trained feature distribution in the initial stage. Then, a dynamic condition judgment mechanism is constructed to automatically identify the input mode during the forward propagation of the model. When three-image input is detected, the newly added feature fusion module is activated. The three-way features are dynamically integrated into a dual-way output through channel splicing and compressed convolution to maintain the compatibility of the downstream network structure. At the same time, the newly added parameters are frozen through gradient shielding technology during the training phase to avoid interfering with the original feature extraction capability. Finally, the parameter mapping migration technology is used to automatically match the dual-channel historical parameters and intelligently initialize the newly added module when loading the pre-trained weights. The output stability is maintained through feature space projection, so that the model can seamlessly process the three-image input during inference without retraining, achieving a smooth transition between input dimension expansion and function enhancement.

[0072] 4. Build target detection model (YOLOV11) and model training.

[0073] Secondly, the established binocular vision system is used to collect leaf spring images, and the software LabelImg is used to label them, including information such as target category and bounding box position.

[0074] Then, build the YOLOV11 model, the model is as follows Figure 5As shown. The working principle of the model is as follows: starting from the input image, features are extracted through the backbone network (Backbone), and then multi-scale feature fusion is performed through the feature enhancement network (Neck). Finally, the detection head (Head) completes the target classification and regression. The input image first undergoes a series of convolution (Conv) operations for preliminary feature extraction, and then enters the C3k2 structure. This structure has two modes: when c3k=False, Bottleneck is used for feature refinement, and when c3k=True, the optimized C3k N=2 structure is used to improve computational efficiency. Subsequently, after the SPPF (Spatial Pyramid Pooling-Fast) module, multiple MaxPool2d operations are used to expand the receptive field while reducing the amount of computation. At the same time, the C2PSA module (including the PSABlock attention mechanism) further enhances the feature extraction capability and improves the model's attention to the target area. Next, the feature fusion stage (Neck) fuses features from different levels using upsampling (Upsample) and feature concatenation (Concat). The C3k2 architecture is again used to enhance the representation of the fused features, enabling the model to account for objects of varying scales. Finally, the detection head (Head) performs object detection through multiple Detect layers. Each detection head employs depthwise separable convolution (DWConv) to reduce computational overhead and improve inference speed. Object bounding box regression uses CloU (Complete Intersection over Union) to optimize the accuracy of the predicted bounding box, while CLSLoss calculates the classification error to optimize object category recognition. Finally, the model outputs the detection results.

[0075] Next, the model is trained. Training hyperparameters are defined, including the learning rate, optimizer (such as SGD or AdamW), batch size, and number of training epochs. Training data is loaded, batched using PyTorch's DataLoader. In the forward propagation phase, the input image is extracted using Backbone, then fed into the Neck for feature fusion. Finally, the Head performs object detection, outputting class predictions, bounding box coordinates, and confidence scores. Loss functions are calculated, including classification loss (CLSLoss), bounding box loss (CloU), and confidence loss. The classification loss measures the error in object class prediction, the bounding box loss optimizes bounding box position, and the confidence loss assesses object presence. Backpropagation then proceeds to calculate gradients and update model weights. Gradient descent is performed using the optimizer to reduce the loss. After each epoch, model performance is evaluated on the validation set, calculating metrics such as mean average precision (mAP), precision, and recall. Hyperparameters, such as learning rate decay and regularization, are adjusted to optimize the model. After training is complete, the test set is used for final evaluation to ensure the generalization ability of the model, and the trained weight file is saved for deployment to the inference environment.

[0076] 5. Build a semantic segmentation model (U-Net) and model training.

[0077] First, all leaf spring ROI regions obtained by target detection are organized into a dataset and subjected to data preprocessing, such as image normalization, data augmentation (flipping, scaling, color adjustment), etc. The LabelMe tool is used to assign a corresponding category label to each pixel and generate a corresponding segmentation mask.

[0078] Next, we build a U-Net model. The U-Net model is based on an encoder-decoder architecture and is specifically designed for semantic segmentation tasks, specifically classifying each pixel in an input image. It consists of three components: downsampling (encoder), upsampling (decoder), and skip connections. First, in the encoding phase, the input image undergoes multiple layers of convolution and max pooling to gradually extract high-level features while reducing spatial resolution. This function is similar to the feature extractor in a CNN. Subsequently, in the decoding phase, deconvolution or upsampling is used to gradually restore the image resolution. This information is combined with information from the encoder to ensure the recovery of fine boundaries. The key innovation of U-Net is skip connections, which directly concatenate feature maps of the same resolution from the encoder with those from the decoder, preventing the loss of detailed information due to continuous downsampling and improving the segmentation of small objects. Finally, a 1×1 convolution is used to generate a pixel-level prediction map of the same size as the input image. The channel value of each pixel represents the probability of belonging to a certain class.

[0079] Finally, the model is trained. At the beginning of training, the input image is fed into the U-Net network. A forward pass is performed to calculate the predicted segmentation map and compare it with the true labels to calculate the loss. Backward propagation then occurs to calculate the gradients and update the model weights. After each epoch, model performance is evaluated on the validation set, calculating metrics such as Intersection over Union (IoU), Dice Score, and Pixel Accuracy to check the model's segmentation performance and perform hyperparameter adjustments. After training is complete, a final evaluation is performed on the test set to ensure the model's generalization ability. The model weights are then saved for subsequent inference and deployment.

[0080] After completing the above pre-processing, the center point of the leaf spring is located. A specific leaf spring center point visual positioning method based on multi-model collaboration includes the following steps:

[0081] S01. By collecting multiple images at different exposure parameters and then fusing them using the IFCNN model, we obtain a single image with the appropriate exposure. This overcomes the difficulty of preserving both dark texture and bright detail in a single exposure image. Non-deep learning image fusion algorithms have poor real-time performance and require that highlights are not overexposed and shadows are not overdarkened.

[0082] The brightness of the current lighting environment is determined by the following formula, the average pixel value of the image is calculated, and the degree of lighting sufficiency of the current working environment is determined. The formula is as follows:

[0083]

[0084] Among them, I avg Represents the average brightness of the image, I(x,y) is the grayscale value of the pixel, M, N are the width and height of the image, according to I avg The value of is used to judge the current exposure situation and decide whether it is necessary to collect additional images with different exposure levels.

[0085] The logic for determining whether additional images with different exposure levels need to be collected is as follows:

[0086] When I avg If the image brightness is within the normal range (i.e., the image brightness is moderate), target detection can be performed directly without collecting additional exposure images;

[0087] When I avg When deviating from the normal range:

[0088] If the current exposure is high (too bright), you need to capture two images with worse exposure (i.e., lower the exposure value);

[0089] If the current exposure is extremely low (too dark), you need to capture two images with better exposure (i.e., increase the exposure value);

[0090] For other situations (such as too dark or too bright), you only need to obtain one picture with better exposure and one picture with worse exposure for subsequent image processing. The details are shown in the following table:

[0091]

[0092]

[0093] Input three images with different exposures into the IFCNN model to obtain the image I with moderate exposure. fusion .

[0094] S02. Use the YOLOV8 model to perform target detection on the image, accurately identify all leaf springs, and extract their regions of interest (ROIs). Each ROI contains only a single leaf spring.

[0095] Image I fusion When input into the YOLOV8 model, the shape of the top leaf spring is an n×6 tensor, where n represents the number of detected targets. Each target contains the following 6 values. The number of leaf springs in each layer in the box is 7. For each leaf spring image, the n output by the YOLOV8 model must be less than or equal to 7.

[0096] x_center, the x coordinate of the center point of the target box; y_center, the y coordinate of the center point of the target box; width, the width of the target box; height, the height of the target box; confidence, the confidence of the target (0-1); class_id, the category index of the target;

[0097] The coordinates of the upper left corner and lower right corner of the bounding box (x min ,y min ) and (x max ,y max ) can be calculated using the following formula:

[0098]

[0099] The ROI area of ​​the leaf spring plate in the image can be taken out according to the acquired coordinates, and there is only one leaf spring plate in each ROI.

[0100] S03. Use U-Net to perform fine segmentation and extract the precise outline of each leaf spring. Feed the leaf spring ROI region into the trained U-Net model to generate a leaf spring segmentation map. In the leaf spring segmentation map, the leaf spring region and other regions are labeled with different RGB color codes. For example, the leaf spring region in the image is labeled with a color of 255, while other regions are labeled with a color of 0. The coordinates of the center point of each leaf spring are then calculated using the following formula.

[0101]

[0102] S04. Use a binocular vision system to achieve accurate three-dimensional positioning of the center point of the leaf spring through dual-camera parallax calculation, and obtain the spatial coordinates (X, Y, Z) of the center point of the leaf spring.

[0103] Due to the different perspectives of the binocular cameras, the position of the same object in the left and right images will be offset. This offset is called disparity (d):

[0104] d=u L -u R

[0105] Among them, u L is the x-coordinate of the center point of the leaf spring in the left image, u R is the x-coordinate of the center point of the leaf spring in the right image, and the depth Z is calculated by triangulation.

[0106]

[0107] Where Z is the depth distance of the spring center point in the camera, f is the camera focal length, B is the camera baseline, and d is the parallax value (the offset of the matched pixels). The X and Y coordinates of the spring center point are calculated as follows:

[0108]

[0109] Finally, the three-dimensional coordinates (X, Y, Z) of the center point of the leaf spring are obtained, which represents the real space position of the target in the left camera coordinate system.

[0110] The above describes the implementation methods of the present application in detail in conjunction with the accompanying drawings, but the present application is not limited to the above implementation methods. Various changes can be made within the scope of knowledge possessed by ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A leaf spring center point visual positioning method based on multi-model collaboration, characterized in that: The following steps are involved: S01, by collecting multiple images under different exposure parameters, and then using the IFCNN model to fuse the multiple exposure images, an image with moderate exposure is obtained; S02. Use the YOLOV8 model to perform target detection on the image, accurately identify all leaf springs, and extract their regions of interest (ROIs). Each ROI contains only a single leaf spring. S03. Use U-Net to perform fine segmentation and extract the precise contour of a single leaf spring; S04. Use a binocular vision system to achieve accurate three-dimensional positioning of the center point of the leaf spring through dual-camera parallax calculation, and obtain the spatial coordinates (X, Y, Z) of the center point of the leaf spring.

2. The leaf spring center point visual positioning method based on multi-model collaboration according to claim 1 is characterized by: In step S01, the brightness of the current lighting environment is determined by the following formula, the average pixel value of the image is calculated, and the degree of lighting sufficiency of the current working environment is determined. The formula is as follows: Among them, I avg Represents the average brightness of the image, I(x,y) is the grayscale value of the pixel, M, N are the width and height of the image, according to I avg The value of is used to judge the current exposure situation and decide whether it is necessary to collect additional images with different exposure levels.

3. The leaf spring center point visual positioning method based on multi-model collaboration according to claim 2 is characterized by: The logic for determining whether additional images with different exposure levels need to be collected is as follows: When I avg If the image brightness is within the normal range (i.e., the image brightness is moderate), target detection can be performed directly without collecting additional exposure images; When I avg When deviating from the normal range: If the current exposure is high (too bright), you need to capture two images with worse exposure (i.e., lower the exposure value); If the current exposure is extremely low (too dark), you need to capture two images with better exposure (i.e., increase the exposure value); For other situations (such as too dark or too bright), it is only necessary to obtain a better-exposed picture and a worse-exposed picture for subsequent image processing.

4. The leaf spring center point visual positioning method based on multi-model collaboration according to claim 2 or 3, characterized in that: In step S01, three images with different exposures are input into the IFCNN model to obtain an image with moderate exposure I fusion .

5. The leaf spring center point visual positioning method based on multi-model collaboration according to claim 4 is characterized by: In step S02, the image I fusion When input into the YOLOV8 model, the shape of the top leaf spring is an n×6 tensor, where n represents the number of detected targets and each target contains the following 6 values: x_center, the x coordinate of the center point of the target box; y_center, the y coordinate of the center point of the target box; width, the width of the target box; height, the height of the target box; confidence, the confidence of the target (0-1); class_id, the category index of the target; The coordinates of the upper left corner and lower right corner of the bounding box (x min ,y min ) and (x max ,y max ) can be calculated using the following formula: The ROI area of ​​the leaf spring plate in the image can be taken out according to the acquired coordinates, and there is only one leaf spring plate in each ROI.

6. The leaf spring center point visual positioning method based on multi-model collaboration according to claim 5 is characterized by: In step S03, the leaf spring ROI region is fed into the trained U-Net model to obtain a segmentation map of the leaf spring.

7. The method for visually locating the center point of a leaf spring based on multi-model collaboration according to claim 6, characterized in that: In the segmentation diagram of the leaf spring, the leaf spring area and other areas are marked with different RGB color codes.

8. The leaf spring center point visual positioning method based on multi-model collaboration according to claim 6 or 7, characterized in that: In step S04, the binocular vision system is calibrated using two steps: monocular calibration and binocular calibration.

9. The leaf spring center point visual positioning method based on multi-model collaboration according to claim 8, characterized in that: Monocular calibration primarily calculates the camera's intrinsic parameters to dedistort the lens and estimate depth. Binocular calibration calculates the extrinsic parameters of the left and right cameras. Knowing these extrinsic parameters allows the left and right cameras to be rotated by a certain angle so that their corresponding points are on the same plane and horizontal line.

10. The leaf spring center point visual positioning method based on multi-model collaboration according to claim 8, characterized in that: Due to the different perspectives of the binocular cameras, the position of the same object in the left and right images will be offset. This offset is called disparity (d): d=u L -u R Among them, u L is the x-coordinate of the center point of the leaf spring in the left image, u R is the x-coordinate of the center point of the leaf spring in the right image, and the depth Z is calculated by triangulation. Where Z is the depth distance of the spring center point in the camera, f is the camera focal length, B is the camera baseline, and d is the parallax value (the offset of the matched pixels). The X and Y coordinates of the spring center point are calculated as follows: Finally, the three-dimensional coordinates (X, Y, Z) of the center point of the leaf spring are obtained, which represents the real space position of the target in the left camera coordinate system.

Images