Nut tightening method, storage medium and tightening device

Abstract

The invention discloses a nut tightening method, a storage medium and a tightening device. The method comprises the steps that a target image is obtained, and the target image is obtained by collecting a cockpit to be assembled with a heat insulation pad; determining stud positioning information based on the target image, wherein the stud positioning information is used for determining the relative position of the stud and the cockpit; obtaining nut classification information; based on the nut classification information and the stud positioning information, classifying the plurality of studs at different positions to obtain an attribute classification result; a first target control instruction is generated based on the attribute classification result, the first target control instruction is used for controlling working parameters of the tightening gun, and the working parameters at least comprise the tightening torque. The technical problem that a nut tightening method is poor in tightening effect is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of nut tightening control, and in particular, to a method for tightening nuts, a storage medium, and a tightening device. Background Art

[0002] Traditional tightening tools (torque wrenches or pneumatic / electric tightening tools) may not be able to accurately identify and adapt to the fastening requirements of nuts in different positions, resulting in poor locking effects. In production lines with low automation levels or when encountering special fastening requirements, manual assistance may be required for nut positioning and fastening to ensure accuracy and fitting effects. However, manual operation is inefficient, and consistency is difficult to guarantee. It may also bring operation risks and increased costs. During the nut fastening process, due to uneven paint thickness, gaps may be generated between the heat insulation pad and the floor, affecting the fitting effect. There are hard supports on the installation fixture to ensure proper installation when installing the heat insulation pad, but gaps may be generated between the heat insulation pad and the floor during the nut fastening process, and it is impossible to press tightly after tightening the nuts.

[0003] In response to the above problems, no effective solutions have been proposed yet. Summary of the Invention

[0004] Embodiments of the present invention provide a method for tightening nuts, a storage medium, and a tightening device to at least solve the technical problem of poor tightening effect of the nut tightening method.

[0005] According to one aspect of the embodiments of the present invention, a method for tightening nuts is provided, including: acquiring a target image, which is obtained by collecting the cockpit of the heat insulation pad to be assembled; determining stud positioning information based on the target image, where the stud positioning information is used to determine the relative position between the stud and the cockpit; acquiring nut classification information, where the nut classification information includes the mapping relationship between the relative position and the attribute category of the stud; classifying a plurality of studs at different positions based on the nut classification information and the stud positioning information to obtain an attribute classification result; generating a first target control instruction based on the attribute classification result, where the first target control instruction is used to control the working parameters of the tightening gun, and the working parameters at least include the tightening torque.

[0006] Further, determining stud positioning information based on the target image includes: extracting stud feature points from the target image using an image processing algorithm to obtain first coordinate information, where the first coordinate information is the pixel coordinates of the stud in the target image; performing coordinate transformation on the first coordinate information to obtain second coordinate information, where the second coordinate information is the spatial coordinates of the stud in the real space coordinate system; determining the second coordinate information as the stud positioning information.

[0007] Further, coordinate transformation is performed on the first coordinate information to obtain second coordinate information, including: calibrating parameters of the camera used to collect the target image, where the parameters at least include internal parameters and external parameters, to obtain a calibration result; determining a normalized camera coordinate based on the calibration result and the first coordinate information; and determining the second coordinate information based on the normalized camera coordinate.

[0008] Further, multiple studs at different positions are classified based on the nut classification information and the stud positioning information to obtain an attribute classification result, including: determining a classification mapping relationship between a spatial coordinate range and an attribute category in a real space coordinate system based on the nut classification information, where the attribute category includes at least one of the following: studs at the lap position of front and rear heat insulation pads, thick paint surface studs, and studs with poor heat insulation pad contact; and classifying the multiple studs based on the second coordinate information and the classification mapping relationship to obtain an attribute classification result.

[0009] Further, before generating a first target control instruction based on the attribute classification result, the method further includes: estimating the pose of the stud based on the target image to obtain a pose estimation result; and generating a second target control instruction in response to the pose estimation result satisfying a preset condition, where the second target control instruction is used to control the tightening gun to stop performing the tightening action and generate an alarm message.

[0010] Further, estimating the pose of the stud based on the target image to obtain a pose estimation result includes: converting the target image into a source point cloud model; performing rigid body transformation on the source point cloud model using an ICP registration algorithm to obtain a transformation matrix; determining a target point cloud model based on the transformation matrix and the source point cloud model; performing point cloud cropping on the target point cloud model to obtain a target stud point cloud; and determining the pose estimation result based on the target stud point cloud.

[0011] Further, determining the pose estimation result based on the target stud point cloud includes: performing cylindrical fitting on the target stud point cloud to obtain a fitted cylinder; and determining that the pose estimation result satisfies the preset condition in response to the axis angle between the fitted cylinder and a reference cylinder being greater than a preset value.

[0012] Further, converting the target image into a source point cloud model includes: performing point cloud sampling on the target image using a voxel grid downsampling method to obtain a source point cloud model.

[0013] According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, where the computer-readable storage medium includes a stored executable program, and when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the methods in the various embodiments of the present invention.

[0014] According to another aspect of the embodiments of the present invention, there is also provided a tightening device, and the tightening device is used to execute the method according to the above.

[0015] In the embodiments of the present invention, a method for determining stud positioning information based on a target image is adopted. By classifying a plurality of studs at different positions based on nut classification information and stud positioning information, an attribute classification result is obtained to adjust the corresponding tightening torque, achieving the purpose of adaptively adjusting tightening parameters for different positions, thereby realizing the technical effect of improving the tightening effect of bolts at various positions, and further solving the technical problem of poor tightening effect of the nut tightening method. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and form a part of this application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0017] Figure 1 is a schematic flowchart of an optional nut tightening method according to an embodiment of the present invention;

[0018] Figure 2 is a schematic block diagram of an optional nut tightening device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0021] According to an embodiment of the present invention, a method embodiment of a method for tightening a nut is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0022] Figure 1 is a method according to an embodiment of the present invention, as Figure 1 shown, the method includes the following steps:

[0023] Step S102, obtaining a target image, which is obtained by collecting the cockpit of the heat insulation pad to be assembled;

[0024] Step S103, determining stud positioning information based on the target image, where the stud positioning information is used to determine the relative position between the stud and the cockpit;

[0025] Step S104, obtaining nut classification information, where the nut classification information includes the mapping relationship between the relative position and the attribute category of the stud;

[0026] Step S105, classifying a plurality of studs at different positions based on the nut classification information and the stud positioning information to obtain an attribute classification result;

[0027] Step S106, generating a first target control instruction based on the attribute classification result, where the first target control instruction is used to control the working parameters of the tightening gun, and the working parameters at least include the tightening torque.

[0028] Through the above steps, by using the method of determining the stud positioning information based on the target image, and classifying a plurality of studs at different positions based on the nut classification information and the stud positioning information to obtain an attribute classification result to adjust the corresponding tightening torque, the purpose of adaptively adjusting the tightening parameters for different positions is achieved, thereby realizing the technical effect of improving the tightening effect of bolts everywhere, and further solving the technical problem of poor tightening effect of the method for tightening nuts.

[0029] Optionally, in step S103, determining the stud positioning information based on the target image includes:

[0030] Step S1031, using an image processing algorithm to extract stud feature points from the target image to obtain first coordinate information, where the first coordinate information is the pixel coordinates of the stud in the target image;

[0031] Step S1032, performing coordinate transformation on the first coordinate information to obtain second coordinate information, where the second coordinate information is the spatial coordinates of the stud in the real space coordinate system;

[0032] Step S1033: Determine that the second coordinate information is the stud positioning information.

[0033] Optionally, in step S1032, perform coordinate transformation on the first coordinate information to obtain the second coordinate information, including:

[0034] Calibrate the parameters of the camera used to collect the target image. The parameters include at least the internal parameters and external parameters to obtain the calibration result;

[0035] Determine the normalized camera coordinates based on the calibration result and the first coordinate information;

[0036] Determine the second coordinate information based on the normalized camera coordinates.

[0037] The actual space coordinates refer to the position of an object in the real three-dimensional space, usually represented by a three-dimensional coordinate system (such as the world coordinate system). The definition of the coordinate system is as follows: Origin: Usually select a fixed point in the scene as the origin (such as a corner of the cockpit). The X-axis and Y-axis are usually defined on the horizontal plane (such as the bottom plane of the cockpit). The Z-axis is perpendicular to the horizontal plane, representing the height direction. Unit: Usually in millimeters (mm) or meters (m). The actual space coordinates are used to describe the exact position of an object in the real world, such as the specific position of a stud at the bottom of the cockpit.

[0038] Optionally, the specific steps to convert the stud coordinates in the image to the actual space coordinates include:

[0039] Step S1: Camera calibration. The purpose of camera calibration is to determine the internal parameters of the camera (such as focal length, principal point, distortion coefficients, etc.) and external parameters (the position and orientation of the camera relative to the world coordinate system). This is the basis for converting image coordinates to actual space coordinates. Step S1.1: Prepare the calibration tool. Use a calibration board with a known size (such as a checkerboard or dot array). The actual coordinates of each corner point or dot on the calibration board are known. Step S1.2: Take calibration images. Take multiple images of the calibration board from different angles (usually 10 - 20 images are required). At this time, ensure that the calibration board is clearly visible in the image and covers the entire field of view. Step S1.3: Calculate the camera parameters. Use the calibration algorithm (in this embodiment, the cv2.calibrateCamera function in OpenCV) to calculate the internal and external parameters of the camera. The internal parameters include: focal length (fx, fy), principal point (cx, cy), distortion coefficients (k1, k2, p1, p2, k3). The external parameters include: rotation matrix (R), translation vector (T). Step S1.4: Save the calibration result. Save the calculated camera parameters for subsequent coordinate transformation.

[0040] Step S2: Conversion from Image Coordinates to Actual Space Coordinates. After camera calibration is completed, the stud coordinates in the image can be converted to actual space coordinates through the following steps. Step S2.1: Obtain the stud coordinates in the image. Use image processing algorithms (such as edge detection, template matching, or deep learning models) to detect the center point of the stud and obtain its pixel coordinates (u, v) in the image. Step S2.2: Determine the actual height (Z coordinate) of the stud. If the stud is located on a plane (such as the bottom of the cockpit), its Z coordinate can be assumed to be 0 (with the plane as the reference). If the heights of the studs are different, the actual height can be measured using a depth sensor (such as a laser sensor or a structured light camera). Step S2.3: Calculate the actual space coordinates. Use the camera calibration results and the stud image coordinates to calculate the actual space coordinates through perspective transformation or inverse perspective mapping. Calculate the actual space coordinates: If the stud is located on a plane (Z = 0), the homography matrix can be used to calculate its actual coordinates.

[0041] Through the above steps, the stud coordinates in the image can be converted to actual space coordinates, providing accurate input data for the subsequent positioning and operation of the tightening gun. Optionally, in step S105, multiple studs at different positions are classified based on the nut classification information and the stud positioning information to obtain an attribute classification result, including:

[0042] Step S1051, determine the classification mapping relationship between the spatial coordinate range and the attribute category in the real space coordinate system based on the nut classification information. The attribute category includes at least one of the following: studs at the overlapping position of the front and rear heat insulation pads, studs with thick paint surfaces, and studs with poor heat insulation pad contact;

[0043] Step S1052, classify the multiple studs based on the second coordinate information and the classification mapping relationship to obtain an attribute classification result.

[0044] In an alternative embodiment, it is determined which position of the cockpit a stud belongs to based on its spatial coordinates by predefined rules: collect the design drawings of the cockpit or actual measurement data to obtain the spatial coordinates (X, Y, Z) of each stud and its corresponding part label (such as front, rear, left, right, etc.). If the design drawings are not available, the coordinates of the stud can be obtained through actual measurement or image recognition. Label the part to which each stud's coordinates belong (such as front, rear, left, right, etc.).

[0045] Example rules: Front part: X ∈ [0, 200], Y ∈ [0, 300]. Rear part: X ∈ [400, 600], Y ∈ [0, 300]. Left side: X ∈ [0, 600], Y ∈ [0, 150]. Right side: X ∈ [0, 600], Y ∈ [450, 600]. For the coordinates (X, Y, Z) of each stud, determine its belonging part according to the defined rules.

[0046] For example, for the coordinates at (x0, y0, z0), since it is within the first coordinate range, it can be determined as the bolt at the lap joint of the front and rear heat insulation pads. At this time, an instruction needs to be sent to the tightening gun to adjust the torque to 6 N.M.

[0047] For example, for the coordinates at (x1, y1, z1), since it is within the second coordinate range, it can be determined as the rear row bolt of the heat insulation pad. The paint thickness of the bolt here is thick, so the torque needs to be increased. At this time, an instruction needs to be sent to the tightening gun to adjust the torque to 8 N.M.

[0048] In an optional embodiment, classification based on a machine learning model can be used to determine the position of the nut. Specifically, the classification methods include:

[0049] Step 1. Feature selection: Use the spatial coordinates (X, Y, Z) of the stud as features. If the part classification has nothing to do with the height (Z coordinate), only (X, Y) can be used as features.

[0050] Step 2. Select a model. Commonly used classification models include: Support Vector Machine (SVM), Decision Tree, Random Forest, K-Nearest Neighbors (KNN), Neural Network.

[0051] Step 3. Use the training set data to train the model.

[0052] Step 4. Use the test set to evaluate the performance of the model, and calculate metrics such as accuracy and confusion matrix. If the model performance is insufficient, try to adjust the model parameters or select other models.

[0053] Finally, use the trained model to classify the coordinates of the new stud.

[0054] Optionally, for the cockpits of different projects, if the cockpit layout is simple and the rules are clear, classification based on predefined rules can be selected. If the cockpit layout is complex or the rules are difficult to define, it is recommended to select classification based on a machine learning model.

[0055] When assembling the heat insulation pad to the bottom of the cockpit, a tightening gun is needed to install nuts and washers onto the studs reserved in the cockpit (there are 8 studs in total in the cockpit that need to install nuts and washers). During the assembly process, it is necessary to take pictures of the cockpit, then perform image recognition to locate the studs, and after positioning, send the positioning information to the tightening gun to tighten the corresponding studs and nuts one by one. However, when tightening nuts on studs at different positions, the torque required by the tightening gun should also be adjusted adaptively. For example, the tightening torque for studs with a thicker paint surface should be greater to ensure the tight fit between the heat insulation pad and the cab.

[0056] The programming control of the servo tightening gun makes the fastening process more precise and consistent, reducing the risk of nut fastening failure or over-tightening caused by improper torque setting; by adjusting the torque setting according to the analysis results, even in areas with a thicker paint surface on the heat insulation pad, effective fastening of the nuts can be achieved, thus ensuring a tight fit between the heat insulation pad and the vehicle body floor, improving the heat insulation performance of the vehicle and the overall assembly quality.

[0057] If there is a failure during the processes such as heat insulation pad shooting and heat insulation pad grasping, it will call for manual handling and the robot will reset. If there is a failure during the nut tightening process, the damaged nuts will be discarded, and at the same time, it will call for manual handling and reset.

[0058] Verification statistics analysis of the tightening consistency of the torque-setting tool and solutions: For the torque-setting tool verification in 2 shifts, verifying 38 vehicles, with the torque set at 7NM (the standard torque for the floor studs given in the design is 6 (tolerance 15%-20%), and the standard torque on the longitudinal beam is 8). As a result, 2 studs at the overlapping position of the front and rear heat insulation pads fell off, and 3 studs in the last row of the rear heat insulation pad could not be tightened due to the thick paint surface.

[0059] By configuring a servo tightening gun, programming can be carried out in the controller of the servo tightening gun. Different tightening torque parameters can be programmed and set, and then through I / O signal control, different torque settings can be adopted for different nut positions to meet the different bolt requirements on site.

[0060] Optionally, before generating the first target control instruction based on the attribute classification result, the method further includes:

[0061] Estimate the pose of the stud based on the target image to obtain a pose estimation result;

[0062] In response to the pose estimation result meeting the preset conditions, generate a second target control instruction, where the second target control instruction is used to control the tightening gun to stop performing the tightening action and generate an alarm message.

[0063] That is, before sending an instruction to the tightening gun, it is necessary to estimate the pose of the bolt. The specific steps are as follows: The image is sampled into a point cloud, and the initial point cloud is rigidly transformed using ICP registration to obtain the target point cloud. Then, the target point cloud is fitted to obtain a fitted stud. If the axis angle deviation of the fitted stud is relatively large, it is determined that the stud is bent and cannot be assembled, and the assembly is stopped and an alarm is sent.

[0064] The ICP (Iterative Closest Point) algorithm is used for point cloud registration and is mainly used to align two or more point cloud data sets. It finds the best rigid transformation (rotation and translation) through iterative optimization to minimize the distance between two point clouds.

[0065] The basic steps of the ICP algorithm:

[0066] Step 1, Initialization: Given two point clouds, the source point cloud and the target point cloud, and assume an initial transformation matrix (usually the identity matrix, indicating no initial transformation).

[0067] Step 2, Find corresponding points: For each point in the source point cloud, find the nearest point in the target point cloud (usually using the Euclidean distance).

[0068] Step 3, Calculate the transformation: Based on the found corresponding point pairs, calculate a rigid transformation (rotation matrix R and translation vector T) to minimize the distance between the source point cloud and the target point cloud. Usually, the least squares method is used to solve it.

[0069] Step 4, Apply the transformation: Apply the calculated transformation to the source point cloud to update the position of the source point cloud.

[0070] Iteration: Repeat steps 2 - 4 until the convergence condition is met (such as the change in the transformation matrix is less than a certain threshold, or the maximum number of iterations is reached).

[0071] Output result: The final transformation matrix is the best transformation that aligns the source point cloud to the target point cloud.

[0072] In an optional embodiment, the variants of the ICP algorithm can adopt the following types: Point-to-Point ICP: The most basic ICP algorithm that directly minimizes the distance between points. Point-to-Plane ICP: Considering the normal vectors of the point cloud, it minimizes the distance from points to planes, usually accelerating the convergence speed and improving the accuracy. Generalized ICP (GICP): Combining the advantages of point-to-point and point-to-plane, it is suitable for more complex point cloud registration tasks. Robust ICP: Introducing a robustness mechanism, it can handle noise and outliers.

[0073] Adopting the technical solution of the present application, a tightening device is provided, which includes the following modules:

[0074] 1. Acquisition module: used to capture images of the bottom of the cockpit. The image acquisition module uses a high-resolution industrial camera to capture images of the bottom of the cockpit. The camera should be installed at a fixed position or on a robotic arm to ensure that all studs can be clearly captured.

[0075] The lighting conditions need to be uniform to avoid shadows or reflections interfering with image recognition.

[0076] 2. Image preprocessing and stud positioning module: locates the positions of the studs through image recognition technology. Image preprocessing includes operations such as denoising and enhancing contrast to improve recognition accuracy.

[0077] Stud detection includes: using a deep learning model (such as YOLO, Faster R-CNN) or traditional image processing algorithms (such as edge detection, template matching) to identify the positions of the studs. Output the center coordinates (x, y) and their bounding boxes of each stud.

[0078] 3. Nut position classification module: determines which part of the cockpit the stud belongs to based on the position information of the stud. Based on the spatial coordinates of the stud, determine which part of the cockpit it belongs to (such as the front part, the rear part, the left side, the right side, etc.). Optionally, predefined rules (coordinate ranges) can be used. In another alternative embodiment, a machine learning model (such as SVM, decision tree) is used for classification.

[0079] In an alternative embodiment, the paint thickness of the nut can also be detected: use a laser sensor or an ultrasonic sensor to measure the paint thickness on the surface of the stud, and combine the paint thickness information with the stud position information for torque adjustment.

[0080] 4. Torque adaptive adjustment module: dynamically adjusts the torque of the tightening gun according to the position and surface characteristics (such as paint thickness) of the stud.

[0081] Optionally, a torque rule library is preset, and torque values are predefined according to the position and surface characteristics of the stud. The rule library can be constructed based on experimental data or empirical values.

[0082] According to the position and paint thickness of the current stud, find the corresponding torque value from the rule library.

[0083] Send the torque value to the tightening gun control system.

[0084] 5. Tightening gun control module: receives the positioning and torque information and performs the tightening operation.

[0085] This module is used to receive the positioning information and torque value of the stud, control the tightening gun to move to the specified position, tighten the nut according to the set torque value, and after the tightening is completed, feedback the tightening result (such as success or failure) to the monitoring system.

[0086] 6. Feedback and Monitoring Module: Monitor the tightening process in real time to ensure the correct completion of the operation.

[0087] This module is used to monitor the working status and tightening result of the tightening gun. If the tightening fails (such as insufficient or excessive torque), the system should issue an alarm and record the fault information. This module is used to record the tightening data (such as position, torque value, tightening result) of each stud for quality traceability and analysis.

[0088] Through the above technical solutions, the intelligent operation of the tightening gun can be realized, ensuring the close fit between the heat insulation pad and the cockpit, and at the same time improving the assembly efficiency and quality. It has the following advantages:

[0089] Advantage 1: Reduce manual intervention and improve assembly efficiency;

[0090] Advantage 2: Ensure the accuracy of stud positioning and torque adjustment through image recognition and sensor technology;

[0091] Advantage 3: Can adapt to the changes of different cockpit models and paint thicknesses;

[0092] Advantage 4: Record the data of each tightening, which is convenient for quality traceability and analysis.

[0093] Optionally, estimate the pose of the stud based on the target image to obtain the pose estimation result, including:

[0094] Convert the target image into a source point cloud model;

[0095] Use the ICP registration algorithm to perform rigid body transformation on the source point cloud model to obtain the transformation matrix;

[0096] Determine the target point cloud model based on the transformation matrix and the source point cloud model;

[0097] Perform point cloud clipping on the target point cloud model to obtain the target stud point cloud;

[0098] Determine the pose estimation result based on the target stud point cloud.

[0099] Optionally, determine the pose estimation result based on the target stud point cloud, including:

[0100] Perform cylindrical fitting on the target stud point cloud to obtain the fitted cylinder;

[0101] In response to the axis angle between the fitted cylinder and the reference cylinder being greater than the preset value, determine that the pose estimation result meets the preset conditions.

[0102] Optionally, converting the target image into a source point cloud model includes:

[0103] Performing point cloud sampling on the target image using a voxel grid downsampling method to obtain a source point cloud model.

[0104] Purpose of point cloud downsampling: Point cloud data usually contains a large number of points (such as millions or even billions of points). Directly processing this data involves huge computational costs. Through downsampling, the number of points can be reduced, thereby reducing the computational complexity. The downsampled point cloud data has a smaller volume and is suitable for real-time processing or scenarios with limited resources. Certain downsampling methods can filter out noise points in the point cloud, improving the data quality. Making the points in the point cloud more evenly distributed and avoiding overly dense or sparse points in certain areas.

[0105] Principle of voxel grid downsampling: Divide the point cloud space into regular voxel grids (three-dimensional cubes), and only retain one point (such as the center point or centroid point) within each voxel.

[0106] Advantages of voxel grid downsampling: Simple and efficient, capable of evenly reducing the number of points.

[0107] Principle of random downsampling: Randomly delete a certain proportion of points from the point cloud. Advantages: Simple to implement and fast in computational speed.

[0108] Principle of curvature-based downsampling: Based on the curvature information of the point cloud, retain points with larger curvatures (usually feature points) and delete points with smaller curvatures. Advantages: Can retain the feature information of the point cloud.

[0109] Principle of farthest point sampling (FPS): Gradually select the point farthest from the selected points from the point cloud until the target number of points is reached. Advantages: Can evenly cover the geometry of the point cloud.

[0110] In an optional embodiment, the method for tightening the nut includes:

[0111] Step 1: Collect images of the cockpit and studs;

[0112] Step 2: Calibrate the camera, and based on the calibration result, convert the image coordinates of the studs into normalized camera coordinates;

[0113] Step 3: Convert the normalized camera coordinates into real coordinates (x, y, z) in the real space coordinate system;

[0114] Step 4: Obtain the coordinate classification rule. The tightening classification of studs located in different coordinate ranges is different;

[0115] In summary, the torque of bolts at different positions is adaptively adjusted.

[0116] In an optional embodiment, before the tightening operation starts, the collaborative robot arm carries the tightening gun and moves to the preset operation point. The nut is automatically sent to the end of the tightening gun through the feeder and the blowing device, and the vision system confirms that the nut is in place.

[0117] The 3D vision sensor takes pictures and identifies the studs on the cab floor, and determines the exact position of each stud through the stitching and analysis of point cloud data. If the cab moves during the vision recognition process, the vision system will update the stud position in real time to ensure the accuracy of the positioning information.

[0118] According to the position information of the stud, the controller of the tightening gun adjusts the angle and position of the tightening gun through the servo motor to ensure that the tightening head of the tightening gun is precisely aligned with the stud. This process requires precise robotic arm control and close coordination of the tightening gun attitude adjustment.

[0119] The controller of the tightening gun starts the tightening operation according to the preset torque parameters. During the tightening process, the controller monitors the torque output of the tightening gun to ensure that the torque reaches the set value. If it is detected that the torque does not meet the requirement, the tightening gun will continue to tighten until the torque requirement is met.

[0120] After the tightening is completed, the controller of the tightening gun will feedback the tightening status (including whether it is successful, the torque value, etc.) to the control system in real time through the signal connection. This feedback information is used for subsequent data analysis and adjustment of the operation process.

[0121] During the tightening process, if abnormal situations such as nut damage, stud identification error, or unqualified tightening torque occur, the tightening gun and the collaborative robot will immediately stop the operation and call for manual intervention through the control system. After the manual processing is completed, the robot and the tightening gun are reset and ready for the next tightening operation.

[0122] At Station 1 and Station 2, the tightening of nuts by the tightening gun adopts parallel operation, that is, while installing the left and right side heat insulation pads, the tightening gun can start to tighten the nuts on the front or rear side. This optimization method utilizes the waiting time between operations and further improves the assembly efficiency by reasonably planning the operation process.

[0123] During the tightening operation, it is necessary to regularly verify and adjust the torque consistency of the tightening gun. By comparing with the standard torque and statistically analyzing the actual tightening effect, the torque setting of the tightening gun is finely adjusted to meet the tightening requirements under different working conditions and ensure the consistency of the tightening quality.

[0124] As Figure 2 shown, according to an embodiment of the present invention, an apparatus embodiment of a tightening device is provided. It should be noted that this device can be used to execute the above-mentioned method for tightening nuts. The tightening device includes:

[0125] A first acquisition module 40, configured to acquire a target image, which is obtained by collecting the cockpit of the heat insulation pad to be assembled;

[0126] A determination module 42, configured to determine stud positioning information based on the target image, and the stud positioning information is used to determine the relative position between the stud and the cockpit;

[0127] A second acquisition module 44, configured to acquire nut classification information, where the nut classification information includes a mapping relationship between the relative position and the attribute category of the stud;

[0128] A classification module 46, configured to classify a plurality of studs at different positions based on the nut classification information and the stud positioning information to obtain an attribute classification result;

[0129] A generation module 48, configured to generate a first target control instruction based on the attribute classification result, and the first target control instruction is used to control the working parameters of the tightening gun, and the working parameters at least include the tightening torque.

[0130] An embodiment of the present application further provides an electronic device, including: a memory storing an executable program; a processor for running the program, where when the program runs, it executes the methods in the various embodiments of the present invention.

[0131] An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium includes a stored executable program, and when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the methods in the various embodiments of the present invention.

[0132] An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the methods in the various embodiments of the present invention.

[0133] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium for storing a computer program, and when the computer program is executed by a processor, it implements the methods in the various embodiments of the present invention.

[0134] In the above embodiments of the present invention, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0135] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0136] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0138] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The foregoing storage medium includes: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks or optical discs that can store program codes.

[0139] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for tightening a nut, characterized in that: include: Acquire a target image, wherein the target image is acquired by collecting the cockpit on which the thermal insulation pad is to be installed; Determining stud positioning information based on the target image, wherein the stud positioning information is used to determine a relative position of the stud and the cockpit; Acquire nut classification information, wherein the nut classification information includes a mapping relationship between the relative position and the attribute category of the stud; Classifying the plurality of studs located at different positions based on the nut classification information and the stud positioning information to obtain a property classification result; A first target control instruction is generated based on the attribute classification result, where the first target control instruction is used to control working parameters of the tightening gun, where the working parameters at least include tightening torque.

2. The method for tightening a nut according to claim 1, characterized in that: Determining stud location information based on the target image includes: Extracting stud feature points from a target image using an image processing algorithm to obtain first coordinate information, where the first coordinate information is a pixel coordinate of the stud in the target image; Performing coordinate conversion on the first coordinate information to obtain second coordinate information, where the second coordinate information is the spatial coordinate of the stud in a real space coordinate system; The second coordinate information is determined to be the stud positioning information.

3. The method for tightening a nut according to claim 2, characterized in that: The first coordinate information is subjected to coordinate conversion to obtain second coordinate information, including: Calibrate parameters of a camera used to acquire the target image, wherein the parameters include at least internal parameters and external parameters, and obtain a calibration result; Determine a normalized camera coordinate based on the calibration result and the first coordinate information; The second coordinate information is determined based on the normalized camera coordinates.

4. The method for tightening a nut according to claim 2, characterized in that: Based on the nut classification information and the stud location information, a plurality of studs located at different positions are classified to obtain attribute classification results, including: Determine the classification mapping relationship between the spatial coordinate range in the real space coordinate system and the attribute category based on the nut classification information, wherein the attribute category includes at least one of the following: studs at the overlap positions of the front and rear thermal insulation pads, studs with thick paint, and studs with poor contact of the thermal insulation pads; The plurality of studs are classified based on the second coordinate information and the classification mapping relationship to obtain a property classification result.

5. The method for tightening a nut according to claim 1, characterized in that: Before generating the first target control instruction based on the attribute classification result, the method further includes: estimating the position and posture of the stud based on the target image to obtain a position and posture estimation result; In response to the pose estimation result satisfying a preset condition, a second target control instruction is generated, where the second target control instruction is used to control the tightening gun to stop performing the tightening action and generate an alarm message.

6. The method for tightening a nut according to claim 5, characterized in that: The pose of the stud is estimated based on the target image to obtain a pose estimation result, including: Converting the target image into a source point cloud model; Using the ICP registration algorithm to perform a rigid body transformation on the source point cloud model to obtain a transformation matrix; Determine a target point cloud model based on the transformation matrix and the source point cloud model; Performing point cloud cropping on the target point cloud model to obtain a target stud point cloud; The pose estimation result is determined based on the target stud point cloud.

7. The method for tightening a nut according to claim 6, characterized in that: Determining the pose estimation result based on the target stud point cloud includes: Performing cylinder fitting on the target stud point cloud to obtain a fitted cylinder; In response to the angle between the axes of the fitting cylinder and the reference cylinder being greater than a preset value, it is determined that the pose estimation result meets a preset condition.

8. The method for tightening a nut according to claim 6, characterized in that: Converting the target image into a source point cloud model includes: The target image is sampled using a voxel grid downsampling method to obtain the source point cloud model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 8.

10. A tightening device, characterized in that: The tightening device is used to perform the method according to any one of claims 1 to 8.

Images