Nut tightening methods, storage media, and tightening devices

Abstract

This invention discloses a method, storage medium, and tightening device for tightening nuts. The method includes: acquiring a target image obtained from the cockpit of the area to be fitted with a heat insulation pad; determining stud positioning information based on the target image, the stud positioning information being used to determine the relative position of the studs to the cockpit; acquiring nut classification information; classifying multiple studs located at different positions based on the nut classification information and the stud positioning information to obtain attribute classification results; and generating a first target control command based on the attribute classification results, the first target control command being used to control the operating parameters of the tightening gun, the operating parameters including at least the tightening torque. This invention solves the technical problem of poor tightening effect in nut tightening methods.

Description

Technical Field

[0001] This invention relates to the field of nut tightening control technology, and more specifically, to a nut tightening method, storage medium, and tightening device. Background Technology

[0002] Traditional tightening tools (torque wrenches or pneumatic / electric tightening tools) may not accurately identify and adapt to the tightening requirements of nuts in different positions, resulting in poor locking effects. In production lines with low automation, or when encountering special tightening requirements, manual assistance may be needed for nut positioning and tightening to ensure accuracy and fit. However, manual operation is inefficient, inconsistent, and may also lead to operational risks and increased costs. During nut tightening, uneven paint thickness may cause gaps between the heat insulation pad and the floor, affecting the fit. While rigid supports in the mounting fixture can ensure proper installation of the heat insulation pad, gaps may still occur between the heat insulation pad and the floor during nut tightening, preventing proper compression after tightening the nut.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a method for tightening a nut, a storage medium, and a tightening device to at least solve the technical problem of poor tightening effect in nut tightening methods.

[0005] According to one aspect of the present invention, a method for tightening a nut is provided, comprising: acquiring a target image, the target image being acquired from a cockpit of a heat insulation pad to be assembled; determining stud positioning information based on the target image, the stud positioning information being used to determine the relative position of the stud and the cockpit; acquiring nut classification information, wherein the nut classification information includes a mapping relationship between the relative position and the attribute category of the stud; classifying multiple studs located at different positions based on the nut classification information and the stud positioning information to obtain an attribute classification result; and generating a first target control command based on the attribute classification result, the first target control command being used to control the operating parameters of a tightening gun, the operating parameters including at least a tightening torque.

[0006] Further, determining stud positioning information based on the target image includes: extracting stud feature points from the target image using an image processing algorithm to obtain first coordinate information, which is the pixel coordinate of the stud in the target image; performing coordinate transformation on the first coordinate information to obtain second coordinate information, which is the spatial coordinate of the stud in the real space coordinate system; and determining the second coordinate information as stud positioning information.

[0007] Further, the first coordinate information is transformed to obtain the second coordinate information, including: calibrating the parameters of the camera used to acquire the target image, the parameters including at least intrinsic and extrinsic parameters, and obtaining the calibration result; determining the normalized camera coordinates based on the calibration result and the first coordinate information; and determining the second coordinate information based on the normalized camera coordinates.

[0008] Furthermore, based on nut classification information and stud positioning information, multiple studs located at different positions are classified to obtain attribute classification results, including: determining the classification mapping relationship between the spatial coordinate range in the real spatial coordinate system and the attribute category based on nut classification information, where the attribute category includes at least one of the following: studs at the overlap position of the front and rear heat insulation pads, studs with thick paint, and studs with poor contact of the heat insulation pads; and classifying multiple studs based on the second coordinate information and the classification mapping relationship to obtain attribute classification results.

[0009] Furthermore, before generating the first target control command based on the attribute classification result, the method further includes: estimating the pose of the stud based on the target image to obtain a pose estimation result; and generating a second target control command in response to the pose estimation result meeting a preset condition. The second target control command is used to control the tightening gun to stop performing the tightening action and generate an alarm message.

[0010] Furthermore, the pose of the stud is estimated based on the target image to obtain the pose estimation result, including: converting the target image into a source point cloud model; performing a rigid body transformation on the source point cloud model using the ICP registration algorithm to obtain a transformation matrix; determining the target point cloud model based on the transformation matrix and the source point cloud model; cropping the target point cloud model to obtain the target stud point cloud; and determining the pose estimation result based on the target stud point cloud.

[0011] Furthermore, the pose estimation result is determined based on the target stud point cloud, including: performing cylindrical fitting on the target stud point cloud to obtain a fitted cylinder; and determining that the pose estimation result meets the preset condition in response to the angle between the axes of the fitted cylinder and the reference cylinder being greater than a preset value.

[0012] Furthermore, the target image is transformed into a source point cloud model, including: using a voxel grid downsampling method to sample the target image into a point cloud to obtain a source point cloud model.

[0013] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is executed, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0014] According to another aspect of the present invention, a tightening device is also provided for performing the method described above.

[0015] In this embodiment of the invention, a method based on target image to determine stud positioning information is adopted. Multiple studs located at different positions are classified based on nut classification information and stud positioning information to obtain attribute classification results and adjust the corresponding tightening torque. This achieves the purpose of adaptively adjusting tightening parameters for different positions, thereby improving the tightening effect of bolts at all locations and solving the technical problem of poor tightening effect of nut tightening methods. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0017] Figure 1 This is a schematic flowchart of an optional nut tightening method according to an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of an optional nut tightening device according to an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] According to an embodiment of the present invention, a method embodiment for tightening a nut is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0022] Figure 1 This is a method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0023] Step S102: Acquire the target image, which is obtained by collecting data from the cockpit of the area to be fitted with the heat insulation pad;

[0024] Step S103: Determine stud positioning information based on the target image. The stud positioning information is used to determine the relative position of the stud and the cockpit.

[0025] Step S104: Obtain nut classification information, wherein the nut classification information includes the mapping relationship between relative position and attribute category of stud;

[0026] Step S105: Based on nut classification information and stud positioning information, classify multiple studs located at different positions to obtain attribute classification results;

[0027] Step S106: Generate a first target control command based on the attribute classification result. The first target control command is used to control the working parameters of the tightening gun. The working parameters include at least the tightening torque.

[0028] Through the above steps, the positioning information of the studs is determined based on the target image. Multiple studs located at different positions are classified based on the nut classification information and the stud positioning information. The attribute classification results are used to adjust the corresponding tightening torque, thereby achieving the purpose of adaptively adjusting the tightening parameters for different positions. This improves the tightening effect of bolts at all locations and solves the technical problem of poor tightening effect of nut tightening methods.

[0029] Optionally, in step S103, determining stud positioning information based on the target image includes:

[0030] Step S1031: Use an image processing algorithm to extract stud feature points from the target image to obtain first coordinate information, which is the pixel coordinates of the stud in the target image;

[0031] Step S1032: The first coordinate information is transformed to obtain the second coordinate information, which is the spatial coordinate of the stator in the real space coordinate system;

[0032] Step S1033: Determine that the second coordinate information is stud positioning information.

[0033] Optionally, in step S1032, the first coordinate information is transformed to obtain the second coordinate information, including:

[0034] The camera used to acquire target images is calibrated with parameters including at least intrinsic and extrinsic parameters, and the calibration results are obtained.

[0035] The normalized camera coordinates are determined based on the calibration results and the first coordinate information.

[0036] The second coordinate information is obtained based on the normalized camera coordinates.

[0037] Actual spatial coordinates refer to the position of an object in real three-dimensional space, usually represented by a three-dimensional coordinate system (such as the world coordinate system). The coordinate system is defined as follows: Origin: Usually a fixed point in the scene is chosen as the origin (such as a corner of the cockpit). X-axis and Y-axis are usually defined on a horizontal plane (such as the bottom plane of the cockpit). Z-axis is perpendicular to the horizontal plane, representing the height direction. Units: Usually in millimeters (mm) or meters (m). Actual spatial coordinates are used to describe the precise position of an object in the real world, such as the specific location of a stud on the bottom of the cockpit.

[0038] Optionally, the specific steps for converting stud coordinates in the image to actual spatial coordinates include:

[0039] Step S1: Camera Calibration. The purpose of camera calibration is to determine the camera's intrinsic parameters (such as focal length, principal point, distortion coefficients, etc.) and extrinsic parameters (the camera's position and orientation relative to the world coordinate system). This is the basis for converting image coordinates to actual spatial coordinates. Step S1.1: Prepare Calibration Tools. Use a calibration board of known size (such as a checkerboard or dot array). The actual coordinates of each corner or dot of the calibration board are known. Step S1.2: Capture Calibration Images. Take multiple images of the calibration board from different angles (usually 10-20 images are needed). Ensure the calibration board is clearly visible in the images and covers the entire field of view. Step S1.3: Calculate Camera Parameters. Use a calibration algorithm (in this embodiment, the cv2.calibrateCamera function in OpenCV) to calculate the camera's intrinsic and extrinsic parameters. Intrinsic parameters include: focal length (fx, fy), principal point (cx, cy), and distortion coefficients (k1, k2, p1, p2, k3). The extrinsic parameters include: rotation matrix (R) and translation vector (T). Step S1.4: Save the calibration results. Save the calculated camera parameters for subsequent coordinate transformations.

[0040] Step S2: Conversion from Image Coordinates to Actual Spatial Coordinates. After camera calibration, the stud coordinates in the image can be converted to actual spatial coordinates using the following steps: Step S2.1: Obtain the stud coordinates in the image. Use image processing algorithms (such as edge detection, template matching, or deep learning models) to detect the center point of the stud and obtain its pixel coordinates (u,v) in the image. Step S2.2: Determine the actual height (Z coordinate) of the stud. If the stud is located on a plane (such as the bottom of the cockpit), its Z coordinate can be assumed to be 0 (with the plane as a reference). If the stud height varies, its actual height can be measured using a depth sensor (such as a laser sensor or structured light camera). Step S2.3: Calculate the actual spatial coordinates. Using the camera calibration results and the image coordinates of the stud, calculate the actual spatial coordinates through perspective transformation or inverse perspective mapping. Calculating the actual spatial coordinates: If the stud is located on a plane (Z=0), its actual coordinates can be calculated using homography.

[0041] Through the above steps, the stud coordinates in the image can be converted into actual spatial coordinates, providing accurate input data for subsequent tightening gun positioning and operation. Optionally, in step S105, multiple studs located at different positions are classified based on nut classification information and stud positioning information to obtain attribute classification results, including:

[0042] Step S1051: Based on the nut classification information, determine the classification mapping relationship between the spatial coordinate range and the attribute category in the real spatial coordinate system. The attribute category includes at least one of the following: stud at the overlap position of the front and rear heat insulation pads, stud with thick paint, and stud with poor contact of the heat insulation pad.

[0043] Step S1052: Classify multiple studs based on the second coordinate information and classification mapping relationship to obtain attribute classification results.

[0044] In an optional embodiment, the location of a stud within the cockpit is determined based on its spatial coordinates using predefined rules: Design drawings or actual measurement data of the cockpit are collected to obtain the spatial coordinates (X, Y, Z) of each stud and its corresponding location label (e.g., front, rear, left, right, etc.). If design drawings are unavailable, the stud coordinates can be obtained through actual measurement or image recognition. The location (e.g., front, rear, left, right, etc.) of each stud is then labeled according to its coordinates.

[0045] Example rules: Front: X∈[0,200], Y∈[0,300]. Back: X∈[400,600], Y∈[0,300]. Left: X∈[0,600], Y∈[0,150]. Right: X∈[0,600], Y∈[450,600]. For the coordinates (X,Y,Z) of each stud, determine its location according to the defined rules.

[0046] For example, the coordinates at (x0, y0, z0) are within the first coordinate range, which can be identified as the bolt at the overlap of the front and rear heat insulation pads. At this time, it is necessary to send a command to the tightening gun to adjust the torque to 6 N.M.

[0047] For example, the coordinates at (x1, y1, z1) are located in the second coordinate range, which can be identified as the rear bolts of the heat insulation pad. The paint on the bolts here is thick, so the torque needs to be increased. At this time, it is necessary to send a command to the tightening gun to adjust the torque to 8 N.M.

[0048] In an optional embodiment, classification based on a machine learning model can be used to determine the position of the nut. Specifically, the classification method includes:

[0049] Step 1: Feature Selection. Use the stator's spatial coordinates (X, Y, Z) as features. If the part classification is unrelated to height (Z coordinate), you can use only (X, Y) as features.

[0050] Step 2: Select a model. Commonly used classification models include: Support Vector Machine (SVM), Decision Tree, Random Forest, K-Nearest Neighbors (KNN), and Neural Network.

[0051] Step 3: Train the model using the training set data.

[0052] Step 4: Evaluate the model's performance using the test set, calculating metrics such as accuracy and confusion matrix. If the model's performance is insufficient, try adjusting the model parameters or selecting a different model.

[0053] Finally, the trained model is used to classify the coordinates of the new stud.

[0054] Optionally, for different project cockpits, if the cockpit layout is simple and the rules are clear, a classification based on predefined rules can be selected. If the cockpit layout is complex or the rules are difficult to define, it is recommended to choose a classification based on a machine learning model.

[0055] When installing the heat insulation pad onto the bottom of the cockpit, a tightening gun is needed to install the nuts and washers onto the pre-installed studs in the cockpit (there are a total of 8 studs in the cockpit that require nuts and washers). During the assembly process, the cockpit needs to be photographed, and then the image recognition is used to locate the studs. After positioning, the positioning information is sent to the tightening gun to tighten the corresponding studs and nuts one by one. However, when tightening the nuts on studs in different locations, the torque required by the tightening gun should be adjusted accordingly. For example, the tightening torque for studs with thicker paint should be greater to ensure a tight fit between the heat insulation pad and the cockpit.

[0056] The servo tightening gun's programmed control makes the tightening process more precise and consistent, reducing the risk of nut tightening failure or over-tightening due to improper torque setting. By adjusting the torque setting based on analysis results, the nuts can be effectively tightened even in areas with thick paint on the heat insulation pad, thus ensuring a tight fit between the heat insulation pad and the vehicle floor, improving the vehicle's heat insulation performance and overall assembly quality.

[0057] If the process of photographing or grasping the heat insulation pad fails, a manual intervention will be called and the robot will reset. If the process of tightening the nut fails, the damaged nut will be discarded, and a manual intervention will be called to reset the robot.

[0058] The consistency verification of the torque-controlled tool tightening was statistically analyzed and resolved. The torque-controlled tool was verified on 38 vehicles in 2 shifts. The torque was set to 7 NM (the standard torque of the floor studs given by the design is 6 (tolerance 15%-20%), and the standard torque on the longitudinal beam is 8). The results showed that 2 studs at the overlap of the front and rear heat insulation pads fell off, and 3 studs in the last row of the rear heat insulation pads could not be tightened due to the thick paint.

[0059] By configuring a servo tightening gun, and then programming the controller of the servo tightening gun, different tightening torque parameters can be set. Then, through I / O signals, different torque settings can be used for different nut positions to meet different bolt requirements on site.

[0060] Optionally, before generating the first target control instruction based on the attribute classification results, the method further includes:

[0061] The pose of the stud is estimated based on the target image, and the pose estimation result is obtained.

[0062] In response to the pose estimation result meeting the preset conditions, a second target control command is generated. The second target control command is used to control the tightening gun to stop performing the tightening action and generate an alarm message.

[0063] In other words, before sending a command to the tightening gun, the pose of the bolt needs to be estimated. The specific steps are: the image is sampled as a point cloud, and the initial point cloud is rigidly transformed using ICP registration to obtain the target point cloud. Then, the target point cloud is fitted to obtain the fitted stud. If the included angle deviation of the axis of the fitted stud is large, it is determined that the stud is bent and cannot be assembled, and the assembly is stopped and an alarm is sent.

[0064] The ICP (Iterative Closest Point) algorithm is used for point cloud registration, primarily to align two or more point cloud datasets. It iteratively optimizes the algorithm to find the optimal rigid body transformation (rotation and translation) that minimizes the distance between two point clouds.

[0065] Basic steps of the ICP algorithm:

[0066] Step 1, Initialization: Given two point clouds, a source point cloud and a target point cloud, assume an initial transformation matrix (usually an identity matrix, indicating no initial transformation).

[0067] Step 2: Find the corresponding point: For each point in the source point cloud, find the nearest point in the target point cloud (usually using Euclidean distance).

[0068] Step 3: Calculate the transformation: Based on the found corresponding point pairs, calculate a rigid body transformation (rotation matrix R and translation vector T) that minimizes the distance between the source point cloud and the target point cloud. The least squares method is typically used to solve this.

[0069] Step 4: Apply the transformation: Apply the calculated transformation to the source point cloud and update the position of the source point cloud.

[0070] Iteration: Repeat steps 2-4 until the convergence condition is met (e.g., the change in the transformation matrix is ​​less than a certain threshold, or the maximum number of iterations is reached).

[0071] Output: The final transformation matrix is ​​the optimal transformation that aligns the source point cloud to the target point cloud.

[0072] In an optional embodiment, variants of the ICP algorithm can take the following types: Point-to-Point ICP: The most basic ICP algorithm, directly minimizing the distance between points. Point-to-Plane ICP: Considering the normal vector of the point cloud, minimizing the distance from the point to the plane, which usually speeds up convergence and improves accuracy. Generalized ICP (GICP): Combining the advantages of point-to-point and point-to-plane ICP, suitable for more complex point cloud registration tasks. Robust ICP: Introducing robustness mechanisms to handle noise and outliers.

[0073] The present application provides a tightening device comprising the following modules:

[0074] 1. Image Acquisition Module: Used to capture images of the bottom of the cockpit. The image acquisition module uses a high-resolution industrial camera to capture images of the bottom of the cockpit. The camera should be mounted in a fixed position or on a robotic arm to ensure that all studs can be clearly captured.

[0075] Lighting conditions should be uniform to avoid shadows or reflections that could interfere with image recognition.

[0076] 2. Image Preprocessing and Stud Positioning Module: This module uses image recognition technology to locate the stud position. Image preprocessing includes operations such as noise reduction and contrast enhancement to improve recognition accuracy.

[0077] Stud detection involves identifying the location of studs using deep learning models (such as YOLO, Faster R-CNN) or traditional image processing algorithms (such as edge detection, template matching). The output is the center coordinates (x, y) of each stud and its bounding box.

[0078] 3. Nut Position Classification Module: Based on the stud's position information, determine which part of the cockpit it belongs to. Based on the stud's spatial coordinates, determine its location within the cockpit (e.g., front, rear, left, right, etc.). Optionally, predefined rules (coordinate ranges) can be used. In another optional embodiment, a machine learning model (e.g., SVM, decision tree) is used for classification.

[0079] In an optional embodiment, the paint thickness of the nut can also be detected: the paint thickness on the stud surface is measured using a laser sensor or an ultrasonic sensor, and the paint thickness information is combined with the stud position information for torque adjustment.

[0080] 4. Torque adaptive adjustment module: Dynamically adjusts the torque of the tightening gun according to the position of the stud and surface characteristics (such as paint thickness).

[0081] Optionally, a torque rule library is pre-defined, with torque values ​​predefined based on the stud's position and surface characteristics. The rule library can be built based on experimental data or empirical values.

[0082] Based on the current position of the stud and the thickness of the paint, the corresponding torque value is retrieved from the rule base.

[0083] The torque value is sent to the tightening gun control system.

[0084] 5. Tightening gun control module: Receives positioning and torque information and executes tightening operations.

[0085] This module is used to receive the positioning information and torque value of the stud, control the tightening gun to move to the designated position, tighten the nut according to the set torque value, and after tightening is completed, feed back the tightening result (such as success or failure) to the monitoring system.

[0086] 6. Feedback and monitoring module: Monitors the tightening process in real time to ensure the operation is completed correctly.

[0087] This module monitors the working status and tightening results of the tightening gun. If tightening fails (e.g., insufficient or excessive torque), the system should issue an alarm and record the fault information. This module records the tightening data (such as position, torque value, and tightening result) for each stud, for quality traceability and analysis.

[0088] The above technical solution enables intelligent operation of the tightening gun, ensuring a tight fit between the heat insulation pad and the cockpit, while improving assembly efficiency and quality. It offers the following advantages:

[0089] Advantage 1: Reduces manual intervention and improves assembly efficiency;

[0090] Advantage 2: Image recognition and sensor technology ensure the accuracy of stud positioning and torque adjustment;

[0091] Advantage 3: It can adapt to different cockpit models and variations in paint thickness;

[0092] Advantage 4: Records data for each tightening, facilitating quality traceability and analysis.

[0093] Optionally, the pose of the stud is estimated based on the target image to obtain the pose estimation result, including:

[0094] Convert the target image into a source point cloud model;

[0095] The source point cloud model is subjected to rigid body transformation using the ICP registration algorithm to obtain the transformation matrix;

[0096] The target point cloud model is determined based on the transformation matrix and the source point cloud model.

[0097] The target point cloud model is cropped to obtain the target stud point cloud;

[0098] The pose estimation results are determined based on the target stud point cloud.

[0099] Optionally, the pose estimation results are determined based on the target stud point cloud, including:

[0100] Perform cylindrical fitting on the point cloud of the target stud to obtain a fitted cylinder;

[0101] If the angle between the axes of the fitted cylinder and the reference cylinder is greater than a preset value, the pose estimation result is determined to meet the preset condition.

[0102] Optionally, the target image is converted into a source point cloud model, including:

[0103] The target image is sampled using a voxel grid downsampling method to obtain the source point cloud model.

[0104] The purpose of point cloud downsampling is to address the massive computational demands of processing point cloud data, which typically contains millions or even billions of points. Downsampling reduces the number of points, thereby decreasing computational complexity. Downsampled point cloud data is also smaller, making it suitable for real-time processing or resource-constrained scenarios. Some downsampling methods can filter out noisy points in the point cloud, improving data quality. Finally, it makes the point cloud distribution more uniform, preventing areas from being too densely or sparsely populated.

[0105] The principle of voxel grid downsampling is to divide the point cloud space into a regular voxel grid (3D cube), and retain only one point (such as the center point or centroid) in each voxel.

[0106] Advantages of voxel grid downsampling: simple and efficient, capable of uniformly reducing the number of points.

[0107] The principle of random downsampling is to randomly delete a certain proportion of points from a point cloud. Advantages: simple to implement and fast computation.

[0108] The principle of curvature-based downsampling is to retain points with higher curvature (usually feature points) and delete points with lower curvature based on the curvature information of the point cloud. Its advantage is that it preserves the feature information of the point cloud.

[0109] The principle of Farthest Point Sampling (FPS) is to progressively select the points in the point cloud that are furthest from the selected points until the target number of points is reached. Its advantage is that it can uniformly cover the geometry of the point cloud.

[0110] In an optional embodiment, the method for tightening the nut includes:

[0111] Step 1: Acquire images of the cockpit and studs;

[0112] Step 2: Calibrate the camera and convert the image coordinates of the stud into normalized camera coordinates based on the calibration results;

[0113] Step 3: Convert the normalized camera coordinates into true coordinates (x, y, z) in the real space coordinate system;

[0114] Step 4: Obtain the coordinate classification rules. Studs located in different coordinate ranges have different tightening classifications.

[0115] In summary, this achieves adaptive torque adjustment for bolts in different positions.

[0116] In an optional embodiment, before the tightening operation begins, the collaborative robot arm carrying the tightening gun moves to a preset work point, and the nut is automatically fed to the end of the tightening gun by a feeder and a blowing device, and the vision system confirms that the nut is in place.

[0117] 3D vision sensors capture and identify studs on the cab floor, determining the precise location of each stud through point cloud data stitching and analysis. If the cab moves during the visual recognition process, the vision system updates the stud positions in real time to ensure the accuracy of the positioning information.

[0118] Based on the position information of the stud, the controller of the tightening gun adjusts the angle and position of the tightening gun via a servo motor to ensure that the tightening head of the tightening gun is precisely aligned with the stud. This process requires close coordination between precise robotic arm control and tightening gun posture adjustment.

[0119] The tightening gun's controller initiates the tightening operation based on preset torque parameters. During the tightening process, the controller monitors the tightening gun's torque output to ensure the torque reaches the set value. If the torque is detected as insufficient, the tightening gun will continue tightening until the required torque is met.

[0120] After tightening is complete, the tightening gun's controller sends real-time feedback on the tightening status (including success rate and torque magnitude) to the control system via a signal connection. This feedback information is used for subsequent data analysis and adjustments to the work process.

[0121] During the tightening process, if any abnormalities occur such as a damaged nut, incorrect stud identification, or insufficient tightening torque, the tightening gun and collaborative robot will immediately stop operating and call for manual intervention via the control system. After the manual intervention is completed, the robot and tightening gun will reset and prepare for the next tightening operation.

[0122] At workstations one and two, the tightening guns operate in parallel, meaning that while the left and right heat insulation pads are being installed, the tightening guns can begin tightening the nuts on the front or rear sides. This optimized method utilizes the waiting time between workstations and further improves assembly efficiency through rational planning of the workflow.

[0123] During tightening operations, it is necessary to periodically verify and adjust the consistency of the tightening gun's torque. By comparing it with the standard torque and statistically analyzing the actual tightening effect, the torque setting of the tightening gun is fine-tuned to adapt to the tightening requirements under different working conditions and ensure consistent tightening quality.

[0124] like Figure 2 As shown in the embodiment of the present invention, a tightening device is provided. It should be noted that this device can be used to perform the above-described nut tightening method. The tightening device includes:

[0125] The first acquisition module 40 is used to acquire the target image, which is obtained by collecting data from the cockpit of the heat insulation pad to be assembled.

[0126] The determination module 42 is used to determine the stud positioning information based on the target image. The stud positioning information is used to determine the relative position of the stud and the cockpit.

[0127] The second acquisition module 44 is used to acquire nut classification information, wherein the nut classification information includes the mapping relationship between relative position and attribute category of stud;

[0128] Classification module 46 is used to classify multiple studs located at different positions based on nut classification information and stud positioning information to obtain attribute classification results;

[0129] The generation module 48 is used to generate a first target control command based on the attribute classification result. The first target control command is used to control the working parameters of the tightening gun, and the working parameters include at least the tightening torque.

[0130] Embodiments of this application also provide an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention during runtime.

[0131] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of the present invention.

[0132] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of the present invention.

[0133] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of the present invention.

[0134] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0138] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0139] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for tightening a nut, characterized in that, include: Acquire a target image, which is obtained from the cockpit of the area to be fitted with the heat insulation pad; Based on the target image, stud positioning information is determined, and the stud positioning information is used to determine the relative position of the stud and the cockpit; Obtain nut classification information, wherein the nut classification information includes the mapping relationship between the relative position and the attribute category of the stud; Based on the nut classification information and the stud positioning information, multiple studs located at different positions are classified to obtain attribute classification results; Based on the attribute classification results, a first target control command is generated. The first target control command is used to control the working parameters of the tightening gun. The working parameters include at least the tightening torque. Determining stud positioning information based on the target image includes: Image processing algorithms are used to extract stud feature points from the target image to obtain first coordinate information, which is the pixel coordinate of the stud in the target image. The first coordinate information is transformed to obtain the second coordinate information, which is the spatial coordinate of the stud in the real space coordinate system; The second coordinate information is determined to be the stud positioning information; The first coordinate information is transformed to obtain the second coordinate information, including: The camera used to acquire the target image is calibrated with parameters including at least intrinsic and extrinsic parameters, and the calibration results are obtained. Based on the calibration results and the first coordinate information, the normalized camera coordinates are determined. The second coordinate information is obtained based on the normalized camera coordinates; Based on the nut classification information and the stud positioning information, multiple studs located at different positions are classified to obtain attribute classification results, including: Based on the nut classification information, a classification mapping relationship is determined between the spatial coordinate range in the real spatial coordinate system and the attribute category. The attribute category includes at least one of the following: stud at the overlap position of the front and rear heat insulation pads, stud with thick paint, and stud with poor contact of the heat insulation pad. Based on the second coordinate information and the classification mapping relationship, the multiple studs are classified to obtain the attribute classification results; Before generating the first target control instruction based on the attribute classification result, the method further includes: The pose of the stud is estimated based on the target image to obtain the pose estimation result; In response to the pose estimation result meeting the preset conditions, a second target control command is generated. The second target control command is used to control the tightening gun to stop performing the tightening action and generate an alarm message. The pose of the stud is estimated based on the target image to obtain the pose estimation result, including: The target image is converted into a source point cloud model; The source point cloud model is subjected to rigid body transformation using the ICP registration algorithm to obtain the transformation matrix; The target point cloud model is determined based on the transformation matrix and the source point cloud model. The target point cloud model is cropped to obtain the target stud point cloud; The pose estimation result is determined based on the target stud point cloud.

2. The method for tightening a nut according to claim 1, characterized in that, Determining the pose estimation result based on the target stud point cloud includes: Perform cylindrical fitting on the target stud point cloud to obtain a fitted cylinder; In response to the angle between the axes of the fitted cylinder and the reference cylinder being greater than a preset value, it is determined that the pose estimation result meets the preset condition.

3. The method for tightening a nut according to claim 1, characterized in that, Converting the target image into a source point cloud model includes: The target image is sampled using a voxel grid downsampling method to obtain the source point cloud model.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 3.

5. A tightening device, characterized in that, The tightening device is used to perform the method according to any one of claims 1 to 3.

Images