Bolt screwing visual detection method and system
By obtaining the relative coordinate set and attitude offset angle correction of the tightening tool on the battery pack, and combining it with the area matching method, the problem of missing tightening position detection is solved, and a stable tightening sequence verification and low-cost detection scheme are realized.
Patent Information
- Application Number
- CN202511779930.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-02-27
AI Technical Summary
Existing technologies cannot directly detect the tightening position, resulting in a lack of assembly sequence verification, which leads to batch problems such as poor battery sealing and short circuits.
The relative coordinate set of the tightening tool is obtained based on the battery pack reference point. The attitude offset angle is calculated for coordinate projection correction. The tightening sequence is verified by area matching and detection is performed using a low-precision industrial camera.
It achieves stable position detection and sequential output even when the tightening tool is partially obstructed, reducing hardware costs and improving the automation accuracy and reliability of the production line.
Smart Images
Figure CN121582222A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of automotive power batteries, specifically to a visual inspection method and system for bolt tightening. Background Technology
[0002] During the assembly of automotive power battery covers, it is crucial to ensure uniform stress distribution when tightening large flat components such as the battery pack cover. Therefore, a specific tightening sequence is required. By controlling the order in which the bolts are tightened, the stress distribution can be made as uniform as possible, which also helps to mitigate torque decay to some extent. Typically, factories use ergonomic lever arms with encoders to control the position of the tightening sequence. While this method is conventional, it is not optimal for positioning the bolts on the battery pack cover. This is because battery pack covers are usually quite large, making it difficult to cover and arrange using lever arms, hindering manual operation, and potentially creating blind spots.
[0003] In addition, manual tightening also presents the problem of unpredictable bolt tightening sequence, which can lead to poor battery sealing, intermittent contact of high-voltage copper contacts causing short circuits, and other batch-related issues, easily resulting in batch quality problems on the production line. Currently, the production line uses a manual tightening gun to tighten the screws and identifies whether they are tightened by the torque range. However, this method lacks the ability to directly detect the actual spatial position of the tightening gun, leading to incorrect assembly sequence. This is the primary cause of abnormal bolt rework (the number one bolt batch incident). Summary of the Invention
[0004] This application provides a visual inspection method and system for bolt tightening, which can solve the problem of missing assembly sequence verification caused by the inability to directly detect the tightening position in the prior art.
[0005] In a first aspect, embodiments of this application provide a visual inspection method for bolt tightening, comprising: Based on the reference point of the battery pack and the set of pixel coordinates of the tightening tool at the working bolt, obtain the set of relative coordinates of the tightening tool at the working bolt relative to the battery pack. Based on the relative coordinate set and the reference coordinate set, obtain the set of attitude offset angles of the tightening tool at the working bolt; Based on the set of attitude offset angles, the set of relative coordinates is corrected by coordinate projection to obtain the corrected set of projected coordinates; The projected coordinate set is matched with the reference coordinate set to verify whether the tightening operation of the working bolts conforms to the preset tightening sequence, and the verification results are recorded.
[0006] In conjunction with the first aspect, in one embodiment, before obtaining the set of relative coordinates of the tightening tool relative to the battery pack based on the reference point of the battery pack and the set of pixel coordinates of the tightening tool corresponding to the working bolt, the method further includes the step of obtaining the reference coordinate set: Tighten each bolt on the calibrated battery pack and obtain the calibration pixel coordinates of the tightening tool at each bolt; Based on the reference point coordinates of the battery pack and the coordinates of each calibrated pixel, the set of calibrated relative coordinates of the tightening tool at each bolt relative to the battery pack is obtained, and the set of calibrated relative coordinates is used as the reference coordinate set.
[0007] In conjunction with the first aspect, in one embodiment, when tightening each bolt on the calibration battery pack, the tightening tool is perpendicular to the edge of the calibration battery pack.
[0008] In conjunction with the first aspect, in one implementation, based on the relative coordinate set and the reference coordinate set, a set of attitude offset angles of the tightening tool at the working bolt is obtained, specifically including: Project the relative coordinate set onto the coordinate angles, and based on the reference coordinate set, obtain the set of attitude offset angles of the tightening tool at the working bolt.
[0009] In conjunction with the first aspect, in one implementation, the relative coordinate set is projected by coordinate angles, and based on the reference coordinate set, the set of attitude offset angles of the tightening tool at the working bolt is obtained, specifically including: Determine the relative position of the tightening tool to the battery pack; Project the relative coordinates of the tightening tool located on the first side of the battery pack into the Y-axis coordinate angle, and obtain the Y-axis attitude offset angle set based on the reference coordinate set. Project the relative coordinates of the tightening tool located on the second side of the battery pack into the X-axis coordinate angle, and obtain the X-axis attitude offset angle set based on the reference coordinate set. By integrating the Y-axis attitude offset angle set and the X-axis attitude offset angle set, the attitude offset angle set of the tightening tool at the working bolt is obtained.
[0010] In conjunction with the first aspect, in one implementation, based on the set of attitude offset angles, the set of relative coordinates is subjected to coordinate projection correction to obtain a corrected set of projected coordinates, specifically including: The relative coordinate set is corrected by Y-axis coordinate projection based on the Y-axis attitude offset angle set; The relative coordinate set is corrected by X-axis coordinate projection based on the X-axis attitude offset angle set; Integrate the Y-axis coordinate projection correction results and the X-axis coordinate projection correction results to obtain the corrected coordinate set.
[0011] In conjunction with the first aspect, in one implementation, the projected coordinate set is matched with the reference coordinate set to verify whether the tightening operation of the working bolt conforms to the preset tightening sequence, and the verification result is recorded. Specifically, this includes: Match the projected coordinate set with the reference coordinate set to determine the current tightening sequence of the bolts. Compare the current tightening sequence of the bolts with the corresponding preset tightening sequence; If the current tightening sequence is different from the preset tightening sequence, the verification result is recorded as an incorrect tightening sequence. If the current tightening sequence is the same as the preset tightening sequence, then the verification result is recorded as "tightening sequence is correct".
[0012] In conjunction with the first aspect, in one implementation, the projected coordinate set is matched with the reference coordinate set to determine the current tightening sequence of the working bolts, specifically including: Calculate the intersection of the projected area of the tightening tool in the projected coordinate set and the projected area of each reference coordinate point in the reference coordinate set to obtain the percentage of the maximum overlap area corresponding to the tightening tool; Calculate the pixel coordinate distance between the center point of the projected area of the tightening tool in the projected coordinate set and the center point of the corresponding reference coordinate point in the reference coordinate set, and obtain the minimum pixel coordinate center point distance corresponding to the tightening tool; Based on the percentage of the maximum overlap area and the minimum pixel coordinate center point distance corresponding to the tightening tool, the current tightening sequence of the working bolts is determined.
[0013] In conjunction with the first aspect, in one implementation, the current tightening sequence of the bolts is determined based on the percentage of the maximum overlap area corresponding to the tightening tool and the minimum pixel coordinate center point distance, specifically including: Based on the percentage of the maximum overlapping area and the minimum pixel coordinate center point distance corresponding to the tightening tool, the working bolt corresponding to the current tightening operation of the tightening tool is identified through matching degree calculation; Determine the current tightening sequence of the bolts based on the number of tightening passes performed by the tightening tool.
[0014] Secondly, embodiments of this application provide a bolt tightening visual inspection system, comprising: a first module, a second module, a third module, and a fourth module. The first module is used to obtain a set of relative coordinates of the tightening tool relative to the battery pack at the working bolt, based on a reference point of the battery pack and a set of pixel coordinates of the tightening tool at the working bolt. The second module is used to obtain a set of attitude offset angles of the tightening tool at the working bolt based on the set of relative coordinates and the reference coordinates. The third module is used to perform coordinate projection correction on the set of relative coordinates based on the set of attitude offset angles to obtain a corrected set of projected coordinates. The fourth module is used to perform area matching between the set of projected coordinates and the reference coordinates to verify whether the tightening operation of the working bolt conforms to a preset tightening sequence and to record the verification results.
[0015] The beneficial effects of the technical solutions provided in this application include: This application provides a visual inspection method and system for bolt tightening. By obtaining the relative coordinate set of the tightening tool at the working bolt based on the battery pack reference point, and calculating the posture offset angle for coordinate projection correction, the interference of the tightening tool posture change (such as tilting or rotation) on the coordinate data is effectively eliminated, so that the corrected projection coordinate set can more accurately represent the actual spatial position of the tightening tool. By performing area matching between the corrected projected coordinate set and the reference coordinate set, the sensitivity of traditional point matching methods to minor positional deviations is avoided. By statistically analyzing the area of the overlapping region, stable position detection and tightening sequence output can be achieved based on the remaining effective overlapping area even when the tightening tool is partially occluded. Furthermore, area matching has low dependence on image details, eliminating the need for a high-precision camera. A low-precision industrial camera is sufficient to meet inspection requirements, and this camera can be shared with vision inspection systems at other workstations, reducing hardware deployment costs and optimizing equipment resource utilization. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the visual inspection method for bolt tightening in this application; Figure 2 This is a schematic diagram illustrating the tightening operation of all bolts on the edge of the battery in a direction perpendicular to the battery edge, as per this application. Figure 3 This is a schematic diagram of the tightening operation when the screw tightening direction can be changed arbitrarily in this application; Figure 4 This is a schematic diagram of the projection coordinate set of this application. Detailed Implementation
[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0018] This application provides a visual inspection method and system for bolt tightening, which can solve the problem of missing assembly sequence verification caused by the inability to directly detect the tightening position in the prior art.
[0019] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0020] In a first aspect, embodiments of this application provide a visual inspection method for bolt tightening, comprising: 101: Based on the reference point of the battery pack and the set of pixel coordinates of the tightening tool at the working bolt, obtain the set of relative coordinates of the tightening tool at the working bolt relative to the battery pack; 102: Based on the relative coordinate set and the reference coordinate set, obtain the set of attitude offset angles of the tightening tool at the working bolt; 103: Based on the set of attitude offset angles, the relative coordinate set is corrected by coordinate projection to obtain the corrected projected coordinate set; 104: Match the area of the projected coordinate set with the reference coordinate set to verify whether the tightening operation of the working bolts conforms to the preset tightening sequence, and record the verification results.
[0021] This application obtains the relative coordinate set of the tightening tool at the working bolt based on the battery pack reference point, and calculates the attitude offset angle for coordinate projection correction. This effectively eliminates the interference of the tightening tool's attitude change (such as tilting or rotation) on the coordinate data, so that the corrected projected coordinate set can more accurately represent the actual spatial position of the tightening tool. By performing area matching between the corrected projected coordinate set and the reference coordinate set, the sensitivity of traditional point matching methods to minor positional deviations is avoided. By statistically analyzing the area of the overlapping region, stable position detection and tightening sequence output can be achieved based on the remaining effective overlapping area even when the tightening tool is partially occluded. Furthermore, area matching has low dependence on image details, eliminating the need for a high-precision camera. A low-precision industrial camera is sufficient to meet inspection requirements, and this camera can be shared with vision inspection systems at other workstations, reducing hardware deployment costs and optimizing equipment resource utilization.
[0022] Building upon the above embodiments, this embodiment further refines the process of constructing the reference coordinate set to ensure the accuracy and reliability of the tightening operation verification. Specifically, before obtaining the relative coordinate set of the tightening tool relative to the battery pack based on the reference point of the battery pack and the pixel coordinate set of the tightening tool corresponding to the working bolt, this embodiment further includes the step of obtaining the reference coordinate set: This step first involves performing a pre-set calibration tightening operation on each bolt on the calibrated battery pack. This involves tightening the bolts under unobstructed or standard operating conditions and acquiring the calibration pixel coordinates of the tightening tool at each bolt in real time. In practical applications, a representative standard battery pack sample is selected for calibration to avoid using battery packs with deformation or assembly errors, ensuring the reliability of the calibration data. Subsequently, based on the battery pack reference point coordinates (such as a fixed reference point on the edge of the battery pack; in this embodiment, the pixel coordinates O1(X,Y) at the intersection of the lower left corner long and wide edges of the battery pack are used as the origin, i.e., the reference point coordinates) and the coordinates of each calibration pixel, a coordinate system transformation is performed to obtain the set of calibration relative coordinates of the tightening tool at each bolt relative to the battery pack. This set of calibration relative coordinates is then used as the reference coordinate set for subsequent tightening sequence verification. Through this calibration process, the system pre-establishes the correspondence between bolt positions and tightening tool pixel coordinates, providing a benchmark reference for subsequent steps to calculate the matching degree of area overlap rate and center point distance. This effectively avoids recognition deviations caused by tightening gun obstruction or environmental interference, thereby achieving accurate identification of the current bolt and strict verification of the preset sequence in subsequent tightening operations. In this application, the intersection of the extension lines of the long and wide sides of the lower left corner of the battery pack is taken as the reference point O. The wide side is set as the y-axis, and the long side is set as the x-axis. Based on this reference coordinate system XOY, the approach of relative coordinate systems avoids the pain point of lacking a fixed coordinate system for each battery pack of the same category. The entire detection method is divided into two steps. The first step is the calibration step, in which the position of the tightening tool relative to the length and width of the calibrated battery pack is corrected when the tightening tool is at the working bolt, so that the tightening tool is perpendicular to the length and width of the calibrated battery pack as a standard tightening posture, and the calibration pixel coordinates of the tightening tool and the reference point of each battery pack are obtained when the tightening tool is at the working bolt. By subtracting these two coordinates, we can obtain the set of relative coordinates for all bolt positions requiring tightening, based on the reference coordinate system. This completes the battery pack calibration; each battery pack requires only one calibration during operation. The second part describes the operational steps, which will be detailed below.
[0023] It should be noted that when the calibration battery pack enters the tightening station from the previous process, the MES system automatically identifies the battery pack's positioning and triggers the camera to perform exposure imaging. When tightening each bolt on the calibration battery pack, the tightening tool is perpendicular to the edge of the battery pack, meaning it tightens all bolts at a 90-degree angle to the battery edge: when the bolt hole is on the long side, tighten perpendicularly to the long side; when it is on the wide side, tighten perpendicularly to the wide side. This ensures the tightening action is consistent with the edge normal direction, effectively eliminating coordinate measurement errors caused by tightening angle deviations and improving the stability of subsequent data acquisition. The tightening tool in this application is a tightening gun.
[0024] Subsequently, the camera triggers exposure sequentially according to the preset screw tightening order (e.g., from the lower left corner to the upper right corner), and completes the following steps through a real-time image processing algorithm: identifying the calibrated pixel coordinates N of the tightening gun. i (X,Y) (i.e., the pixel position of the center point of the tightening gun in the image), and simultaneously locate the pixel coordinates O1(X,Y) of the intersection of the lower left corner long side and wide side edge of the battery pack. This point serves as the origin of the system's global coordinate system, i.e., the reference point. Based on this, the system calculates the calibration relative coordinate system set P between the tightening gun and the battery edge. 1，2，3，……，i (X,Y)=(N1(X,Y)-O1(X,Y),...,N i (X,Y)-O1(X,Y)), where i is the total number of bolts that need to be tightened in the calibrated battery pack. This calibration relative coordinate set P is defined as the reference coordinate set, which is used for calibration and verification of subsequent tightening operations. When a new battery pack enters the workstation, the system will match the real-time acquired projection coordinates of the tightening gun with the reference coordinate set P (through area overlap rate calculation and center point distance optimization) to accurately identify the position of the currently tightened bolts and strictly verify whether they conform to the preset tightening sequence (such as "bolt 1 first, then bolt 2"), effectively solving the problem of sequence misjudgment caused by tightening gun obstruction or environmental interference, and significantly improving the automation accuracy and reliability of the production line.
[0025] Building upon the above embodiments, during subsequent battery pack operations, due to worker deviations in posture relative to calibration during tightening operations to ensure efficiency, this example further introduces a dynamic calculation and coordinate correction mechanism for the posture deviation angle. The screw tightening direction can be arbitrarily changed during operation to address coordinate recognition deviations caused by incorrect tightening gun posture (such as tilting or rotation), significantly improving the system's robustness under complex working conditions. Based on the relative coordinate set and the reference coordinate set (bolt position reference data generated during calibration), the posture deviation angle set of the tightening tool at the working bolt is obtained. Specifically, this includes projecting the relative coordinate set onto coordinate angles and obtaining the posture deviation angle set of the tightening tool at the working bolt based on the reference coordinate set.
[0026] Specifically: First, determine the relative position of the tightening tool relative to the battery pack; then, project the relative coordinates of the tightening tool located on the first side (long side, defined as the first side in this embodiment, Y-axis direction) of the battery pack onto the Y-axis coordinate angle, and obtain the Y-axis attitude offset angle set based on the reference coordinate set, that is, by calculating the angle difference between the relative coordinate point and the reference coordinate point in the Y-axis direction; next, project the relative coordinates of the tightening tool located on the second side (wide side, defined as the first side in this embodiment, X-axis direction) of the battery pack onto the X-axis coordinate angle, and obtain the X-axis attitude offset angle set based on the reference coordinate set, that is, by calculating the angle difference between the relative coordinate point and the reference coordinate point in the X-axis direction; finally, integrate the Y-axis attitude offset angle set and the X-axis attitude offset angle set to obtain the complete attitude offset angle set of the tightening tool at the working bolt. This set quantifies the degree of offset of the tightening gun relative to the reference attitude of the battery pack (such as Y-axis tilt angle and X-axis rotation angle).
[0027] Specifically, based on the set of attitude offset angles, the relative coordinate set is subjected to coordinate projection correction to obtain the corrected coordinate set, which includes: First, the relative coordinate set is corrected by Y-axis coordinate projection based on the Y-axis attitude offset angle set, that is, the relative coordinate set is corrected by Y-axis coordinate projection using the Y-axis attitude offset angle set to eliminate vertical offset error. Then, the relative coordinate set is corrected by X-axis coordinate projection based on the X-axis attitude offset angle set, that is, the relative coordinate set is corrected by X-axis coordinate projection using the X-axis attitude offset angle set to eliminate horizontal offset error. Finally, the Y-axis coordinate projection correction results and the X-axis coordinate projection correction results are integrated to obtain the corrected coordinate set.
[0028] In a specific embodiment, after the subsequent battery pack smoothly enters the tightening station from the previous process, the MES system automatically triggers the camera to perform sequential exposure of the battery pack. The system exposes the tightening gun one by one according to the preset bolt order (i=1,2,3,...,n, where n is the total number of bolts), and identifies the pixel coordinate set M of the tightening gun in real time. 1，2，3，……，i The system uses (X,Y) as the origin, and simultaneously locates the pixel coordinates O2(X,Y) at the intersection of the lower left corner of the battery pack's long and short edges as the global origin. Based on this, the system calculates the set of relative coordinate systems Q between the tightening gun and the battery edge. 1，2，3，……，i (X,Y)=(M1(X,Y)-O2(X,Y),...,M i (X,Y)-O2(X,Y)), where i is the total number of bolts. This relative coordinate set Q... 1，2，3，……，i The (X,Y) coordinate system, with the lower left corner of the battery pack as the origin, constructs real-time operational data in a vertical coordinate system, providing basic input for subsequent attitude correction and sequence verification.
[0029] After obtaining the set of relative coordinates Q between the tightening gun and the battery edge during subsequent battery pack operations, refer to... Figure 3 As shown, the system performs coordinate angle projection analysis on the long side (first side, Y-axis direction) and wide side (second side, X-axis direction) of the battery pack. For the long side bolt (i is the index of the long side bolt), the relative coordinate set Q is... i The formula for calculating the offset angle along the Y-axis from (X,Y) is as follows: α i =arctan[(Q iY -P iY ) / (Q iX -P iX )]; Where P i Let α be the coordinate of the corresponding bolt in the reference coordinate set. i This indicates the offset angle of the tightening gun relative to the reference Y-axis.
[0030] Similarly, for wide-side bolts (where k is the index of the wide-side bolt), the relative coordinate set Q is... K The offset angle along the X-axis is calculated using the formula: β K =arctan[(Q KY -P KY ) / (Q KX -P KX )]; Where β K This represents the offset angle of the tightening gun relative to the reference X-axis. Through this calculation, the system dynamically obtains the set of Y-axis attitude offset angles {α...} i} and the set of X-axis attitude offset angles {β K The degree of tilt of the tightening gun during actual operation was quantified (e.g., the Y-axis offset angle α of the long-side bolt). i Reflecting the vertical tilt, the X-axis offset angle β of the wide-side bolt. K (Reflects horizontal rotation).
[0031] Then, based on the acquired set of attitude offset angles, the system performs multi-dimensional coordinate correction to eliminate attitude errors. For the long-side bolt, the Y-axis attitude offset angle α is applied. i The formula for coordinate projection correction is as follows: ; This formula corrects the coordinate system to the reference Y-axis direction using a rotation matrix, eliminating coordinate deviations caused by vertical tilt.
[0032] For wide-side bolts, apply the X-axis attitude offset angle β. K The formula for coordinate projection correction is as follows: ; That is, based on the set of pixel coordinates of attitude offset, after obtaining the angle α between the pixel coordinates and the length and width of the battery, since the tightening tool is perpendicular to the length and width when tightening during the calibration step, the offset angle β = 90° - α between the pixel coordinates of the tightening tool during calibration and the set of pixel coordinates of attitude offset can be obtained. The set of pixel coordinates of attitude offset is then corrected by coordinate projection based on the relative coordinate set of the tightening tool under the reference coordinate system in the calibration step using the offset angle β, and the corrected set of projected coordinates of the tightening tool is obtained.
[0033] After correction, the system integrates the Y-axis and X-axis correction results to generate a set of projected coordinates. }and{ This correction mechanism effectively solves the coordinate distortion problem caused by changes in the operating angle of the tightening gun (such as worker tilting or equipment vibration), thus improving the relative coordinate accuracy. Finally, the corrected coordinate set is incorporated into the matching degree calculation in step 104, ensuring that even under complex working conditions where the screw tightening direction changes arbitrarily (such as alternating tightening of the long / wide side), the system can still accurately identify the currently tightened bolt and strictly verify whether it conforms to the preset sequence (such as "bolt 1 → bolt 2"), thereby significantly reducing the sequence misjudgment rate caused by posture errors and providing a guarantee for the automation and reliability of the battery pack tightening process.
[0034] Based on the above embodiments, this embodiment further improves the dynamic verification mechanism for the tightening sequence. Sequence verification is achieved through area matching between the projected coordinate set and the reference coordinate set: the projected coordinate set is matched with the reference coordinate set in the calibration step to verify whether the tightening operation of the working bolt conforms to the preset tightening sequence, and the verification result is recorded. Specifically, after obtaining the corrected coordinate set, the system will use the real-time projected coordinate set { }and{ } and the reference coordinate set P 1，2，3，……，i A calibration is performed at (X,Y) for area matching. This involves intersecting the pixel areas formed by the two coordinate sets to confirm whether the current tightening sequence of the bolts conforms to the preset sequence. The verification process is as follows: First, the intersection of the real-time projected area of the tightening tool in the projected coordinate set and the projected area of each reference coordinate point in the reference coordinate set is calculated to obtain the percentage of the maximum overlap area corresponding to the tightening tool. This percentage of the maximum overlap area is then used as the matching index (P1 = (A...). 投影面积 / P 投影面积i (max), this value quantifies the geometric fit between the current tightening operation and the reference position, where i is the number of screws that need to be tightened, and the pixel coordinate center point d1 is identified based on this. i d2 iThen, for the center point (d1) of the projected area of the tightening tool in the projected coordinate set... i ) and the center point (d2) of the corresponding reference coordinate point in the reference coordinate set ij The pixel coordinate distance is calculated to obtain the minimum pixel coordinate center point distance (d1) corresponding to the tightening tool. i -d2 ij min is used as a constraint condition, where d1 i The center point of the projected area of the tightening gun onto the long / wide side after each screw is tightened, d2 ij In the step of obtaining the reference coordinate set, the center point of the relative coordinate system reference set of all tightening guns and the edge of the battery after each screw is tightened; Finally, based on the maximum overlap area percentage P1 corresponding to the tightening tool and the minimum pixel coordinate center point distance, the current tightening sequence of the bolts is determined. This process is divided into: first, based on the maximum overlap area percentage and the minimum pixel coordinate center point distance corresponding to the tightening tool, the bolts corresponding to the current tightening operation of the tightening tool are identified through matching degree calculation; then, based on the number of tightening operations of the tightening tool, the current tightening sequence of the bolts is determined. A comprehensive matching degree (P2=P1∪(d1)) is constructed based on P1 and the minimum distance. i -d2 ij The system identifies the bolt corresponding to the current tightening operation by using a matching threshold (e.g., P2>95% and distance<0.5px). Based on this, the system automatically associates the real-time tightening count of the tightening tool (e.g., the second operation) to confirm the current tightening sequence of the bolt.
[0035] Then compare the current tightening sequence of the bolts with the corresponding preset tightening sequence (the preset tightening sequence is defined as "bolt 1 → bolt 2 → bolt 3"). If the current tightening sequence is different from the preset tightening sequence (e.g., the second operation identifies bolt 3), then record the verification result as an incorrect tightening sequence; if the current tightening sequence is the same as the preset tightening sequence (e.g., the second operation corresponds to bolt 2), then record the verification result as a correct tightening sequence.
[0036] This mechanism effectively avoids misjudgment problems caused by tightening gun obstruction, ambient light interference, or operational posture deviation by matching the area overlap rate and the center point distance in two dimensions. It improves the accuracy of sequential verification, ensures that the battery pack tightening process meets the process specifications, and reduces the sequential error rate, significantly improving the automation reliability and quality traceability of the production line.
[0037] In summary, this application can perform position detection and output even when the tightening gun is covered by more than 50% of the area, dynamically compensate for feature loss caused by partial occlusion, ensure detection continuity, significantly improve the robustness of the system under dynamic conditions, and avoid downtime due to misjudgment caused by temporary occlusion; in terms of hardware, only a low-precision industrial camera is needed for detection, and it can be shared with cameras at other workstations; image tracing is performed through tightening signals, and multiple photos in the scene are processed optimally to greatly reduce the impact of occlusion: based on the area overlap rate (P1) and the center point distance ((d1) i -d2 ij The system calculates the matching degree of each image and selects the image with the highest matching degree as the final recognition result. Even when the tightening gun is 50% obscured by the hand, it can still complete the correct recognition through backup images (such as frames with less obscuration), which greatly improves the robustness under dynamic working conditions. It establishes a coordinate cluster of a large reference coordinate system and multiple small reference coordinate systems for bolt positions to avoid positional errors in material loading and AGV trolleys. It supports real-time recognition of multiple battery models and only requires quick calibration during the first operation without changing the software reference.
[0038] Secondly, embodiments of this application provide a bolt tightening visual inspection system, comprising: a first module, a second module, a third module, and a fourth module. The first module is used to obtain a set of relative coordinates of the tightening tool relative to the battery pack at the working bolt, based on a reference point of the battery pack and a set of pixel coordinates of the tightening tool at the working bolt. The second module is used to obtain a set of attitude offset angles of the tightening tool at the working bolt based on the set of relative coordinates and the reference coordinates. The third module is used to perform coordinate projection correction on the set of relative coordinates based on the set of attitude offset angles to obtain a corrected set of projected coordinates. The fourth module is used to perform area matching between the set of projected coordinates and the reference coordinates to verify whether the tightening operation of the working bolt conforms to a preset tightening sequence and to record the verification results.
[0039] This application obtains the relative coordinate set of the tightening tool at the working bolt based on the battery pack reference point, and calculates the attitude offset angle for coordinate projection correction. This effectively eliminates the interference of the tightening tool's attitude change (such as tilting or rotation) on the coordinate data, so that the corrected projected coordinate set can more accurately represent the actual spatial position of the tightening tool. By performing area matching between the corrected projected coordinate set and the reference coordinate set, the sensitivity of traditional point matching methods to minor positional deviations is avoided. By statistically analyzing the area of the overlapping region, stable position detection and tightening sequence output can be achieved based on the remaining effective overlapping area even when the tightening tool is partially occluded. Furthermore, area matching has low dependence on image details, eliminating the need for a high-precision camera. A low-precision industrial camera is sufficient to meet inspection requirements, and this camera can be shared with vision inspection systems at other workstations, reducing hardware deployment costs and optimizing equipment resource utilization.
[0040] The functions of each module in the above-mentioned bolt tightening visual inspection system correspond to the steps in the above-mentioned bolt tightening visual inspection method embodiment, and their functions and implementation processes will not be described in detail here.
[0041] Thirdly, embodiments of this application provide a bolt tightening visual inspection device, which can be a personal computer (PC), laptop computer, server or other device with data processing capabilities.
[0042] In this embodiment, the bolt tightening visual inspection device may include a processor, a memory, a communication interface, and a communication bus.
[0043] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.
[0044] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the bolt tightening visual inspection equipment, as well as interfaces used for interconnecting the bolt tightening visual inspection equipment with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.
[0045] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.
[0046] The processor can be a general-purpose processor, which can call the bolt tightening visual inspection program stored in the memory and execute the bolt tightening visual inspection method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the bolt tightening visual inspection program is called can be referred to in the various embodiments of the bolt tightening visual inspection method of this application, and will not be described again here.
[0047] Fourthly, embodiments of this application also provide a computer-readable storage medium.
[0048] The present application has a computer-readable storage medium storing a bolt tightening visual inspection program, wherein when the bolt tightening visual inspection program is executed by a processor, it implements the steps of the bolt tightening visual inspection method as described above.
[0049] The method implemented when the bolt tightening visual inspection program is executed can be referred to in various embodiments of the bolt tightening visual inspection method of this application, and will not be repeated here.
[0050] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0051] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.
[0052] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.
[0053] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.
[0054] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.
[0055] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.
[0056] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A visual inspection method for bolt tightening, characterized in that, It includes: Based on the reference point of the battery pack and the set of pixel coordinates of the tightening tool at the working bolt, obtain the set of relative coordinates of the tightening tool at the working bolt relative to the battery pack. Based on the relative coordinate set and the reference coordinate set, obtain the set of attitude offset angles of the tightening tool at the working bolt; Based on the set of attitude offset angles, the set of relative coordinates is corrected by coordinate projection to obtain the corrected set of projected coordinates; The projected coordinate set is matched with the reference coordinate set to verify whether the tightening operation of the working bolts conforms to the preset tightening sequence, and the verification results are recorded.
2. The visual inspection method for bolt tightening as described in claim 1, characterized in that, Before obtaining the relative coordinate set of the tightening tool relative to the battery pack based on the reference point of the battery pack and the pixel coordinate set of the tightening tool corresponding to the working bolt, the method further includes the step of obtaining the reference coordinate set: Tighten each bolt on the calibrated battery pack and obtain the calibration pixel coordinates of the tightening tool at each bolt; Based on the reference point coordinates of the battery pack and the coordinates of each calibrated pixel, the set of calibrated relative coordinates of the tightening tool at each bolt relative to the battery pack is obtained, and the set of calibrated relative coordinates is used as the reference coordinate set.
3. The visual inspection method for bolt tightening as described in claim 2, characterized in that: When tightening each bolt on the calibration battery pack, the tightening tool is perpendicular to the edge of the calibration battery pack.
4. The visual inspection method for bolt tightening as described in claim 1, characterized in that, Based on the relative coordinate set and the reference coordinate set, the set of attitude offset angles of the tightening tool at the working bolt is obtained, specifically including: Project the relative coordinate set onto the coordinate angles, and based on the reference coordinate set, obtain the set of attitude offset angles of the tightening tool at the working bolt.
5. The visual inspection method for bolt tightening as described in claim 4, characterized in that, Projecting the relative coordinate set onto coordinate angles, and based on the reference coordinate set, obtaining the set of attitude offset angles of the tightening tool at the working bolt, specifically including: Determine the relative position of the tightening tool to the battery pack; Project the relative coordinates of the tightening tool located on the first side of the battery pack into the Y-axis coordinate angle, and obtain the Y-axis attitude offset angle set based on the reference coordinate set. Project the relative coordinates of the tightening tool located on the second side of the battery pack into the X-axis coordinate angle, and obtain the X-axis attitude offset angle set based on the reference coordinate set. By integrating the Y-axis attitude offset angle set and the X-axis attitude offset angle set, the attitude offset angle set of the tightening tool at the working bolt is obtained.
6. The visual inspection method for bolt tightening as described in claim 5, characterized in that, Based on the set of attitude offset angles, the relative coordinate set is subjected to coordinate projection correction to obtain the corrected projected coordinate set, specifically including: The relative coordinate set is corrected by Y-axis coordinate projection based on the Y-axis attitude offset angle set; The relative coordinate set is corrected by X-axis coordinate projection based on the X-axis attitude offset angle set; Integrate the Y-axis coordinate projection correction results and the X-axis coordinate projection correction results to obtain the corrected coordinate set.
7. The visual inspection method for bolt tightening as described in claim 1, characterized in that, The projected coordinate set is matched with the reference coordinate set to verify whether the tightening operation of the working bolts conforms to the preset tightening sequence, and the verification results are recorded, specifically including: Match the projected coordinate set with the reference coordinate set to determine the current tightening sequence of the bolts. Compare the current tightening sequence of the bolts with the corresponding preset tightening sequence; If the current tightening sequence is different from the preset tightening sequence, the verification result is recorded as an incorrect tightening sequence. If the current tightening sequence is the same as the preset tightening sequence, then the verification result is recorded as "tightening sequence is correct".
8. The visual inspection method for bolt tightening as described in claim 7, characterized in that, The projected coordinate set is matched with the reference coordinate set to determine the current tightening sequence of the bolts, specifically including: Calculate the intersection of the projected area of the tightening tool in the projected coordinate set and the projected area of each reference coordinate point in the reference coordinate set to obtain the percentage of the maximum overlap area corresponding to the tightening tool; Calculate the pixel coordinate distance between the center point of the projected area of the tightening tool in the projected coordinate set and the center point of the corresponding reference coordinate point in the reference coordinate set, and obtain the minimum pixel coordinate center point distance corresponding to the tightening tool; Based on the percentage of the maximum overlap area and the minimum pixel coordinate center point distance corresponding to the tightening tool, the current tightening sequence of the working bolts is determined.
9. The visual inspection method for bolt tightening as described in claim 8, characterized in that, Based on the percentage of the maximum overlap area corresponding to the tightening tool and the minimum pixel coordinate center point distance, the current tightening sequence of the bolts is determined, specifically including: Based on the percentage of the maximum overlapping area and the minimum pixel coordinate center point distance corresponding to the tightening tool, the working bolt corresponding to the current tightening operation of the tightening tool is identified through matching degree calculation; Determine the current tightening sequence of the bolts based on the number of tightening passes performed by the tightening tool.
10. A visual inspection system for bolt tightening, characterized in that, It includes: The first module is used to obtain the set of relative coordinates of the tightening tool relative to the battery pack at the working bolt, based on the reference point of the battery pack and the set of pixel coordinates of the tightening tool at the working bolt. The second module is used to obtain the set of attitude offset angles of the tightening tool at the working bolt based on the relative coordinate set and the reference coordinate set; The third module is used to perform coordinate projection correction on the relative coordinate set based on the attitude offset angle set, and obtain the corrected projected coordinate set. The fourth module is used to match the area of the projected coordinate set with the reference coordinate set to verify whether the tightening operation of the working bolts conforms to the preset tightening sequence, and to record the verification results.