A general rivet operation method and system based on visual positioning

Abstract

This application relates to the field of automated riveting operations and discloses a universal riveting operation method and system based on vision positioning. It includes: acquiring a riveting recipe package corresponding to the target product, the riveting recipe package including visual positioning parameters, image capture parameters, height parameters, rivet hole sequence, coordinate conversion parameters, and rivet distribution parameters; acquiring an image of the target product through a CCD vision acquisition mechanism to determine the actual reference position and posture of the target product; identifying the rivet hole positions based on the image capture template, image capture position, and height parameters, and converting them into mechanical execution coordinates; generating a riveting execution queue based on the rivet distribution parameters and rivet hole sequence; and controlling a motion execution mechanism to drive the riveting execution mechanism to the corresponding rivet hole position and complete the riveting operation. This application can realize recipe calling, visual positioning compensation, and riveting queue scheduling for multiple product models, and is suitable for the universal operation of automated riveting equipment.

Description

Technical Field

[0001] This invention relates to the field of automated riveting operations, and specifically to a universal riveting operation method and system based on vision positioning. Background Technology

[0002] In product assembly, riveting is typically used to securely connect two or more workpieces. For products with a large number of rivet holes and a dense distribution of holes, manual positioning and riveting of each hole is not only time-consuming, but also requires operators to repeatedly pick up, place, align, and press the rivets. This is easily affected by the operator's skill level, observation angle, and operational stability, leading to insufficient consistency in the riveting positions. As automated production lines demand higher cycle times and consistency, replacing manual riveting with automated riveting equipment has become a common technological trend.

[0003] Existing automatic riveting equipment typically performs riveting operations based on pre-set tooling positioning references and motion programs. Some equipment also incorporates CCD vision acquisition mechanisms to identify product hole positions or reference features to correct product placement deviations. However, such equipment often establishes operating programs for fixed product models. When product specifications, number of holes, hole distribution, or riveting height change, it is often necessary to re-teach, reset the photographing position, or readjust the riveting path. For stand-alone automatic riveting equipment that needs to be compatible with multiple products, the lack of a unified product import and formula call mechanism means that product changeover still relies on manual machine adjustments, affecting the equipment's continuous production capacity.

[0004] Furthermore, the rivet distribution varies significantly across different products. Some products have a large number of rivet holes on the front, some on the sides, and some with fewer on both sides. Existing equipment typically operates according to fixed program segments or a fixed hole position sequence during riveting operations, making it difficult to automatically generate corresponding riveting execution queues based on the actual rivet distribution on the front and sides of the product. When a side has few or no rivet holes, the equipment may still execute invalid program segments or redundant positioning actions. Inconsistencies in data between hole position recognition, coordinate conversion, and riveting execution can also easily arise when there are product posture deviations, camera installation deviations, or differences in height parameters. Therefore, there is an urgent need for a universal riveting operation solution that can uniformly manage visual positioning, camera position parameters, height parameters, coordinate conversion, rivet hole sequence, and rivet distribution scheduling to meet the automated riveting needs of multiple product models. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a universal riveting operation method and system based on vision positioning, thereby solving the technical problems existing in the prior art.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: A general riveting operation method based on vision positioning includes the following steps: S1: Obtain the riveting recipe package corresponding to the target product. The riveting recipe package includes visual positioning parameters, photo position parameters, height parameters, rivet hole sequence, coordinate conversion parameters, and rivet distribution parameters. The photo position parameters include the photo position template and photo position. S2: Acquire images of the target product through a CCD vision acquisition mechanism, and determine the actual reference position and posture of the target product based on visual positioning parameters; S3: Identify the rivet hole positions on the target product based on the photo template, photo position, and height parameters, and convert the identified rivet hole positions into mechanical execution coordinates based on coordinate conversion parameters; S4: Generate a riveting execution queue based on rivet distribution parameters and rivet hole sequence; S5: Control the motion actuator to reach the corresponding rivet hole position according to the riveting execution queue, mechanical execution coordinates and height parameters, and the riveting actuator completes the riveting operation.

[0007] Preferably, when creating the riveting formula package, the product to be imported is placed in the positioning and clamping mechanism of the automatic riveting equipment, and the positioning and clamping mechanism clamps and corrects the product to be imported. The CCD vision acquisition mechanism is controlled to acquire the reference feature image of the product to be imported, a material finding template is generated based on the reference feature image, and a local coordinate system of the product is established based on the reference feature position identified in the material finding template. The CCD vision acquisition mechanism is controlled to acquire images of the riveting area of ​​the product to be imported, and a photo position template is generated based on the riveting area image. Associate the material locating template, photo position template, height parameter, coordinate conversion parameter, rivet hole sequence, and rivet distribution parameter with the same product number to form a riveting formula package corresponding to that product number and store it.

[0008] Preferably, the riveting area includes a front riveting area and a side riveting area; When generating the photo position template, the CCD vision acquisition mechanism is controlled to acquire the front riveting area image and the side riveting area image respectively, and generate the front photo position template and the side photo position template. The rivet hole sequence includes a front rivet hole sequence and a side rivet hole sequence. The front photo position template is stored in correspondence with the front rivet hole sequence, and the side photo position template is stored in correspondence with the side rivet hole sequence. Both the front rivet hole sequence and the side rivet hole sequence include hole position number, face identification, theoretical rivet hole coordinates, and riveting operation height.

[0009] Preferably, the height parameters include the height for clear photography and the height for riveting operations; The image clarity height is the Z-axis position when the CCD vision acquisition mechanism acquires the corresponding image template image, and is stored according to the image template number. The riveting operation height is the Z-axis position of the riveting actuator when performing riveting operation on the corresponding rivet hole, and is stored according to the hole position number of the rivet hole; When the motion actuator performs the riveting operation, the motion actuator drives the riveting actuator to the corresponding rivet hole position according to the mechanical execution coordinates and the corresponding riveting operation height.

[0010] Preferably, the visual positioning parameters include a material-finding template and a reference feature type; The reference feature type is a product mark point, a reference hole, an outer contour edge, or a corner contour; When the reference feature type is a product Mark point, the center of the product Mark point is used as the reference origin of the product's local coordinate system; When the reference feature type is a reference hole, the center of the reference hole is used as the reference origin of the local coordinate system of the product. When the reference feature type is an outer contour edge or corner contour, extract the contour point set of the outer contour edge or corner contour, determine the center of the circumscribed rectangle based on the contour point set, and use the center of the circumscribed rectangle as the reference origin of the product's local coordinate system.

[0011] Preferably, the coordinate conversion parameters include pixel ratio coefficient, X-axis installation offset, Y-axis installation offset, product attitude angle, and coordinate rotation parameters; When converting the rivet hole position into mechanical execution coordinates, the visual pixel coordinates of the rivet hole are first converted into local coordinates of the product based on the pixel ratio coefficient. Then, the local coordinates of the product are converted into mechanical execution coordinates based on the X-axis installation offset, Y-axis installation offset, product attitude angle, and coordinate rotation parameters.

[0012] Preferably, the rivet distribution parameters include the number of front rivet holes, the number of side rivet holes, the front program activation threshold, and the side program activation threshold. When generating the riveting execution queue, if the number of front rivet holes is greater than or equal to the front program activation threshold and the number of side rivet holes is less than the side program activation threshold, only the front rivet hole sequence is written into the riveting execution queue. When the number of side rivet holes is greater than or equal to the threshold for the number of side program activations, and the number of front rivet holes is less than the threshold for the number of front program activations, only the side rivet hole sequence is written into the riveting execution queue. When the number of front rivet holes is less than the front program activation threshold and the number of side rivet holes is less than the side program activation threshold, one rivet hole is taken from the front rivet hole sequence and one from the side rivet hole sequence according to the hole position number order and written alternately into the riveting execution queue. When the rivet holes in one sequence are written, the remaining rivet holes in the other sequence are written into the riveting execution queue according to the hole position number order. When the number of front rivet holes is greater than or equal to the front program activation threshold, and the number of side rivet holes is greater than or equal to the side program activation threshold, the front rivet hole sequence is first written into the riveting execution queue in the order of hole position number, and then the side rivet hole sequence is written into the riveting execution queue in the order of hole position number.

[0013] Preferably, when only the front rivet hole sequence is written to the riveting execution queue, the controller enables the front riveting program segment and skips the side riveting program segment; When only the side rivet hole sequence is written to the riveting execution queue, the controller enables the side riveting program segment and skips the front riveting program segment. When the front rivet hole sequence and the side rivet hole sequence are alternately written into the riveting execution queue, the controller calls the front riveting program segment and the side riveting program segment in sequence according to the riveting execution queue. When the number of front rivet holes and the number of side rivet holes are both greater than or equal to the corresponding threshold, the controller first enables the front riveting program segment to execute the front rivet hole sequence, and then enables the side riveting program segment to execute the side rivet hole sequence. The riveting recipe package also includes riveting mode parameters, which include origin riveting mode parameters, multi-point feature riveting mode parameters, and contour global riveting mode parameters. When the reference feature type recorded in the riveting recipe package is a product Mark point or a reference hole, the origin riveting mode parameters are called. When the number of reference features recorded in the riveting recipe package is more than two, the multi-point feature riveting mode parameters are called, and the product attitude angle is determined based on the two or more reference features. When the reference feature type recorded in the riveting recipe package is an outer contour edge or corner contour, the contour global riveting mode parameters are called, and the reference origin of the product local coordinate system is determined according to the outer contour edge or corner contour.

[0014] Preferably, the riveting recipe package further includes an identification confidence threshold, a coordinate deviation threshold, and a positioning deviation threshold; Before performing the riveting operation, the rivet hole position identified by the CCD vision acquisition mechanism is compared with the theoretical rivet hole coordinates of the corresponding hole position number. When the coordinate deviation between the two is greater than the coordinate deviation threshold, the image corresponding to the photo template of the rivet hole is re-acquired. When the recognition confidence level after re-collection is less than the recognition confidence level threshold, stop executing the riveting action corresponding to the rivet hole and output the hole position number of the rivet hole; After the motion actuator drives the riveting actuator to the corresponding rivet hole position, the actual positioning coordinates fed back by the motion actuator are compared with the mechanical execution coordinates corresponding to the rivet hole. When the positioning deviation between the two is greater than the positioning deviation threshold, the riveting action corresponding to the rivet hole is stopped and the hole position number of the rivet hole is output.

[0015] A universal riveting operation system based on vision positioning includes a positioning and clamping mechanism, a CCD vision acquisition mechanism, a motion execution mechanism, a riveting execution mechanism, a controller, and a human-machine interaction module; The controller is communicatively connected to the CCD vision acquisition mechanism, motion execution mechanism, riveting execution mechanism and human-computer interaction module respectively; The positioning and clamping mechanism is used to carry and clamp the product, so that the product is within the field of view of the CCD vision acquisition mechanism; The CCD vision acquisition mechanism is used to acquire product images and output the image acquisition results to the controller. The human-computer interaction module is used to receive the product number of the target product and send the product number to the controller; The controller is used to obtain the corresponding riveting recipe package according to the product number, determine the actual reference position, attitude and rivet hole position of the target product according to the image acquisition results of the CCD vision acquisition mechanism, convert the rivet hole position into mechanical execution coordinates according to the coordinate conversion parameters in the riveting recipe package, and generate a riveting execution queue according to the rivet distribution parameters and rivet hole sequence in the riveting recipe package. The controller includes a template generation module, a coordinate conversion module, a recipe management module, a queue generation module, and a mode scheduling module; The template generation module is used to generate a material finding template based on the product's reference feature image and to generate a photo taking position template based on the product's riveting area image. The coordinate conversion module is used to convert the visual pixel coordinates identified by the CCD vision acquisition mechanism into mechanical execution coordinates according to the coordinate conversion parameters. The recipe management module is used to associate and store the material finding template, photo position template, clear photo height, riveting operation height, coordinate conversion parameters, rivet hole sequence, riveting mode parameters and rivet distribution parameters as a riveting recipe package corresponding to the product number, and call the corresponding riveting recipe package according to the product number selected by the human-computer interaction module. The queue generation module is used to generate a riveting execution queue based on rivet distribution parameters and rivet hole sequence. The mode scheduling module is used to enable the front riveting program segment and the side riveting program segment according to the riveting execution queue, or to call the front riveting program segment and the side riveting program segment in sequence. The CCD vision acquisition mechanism includes a front vision acquisition unit and a side vision acquisition unit. The front vision acquisition unit is positioned facing the front of the product, and the side vision acquisition unit is positioned facing the side of the product. The motion actuator is used to drive the riveting actuator to the corresponding rivet hole position according to the motion control command output by the controller, and to feed back the actual positioning coordinates to the controller. The controller is also used to output a continue riveting command or a stop riveting command to the riveting actuator based on the deviation between the actual positioning coordinates and the mechanical execution coordinates. The riveting actuator is used to perform riveting operations at the corresponding rivet hole position according to the riveting control command output by the controller.

[0016] In summary, the present invention has the following main beneficial effects: By associating and storing visual positioning parameters, image position parameters, height parameters, rivet hole sequence, coordinate conversion parameters, and rivet distribution parameters as a riveting recipe package corresponding to the target product, and directly calling the riveting recipe package based on the product number during production, the same automatic riveting equipment can quickly import and switch between different product models. Compared to riveting equipment that relies on fixed programs or manual re-teaching, this application completes the binding and storage of material finding templates, image position templates, image clarity height, riveting operation height, and hole sequence during the product import stage. This allows subsequent production to complete positioning, identification, and execution based on the saved recipe data, reducing repeated machine adjustments and modeling processes, and improving the equipment's compatibility with multiple product models.

[0017] The CCD vision acquisition mechanism identifies the actual reference position and posture of the target product. Combined with pixel ratio coefficients, installation offset, product posture angle, and coordinate rotation parameters, the visual pixel coordinates of the rivet holes are converted into mechanical execution coordinates. This achieves unified compensation for product placement offset, angle deviation, and camera installation deviation. This method does not simply perform riveting according to a fixed teaching point. Instead, it redefines the reference position, hole position, and execution coordinates based on the actual image of the current product in each production run. This allows the riveting execution mechanism to move to the corresponding hole position according to the actual placement state of the target product, reducing the risk of hole position offset caused by product placement skew, fixture repetitive positioning errors, or camera installation deviations.

[0018] By setting rivet distribution parameters in the riveting recipe package and generating a riveting execution queue based on the number of rivet holes on the front and sides, and the corresponding program activation threshold, the system automatically selects the front program segment, side program segment, alternating execution queue, or sequential execution queue according to the product's rivet distribution. This method links visual recognition results, hole position sequences, and program segment scheduling within the same control flow. It avoids executing invalid program segments when there are few or no rivet holes on a certain side, and establishes a clear execution order when riveting is required on both the front and sides. This ensures that the riveting path matches the actual rivet distribution of the product, improving the continuity and stability of automated riveting operations. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method of the present invention.

[0020] Figure 2 This is a system block diagram of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1 refer to Figure 1 A general riveting operation method based on vision positioning includes the following steps: S1: Obtain the riveting recipe package corresponding to the target product. The riveting recipe package includes visual positioning parameters, photo position parameters, height parameters, rivet hole sequence, coordinate conversion parameters, and rivet distribution parameters. The photo position parameters include the photo position template and photo position. S2: Acquire images of the target product through a CCD vision acquisition mechanism, and determine the actual reference position and posture of the target product based on visual positioning parameters; S3: Identify the rivet hole positions on the target product based on the photo template, photo position, and height parameters, and convert the identified rivet hole positions into mechanical execution coordinates based on coordinate conversion parameters; S4: Generate a riveting execution queue based on rivet distribution parameters and rivet hole sequence; S5: Control the motion actuator to reach the corresponding rivet hole position according to the riveting execution queue, mechanical execution coordinates and height parameters, and the riveting actuator completes the riveting operation.

[0023] This application pertains to riveting operations on different product models using the same automatic riveting equipment. The automatic riveting equipment includes a positioning and clamping mechanism, a CCD vision acquisition mechanism, a motion execution mechanism, a riveting execution mechanism, a controller, and a human-machine interface module. The positioning and clamping mechanism carries the product and performs clamping and correction; the CCD vision acquisition mechanism acquires images of the product's reference features, the area to be riveted, and the rivet holes; the motion execution mechanism moves the riveting execution mechanism to the corresponding rivet hole position; the riveting execution mechanism performs one of the following riveting operations: press riveting, pull riveting, or spin riveting; the controller handles vision template invocation, coordinate conversion, riveting execution queue generation, program segment scheduling, and anomaly detection; the human-machine interface module allows selection of product number, loading of riveting recipe packages, and display of positioning status, coordinate data, and fault information.

[0024] In this embodiment, the target product refers to the product currently to be produced; the product to be imported refers to the product for which a riveting recipe package is established for the first time on the automatic riveting equipment. The target product and the product to be imported can be the same model or the same product from different batches. The product number is used to distinguish different model products, and each product number corresponds to a riveting recipe package. The riveting recipe package includes at least visual positioning parameters, image position parameters, height parameters, rivet hole sequence, coordinate conversion parameters, and rivet distribution parameters. Among them, the visual positioning parameters include the material finding template, the type of reference feature, the number of reference features, and the origin calibration data; the image position parameters include the image position template, the image position, the image position number, and the corresponding CCD vision acquisition mechanism position; the height parameters include the image clarity height and the riveting operation height; the rivet hole sequence includes the hole position number, the surface identification, the theoretical rivet hole coordinates, and the riveting operation height; the coordinate conversion parameters include the pixel ratio coefficient, the X-axis installation offset, the Y-axis installation offset, the product attitude angle, and the coordinate rotation parameters; the rivet distribution parameters include the number of front rivet holes, the number of side rivet holes, the front program activation threshold, and the side program activation threshold.

[0025] When creating the riveting formula package, the operator places the product to be imported into the positioning and clamping mechanism. This mechanism can employ pneumatic grippers, clamping blocks, positioning pins, limiting edges, or combinations thereof, to maintain the product's stable position within the field of view of the CCD vision acquisition mechanism. After the positioning and clamping mechanism completes clamping and calibration, the controller controls the CCD vision acquisition mechanism to acquire a reference feature image of the product. Reference features can be product mark points, reference holes, outer contour edges, or corner contours. For regular products with fixed round holes or mark points, the controller extracts the center of the product mark point or the center of the reference hole and uses it as the reference origin of the product's local coordinate system. For products without fixed hole positions but with stable outlines, the controller extracts the contour point set of the outer contour edge or corner contour, determines the center of the circumscribed rectangle based on the contour point set, and uses the center of the circumscribed rectangle as the reference origin of the product's local coordinate system. This reference origin serves as a common reference for subsequent rivet hole coordinates, photographic positions, and motion coordinate conversions.

[0026] The material finding template is used to determine the actual reference position and orientation of the target product during the production stage. When establishing the template, the controller delineates the reference feature region in the reference feature image as the material finding area, recording the corresponding image features, template center, template direction, and the origin of the product's local coordinate system. For product Mark points or reference holes, the template center is the center of the Mark point or the center of the reference hole; for outer contour edges or corner contours, the template center is the center of the circumscribed rectangle. If the same product has more than two reference features, the controller records the relative positional relationship of each reference feature and determines the product's orientation angle based on these two or more reference features during the production stage. The material finding template is stored in association with the product number.

[0027] After the material selection template is completed, the controller controls the CCD vision acquisition mechanism to acquire images of the area to be riveted. The area to be riveted is divided into a front riveting area and a side riveting area based on the location of the rivet holes on the product. The CCD vision acquisition mechanism can include a front vision acquisition unit and a side vision acquisition unit, where the front vision acquisition unit's viewpoint faces the front of the product, and the side vision acquisition unit's viewpoint faces the side of the product. For devices equipped with only one camera, the front and side image acquisition can also be completed by the same CCD vision acquisition mechanism at different shooting positions. The controller acquires images of the front and side riveting areas respectively, and generates front and side shooting position templates based on the rivet hole edges, hole centers, hole perimeter contours, or preset hole position areas.

[0028] The front-facing photo template is stored corresponding to the front-facing rivet hole sequence, and the side-facing photo template is stored corresponding to the side-facing rivet hole sequence. Each rivet hole in the sequence is configured with a hole position number, a face identifier, theoretical rivet hole coordinates, and a riveting operation height. The hole position number is used to determine the riveting sequence and abnormal output objects; the face identifier is used to distinguish between front and side rivet holes; the theoretical rivet hole coordinates are derived from the hole center coordinates identified by the CCD vision acquisition mechanism during the product introduction phase or calculated from product drawings and fixture positioning references; the riveting operation height is derived from the Z-axis position of the riveting actuator when it can effectively rivet at that hole position. The image clarity height is the Z-axis position when the CCD vision acquisition mechanism acquires the image of the corresponding photo template, and its source is the equipment Z-axis coordinate after focusing confirmation of that photo position during the product introduction phase. The image clarity height is stored according to the photo template number, and the riveting operation height is stored according to the rivet hole position number.

[0029] This implementation method does not limit the specific values ​​of the image clarity height, riveting operation height, coordinate deviation threshold, and positioning deviation threshold. These values ​​are determined by product drawing tolerances, rivet specifications, riveting process specifications, fixture positioning repeatability, CCD vision acquisition mechanism resolution, and equipment debugging records. For the same product model, these values ​​are entered into the riveting formula package after confirmation during the product introduction phase; for different product models, they are stored separately under their respective product numbers. This avoids inaccurate riveting positions caused by sharing height parameters or error thresholds between different products.

[0030] When producing the target product, the operator selects the product number of the target product through the human-machine interface module. The controller calls the corresponding riveting recipe package based on the product number and loads the material finding template, image capture template, height parameters, coordinate conversion parameters, rivet hole sequence, rivet distribution parameters, and riveting mode parameters from the riveting recipe package into the current production task. After loading, the controller controls the positioning and clamping mechanism to hold the target product and controls the CCD vision acquisition mechanism to acquire an image of the target product. The controller identifies the actual reference position and posture of the target product based on the material finding template, obtaining the translational and angular deviations of the target product relative to the product introduction stage. Subsequently, the controller identifies the front rivet hole positions of the target product based on the front image capture template and the side rivet hole positions based on the side image capture template, converting the identified visual pixel coordinates into mechanical execution coordinates.

[0031] In this embodiment, the coordinate conversion employs a three-level conversion: pixel coordinates, product local coordinates, and mechanical execution coordinates. Let the visual pixel coordinates of the rivet hole in the image be... The pixel coordinates of the reference origin obtained by the material finding template in the image are: The pixel ratio factor is and ,in This represents the physical length corresponding to a single pixel in the X direction. This represents the physical length corresponding to a single pixel in the Y direction. The coordinates of the rivet hole in the product's local coordinate system. Determine using the following formula: in, Indicates the hole position number of the rivet hole; and They represent the first The horizontal and vertical pixel coordinates of each rivet hole in the image coordinate system; and These represent the horizontal and vertical pixel coordinates of the reference origin in the image coordinate system, respectively. and They represent the first The X and Y coordinates of each rivet hole in the product's local coordinate system. Let the reference coordinates in the equipment's mechanical coordinate system be... The X-axis installation offset is The Y-axis installation offset is The product's attitude angle is The rotation compensation angle of the CCD vision acquisition mechanism relative to the mechanical coordinate system is: , No. The mechanical coordinates corresponding to each rivet hole are: The mechanical actuator coordinates are then determined by the following formula: in, and They represent the first The X-axis and Y-axis coordinates of each rivet hole in the equipment's mechanical coordinate system; and These represent the reference coordinates in the equipment's mechanical coordinate system corresponding to the product's reference origin; and Used to compensate for the static offset between the installation position of the CCD vision acquisition mechanism and the coordinates of the mechanical actuator; It is identified by the material finding template and is used to compensate for the target product placement angle deviation; Obtained through equipment calibration, this is used to compensate for the installation angle deviation of the camera coordinate system relative to the mechanical coordinate system. The Z-axis execution coordinate of each rivet hole is determined by the riveting operation height of the corresponding hole number in the riveting recipe location package.

[0032] When generating the riveting execution queue, the controller reads the rivet distribution parameters. These parameters include the number of front rivet holes, the number of side rivet holes, the front program activation threshold, and the side program activation threshold. The front and side program activation thresholds are not uniformly fixed values; instead, they are written into the riveting formula package during the product introduction phase based on equipment cycle time, product rivet hole distribution, riveting actuator reversal time, and the production strategy confirmed by process engineers. These thresholds are used to determine whether the current product is suitable for a front riveting program segment, a side riveting program segment, an alternating riveting program segment, or a sequential riveting program segment.

[0033] When the number of front rivet holes is greater than or equal to the front program activation threshold, and the number of side rivet holes is less than the side program activation threshold, the controller only writes the front rivet hole sequence into the riveting execution queue, activates the front riveting program segment, and skips the side riveting program segment. This mode is suitable for products with a concentrated number of front rivet holes and a small number or no side rivet holes.

[0034] When the number of side rivet holes is greater than or equal to the side program activation threshold, and the number of front rivet holes is less than the front program activation threshold, the controller only writes the side rivet hole sequence into the riveting execution queue, activates the side riveting program segment, and skips the front riveting program segment. This mode is suitable for products with a concentrated number of side rivet holes and a relatively small number of front rivet holes.

[0035] When the number of front rivet holes is less than the front program activation threshold, and the number of side rivet holes is less than the side program activation threshold, the controller takes one rivet hole from each of the front and side rivet hole sequences and writes them alternately into the riveting execution queue according to their hole position numbers. After the rivet holes in one sequence are written, the remaining rivet holes in the other sequence are written into the riveting execution queue according to their hole position numbers. This mode is suitable for products with a small number of front and side rivet holes. The controller calls the front riveting program segment and the side riveting program segment sequentially according to the riveting execution queue.

[0036] When the number of front rivet holes is greater than or equal to the front program activation threshold, and the number of side rivet holes is greater than or equal to the side program activation threshold, the controller first writes the front rivet hole sequence into the riveting execution queue in hole position number order, and then writes the side rivet hole sequence into the riveting execution queue in hole position number order. The controller first activates the front riveting program segment to execute the front rivet hole sequence, and then activates the side riveting program segment to execute the side rivet hole sequence. The above four branches cover all combinations of the number of front and side rivet holes relative to their respective thresholds, and will not result in undefined queue generation.

[0037] After generating the riveting execution queue, the controller sends the mechanical execution coordinates, riveting operation height, surface identifier, and program segment type corresponding to each hole position number in the queue to the motion actuator. The motion actuator moves according to the riveting execution queue. Upon reaching the corresponding rivet hole position, the motion actuator feeds back the actual positioning coordinates to the controller. The controller compares the actual positioning coordinates with the mechanical execution coordinates corresponding to that hole position number. When the positioning deviation is not greater than the positioning deviation threshold, the controller outputs a continue riveting command to the riveting actuator; when the positioning deviation is greater than the positioning deviation threshold, the controller stops executing the riveting action corresponding to that rivet hole and outputs the hole position number of that rivet hole.

[0038] Position deviation can be determined by the following formula: in, Indicates the first The positioning deviation of each rivet hole; , , These represent the feedback from the motion actuator. The actual coordinates of each rivet hole; , , These represent the riveting formula package and the coordinate conversion result, respectively, determining the first... Mechanical actuator coordinates for each rivet hole; The riveting height is determined by the riveting position number corresponding to that hole in the riveting recipe package. The positioning deviation threshold is determined by the equipment's repeatability accuracy, rivet specifications, and product process tolerances, and is written into the riveting recipe package or equipment process parameter table during the product introduction phase.

[0039] Before performing the riveting operation, the controller can also perform hole position recognition verification. The controller compares the rivet hole position identified by the CCD vision acquisition mechanism with the theoretical rivet hole coordinates of the corresponding hole position number in the riveting formula package to obtain the coordinate deviation. When the coordinate deviation is greater than the coordinate deviation threshold, the controller re-acquires the image corresponding to the photo template of the rivet hole location; when the recognition confidence after re-acquisition is less than the recognition confidence threshold, the controller stops executing the riveting action corresponding to the rivet hole and outputs the hole position number of the rivet hole. The recognition confidence threshold and the coordinate deviation threshold are derived from the resolution of the CCD vision acquisition mechanism, the illumination stability, the stability of hole edge recognition, and the allowable deviation of the product process, and are not set as fictitious fixed values ​​in this embodiment.

[0040] Coordinate deviation can be determined by the following formula: in, Indicates the first Coordinate deviation of drilling on individual ships; and These represent the first and second images obtained by recognizing and converting the currently acquired image. Coordinates of each rivet hole; and These represent the theoretical rivet hole coordinates for the corresponding hole positions in the rivet assembly kit. If... If the deviation exceeds the coordinate deviation threshold, it means that the deviation between the hole position obtained by the current image recognition and the theoretical hole position recorded during the product introduction stage exceeds the allowable range. The controller will then execute a re-photograph or stop the riveting process.

[0041] In this embodiment, none of the threshold values ​​are limited to fixed values. The threshold values ​​for the number of front and side program activations are derived from the product's rivet hole distribution, equipment cycle time, and program segment switching time; the recognition confidence threshold value is derived from the test shooting results of qualified samples and visual recognition stability; the coordinate deviation threshold value is derived from the product's hole position tolerance, rivet specifications, and visual recognition accuracy; and the positioning deviation threshold value is derived from the repeatability accuracy of the motion actuator, the allowable eccentricity of the rivet joint, and the product's process tolerance. The above threshold values ​​are stored as product formula parameters or equipment process parameters and are not fixed values ​​applicable to all products.

[0042] This embodiment also provides a universal riveting operation system based on vision positioning. The system includes a positioning and clamping mechanism, a CCD vision acquisition mechanism, a motion execution mechanism, a riveting execution mechanism, a controller, and a human-machine interface module. The controller is communicatively connected to the CCD vision acquisition mechanism, the motion execution mechanism, the riveting execution mechanism, and the human-machine interface module. After acquiring a product image, the CCD vision acquisition mechanism sends the image acquisition result to the controller; the human-machine interface module receives the product number of the target product and sends the product number to the controller; the controller obtains the corresponding riveting formula package based on the product number, determines the actual reference position, posture, and rivet hole position of the target product based on the image acquisition result, converts the rivet hole position into mechanical execution coordinates based on coordinate conversion parameters, and generates a riveting execution queue based on rivet distribution parameters and rivet hole sequence. The motion execution mechanism moves the riveting execution mechanism to the corresponding rivet hole position according to the motion control command output by the controller and feeds back the actual positioning coordinates to the controller. Based on the deviation between the actual positioning coordinates and the mechanical execution coordinates, the controller outputs a continue riveting command or a stop riveting command to the riveting execution mechanism. The riveting actuator performs riveting operations at the corresponding rivet hole position according to the riveting control command output by the controller.

[0043] The controller includes a template generation module, a coordinate conversion module, a recipe management module, a queue generation module, and a mode scheduling module. The template generation module receives the product reference feature image and the riveting area image output by the CCD vision acquisition mechanism. It generates a material finding template based on the product reference feature image and a photo-taking position template based on the photo-taking area image. The coordinate conversion module receives the visual pixel coordinates identified by the CCD vision acquisition mechanism and outputs mechanical execution coordinates based on the pixel ratio coefficient, X-axis installation offset, Y-axis installation offset, product attitude angle, and coordinate rotation parameters. The recipe management module associates and stores the material finding template, photo-taking position template, photo clarity height, riveting operation height, coordinate conversion parameters, rivet hole sequence, riveting mode parameters, and rivet distribution parameters as a riveting recipe package corresponding to the product number. It then calls the corresponding riveting recipe package based on the product number selected by the human-machine interaction module. The queue generation module generates a riveting execution queue based on the rivet distribution parameters and rivet hole sequence. The mode scheduling module activates the front riveting program segment and the side riveting program segment based on the riveting execution queue, or sequentially calls the front riveting program segment and the side riveting program segment.

[0044] Taking a specific application process as an example, when a certain model of product is first introduced into the equipment, the operator places the product in the positioning and clamping mechanism. After the positioning and clamping mechanism clamps the product, the controller controls the CCD vision acquisition mechanism to acquire the image of the reference hole, extracts the center of the reference hole as the reference origin of the product's local coordinate system, and generates a material finding template. Subsequently, the controller acquires images of the front and side riveting areas of the product, respectively, generating front and side photo templates, and records the clear photo height for each photo template, and records the hole number, face identification, theoretical rivet hole coordinates, and riveting operation height for each rivet hole. The controller writes the pixel ratio coefficient, X-axis installation offset, Y-axis installation offset, and coordinate rotation parameters according to the equipment calibration results, and writes the front program activation quantity threshold and side program activation quantity threshold according to the number of front and side rivet holes of the product and the equipment production strategy. The above data together form the riveting formula package corresponding to the product number.

[0045] When producing the same model of product, the operator selects the corresponding product number in the human-machine interface module, and the controller calls the riveting recipe package corresponding to that product number. The CCD vision acquisition mechanism identifies the actual reference position and posture of the current target product based on the material finding template, and identifies the rivet hole positions based on the front and side photo position templates. The controller converts the visual pixel coordinates of the rivet holes into mechanical execution coordinates, and generates a riveting execution queue based on the number of front and side rivet holes and the corresponding thresholds. The motion execution mechanism moves to each rivet hole position according to the riveting execution queue. When the positioning deviation meets the requirements, the controller controls the riveting execution mechanism to complete the riveting operation of the corresponding hole position; when the recognition fails, the coordinate deviation exceeds the limit, or the positioning deviation exceeds the limit, the controller outputs the corresponding hole position number and stops the riveting action at that hole position. This application process corresponds to the riveting recipe package acquisition, visual positioning, photo position template recognition, coordinate conversion, riveting execution queue generation, program segment scheduling, and fault-tolerant control process. Those skilled in the art can implement the above method based on a CCD camera, motion controller, servo motion mechanism, and riveting execution mechanism.

[0046] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A universal riveting operation method based on vision positioning, characterized in that, Includes the following steps: S1: Obtain the riveting recipe package corresponding to the target product. The riveting recipe package includes visual positioning parameters, photo position parameters, height parameters, rivet hole sequence, coordinate conversion parameters, and rivet distribution parameters. The photo position parameters include the photo position template and photo position. S2: Acquire images of the target product through a CCD vision acquisition mechanism, and determine the actual reference position and posture of the target product based on visual positioning parameters; S3: Identify the rivet hole positions on the target product based on the photo template, photo position, and height parameters, and convert the identified rivet hole positions into mechanical execution coordinates based on coordinate conversion parameters; S4: Generate a riveting execution queue based on rivet distribution parameters and rivet hole sequence; S5: Control the motion actuator to reach the corresponding rivet hole position according to the riveting execution queue, mechanical execution coordinates and height parameters, and the riveting actuator completes the riveting operation.

2. The universal riveting operation method based on vision positioning according to claim 1, characterized in that, When creating the riveting formula package, the product to be imported is placed in the positioning and clamping mechanism of the automatic riveting equipment, and the positioning and clamping mechanism clamps and corrects the product to be imported. The CCD vision acquisition mechanism is controlled to acquire the reference feature image of the product to be imported, a material finding template is generated based on the reference feature image, and a local coordinate system of the product is established based on the reference feature position identified in the material finding template. The CCD vision acquisition mechanism is controlled to acquire images of the riveting area of ​​the product to be imported, and a photo position template is generated based on the riveting area image. Associate the material locating template, photo position template, height parameter, coordinate conversion parameter, rivet hole sequence, and rivet distribution parameter with the same product number to form a riveting formula package corresponding to that product number and store it.

3. The universal riveting operation method based on vision positioning according to claim 2, characterized in that, The riveting area includes a front riveting area and a side riveting area; When generating the photo position template, the CCD vision acquisition mechanism is controlled to acquire the front riveting area image and the side riveting area image respectively, and generate the front photo position template and the side photo position template. The rivet hole sequence includes a front rivet hole sequence and a side rivet hole sequence. The front photo position template is stored in correspondence with the front rivet hole sequence, and the side photo position template is stored in correspondence with the side rivet hole sequence. Both the front rivet hole sequence and the side rivet hole sequence include hole position number, face identification, theoretical rivet hole coordinates, and riveting operation height.

4. The universal riveting operation method based on vision positioning according to claim 3, characterized in that, The height parameters include the height at which the photo can be taken clearly and the height at which the riveting operation is performed; The image clarity height is the Z-axis position when the CCD vision acquisition mechanism acquires the corresponding image template image, and is stored according to the image template number. The riveting operation height is the Z-axis position of the riveting actuator when performing riveting operation on the corresponding rivet hole, and is stored according to the hole position number of the rivet hole; When the motion actuator performs the riveting operation, the motion actuator drives the riveting actuator to the corresponding rivet hole position according to the mechanical execution coordinates and the corresponding riveting operation height.

5. A universal riveting operation method based on vision positioning according to claim 4, characterized in that, The visual positioning parameters include the material search template and the reference feature type; The reference feature type is a product mark point, a reference hole, an outer contour edge, or a corner contour; When the reference feature type is a product Mark point, the center of the product Mark point is used as the reference origin of the product's local coordinate system; When the reference feature type is a reference hole, the center of the reference hole is used as the reference origin of the local coordinate system of the product. When the reference feature type is an outer contour edge or corner contour, extract the contour point set of the outer contour edge or corner contour, determine the center of the circumscribed rectangle based on the contour point set, and use the center of the circumscribed rectangle as the reference origin of the product's local coordinate system.

6. A universal riveting operation method based on vision positioning according to claim 5, characterized in that, The coordinate conversion parameters include pixel ratio coefficient, X-axis installation offset, Y-axis installation offset, product attitude angle, and coordinate rotation parameters; When converting the rivet hole position into mechanical execution coordinates, the visual pixel coordinates of the rivet hole are first converted into local coordinates of the product based on the pixel ratio coefficient. Then, the local coordinates of the product are converted into mechanical execution coordinates based on the X-axis installation offset, Y-axis installation offset, product attitude angle, and coordinate rotation parameters.

7. A universal riveting operation method based on vision positioning according to claim 6, characterized in that, The rivet distribution parameters include the number of rivet holes on the front, the number of rivet holes on the side, the threshold for the number of rivet holes enabled on the front, and the threshold for the number of rivet holes enabled on the side. When generating the riveting execution queue, if the number of front rivet holes is greater than or equal to the front program activation threshold and the number of side rivet holes is less than the side program activation threshold, only the front rivet hole sequence is written into the riveting execution queue. When the number of side rivet holes is greater than or equal to the threshold for the number of side program activations, and the number of front rivet holes is less than the threshold for the number of front program activations, only the side rivet hole sequence is written into the riveting execution queue. When the number of front rivet holes is less than the front program activation threshold and the number of side rivet holes is less than the side program activation threshold, one rivet hole is taken from the front rivet hole sequence and one from the side rivet hole sequence according to the hole position number order and written alternately into the riveting execution queue. When the rivet holes in one sequence are written, the remaining rivet holes in the other sequence are written into the riveting execution queue according to the hole position number order. When the number of front rivet holes is greater than or equal to the front program activation threshold, and the number of side rivet holes is greater than or equal to the side program activation threshold, the front rivet hole sequence is first written into the riveting execution queue in the order of hole position number, and then the side rivet hole sequence is written into the riveting execution queue in the order of hole position number.

8. A universal riveting operation method based on vision positioning according to claim 7, characterized in that, When only the front rivet hole sequence is written to the riveting execution queue, the controller enables the front riveting program segment and skips the side riveting program segment. When only the side rivet hole sequence is written to the riveting execution queue, the controller enables the side riveting program segment and skips the front riveting program segment. When the front rivet hole sequence and the side rivet hole sequence are alternately written into the riveting execution queue, the controller calls the front riveting program segment and the side riveting program segment in sequence according to the riveting execution queue. When the number of front rivet holes and the number of side rivet holes are both greater than or equal to the corresponding threshold, the controller first enables the front riveting program segment to execute the front rivet hole sequence, and then enables the side riveting program segment to execute the side rivet hole sequence. The riveting recipe package also includes riveting mode parameters, which include origin riveting mode parameters, multi-point feature riveting mode parameters, and contour global riveting mode parameters. When the reference feature type recorded in the riveting recipe package is a product Mark point or a reference hole, the origin riveting mode parameters are called. When the number of reference features recorded in the riveting recipe package is more than two, the multi-point feature riveting mode parameters are called, and the product attitude angle is determined based on the two or more reference features. When the reference feature type recorded in the riveting recipe package is an outer contour edge or corner contour, the contour global riveting mode parameters are called, and the reference origin of the product local coordinate system is determined according to the outer contour edge or corner contour.

9. A universal riveting operation method based on vision positioning according to claim 8, characterized in that, The riveting recipe package also includes an identification confidence threshold, a coordinate deviation threshold, and a positioning deviation threshold; Before performing the riveting operation, the rivet hole position identified by the CCD vision acquisition mechanism is compared with the theoretical rivet hole coordinates of the corresponding hole position number. When the coordinate deviation between the two is greater than the coordinate deviation threshold, the image corresponding to the photo template of the rivet hole is re-acquired. When the recognition confidence level after re-collection is less than the recognition confidence level threshold, stop executing the riveting action corresponding to the rivet hole and output the hole position number of the rivet hole; After the motion actuator drives the riveting actuator to the corresponding rivet hole position, the actual positioning coordinates fed back by the motion actuator are compared with the mechanical execution coordinates corresponding to the rivet hole. When the positioning deviation between the two is greater than the positioning deviation threshold, the riveting action corresponding to the rivet hole is stopped and the hole position number of the rivet hole is output.

10. A universal riveting system based on vision positioning, characterized in that, The method for performing the universal riveting operation based on vision positioning as described in any one of claims 1 to 9 includes a positioning and clamping mechanism, a CCD vision acquisition mechanism, a motion execution mechanism, a riveting execution mechanism, a controller, and a human-machine interaction module. The controller is communicatively connected to the CCD vision acquisition mechanism, motion execution mechanism, riveting execution mechanism and human-computer interaction module respectively; The positioning and clamping mechanism is used to carry and clamp the product, so that the product is within the field of view of the CCD vision acquisition mechanism; The CCD vision acquisition mechanism is used to acquire product images and output the image acquisition results to the controller. The human-computer interaction module is used to receive the product number of the target product and send the product number to the controller; The controller is used to obtain the corresponding riveting recipe package according to the product number, determine the actual reference position, attitude and rivet hole position of the target product according to the image acquisition results of the CCD vision acquisition mechanism, convert the rivet hole position into mechanical execution coordinates according to the coordinate conversion parameters in the riveting recipe package, and generate a riveting execution queue according to the rivet distribution parameters and rivet hole sequence in the riveting recipe package. The controller includes a template generation module, a coordinate conversion module, a recipe management module, a queue generation module, and a mode scheduling module; The template generation module is used to generate a material finding template based on the product's reference feature image and to generate a photo taking position template based on the product's riveting area image. The coordinate conversion module is used to convert the visual pixel coordinates identified by the CCD vision acquisition mechanism into mechanical execution coordinates according to the coordinate conversion parameters. The recipe management module is used to associate and store the material finding template, photo position template, clear photo height, riveting operation height, coordinate conversion parameters, rivet hole sequence, riveting mode parameters and rivet distribution parameters as a riveting recipe package corresponding to the product number, and call the corresponding riveting recipe package according to the product number selected by the human-computer interaction module. The queue generation module is used to generate a riveting execution queue based on rivet distribution parameters and rivet hole sequence. The mode scheduling module is used to enable the front riveting program segment and the side riveting program segment according to the riveting execution queue, or to call the front riveting program segment and the side riveting program segment in sequence. The CCD vision acquisition mechanism includes a front vision acquisition unit and a side vision acquisition unit. The front vision acquisition unit's view is directed towards the front of the product, and the side vision acquisition unit's view is directed towards the side of the product. The motion actuator is used to drive the riveting actuator to the corresponding rivet hole position according to the motion control command output by the controller, and to feed back the actual positioning coordinates to the controller. The controller is also used to output a continue riveting command or a stop riveting command to the riveting actuator based on the deviation between the actual positioning coordinates and the mechanical execution coordinates. The riveting actuator is used to perform riveting operations at the corresponding rivet hole position according to the riveting control command output by the controller.

Images