Machine vision-based mechanical parachute stringing operation method

Abstract

The application discloses a kind of based on machine vision's mechanical operation method of parachute stringing, including the shooting of industrial camera in starting side, image is transmitted to visual guidance and control software and is detected and analyzed;Learn the rope loop state, confirm rope loop pose, and image coordinates are converted into world coordinates in combination with calibration result;Visual guidance and control software runs action instruction module, formulates stringing action parameter according to rope loop pose information, compiles program instruction and sends to lower computer controller;The application combines stringing operation process and rope loop form characteristics, proposes a kind of based on machine vision's mechanical operation process of parachute stringing, applies machine vision technology to the mechanical operation of parachute stringing, realizes the identification positioning and attitude determination of rope loop, according to this, the movement stroke of each driving component is determined, to reach the purpose of optimizing operation process, ensuring stringing quality, improving the degree of automation of such equipment, to fill the blank of stringing automatic operation process.

Description

Technical Field

[0001] This invention relates to the field of automated equipment technology, specifically to a method for mechanized parachute rope threading based on machine vision. Background Technology

[0002] Several parachute lines are fixed to the edge of the parachute canopy, and the other end of the parachute lines is tied to the parachute pack straps through rope loops; two rows of rope loops are designed on the surface of the parachute pack, which are used to bundle the parachute lines and pass them through the rope loops to store and organize the parachute lines; manual rope threading is time-consuming, labor-intensive and quite tedious, so adopting mechanical operation to replace manual operation has great engineering application value.

[0003] Obtaining the position and orientation information of the rope loops is a technical challenge in achieving automated rope threading. The intuitive solution is to preset a fixed feed distance of the execution platform based on the average size and spacing of the rope loops, so that the threading mechanism moves approximately to the vicinity of the rope loops before performing the threading action. However, in actual operation, because the rope loops are made of flexible materials, their size and spacing are not uniform. Furthermore, after the paracord is threaded into the rope loop, the current rope loop will deform and shift, causing the position and orientation of subsequent rope loops to change accordingly. This results in cumulative errors, which seriously affect the quality of rope threading. Summary of the Invention

[0004] The purpose of this invention is to provide a machine vision-based mechanized parachute rope threading method to solve the problems mentioned in the background art. In actual operation, the rope loops are made of flexible materials, and their sizes and spacing are not the same. After the parachute rope is threaded into the rope loop, the current rope loop will deform and shift, causing the position and attitude of the subsequent rope loops to change accordingly. This results in cumulative errors and seriously affects the quality of rope threading.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A machine vision-based mechanized parachute rope threading method includes the following steps:

[0007] Preliminary work: Install the industrial camera in the designated position, and then complete the camera calibration according to the instructions of the vision guidance and control software;

[0008] Step 1: The industrial camera on the starting side takes pictures and transmits the images to the vision guidance and control software for detection and analysis; the state of the rope loop is known, the position and pose of the rope loop are confirmed, and the image coordinates are converted into world coordinates based on the calibration results;

[0009] Step 2: The visual guidance and control software runs the action instruction module, formulates the rope threading action parameters based on the rope loop posture information, compiles the program instructions based on the action parameters and the working principle of the equipment, and sends them to the lower-level controller.

[0010] Step 3: The lower-level controller issues an action command, and the current actuator moves to the positioning position of the loop to be threaded;

[0011] Step 4: Based on the rope loop position information, the actuator adjusts the rope loop's posture to keep it upright and ensure that the loop opening is large enough for the paracord to pass through smoothly.

[0012] Step 5: The actuator pulls the rope loop while grabbing the paracord and passing it through the rope loop for a certain length to tidy and fix the paracord.

[0013] Step 6: Repeat steps 2 to 5 for the industrial camera and actuator on the opposite side until the paracord passes through all the rope loops on both sides and is fixed to the surface of the parachute in an S-shape;

[0014] Step 7: Check if the number of rope threading attempts matches the preset number of rope loops. Once confirmed, the entire rope threading process is complete.

[0015] Preferably, the installation position of the industrial camera in the pre-processing stage ensures that the rope loops are arranged horizontally, the number of rope loops in the image is controlled to be four to five, and the shooting effect of the cameras on both sides is consistent.

[0016] Preferably, the visual guidance and control software in step two includes a camera calibration module, an image acquisition module, a visual detection module, and an action command module.

[0017] Preferably, the visual detection module is implemented based on the YOLOv8-Seg model. The detection model analyzes the state and pose information of the rope loop in the image, and formulates corresponding correction actions based on the rope loop posture in combination with the working principle of the device.

[0018] Preferably, the visual detection module selects an appropriate positioning coordinate selection strategy based on the equipment conditions, which selects the horizontal center point of the largest envelope rectangle of the rope loop to be threaded as the positioning coordinate of the rope loop or selects the edge contour of the rope loop to be threaded as the positioning information.

[0019] Preferably, after the lower-level controller receives the rope threading action parameters in step three, the upper-level controller simultaneously issues a shooting command to the industrial camera on the other side and executes step 2 for visual detection and action parameter setting. After the current actuator completes the rope threading action, the action parameters on the other side are promptly sent to the lower-level controller to improve the overall operating efficiency of the rope threading equipment.

[0020] Compared with existing technologies, this invention provides a machine vision-based mechanized parachute rope threading method with the following advantages: This invention combines the rope threading process with the characteristics of the rope loop shape, proposing a machine vision-based mechanized parachute rope threading process. By applying machine vision technology to the mechanized parachute rope threading operation, it achieves the identification, positioning, and attitude judgment of the rope loops, thereby determining the movement stroke of each driving component. This optimizes the operation process, ensures rope threading quality, and improves the automation level of such equipment, thus filling a gap in automated rope threading processes. The machine vision system can accurately identify the actual size and spacing of each rope loop, overcoming the problem of difficulty in precise positioning due to the size differences of rope loops made of flexible materials. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the rope threading device and vision system of the present invention;

[0022] Figure 2 This is a functional diagram of the visual guidance and control software of the present invention;

[0023] Figure 3 This is a schematic diagram of the output results of a portion of the visual detection module in an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of the structure and operation of the parachute rope threading device according to an embodiment of the present invention;

[0025] Figure 5 For the present invention Figure 4 A schematic diagram of the rope-threading mechanism;

[0026] Figure 6 For the present invention Figure 4 A schematic diagram of the translational rope-fastening mechanism;

[0027] Figure 7 For the present invention Figure 6 A schematic diagram of the buckle assembly;

[0028] Figure 8 This is a schematic diagram of the rope threading operation process according to a preferred embodiment of the present invention;

[0029] Figure 9 For the present invention Figure 8 The diagram shows the specific process of threading the rope. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Please see Figure 1-9 The present invention provides an embodiment of a mechanized parachute rope threading method based on machine vision.

[0032] The image acquisition component includes an industrial camera, lens, auxiliary light source, acquisition card, and data transmission cable; it is used to acquire images of the rope loops in the current working area according to the shooting instructions issued by the controller, and transmit them to the host computer for storage and analysis.

[0033] This visual guidance and control software is used for camera installation calibration, issuing acquisition commands during operation, detecting and analyzing acquired images, designing rope-threading actions, and sending the compiled action commands to the lower-level controller.

[0034] The visual guidance and control software is divided into four functional modules: camera calibration, image acquisition, visual detection, and motion commands.

[0035] The camera calibration module is used to assist staff in quickly completing camera calibration after the camera is installed or when the installation location changes. The camera calibration module can be implemented based on Zhang Zhengyou's calibration method or other methods. The purpose of calibration is to calculate the camera's intrinsic and extrinsic parameter matrices and realize the coordinate mapping between image pixel coordinates and world space coordinates.

[0036] The image acquisition module sends acquisition commands to the industrial camera in a timely manner based on sensor signals, and selectively performs image preprocessing on the original image as needed. The image preprocessing methods used by the image acquisition module include image operations such as size transformation, grayscale conversion, and filtering and sharpening. The preprocessing method can be manually selected based on the imaging quality of the industrial camera, the performance of the host computer hardware, and the actual detection effect.

[0037] The visual inspection module analyzes the input image by calling a deep learning model, perceiving the state and pose of the rope loops in the image, and combining this with camera calibration results to obtain the spatial pose information of the rope loop to be threaded. The rope loop state in the visual inspection module includes three categories: whether threading is complete, whether there is damage, and whether there is a missed threading. Whether threading is complete is used to determine the next rope loop to be threaded in the image; damage and missed threading serve as conditions for the system to interrupt the operation process. The visual inspection module can select an appropriate detection model and model parameters based on the hardware performance of the host computer and the operation time of the threading equipment. This achieves the goal of shortening the running time of the visual inspection module while meeting the required detection accuracy, ensuring the overall efficiency of the threading equipment. The visual inspection module can also enable online learning of the detection model, allowing staff to supervise the image detection results during the threading process. Images with correct detection results are added to the database for retraining the detection model.

[0038] The motion command module designs the rope-threading action based on the rope loop's position and the actuator's principle, and compiles this action into commands to send to the lower-level controller. When specifying the rope-threading action, the module divides it into three stages: positioning, correction, and rope threading. Positioning involves moving the actuator to the position directly opposite the rope loop based on the spatial coordinates of the largest envelope rectangle. Correction involves performing necessary attitude corrections on the rope loop based on attitude information to facilitate smooth rope passage. Rope threading involves pulling the parachute ropes according to the actuator's design principle and threading them through the rope loop to complete the fixation. The motion command module can select a suitable positioning strategy based on the actuator's motion design: for actuators capable only of simple linear motion, the center line of the envelope rectangle in the horizontal direction can be used as the feed endpoint; for actuators capable of complex rope-threading actions, the module can provide more precise rope loop position and spatial attitude information based on the edge contour of the rope loop.

[0039] Example 1: As per the instruction manual Figure 2 As shown, the visual guidance and control software comprises four main modules: camera calibration, image acquisition, visual detection, and motion commands. Its main functions are as follows:

[0040] After the image acquisition components are installed, the camera calibration module is operated by the operator using a calibration board and following the software instructions to calibrate the camera and map the image coordinate system to the world coordinate system.

[0041] The image acquisition module is used by the host computer to send acquisition commands to the industrial camera and to perform size transformation and feature enhancement on the returned raw image so that the image to be detected meets the input size requirements of the detection model, while optimizing image quality and improving detection efficiency and accuracy.

[0042] The visual inspection module will provide an image input interface for the deep learning model, and output the rope loop state and pose information after analysis; this embodiment uses the YOLOV8-Seg model to realize the visual inspection task of the rope loop.

[0043] As per the instruction manual Figure 3 As shown, this is an example of the output of the detection model. Targets marked "Finished" indicate that the detection result is that the rope loop has been threaded, targets marked "Unfinished" indicate that the rope loop is to be threaded; rope loop targets marked "Unfinished_Lie" indicate that the rope loop is currently in a state to be threaded, but is in a lying or tilted position, which is not conducive to the paracord passing through.

[0044] The motion command module generates a motion parameter vector (L1…L) adapted to the current loop to be threaded, based on the spatial position information of the loop. m ,θ1…θ n The system compiles control commands based on these parameters and sends them to the lower-level controller to achieve the goal of vision-assisted rope threading. The motion parameter vector represents the stroke size and direction of each moving part in the rope threading action. The order of elements in the vector determines the sequence of actions of each driving component.

[0045] As per the instruction manual Figure 4 As shown, it is a schematic diagram of the structure and working scenario of a rope threading device. The device's workbench 101 is used to place the parachute pack 105; 102 is the industrial camera, lens, and auxiliary light source in the image acquisition component; 103 is the rope threading mechanism, which is responsible for grabbing and transporting the parachute rope and adjusting the rope loop posture; 104 is the translational rope fastening mechanism. After the rope threading mechanism 103 delivers the parachute rope to the opening of the rope loop, this mechanism pulls the parachute rope to pass through the rope loop and fix it.

[0046] This embodiment is based on the working principle of the existing rope threading equipment and describes the rope threading operation process under machine vision, thereby demonstrating the technical features of the present invention. It is cited from the authorized announcement number CN216035149U "An Automatic Parachute Rope Threading Device Applicable to Parachutes".

[0047] The installation and debugging of the data acquisition system are as follows: Prepare two sets of industrial cameras, camera lenses, and auxiliary light sources with identical configurations; install the acquisition components in the same positions on both sides of the actuators; adjust the lens direction so that the acquired image is as shown in the attached manual. Figure 3 As shown, the rope loop sequence is horizontal, with four to five loops; run the camera calibration module of the vision guidance and control software and complete the camera calibration according to the instructions;

[0048] After completing camera calibration and parachute placement, you can begin threading the ropes; as per the instruction manual. Figure 8As shown, the machine vision-based rope threading process of the present invention is as follows:

[0049] Step S1: The industrial camera of the starting side actuator receives the acquisition command, captures the current scene, and transmits the acquired image to the vision guidance and control software, and runs the vision inspection module to obtain the pose information of the rope loop to be threaded; according to the intrinsic and extrinsic parameter matrix calibrated by the camera, the pose information is mapped from image coordinates to world coordinates.

[0050] Step S2: The visual guidance and control software runs the motion instruction module. Based on the spatial position information of the rope loop, combined with the working principle of the equipment and the current position of each moving part, it calculates the motion stroke of each prime mover, outputs the motion parameter vector, compiles it into control instructions, and sends them to the lower-level controller.

[0051] Step S3: The lower-level controller sends an action signal to start the rope threading action; after the first drive motor of the current rope threading mechanism 103 rotates by an angle θ1, it arrives at the positioning position of the rope loop to be threaded, and the second drive motor rotates by an angle θ2, causing the rope threading mechanism to move closer to the rope loop.

[0052] Step S4: Based on the position information of the loop to be threaded, the threading mechanism 103 adjusts the attitude of the loop to be threaded, and performs the following sub-steps in sequence according to the attitude:

[0053] Step S4.1: If the rope loop is in a forward and backward lying posture, the rope threading mechanism 103 rotates the No. 2 drive motor θ'2 according to the lying direction and angle provided by the machine vision, so that the rope hook moves to the position of the rope loop; the first cylinder 203 connected to the stop lever 209 moves L'1, and the first rope hook 206 will pull up the rope loop; then the No. 2 drive motor rotates θ'2' to correct the lying posture;

[0054] Step S4.2: If the rope loop is tilted to the left or right, the rope threading mechanism 103 rotates the No. 1 drive motor θ'1 according to the tilt direction and angle provided by the machine vision, so that the rope hook moves to the position of the rope loop; the first cylinder 203 connected to the stop lever 209 moves L'1, and the first rope hook 206 will pull up the rope loop; then the No. 1 drive motor rotates θ'1' to correct the tilt posture.

[0055] Step S4.3: After the above sub-steps, the rope loop has achieved the lying down and tilting posture correction and is in the upright posture to be threaded; the first cylinder 203 connected to the push plate 205 moves L2, and the paracord will pass through the rope loop under the push of the push plate 205.

[0056] Step S5: As per the instruction manual Figure 6 As shown, stepper motor 301 rotates by an angle θ3, driving the motor as shown in the instruction manual. Figure 7The slide block 404 is moved to the vicinity of the push plate 205, aligning the second rope hook 403 with the through groove of the push plate 205. Then, the horizontally moving third cylinder 401 extends L3, bringing the second rope hook 403 directly above the through groove of the push plate 205. Then, another vertically moving third cylinder 401 shortens L4, causing the second rope hook 403 to insert into the through groove. Finally, the horizontally moving third cylinder 401 shortens L5, controlling the second rope hook 403 to pull back, while fixing the paracord in the rope loop. At this point, the rope threading work of the rope loop is completed.

[0057] Step S6: After the rope threading is completed by the actuator on the starting side, the actuator on the opposite side performs the next rope threading, and steps S1 to S5 are repeated; the paracords are alternately fixed to the surface of the parachute pack in an "S" shape on both sides.

[0058] Step S7: The vision guidance and control software compares the number of rope threading attempts with the preset number of rope loops. If they match, the entire rope threading process ends. Otherwise, the industrial camera and vision guidance and control software on this side scan and detect the rope loop sequence, find the missing threading positions, and signal the staff to handle them.

[0059] It is easy to understand that the shooting time of the industrial camera on the opposite side in step S6 can be selected between steps S2 and S3. That is, after the host computer completes the visual inspection and sends the command on the current side, it can continue to carry out the inspection work on the opposite side. This saves or even eliminates the interval time for the two actuators to perform the rope threading action, thereby improving work efficiency.

[0060] This invention patent aims to protect a method for mechanized parachute rope threading based on machine vision. It applies machine vision technology to mechanized parachute rope threading to identify, locate, and judge the attitude of the rope loops, thereby determining the movement stroke of each driving component. This optimizes the operation process, ensures the quality of rope threading, and improves the automation level of such equipment, thus filling the gap in automated rope threading operations.

[0061] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for mechanized parachute rope threading based on machine vision, characterized in that, Includes the following steps: Preliminary work: Install the industrial camera in the designated position, and then complete the camera calibration according to the instructions of the vision guidance and control software; Step 1: The industrial camera on the starting side takes pictures and transmits the images to the vision guidance and control software for detection and analysis; the state of the rope loop is known, the position and pose of the rope loop are confirmed, and the image coordinates are converted into world coordinates based on the calibration results; Step 2: The visual guidance and control software runs the action instruction module, formulates the rope threading action parameters based on the rope loop posture information, compiles the program instructions based on the action parameters and the working principle of the equipment, and sends them to the lower-level controller. Step 3: The lower-level controller issues an action command, and the current actuator moves to the positioning position of the loop to be threaded; Step 4: Based on the rope loop position information, the actuator adjusts the rope loop's posture to keep it upright and ensure that the loop opening is large enough for the paracord to pass through smoothly. Step 5: The actuator pulls the rope loop while grabbing the paracord and passing it through the rope loop for a certain length to tidy and fix the paracord. Step 6: Repeat steps 2 to 5 for the industrial camera and actuator on the opposite side until the paracord passes through all the rope loops on both sides and is fixed to the surface of the parachute in an S-shape; Step 7: Check if the number of rope threading attempts matches the preset number of rope loops. Once confirmed, the entire rope threading process is complete.

2. The method for mechanized parachute rope threading based on machine vision according to claim 1, characterized in that: The installation position of the industrial camera in the pre-processing stage ensures that the rope loops are arranged horizontally, the number of rope loops in the image is controlled to be four to five, and the shooting effect of the cameras on both sides is consistent.

3. The method for mechanized parachute rope threading based on machine vision according to claim 1, characterized in that: The visual guidance and control software mentioned in step two includes a camera calibration module, an image acquisition module, a visual detection module, and an action command module.

4. The method for mechanized parachute rope threading based on machine vision according to claim 3, characterized in that: The visual detection module is based on the YOLOv8-Seg model. The detection model analyzes the state and pose information of the rope loop in the image, and, in conjunction with the working principle of the device, formulates corresponding correction actions based on the rope loop posture.

5. The method for mechanized parachute rope threading based on machine vision according to claim 3, characterized in that: The visual detection module selects an appropriate positioning coordinate selection strategy based on the equipment conditions. It selects the horizontal center point of the largest envelope rectangle of the rope loop to be threaded as the positioning coordinate of the rope loop or selects the edge contour of the rope loop to be threaded as the positioning information.

6. The method for mechanized parachute rope threading based on machine vision according to claim 1, characterized in that: After receiving the rope threading action parameters in step three, the host computer simultaneously issues a shooting command to the industrial camera on the other side and executes step 2 for visual detection and action parameter setting. After the current actuator completes the rope threading action, the action parameters on the other side are promptly sent to the host computer to improve the overall operating efficiency of the rope threading equipment.

Images