Double-arm traffic cone automatic collecting and releasing method based on machine vision

Abstract

The present application relates to the field of highway construction traffic control auxiliary, especially to a double-arm traffic cone cylinder automatic collecting and releasing method based on machine vision. The machine vision camera identifies the cone cylinder through a vision sensing system and image analysis; the automatic tracking and dynamic measurement of the cone cylinder contour size; once the cone cylinder enters the included angle between the scissor type manipulator, the touch rod captures and rightens the cone cylinder, the sensor below the clamp sends a feedback signal to the programmable controller; the rotary mechanical arm then goes down, reciprocating cycle, controls the rotary mechanical arm to grab the cone cylinder and up, down and steering, to complete the collection work of different specifications and sizes of cone cylinder, so as to enhance the automation level of the automatic collecting and releasing machine of the cone cylinder, and improve the work efficiency and the safety factor of the driver.

Description

Technical Field

[0001] This invention relates to the field of traffic control assistance during highway construction, and more particularly to a machine vision-based method for the automatic deployment and retraction of dual-arm traffic cones. Background Technology

[0002] Traffic cones are hollow cones made of plastic or rubber, used for road closures or emergency construction, as well as lane separation and traffic diversion signs. As of the end of 2020, the total length of highways open to traffic in China reached 5.1981 million kilometers, including 161,000 kilometers of national expressways, ranking first in the world.

[0003] Highways require routine maintenance, most of which is carried out without disrupting traffic. Due to the high volume and speed of traffic on highways, there are significant safety risks during maintenance. Therefore, traffic cones need to be placed in advance along the work section to demarcate the work area and to divert traffic from the highway.

[0004] Highway maintenance is characterized by its large workload and high level of danger, making it a high-risk operation. Currently, most domestic and international highway traffic systems still rely on manual placement and collection of traffic cones, which is not only time-consuming, labor-intensive, and inefficient, but also environmentally hazardous. Therefore, to improve the efficiency of highway maintenance work and reduce the probability of accidents for highway maintenance workers, automated traffic cone placement and collection equipment for highway maintenance has gradually attracted close attention from research institutions and enterprises.

[0005] While China has made some progress in deploying and recycling cone-shaped equipment, there are still many areas for improvement, as detailed below:

[0006] First, most of these robotic arms use hydraulic or pneumatic methods to control their grasping and placement actions. The overall design of the product is too complex, which not only increases the overall design cost of the device, but also results in a large footprint, high application and maintenance costs, and reduces placement efficiency due to the overly complex motion trajectory.

[0007] Secondly, these are basically single-function cone placement products. For example, the sliding rail semi-automatic cone placement vehicle completes the cone placement work by sliding the cones out from the chute connected to the rear of the vehicle, but manual collection of the cones from the road is still required, making the product's performance limited.

[0008] Third, there is a lack of research on machine vision recognition of traffic cones. The current method of locating and collecting traffic cones relies entirely on the driver's control of the vehicle's path, which not only significantly distracts the driver and reduces driving safety, but also makes it difficult to universally collect cones of different sizes.

[0009] Current cone placement and recovery equipment can no longer meet the needs of modern highway maintenance operations, and its efficiency, practicality, reliability, automation and intelligence still need to be improved.

[0010] On September 19, 2024, a search was conducted in the China Patent Publication Database using "cone and vision and image and sensor and recognition" as the abstract keywords and with the option to allow synonym expansion. No relevant literature was found.

[0011] On September 19, 2024, an abstract search was conducted on CNKI (China National Knowledge Infrastructure) for the keyword "cone and vision and image and sensor and recognition", but no relevant literature was found.

[0012] On September 19, 2024, a search was conducted on the website of the United States Patent and Trademark Office for the term "Cone with vision with image with sensor with recognition," but no relevant literature was found; the search URL is https: / / ppubs.uspto.gov / pubwebapp / .

[0013] On September 19, 2024, a search was conducted at the Korean Intellectual Property Office using the search term "Cone and vision and image and sensor and recognition"; the search URL was:

[0014] http: / / eng.kipris.or.kr / enghome / main.jsp. Twelve irrelevant documents were found: Applicant: Cornell University, Application No.: 1020137005335 Image Processing Systems, Methods, and Applications Based on Angle-Sensitive Pixels (ASP);

[0015] Applicant: Aurora Flight Sciences Corporation; Application No.: 1020180071987; Report on Systems and Methods for Detecting Obstacles in Airborne Systems;

[0016] Applicant: Microsoft Technology Licensing, LLC Application No.: 1020147011240 Report: Illustration of video display modification based on sensor input for transparent near-eye display.

[0017] Applicant: EYEFLUENCE, INC. Application No.: 1020167034649 Systems and methods for biomechanical eye signaling for interaction with real and virtual objects. Report on systems and methods for biomechanical eye signaling for interaction with real and virtual objects.

[0018] Applicant: BLACKMORE SENSORS AND ANALYTICS, LLC Application No.: 1020197018575 Report on methods and systems for classifying objects in point cloud datasets. Illustrations of methods and systems for classifying objects in point cloud datasets.

[0019] Applicant: PRIMESENSE LTD. Application No.: 1020087008228 Methods and Systems for Object Reconstruction Report and Drawings of Object Recovery Methods and Systems;

[0020] Applicant: BLACKMORE SENSORS AND ANALYTICS, LLC Application No.: 1020227009625 Report on methods and systems for classifying objects in point cloud datasets. Illustrations of methods and systems for classifying objects in point cloud datasets.

[0021] Applicant: Navigate Surgical Technologies, Inc. Application No.: 1020157015076 Surgical Position Monitoring System and Method Using Natural Markers Report on the mapping of the industrial chain using a surgical position monitoring system and method using natural markers;

[0022] Applicant: Motional AD LLC; Application No.: 1020200149202; Sequential Fusion Report of 3D Object Detection; Sequential Fusion Illustration of 3D Object Detection for Industrial Control Computers;

[0023] Applicant: GlaxoSmithKline Group Limited; Application No.: 1020107015248; Antigen-binding protein trial report; Antigen-binding protein diagram;

[0024] Applicant: GlaxoSmithKline Group Limited; Application No.: 1020157022610; Antigen-binding protein reporter antigen-binding protein diagram.

[0025] On September 19, 2024, a search was conducted on WIPO's website https: / / patentscope2.wipo.int / with the search term "Cone and vision and image and sensor and recognition". The following irrelevant documents were found: 102016001070732 Method for solving the mirror parameters of a cone mirror refraction camera using a straight line; 201611018620.3 Surround view provision device for vehicles and vehicles; 102017000014391 Steering device and vehicles; 200510009294.5 Visual audio tape playback device with memory; 02130013.5 Method for generating and encoding image and text data of visual audio tape and image and text data playback device; PCT / CN2002 / 000549 TELETEXT DATA GENERATING AND ENCODINGMETHOD ON THE AUDIOVISUAL MAGNETIC TAPE & THE TELETEXT DATA PLAYER.

[0026] On September 19, 2024, a search was conducted on the website of the Japan Patent Office (https: / / www.j-platpat.inpit.go.jp / ) for the term "Cone and vision and image and sensor and recognition," but no relevant literature was found.

[0027] It is completely different from the concept of this patent. Summary of the Invention

[0028] Purpose of the invention: To provide a more effective machine vision-based automatic deployment and retraction method for dual-arm traffic cones, the specific purpose of which is described in the detailed implementation section for several substantial technical effects.

[0029] To achieve the above objectives, the present invention adopts the following technical solution:

[0030] A machine vision-based automatic deployment and retraction method for dual-arm traffic cones, characterized in that...

[0031] It includes the following steps:

[0032] Machine vision cameras identify cones through visual sensing systems and image analysis;

[0033] Automatic tracking and dynamic measurement of the cone's external profile dimensions;

[0034] Once the cone enters the angle between the scissor-type robotic arms, the contact bar captures and straightens the cone, and the sensor below the gripper sends a feedback signal to the programmable controller.

[0035] The programmable logic controller will dynamically adjust the outreach height, lifting height and the angle of the electronic push rod of the manipulator according to information such as the position and profile of the cone barrel, and automatically import the original upright cone barrels on the road into the cone barrel through the double-arm guiding and adjusting mechanism at different positions and attitudes. The piston rod of the electric push and pull rod retracts, the fixture clamps the cone barrel, and the robotic arm moves upward;

[0036] When the robotic arm moves to the upper limit photoelectric switch, the robotic arm stops moving upward. The piston rod of the electric push and pull rod extends, the fixture opens, and the operator collects the cone barrel and places it in the vehicle;

[0037] Then the rotating robotic arm moves downward and repeats the cycle, controlling the rotating robotic arm to grab the cone barrel, convey it up and down and turn it, so as to complete the collection work of cone barrels of different specifications and sizes, thereby enhancing the automation level of the cone barrel automatic retractor and improving the work efficiency and the safety factor of the driver's driving.

[0038] A further technical solution of the present invention is that the hardware of the traffic cone machine vision positioning system consists of a real-time imaging camera, a Raspberry Pi microcomputer and a lighting and light supplementing system, and the software uses a convolutional neural network algorithm. Deep learning is carried out with the traffic cone as the target to train and generate a weight file, and the weight file with the best effect is selected and deployed to the Raspberry Pi microcomputer; after the device is powered on, the camera supporting the Raspberry Pi can be called to take real-time images. When a single recovery or deployment is completed, the camera starts to take images and identify and locate the position of the traffic cone, and transmits the position information to the PLC control unit. The control system issues a signal to move the fixture to 2 / 3 of the height of the traffic cone. When the infrared sensor in the middle of the fixture acts and gives a feedback signal, the electric push rod acts to clamp the fixture.

[0039] A further technical solution of the present invention is that since the imaging of the camera is affected by factors such as light, physical light supplement is usually required to ensure the clarity of the camera imaging under the above extreme conditions. Therefore, a lighting and light supplementing system is set, and a light supplementing system switch is set on the operation panel. Workers can control the operation of the light supplementing system through simple operations; the light supplementing system is light illumination for light supplement.

[0040] A further technical solution of this invention is that the weight file used for traffic cone recognition and positioning is generated by a YOLOv5 convolutional neural network. The YOLOv5 convolutional neural network uses convolutional neural networks to perform and complete the task of target detection and classification. This network structure consists of four parts: Input, Backbone, Neck, and Head. At the Input end, methods such as Mosaic, Copy-paste, Random Affine Transformation, MixUp, and Cutout are used to augment the dataset, thereby improving the dataset's diversity, model robustness, and generalization ability. The Focus network in the Backbone module slices the input image to obtain feature layers, which are then fed into CSPDarknet for convolution, normalization, and SiLu activation function operations to obtain three different effective feature layers. The three effective feature layers obtained from the Backbone are fed into the Neck module, where they are upsampled using the Feature Pyramid Network (FPN), stacked with the other two effective feature layers, and feature extracted. After upsampling, they are downsampled using the Path Aggregation Network (PAN) to obtain the final prediction result and generate the weight file. Finally, the weight file with the best recognition performance is selected and burned onto a Raspberry Pi. On the Pi microcomputer, algorithms are used to access cameras and weight files to achieve real-time detection and location of traffic cones.

[0041] A further technical solution of the present invention is that the method is based on a double-arm traffic cone automatic deployment and retraction device, the structure of which is as follows: a fixed double-arm traffic cone automatic deployment and retraction device; the structure of the fixed double-arm traffic cone automatic deployment and retraction device is as follows:

[0042] It includes a basic support 15, on which a lifting support 8 is arranged on a vertical track. Two sets of scissor-type robotic arms 4 are arranged on the lifting support 8. A threaded block is also arranged on the lifting support 8, which is threadedly connected to the lifting screw 10. The lifting screw 10 is a worktable lifting motor 11. The lifting support 8 as a whole can be raised and lowered.

[0043] A collection and fixing frame 13 is arranged on the basic support 15; the collection and fixing frame 13 can accommodate multiple sets of nested conical structures;

[0044] The basic support 15 is provided with a fixing and pressing structure 14 that can fix the basic support to the carriage; the fixing and pressing structure 14 can press the basic support 15 onto the carriage by rotating forward.

[0045] A camera is fixed on the basic bracket 15 to acquire image data;

[0046] Multiple position sensors and / or distance sensors are arranged on the basic support 15. The position sensors and / or distance sensors can determine the height and position of the scissor-type manipulator 4. At the same time, the position sensors and / or distance sensors can determine the position of the cone on the ground.

[0047] A further technical solution of the present invention is that the method is based on a double-arm traffic cone automatic deployment and retraction device, the structure of which is as follows: a retractable double-arm traffic cone automatic deployment and retraction device;

[0048] It includes a basic support 15, on which a lifting support 8 is arranged on a vertical track. Two sets of scissor-type robotic arms 4 are arranged on the lifting support 8. A threaded block is also arranged on the lifting support 8, which is threadedly connected to the lifting screw 10. The lifting screw 10 is a worktable lifting motor 11. The lifting support 8 as a whole can be raised and lowered.

[0049] A telescopic structure 22 is fixed on the lifting bracket 8. The upper part of the telescopic structure 22 is hinged to the upper plane 23 of the flipping structure, and the bottom of the telescopic structure 22 is hinged to the flipping bracket 25. The flipping bracket 25 is located on the lifting bracket 8. Two sets of scissor-type robotic arms 4 are located on the flipping bracket 25. The flipping bracket 25 can also rotate around the rotating shaft 27 of the overall fixture structure.

[0050] The bracket where the handle 1 is located can rotate around the pivot 26 of the flip bracket;

[0051] A camera is fixed on the basic bracket 15 to acquire image data;

[0052] Multiple position sensors and / or distance sensors are arranged on the basic support 15. The position sensors and / or distance sensors can determine the height and position of the scissor-type manipulator 4. At the same time, the position sensors and / or distance sensors can determine the position of the cone on the ground.

[0053] A further technical solution of the present invention is that the position sensor and / or distance sensor is any one of an infrared position sensor and an ultrasonic sensor.

[0054] A further technical solution of the present invention is that a rotating mechanical arm 28 is arranged in the middle of the flipping bracket 25, and a power part capable of driving the rotating mechanical arm 28 to rotate is arranged inside the flipping bracket 25. Two sets of scissor-type mechanical hands 4 are fixed on the rotating mechanical arm 28.

[0055] A further technical solution of the present invention is that the scissor-type manipulator 4 is equipped with an electric push-pull rod 16, and a hinge rod 20 is hinged to the telescopic shaft of the electric push-pull rod 16. The hinge rod 20 is hinged to two sets of clamps 18. The electric push-pull rod 16 can drive the two sets of clamps 18 to clamp the cone.

[0056] A further technical solution of the present invention is that a contact rod 9 is also arranged on the basic support.

[0057] The present invention, employing the above technical solution, has the following advantages over the prior art: No vehicle modification is required; the automatic cone deployer can be attached to the side of the vehicle for deployment and retrieval operations. Operation is simple, and the degree of automation and intelligence is high. It innovatively uses visual sensing, eliminating the need for people on moving vehicles or dangerous roads to constantly bend down to check the cones. Attached Figure Description

[0058] To further illustrate the present invention, the following description is provided in conjunction with the accompanying drawings:

[0059] Figure 1 This is a schematic diagram of the structure of the present invention;

[0060] Figure 2 This is a schematic diagram of the rear view structure of the present invention;

[0061] Figure 3 This is a schematic diagram of the robotic gripper structure of the present invention;

[0062] Figure 4 This is a schematic diagram of the network structure of the control system of the present invention;

[0063] Figure 5 This is a flowchart of the process of the present invention;

[0064] Figure 6 This is a schematic diagram of the machine vision recognition structure of the present invention;

[0065] Figure 7 This is a flowchart illustrating the identification and positioning of traffic cones according to the present invention;

[0066] Figure 8 This is a perspective view of the present invention;

[0067] Figure 9 This is a schematic diagram from another perspective of the present invention;

[0068] Figure 10 This is a schematic diagram of a partial structure of the present invention;

[0069] Figure 11 Example diagram of camera and sensor installation;

[0070] Figure 12 A structural diagram of a fixed extrusion structure;

[0071] In the diagram: 1. Handle; 2. Control box; 3. Motor box; 4. Scissor-type robotic arm; 5. Rotary robotic arm; 6. Machine vision camera; 7. Lifting guide groove; 8. Lifting bracket; 9. Contact rod; 10. Lifting screw; 11. Worktable lifting motor; 12. Base frame; 13. Collection fixing frame; 14. Fixed extrusion structure; 15. Basic bracket; 16. Electric push-pull rod; 17. Sensor probe; 18. Fixture; 19. Locking nut; 20. Hinge rod; 21. Infrared sensor; 22. Telescopic structure; 23. Top plane; 24. Tilting bracket; 25. Lifting bracket fixing plate; 26. Tilting bracket pivot; 27. Fixture overall structure rotating pivot; 28. Rotary robotic arm; 29. ​​Camera and sensor; 30. Detachable base. Detailed Implementation

[0072] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," "top," and "bottom," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. In addition, unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication of two components. For those skilled in the art, the specific meaning of the above terms in the present invention can be understood according to the specific circumstances.

[0073] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0074] This patent provides multiple parallel solutions; the different descriptions represent improved solutions or parallel solutions based on the basic solution. Each solution has its own unique characteristics. Furthermore, the technical features involved in the different embodiments of the invention described below can be combined with each other as long as they do not conflict with each other. Fixing methods not described herein can be any type of fixing, such as threaded fixing, bolt fixing, or adhesive bonding.

[0075] Example 1: Referring to all the accompanying drawings; a machine vision-based automatic deployment and retraction method for dual-arm traffic cones, characterized in that,

[0076] It includes the following steps:

[0077] Machine vision cameras identify cones through visual sensing systems and image analysis;

[0078] Automatic tracking and dynamic measurement of the cone's external profile dimensions;

[0079] Once the cone enters the angle between the scissor-type robotic arms, the contact bar captures and straightens the cone, and the sensor below the gripper sends a feedback signal to the programmable controller.

[0080] The programmable controller will dynamically adjust the extension height, lifting height and electronic push rod angle of the robot arm using servo drive based on information such as the position and contour of the cone. The robot arm will automatically guide the existing upright cones on the road into the cones through the double-arm guide adjustment mechanism at different positions and postures. The electric push-pull rod piston rod will retract, the clamp will clamp the cones, and the robot arm will move upward.

[0081] When the robotic arm moves to the upper limit photoelectric switch, the robotic arm stops moving upward, the electric push-pull rod piston rod extends forward, the clamp opens, and the operator collects the cone and places it inside the vehicle;

[0082] The rotating robotic arm then descends and repeats the cycle, controlling the rotating robotic arm to grab cones and transport them up and down and to turn, in order to complete the collection of cones of different sizes. This not only enhances the automation level of the automatic cone take-up and take-down machine, but also improves work efficiency and the safety factor of the driver.

[0083] The substantive technical effects of the technical solution presented herein and its implementation process are as follows:

[0084] In view of the current situation in China's traffic maintenance and road maintenance industry, which mainly relies on manual placement and removal of traffic cones, and has problems such as high risk, frequent accidents and injuries, and slow operation speed, the purpose of this invention patent is to provide a machine vision-based dual-arm automatic traffic cone placement and removal machine to solve the problems of low driving safety during cone collection.

[0085] The technical solution adopted in this invention patent is that the mechanical structure mainly includes a frame support device and a cone guiding and deployment system. This deployment machine is installed on the left or right side of the vehicle. A robotic arm automatically guides the cones through a dual-arm guiding and adjusting mechanism. A servo drive system automatically captures signals from the cones and controls the dual-arm robotic arm to grasp them. The robotic arm transports the cones up and down and steers them according to a set program. The cone deployment and deployment operation is completed when the vehicle is moving forward or backward. The traffic cone machine vision positioning system uses a convolutional neural network algorithm. Taking traffic cones as the object, a self-made traffic cone position dataset is created. A Yolov5 network is used as a deep convolutional neural network for training, and the best training weights are selected. These weights are used to achieve the recognition and positioning of traffic cones, allowing the camera to dynamically acquire the cone position and contour information at any time as it moves with the automatic cone deployment and deployment machine.

[0086] The electrical control system is based on a programmable logic controller (PLC) and is equipped with modules such as an operation panel, machine vision system, servo driver, and sensor probes, specifically as follows: Figure 4 As shown. The entire control system is divided into equipment layer, control layer, and management layer.

[0087] The equipment layer includes various sensors, photoelectric switches, linkage motors, push-pull rod motors, worktable lifting motors, travel / retrieval motors, and various motor drivers. The lifting motors control the left and right swing of the rotary robotic arm; the push-pull rod motors control the piston rod of the electric push-pull rod to extend or retract, controlling the clamping or releasing of the scissor-type mechanical clamp; the worktable lifting motors control and adjust the height of the scissor-type robotic arm off the ground; and the travel / retrieval motors are used for placing and storing the worktable.

[0088] The control layer uses a programmable logic controller (PLC) as the central controller, which is responsible for the logic control and management of cone collection and deployment, machine vision information acquisition and processing, and execution of overall machine coordination control strategies.

[0089] The management system includes an operation panel and a machine vision unit. The operation panel features buttons and switches for start / stop, run / retrieve, raise / lower, collect / release, left / right arm, and left / right light. When collecting cones, the machine vision system captures the cone signal, transmits it to the central controller, and automatically grasps it. The dual-arm balance bar rotates to a photoelectric switch and stops rotating, completing cone collection. When placing cones, the cone is placed into the robotic arm, and the dual-arm balance bar rotates to a photoelectric switch and stops rotating, completing cone placement.

[0090] The workflow diagram of the automatic cone take-up and take-up machine is as follows: Figure 5As shown, the automatic cone deployer is attached to both sides of the pickup truck bed. The operator can select the working direction and mode using buttons on the control panel, including start / stop, run / retract, raise / lower, cone retraction / deployment, left arm / right arm, and left / right light. The working direction is left-right, and one of the two scissor-arm robotic arms can be used for cone deployment and retraction; the working modes include cone deployment and cone collection.

[0091] The deployment process is as follows: When the rotating robotic arm moves to the upper limit photoelectric switch, the piston rod of the electric push-pull rod extends forward, the scissor-type robotic arm opens, and the operator places the traffic cone into the scissor-type robotic arm.

[0092] An infrared sensor located beneath the scissor-type robotic arm sends a feedback signal to the programmable controller, causing the piston rod of the electric push-pull rod to retract, and the scissor-type robotic arm to clamp the cone.

[0093] The rotating robotic arm descends, and when it reaches the lower limit photoelectric switch, the switch sends a feedback signal to the programmable controller. The scissor-type robotic arm then releases the cone, and the contact rod straightens the cone and places it on the ground. The rotating robotic arm then ascends, repeating the cycle to complete the cone placement operation.

[0094] The data collection process is as follows: When the rotating robotic arm descends to the lower limit photoelectric switch, the machine vision camera begins to automatically track and dynamically measure the outer contour dimensions of the cone. Once the cone enters the angle between the scissor-type robotic arms, the contact rod captures and straightens the cone, and the sensor under the fixture sends a feedback signal to the programmable controller.

[0095] The programmable controller will dynamically adjust the extension height, lifting height and electronic push rod angle of the robot arm according to the position and contour of the cone, and automatically guide the existing upright cone on the road into the cone through the double-arm guide adjustment mechanism. The electric push-pull rod piston rod retracts, the clamp clamps the cone, and the robot arm moves upward.

[0096] When the robotic arm reaches the upper limit photoelectric switch, the robotic arm stops moving upward, the electric push-pull rod piston rod extends forward, the clamp opens, and the operator collects the cone and places it inside the vehicle.

[0097] The rotating robotic arm then descends and repeats the cycle, controlling the rotating robotic arm to grab cones and transport them up and down and to turn, in order to complete the collection of cones of different sizes. This not only enhances the automation level of the automatic cone take-up and take-down machine, but also improves work efficiency and the safety factor of the driver, as well as the universal collection of cones of different sizes.

[0098] Embodiment 2: As a further improvable solution, parallel solution or alternative independent solution, the hardware of the traffic cone machine vision positioning system consists of a real-time imaging camera, a Raspberry Pi microcomputer and an illumination and light supplement system, and the software uses a convolutional neural network algorithm. Deep learning is carried out with traffic cones as the target to train and generate a weight file, and the weight file with the best effect is selected and deployed to the Raspberry Pi microcomputer; after the device is powered on, the camera supporting the Raspberry Pi can be called to take real-time images. When a single recycling or placement is completed, the camera starts to take images and identify and locate the position of the traffic cone, and transmits the position information to the PLC control unit. The control system issues a signal to make the fixture move to 2 / 3 of the height of the traffic cone. When the infrared sensor in the middle of the fixture acts and gives a feedback signal, the electric push rod acts to clamp the fixture.

[0099] The substantial technical effects achieved by the technical solution here and its implementation process are as follows:

[0100] The system composition for identifying and positioning traffic cones is as Figure 6 shown. The hardware of the traffic cone machine vision positioning system consists of a real-time imaging camera, a Raspberry Pi microcomputer and an illumination and light supplement system, and the software uses a convolutional neural network algorithm. Deep learning is carried out with traffic cones as the target to train and generate a weight file.

[0101] Embodiment 3: As a further improvable solution, parallel solution or alternative independent solution, since the imaging of the camera is affected by factors such as light, physical light supplement is usually required to ensure the clarity of the camera imaging under the above extreme conditions. Therefore, an illumination and light supplement system is set, and a light supplement system switch is set on the operation panel. Workers can control the operation of the light supplement system through simple operations; the light supplement system is lamp illumination light supplement.

[0102] The weight file used for traffic cone recognition and localization is generated by a YOLOv5 convolutional neural network. The YOLOv5 convolutional neural network performs object detection and classification tasks. This network structure consists of four parts: Input, Backbone, Neck, and Head. At the Input stage, methods such as Mosaic, Copy-paste, Random Affine Transformation, MixUp, and Cutout are used to augment the dataset, improving its diversity, robustness, and generalization ability. The Focus network in the Backbone module slices the input image to obtain feature layers, which are then fed into CSPDarknet for convolution, normalization, and SiLu activation to obtain three different effective feature layers. These three effective feature layers from the Backbone are then fed into the Neck module for upsampling via Feature Pyramid Network (FPN). They are then stacked with the other two effective feature layers for feature extraction. After upsampling, downsampling is performed via Path Aggregation Network (PAN) to obtain the final prediction result and generate the weight file. Finally, the weight file with the best recognition performance is selected and burned onto a Raspberry Pi. On the Pi microcomputer, algorithms are used to access cameras and weight files to achieve real-time detection and location of traffic cones.

[0103] Example 4: As a further improvement, parallel, or optional independent solution, this method is based on a double-arm traffic cone automatic deployment and retraction device. The structure of this device is as follows: It is a fixed double-arm traffic cone automatic deployment and retraction device; the structure of the fixed double-arm traffic cone automatic deployment and retraction device is as follows:

[0104] It includes a basic support 15, on which a lifting support 8 is arranged on a vertical track. Two sets of scissor-type robotic arms 4 are arranged on the lifting support 8. A threaded block is also arranged on the lifting support 8, which is threadedly connected to the lifting screw 10. The lifting screw 10 is a worktable lifting motor 11. The lifting support 8 as a whole can be raised and lowered.

[0105] A collection and fixing frame 13 is arranged on the basic support 15; the collection and fixing frame 13 can accommodate multiple sets of nested conical structures;

[0106] The basic support 15 is provided with a fixing and pressing structure 14 that can fix the basic support to the carriage; the fixing and pressing structure 14 can press the basic support 15 onto the carriage by rotating forward.

[0107] A camera is fixed on the basic bracket 15 to acquire image data;

[0108] Multiple position sensors and / or distance sensors are arranged on the basic support 15. The position sensors and / or distance sensors can determine the height and position of the scissor-type manipulator 4. At the same time, the position sensors and / or distance sensors can determine the position of the cone on the ground.

[0109] The substantive technical effects and implementation process of the technical solution described herein are as follows: (Reference) Figure 1 , Figure 2 , Figure 3 The structure is a lifting scissor-type robotic arm 4. A schematic diagram of the robotic arm gripper structure is shown below. Figure 3 As shown. A power-operated push-pull rod is connected to the rear of the scissor-type robotic arm. The movement of the piston rod on the electric push-pull rod moves the gripper, causing the scissor-type robotic arm to clamp the cone. When the piston rod of the electric push-pull rod extends forward, the scissor-type robotic arm opens to release the cone. A locking nut is used to secure the gripper. An ultrasonic sensor is used to measure the distance between the robotic arm and the cone. When a certain distance is reached, the contact rod will automatically capture and retrieve the cone.

[0110] Product structure diagram as follows: Figure 1 As shown. The handles are easy to move and attach to both sides of the vehicle. The entire workbench has an L-shape, and the control box integrates a programmable controller, motor driver, and machine vision circuitry. The table surface is equipped with buttons and switches for start / stop, run / retract, raise / lower, retract / release, left / right arm, left / right light, etc., to realize the interpretation of control commands.

[0111] The scissor-type robotic arm is connected to an electric push-pull rod. The piston rod of the electric push-pull rod drives the scissor arm to move. The retraction of the scissor arm can grip or place cones, thus utilizing the different directions of vehicle travel to complete the task of collecting and placing cones of different sizes in front, back, left, and right directions, and integrating the collected cones in the vehicle compartment. The collection and placement machine is equipped with a base and rollers for moving the equipment.

[0112] The basic support 15 can be placed on the detachable base 30.

[0113] Example 5: As a further possible improvement, parallel solution, or optional independent solution, this method is based on a double-arm traffic cone automatic deployment and retraction device, which has the following structure: a retractable double-arm traffic cone automatic deployment and retraction device.

[0114] It includes a basic support 15, on which a lifting support 8 is arranged on a vertical track. Two sets of scissor-type robotic arms 4 are arranged on the lifting support 8. A threaded block is also arranged on the lifting support 8, which is threadedly connected to the lifting screw 10. The lifting screw 10 is a worktable lifting motor 11. The lifting support 8 as a whole can be raised and lowered.

[0115] A telescopic structure 22 is fixed on the lifting bracket 8. The upper part of the telescopic structure 22 is hinged to the upper plane 23 of the flipping structure, and the bottom of the telescopic structure 22 is hinged to the flipping bracket 25. The flipping bracket 25 is located on the lifting bracket 8. Two sets of scissor-type robotic arms 4 are located on the flipping bracket 25. The flipping bracket 25 can also rotate around the rotating shaft 27 of the overall fixture structure.

[0116] The bracket where the handle 1 is located can rotate around the pivot 26 of the flip bracket;

[0117] A camera is fixed on the basic bracket 15 to acquire image data;

[0118] Multiple position sensors and / or distance sensors are arranged on the basic support 15. The position sensors and / or distance sensors can determine the height and position of the scissor-type manipulator 4. At the same time, the position sensors and / or distance sensors can determine the position of the cone on the ground.

[0119] The substantive technical effects and implementation process of the technical solution described herein are as follows: (Combined with...) Figure 8 , Figure 9 , Figure 10 , Figure 11 The motor housing contains multiple motors. The rotation of the rotary robotic arm drives the rocker arm to swing left and right, and the two scissor-type robotic arms work alternately.

[0120] Machine vision cameras automatically track and dynamically measure the outline dimensions of the cones. Once the cone enters the angle between the grippers of the robotic arm, the cone signal is transmitted to the control system. Based on the cone's position, orientation, and outline, the control system uses servo drives to dynamically adjust the robotic arm's extension height, lifting height, and electronic push rod angle. This allows the system to automatically guide the existing upright cones on the road into the cones via a dual-arm guide adjustment mechanism, controlling the rotating robotic arm to grasp, transport, and steer the cones. This completes the collection of cones of different sizes, enhancing the automation level of the automatic cone release and take-off machine while improving work efficiency and driver safety.

[0121] The rear view diagram of the cone automatic take-up and take-down machine is shown below. Figure 2 As shown. During runtime,

[0122] The robot's height is controllable. The lifting motor controls the lifting bracket via a screw, and the robot is adjusted to a suitable height via a lifting guide groove. The operator can manually control the working height via a button, and the contact rod is used to capture and straighten the cone.

[0123] Example 6: As a further improvement, parallel solution, or optional independent solution, the position sensor and / or distance sensor is any one of an infrared position sensor or an ultrasonic sensor. The substantial technical effect and implementation process of the technical solution herein are as follows: Similar sensors are all within the protection scope of this patent.

[0124] Example 7: As a further improvement, parallel, or optional independent solution, a rotating robotic arm 28 is arranged in the middle of the flipping bracket 25. Inside the flipping bracket 25, a power unit capable of driving the rotating robotic arm 28 to rotate is arranged. Two sets of scissor-type robotic arms 4 are fixed to the rotating robotic arm 28. The substantial technical effect and implementation process of this technical solution are as follows: The rotating robotic arm 28 is used for the two scissor-type robotic arms 4 to work alternately.

[0125] Example 8: As a further improvement, parallel solution, or optional independent solution, the scissor-type manipulator 4 is equipped with an electric push-pull rod 16. A hinge rod 20 is hinged to the telescopic shaft of the electric push-pull rod 16, and the hinge rod 20 hinges to two sets of clamps 18. The electric push-pull rod 16 can drive the two sets of clamps 18 to clamp the cone. The substantial technical effect and implementation process of this technical solution are as follows: it can achieve clamping, and similar manipulators are all within the protection scope of this patent.

[0126] Example 9: As a further improvement, parallel, or optional independent solution, a contact rod 9 is also arranged on the basic support.

[0127] Innovatively, each of the above effects exists independently, yet a single structure can be used to combine the results.

[0128] It should be noted that the multiple solutions provided in this patent include their own basic solutions, which are independent of each other and do not restrict each other. However, they can also be combined with each other without conflict to achieve multiple effects.

[0129] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claims.

Claims

1. A method for automatic collection and release of double-arm traffic cone based on machine vision, characterized in that, It comprises the following steps: Machine vision camera identifies the cone through visual sensing system and image analysis; Automatic tracking and dynamic measurement of the cone profile size; Once the cone enters the angle between the scissors type manipulator, the touch rod captures and straightens the cone, and the sensor under the clamp sends feedback signal to the programmable controller; The programmable controller will use servo drive to dynamically adjust the manipulator extension height, lifting height and electronic push rod angle according to the cone position and profile information, so as to automatically guide the original upright cone on the road into the cone through the double-arm guide adjustment mechanism, retract the piston rod of the electric push-pull rod, clamp the cone, and make the manipulator go up; When the manipulator moves to the upper limit photoelectric switch, the manipulator stops going up, the piston rod of the electric push-pull rod extends, the clamp opens, and the operator collects the cone and places it in the car; The rotating manipulator then goes down, and the cycle is repeated to control the rotating manipulator to grab the cone and convey it up and down and turn it around to complete the collection of cones of different sizes, thereby enhancing the automation level of the automatic cone collection and release machine, improving the work efficiency and the safety factor of the driver; The method is realized based on the double-arm traffic cone automatic collection and release device, and the structure of the device is as follows: It comprises a basic support (15), a lifting support (8) arranged on the vertical track of the basic support (15), two groups of scissors type manipulators (4) arranged on the lifting support (8), and a threaded block arranged on the lifting support (8), which is in threaded connection with a lifting screw (10) driven by a workbench lifting motor (11); the lifting support (8) can be lifted as a whole; A camera is fixed on the basic support (15) to obtain image data; A plurality of position sensors and / or distance sensors are arranged on the basic support (15), which can obtain the height and position of the scissors type manipulator (4); at the same time, the position sensors and / or distance sensors can obtain the position of the cone on the ground; The subsequent scheme is as follows: a fixed double-arm traffic cone automatic collection and release device or a retractable double-arm traffic cone automatic collection and release device; For the fixed double-arm traffic cone automatic collection and release device, the structure is as follows: A collection fixing frame (13) is arranged on the basic support (15); the collection fixing frame (13) can accommodate a plurality of nested cone structures; A fixing extrusion structure (14) is arranged on the basic support (15) to fix the basic support on the car compartment; the fixing extrusion structure (14) can extrude the basic support (15) on the car compartment by rotating forward; The double-arm traffic cone automatic collecting and releasing device is characterized in that a telescopic structure (22) is fixed on the lifting support (8), the upper part of the telescopic structure (22) is hinged to the upper plane (23) of the turnover structure, the bottom of the telescopic structure (22) is hinged to the turnover support (25), the turnover support (25) is located on the lifting support (8), two groups of scissor type mechanical hands (4) are located on the turnover support (25), and the turnover support (25) can rotate around the rotating shaft (27) of the overall structure of the clamp. The support where the handle (1) is located can rotate around the turnover support rotating shaft (26).

2. The machine vision-based double-arm traffic cone automatic collecting and releasing method according to claim 1 is characterized in that, The traffic cone machine vision positioning system hardware is composed of a real-time image taking camera, a Raspberry Pi microcomputer and an illumination light supplement system, and the software uses a convolution neural network algorithm; traffic cones are taken as targets for deep learning, a weight file is generated through training, and the best weight file is selected and deployed on the Raspberry Pi microcomputer; after the device is started, the camera matched with the Raspberry Pi can be called to take images in real time, when single recycling or releasing is completed, the camera starts to take images and identify the position of the traffic cone, the position information is transmitted to the PLC control unit, the control system sends a signal to move the clamp to the position of 2 / 3 of the height of the traffic cone, and the electric push rod is used to clamp the clamp after the feedback signal of the infrared sensor in the middle of the clamp.

3. The machine vision-based dual-arm cone cylinder automatic collecting and releasing method according to claim 2, characterized in that, Due to the influence of light on the image taking of the camera, physical light supplement is needed to ensure the clarity of the image taking of the camera, therefore, the illumination light supplement system is arranged, a light supplement system switch is arranged on the operation panel, and workers can control the operation of the light supplement system through simple operation; the light supplement system is a light illumination system.

4. The machine vision-based dual-arm cone cylinder automatic collecting and releasing method according to claim 2, wherein, The weight file used for traffic cone cylinder recognition and positioning is generated by a Yolov5 convolutional neural network; the Yolov5 convolutional neural network uses a convolutional neural network to perform and complete the task of target detection and classification; the network structure is composed of four parts of Input, Backbone, Neck and Head, and the data set is subjected to data enhancement by using Mosaic, Copy paste, Random affine transformation, MixUp and Cutout methods at the Input input end, so as to improve the diversity of the data set, the robustness of the model and the generalization ability; the Focus network in the Backbone module slices the input picture to obtain a feature layer, which is then transmitted into the CSPDarknet to obtain three different effective feature layers through convolution, standardization and SiLu activation function operation; the three effective feature layers obtained from the Backbone are transmitted into the Neck module, and are subjected to upsampling by the feature pyramid FPN: stacked and feature extracted with the other two effective feature layers, and after upsampling, are subjected to downsampling by the path aggregation network PAN to obtain the final prediction result and generate the weight file; finally, the best weight file in recognition effect is selected and burned into a Raspberry Pi microcomputer, and the camera and weight file are called by using the algorithm to realize real-time detection and positioning of the traffic cone cylinder.

5. The machine vision-based dual-arm cone cylinder automatic pick-and-place method according to claim 1, wherein, The position sensor and / or distance sensor is any one of an infrared position sensor and an ultrasonic sensor.

6. The machine vision-based dual-arm cone cylinder automatic pick-and-place method according to claim 1, wherein, The rotating mechanical arm (28) is arranged in the middle of the turnover support (25), and a power part capable of driving the rotating mechanical arm (28) to rotate is arranged in the turnover support (25). The rotating mechanical arm (28) is fixed with two groups of scissor type mechanical hands (4).

7. The machine vision-based dual-arm cone cylinder automatic pick-and-place method according to claim 1, wherein, The scissor type mechanical hand (4) is arranged with an electric push-pull rod (16), and the hinge rod (20) is hinged on the telescopic shaft of the electric push-pull rod (16). The hinge rod (20) is hinged with two groups of clamps (18). The electric push-pull rod (16) can drive the two groups of clamps (18) to clamp the cone cylinder.

8. The machine vision-based dual-arm cone cylinder automatic pick-and-place method according to claim 1, wherein, The basic support is also arranged with a touch rod (9).

Images