Automatic folding and unfolding method for double-arm traffic cone based on machine vision

Abstract

The invention relates to the field of road construction traffic control assistance, in particular to a double-arm traffic cone cylinder automatic folding and unfolding method based on machine vision. Machine vision shooting recognizes the conical cylinder through a vision sensing system and image analysis; automatically tracking and dynamically measuring the contour dimension of the cone; once the conical cylinder enters the included angle between the scissor type manipulators, the feeler lever captures and righting the conical cylinder, and the sensor below the clamp sends a feedback signal to the programmable controller; and then the rotary mechanical arm moves downwards and circulates in a reciprocating mode, the rotary mechanical arm is controlled to grab the conical cylinders, convey the conical cylinders up and down and steer, so that collection work of the conical cylinders of different specifications and sizes is completed, and therefore the automation level of the automatic conical cylinder collecting and releasing machine is enhanced, and meanwhile the working efficiency and the driving safety coefficient of a driver are improved.

Description

Technical Field

[0001] The invention relates to the field of highway construction traffic control assistance, and in particular to a method for automatically retracting and releasing double-arm traffic cones based on machine vision. Background Art

[0002] Traffic cones are hollow cones made of plastic or rubber, used for road closures or emergency construction, as well as lane separation and diversion work signs. By the end of 2020, the total mileage of national highways open to traffic reached 5.1981 million kilometers, and the national expressways reached 161,000 kilometers, ranking first in the world.

[0003] Highways need to be maintained on a daily basis, and most maintenance operations are carried out without interrupting traffic. Due to the large volume of traffic on highways and the high speed of vehicles, there are great safety risks during operations. Traffic cones need to be placed in advance on the working section to divide the work area and achieve the purpose of diverting vehicles on the highway.

[0004] Highway maintenance is characterized by heavy workload and high risk, and is a high-risk operation. At present, most domestic and foreign highway traffic systems still use manual placement and collection of traffic cones, which is not only time-consuming and labor-intensive, but also inefficient and environmentally dangerous. Therefore, in order to improve the efficiency of highway maintenance work and reduce the probability of safety accidents among highway maintenance workers, the automatic retracting and releasing equipment for highway maintenance road traffic cones has gradually attracted close attention from research institutions and enterprises.

[0005] Domestic cone deployment and recovery equipment has achieved certain results, but there are still many areas for improvement, as follows:

[0006] First, most of them use hydraulic or pneumatic methods to control the manipulator to grasp and place. The overall design of the product is too complicated, which not only increases the design cost of the entire device, occupies a large volume, and has high application and maintenance costs, but also reduces the efficiency of placement due to the overly complex motion trajectory.

[0007] Second, they are basically products with a single cone placement function. For example, the semi-automatic cone collection and placement vehicle with a slide rail slides the cones out from the slideway connected to the rear of the vehicle to complete the cone road placement work, but it requires manual collection of cones from the road, and the product performance is single.

[0008] Third, there are few research results on cone machine vision recognition. Cones are completely located and collected by the driver according to the vehicle's route, which not only greatly distracts the driver's attention and reduces driving safety, but also makes it difficult to collect cones of different sizes in a universal manner.

[0009] The current cone deployment, release and recovery equipment can no longer meet the needs of modern highway maintenance operations, and its work efficiency, practicality, reliability, automation and intelligence still need to be improved.

[0010] On September 19, 2024, a search was conducted in the Chinese Patent Publication Database with "cone and vision and image and sensor and recognition" as the abstract keywords, and the option of allowing synonym expansion was checked, but no relevant literature was found.

[0011] On September 19, 2024, an abstract search for "cone and vision and image and sensor and recognition" was conducted on China National Knowledge Infrastructure, and no relevant literature was found.

[0012] On September 19, 2024, a search was conducted on the website of the United States Patent and Trademark Office for "Cone with vision with image with sensor with recognition", and no relevant literature was found; the search URL was https: / / ppubs.uspto.gov / pubwebapp / .

[0013] On September 19, 2024, a search was conducted on the Korean Intellectual Property Office for “Cone and vision and image and sensor and recognition”; search URL

[0014] http: / / eng.kipris.or.kr / enghome / main.jsp. Twelve irrelevant documents were found: Applicant: Cornell University Application Number: 1020137005335 Angle-sensitive pixel (ASP)-based image processing systems, methods, and applications;

[0015] Applicant: Aurora Flight Sciences Corporation Application Number: 1020180071987 System and method for detecting obstacles in an aerial system Report System and method for detecting obstacles in an aerial system;

[0016] Applicant: Microsoft Technology Licensing, LLC Application Number: 1020147011240 Video display modification based on sensor input to achieve transparent near-eye display Report Illustration of video display modification based on sensor input for a see-through near-eye display.

[0017] Applicant: EYEFLUENCE, INC. Application Number: 1020167034649 Systems and methods for biomechanically based eye signaling for interacting with real and virtual objects Report Systems and methods for biomechanically based eye signaling for interacting with real and virtual objects

[0018] Applicant: BLACKMORE SENSORS AND ANALYTICS, LLC Application Number: 1020197018575 Method and system for classifying objects in a point cloud dataset An illustration of a method and system for classifying objects in a point cloud dataset is reported.

[0019] Applicant: PRIMESENSE LTD. Application number: 1020087008228 Method and system for object reconstruction Report drawings of method and system for object restoration;

[0020] Applicant: BLACKMORE SENSORS AND ANALYTICS, LLC Application Number: 1020227009625 Method and system for classifying objects in a point cloud dataset An illustration of a method and system for classifying objects in a point cloud dataset is reported.

[0021] Applicant: Navigate Surgical Technologies, Inc. Application Number: 1020157015076 Surgical Position Monitoring System and Method Using Natural Markers Reporting on the Mapping Industrial Chain of Surgical Position Monitoring System and Method Using Natural Markers;

[0022] Applicant: Motional AD LLC Application number: 1020200149202; Sequential fusion report of 3D object detection Sequential fusion diagram of 3D object detection Industrial control computer;

[0023] Applicant: Glaxo Group Limited Application Number: 1020107015248 Antigen Binding Protein Trial Report Antigen Binding Protein Map;

[0024] Applicant: Glaxo Group plc Application number: 1020157022610 Antigen Binding Protein Report Antigen Binding Protein Figure.

[0025] On September 19, 2024, a search was conducted on WIPO's https: / / patentscope2.wipo.int / with "Cone and vision and image and sensor and recognition", and the following unrelated documents were found: 102016001070732 Method for solving the mirror parameters of a cone mirror catadioptric camera using a straight line; 201611018620.3 Surround view providing device for vehicle and vehicle; 102017000014391 Steering device and vehicle; 200510009294.5 Video tape playback device two memory; 02130013.5 Visual audio tape graphic data generation and encoding method and graphic data playback device; PCT / CN2002 / 000549 TELETEXT DATA GENERATING AND ENCODINGMETHOD ON THE AUDIOVISUAL MAGNETIC TAPE&THE TELETEXT DATA PLAYER. 。

[0026] On September 19, 2024, a search was conducted on the Japan Patent Office website https: / / www.j-platpat.inpit.go.jp / for "Cone and vision and image and sensor and recognition", but no relevant literature was found.

[0027] Completely different from the concept of this patent. Summary of the invention

[0028] Purpose of the invention: To provide a better method for automatically retracting and releasing double-arm traffic cones based on machine vision. For specific purposes, see the multiple substantial technical effects in the specific implementation section.

[0029] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0030] The automatic retracting and releasing method of double-arm traffic cones based on machine vision is characterized in that:

[0031] The following steps are included:

[0032] The machine vision camera identifies the cone through the visual sensing system and image analysis;

[0033] Automatically track and dynamically measure the contour dimensions of the cone;

[0034] Once the cone enters the angle between the scissor-type manipulators, the feeler catches and straightens the cone, and the sensor under the fixture sends a feedback signal to the programmable controller;

[0035] The programmable controller will use the servo drive to dynamically adjust the manipulator's outreach height, lifting height, and electronic push rod opening angle according to the cone's position and contour information. The original upright cone on the road will be automatically introduced into the cone through the double-arm guide adjustment mechanism in different positions and postures. The electric push-pull rod piston rod will retract, the fixture will clamp the cone, and the manipulator will move upward.

[0036] When the robot arm moves to the upper limit photoelectric switch, the robot arm stops moving upward, the piston rod of the electric push-pull rod extends forward, the clamp opens, and the operator collects the cones and places them in the car;

[0037] The rotating robotic arm then moves downward in a reciprocating cycle, controlling the rotating robotic arm to grab the cones and transport them up and down and turn them around, so as to complete the collection of cones of different specifications and sizes. This not only enhances the automation level of the cone automatic retracting and unretracting machine, but also improves work efficiency and the safety factor of the driver.

[0038] The further technical solution of the present invention is that the hardware of the traffic cone machine vision positioning system is composed of a real-time imaging camera, a Raspberry Pi microcomputer and a lighting fill light system, and the software uses a convolutional neural network algorithm. Deep learning is performed with traffic cones as the target, and a weight file is generated through training. The weight file with the best effect is selected and deployed on the Raspberry Pi microcomputer; after the device is turned on, the camera supporting the Raspberry Pi can be called for real-time imaging. After a single recovery or deployment is completed, the camera starts to capture images and identify and locate the position of the traffic cone, and transmits the position information to the PLC control unit. The control system sends a signal to move the fixture to 2 / 3 of the height of the traffic cone. When the infrared sensor in the middle of the fixture acts to feedback the signal, the electric push rod acts to clamp the fixture.

[0039] A further technical solution of the present invention is that, since camera imaging is affected by factors such as light, physical fill light is usually required under the above-mentioned extreme conditions to ensure the clarity of the camera imaging. Therefore, a lighting fill light system is provided, and a fill light system switch is provided on the operation panel, so that workers can control the operation of the fill light system through simple operations; the fill light system is light fill light.

[0040] A further technical solution of the present invention is that the weight file used for traffic cone identification and positioning is generated by Yolov5 convolutional neural network; Yolov5 convolutional neural network uses convolutional neural network to execute and complete the task of target detection and classification; the network structure consists of four parts: Input, Backbone, Neck, and Head. Mosaic, Copy paste, Random affine affine transformation, MixUp, and Cutout are used at the Input input end to enhance the data set, thereby improving the diversity of the data set, the robustness of the model, and the generalization ability; the Focus network in the Backbone module slices the input image to obtain a feature layer, and then passes it to CSPDarknet to obtain three different effective feature layers through convolution, normalization, and SiLu activation function operations; the three effective feature layers obtained from Backbone are passed to the Neck module, and upsampled by the feature pyramid FPN: stacked and feature extracted with the other two effective feature layers, downsampled by the path aggregation network PAN after upsampling, and finally the prediction result is obtained, and the weight file is generated; finally, the weight file with the best recognition effect is selected and burned to the Raspberry On the Pi microcomputer, the algorithm is used to call the camera and weight file to achieve real-time detection and positioning of traffic cones.

[0041] A further technical solution of the present invention is that the method is implemented based on a double-arm traffic cone automatic retracting device, and the double-arm traffic cone automatic retracting device has the following structure: a fixed double-arm traffic cone automatic retracting device; the fixed double-arm traffic cone automatic retracting device has the following structure:

[0042] It comprises a basic bracket 15, a lifting bracket 8 is arranged on the vertical track of the basic bracket 15, two groups of scissor-type manipulators 4 are arranged on the lifting bracket 8, and a thread block is also arranged on the lifting bracket 8, the thread block and the lifting screw 10 are threadedly connected, and the lifting screw 10 is a workbench lifting motor 11; the lifting bracket 8 can be lifted and lowered as a whole;

[0043] A collecting and fixing frame 13 is arranged on the basic support 15; the collecting and fixing frame 13 can accommodate multiple groups of overlapping cone structures;

[0044] The basic bracket 15 is provided with a fixed extrusion structure 14 capable of fixing the basic bracket on the carriage; the fixed extrusion structure 14 rotates forward to squeeze the basic bracket 15 on the carriage;

[0045] A camera is fixed on the basic bracket 15 for acquiring image data;

[0046] A plurality of position sensors and / or distance sensors are arranged on the basic support 15 , and the position sensors and / or distance sensors can obtain the height and position of the scissor-type manipulator 4 ; at the same time, the position sensors and / or distance sensors can obtain the position of the cone on the ground.

[0047] A further technical solution of the present invention is that the method is implemented based on a double-arm traffic cone automatic retracting and releasing device, and the double-arm traffic cone automatic retracting and releasing device has the following structure: a retractable double-arm traffic cone automatic retracting and releasing device;

[0048] It comprises a basic bracket 15, a lifting bracket 8 is arranged on the vertical track of the basic bracket 15, two groups of scissor-type manipulators 4 are arranged on the lifting bracket 8, and a thread block is also arranged on the lifting bracket 8, the thread block and the lifting screw 10 are threadedly connected, and the lifting screw 10 is a workbench lifting motor 11; the lifting bracket 8 can be lifted and lowered as a whole;

[0049] A telescopic structure 22 is fixed on the lifting bracket 8, the upper part of the telescopic structure 22 is hinged on the upper plane 23 of the flip structure, and the bottom of the telescopic structure 22 is hinged on the flip bracket 25, and the flip bracket 25 is located on the lifting bracket 8; two groups of scissor-type manipulators 4 are located on the flip bracket 25; the flip bracket 25 can also rotate around the rotating shaft 27 of the overall structure of the fixture;

[0050] The bracket on which the handle 1 is located can rotate around the flip bracket rotation axis 26;

[0051] A camera is fixed on the basic bracket 15 for acquiring image data;

[0052] A plurality of position sensors and / or distance sensors are arranged on the basic support 15 , and the position sensors and / or distance sensors can obtain the height and position of the scissor-type manipulator 4 ; at the same time, the position sensors and / or distance sensors can obtain the position of the cone on the ground.

[0053] A further technical solution of the present invention is that the position sensor and / or the distance sensor is any one of an infrared position sensor and an ultrasonic sensor.

[0054] A further technical solution of the present invention is that a rotating mechanical arm 28 is arranged in the middle of the flip bracket 25, a power part capable of driving the rotating mechanical arm 28 to rotate is arranged inside the flip bracket 25, and two groups of scissor-type manipulators 4 are fixed to the rotating mechanical arm 28.

[0055] A further technical solution of the present invention is that the scissor-type manipulator 4 is provided with an electric push-pull rod 16, a hinged rod 20 is hinged on the telescopic shaft of the electric push-pull rod 16, and the hinged rod 20 is hinged to two groups of clamps 18; the electric push-pull rod 16 can drive the two groups of clamps 18 to clamp the cone.

[0056] A further technical solution of the present invention is that a feeler rod 9 is also arranged on the basic support.

[0057] The present invention adopting the above technical solution has the following beneficial effects compared with the prior art: the vehicle does not need to be modified in any way, and the cone automatic retracting and releasing machine can be hung on the side of the vehicle to perform the retracting and releasing operation, which is simple to operate and has a high degree of automation and intelligence. The innovative use of visual sensing means that people do not need to bend down to check the cones at any time when they are in a moving vehicle or on dangerous roads. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to further illustrate the present invention, the following is further described in conjunction with the accompanying drawings:

[0059] Figure 1 It is a structural schematic diagram of the present invention;

[0060] Figure 2 It is a rear view structural schematic diagram of the present invention;

[0061] Figure 3 It is a schematic diagram of the structure of the manipulator clamp of the present invention;

[0062] Figure 4 It is a schematic diagram of the network structure of the control system of the present invention;

[0063] Figure 5 It is a work flow chart of the present invention;

[0064] Figure 6 It is a schematic diagram of the machine vision recognition structure of the present invention;

[0065] Figure 7 A flow chart of identifying and locating traffic cones according to the present invention;

[0066] Figure 8 A perspective view of the present invention;

[0067] Fig. 9 A schematic diagram of another viewing angle of the present invention;

[0068] Fig.10 It is a schematic diagram of the local structure of the present invention;

[0069] Fig.11 Example diagrams for camera and sensor installation;

[0070] Fig.12 It is a structural diagram of a fixed extrusion structure;

[0071] In the figure: 1. Handle; 2. Control box; 3. Motor box; 4. Scissor-type manipulator; 5. Rotating manipulator arm; 6. Machine vision camera; 7. Lifting guide groove; 8. Lifting bracket; 9. Touch rod; 10. Lifting screw; 11. Workbench lifting motor; 12. Base frame; 13. Collecting fixed frame; 14. Fixed extrusion structure; 15. Basic bracket; 16. Electric push-pull rod; 17. Induction probe; 18. Clamp; 19. Locking nut; 20. Articulated rod; 21 Infrared sensor; 22. Telescopic structure; 23. Upper plane; 24. Flip bracket; 25. Lifting bracket fixing plate; 26. Flip bracket shaft; 27. Rotating shaft of the overall structure of the fixture; 28. Rotating manipulator arm; 29. ​​Camera and sensor; 30. Removable base. DETAILED DESCRIPTION

[0072] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. In the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", "top", "bottom", etc. is based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal connection of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood in specific circumstances.

[0073] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0074] This patent provides a variety of parallel solutions. The different expressions are improved solutions or parallel solutions based on the basic solution. Each solution has its own unique characteristics. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The fixing method not described in the text can be any fixing method such as thread fixing, bolt fixing or glue bonding.

[0075] Embodiment 1: In conjunction with all the accompanying drawings; a method for automatically retracting and releasing double-arm traffic cones based on machine vision, characterized in that:

[0076] The following steps are included:

[0077] The machine vision camera identifies the cone through the visual sensing system and image analysis;

[0078] Automatically track and dynamically measure the contour dimensions of the cone;

[0079] Once the cone enters the angle between the scissor-type manipulators, the feeler catches and straightens the cone, and the sensor under the fixture sends a feedback signal to the programmable controller;

[0080] The programmable controller will use the servo drive to dynamically adjust the manipulator's outreach height, lifting height, and electronic push rod opening angle according to the cone's position and contour information. The original upright cone on the road will be automatically introduced into the cone through the double-arm guide adjustment mechanism in different positions and postures. The electric push-pull rod piston rod will retract, the fixture will clamp the cone, and the manipulator will move upward.

[0081] When the robot arm moves to the upper limit photoelectric switch, the robot arm stops moving upward, the piston rod of the electric push-pull rod extends forward, the clamp opens, and the operator collects the cones and places them in the car;

[0082] The rotating robotic arm then moves downward in a reciprocating cycle, controlling the rotating robotic arm to grab the cones and transport them up and down and turn them around, so as to complete the collection of cones of different specifications and sizes. This not only enhances the automation level of the cone automatic retracting and unretracting machine, but also improves work efficiency and the safety factor of the driver.

[0083] The substantial technical effects and implementation process of the technical solution hereof are as follows:

[0084] In view of the current situation in which domestic traffic maintenance and road maintenance industries mainly rely on manual collection and deployment of cones, which has problems such as high risk, many casualties, and slow operation speed, the purpose of this patent is to provide a double-arm traffic cone automatic collection and deployment machine based on machine vision to solve the problem of low driving safety factor when collecting cones.

[0085] The technical solution adopted by the patent of this invention is that the mechanical structure mainly includes a frame bearing device and a cone guide and retracting system. The retracting machine is installed on the left or right side of the vehicle. The manipulator automatically introduces the cone through the double-arm guide adjustment mechanism. The servo drive system automatically captures the cone signal and controls the double-arm manipulator to capture the cone. The manipulator transports and turns the cone on the mechanical body according to the set program. The cone retracting operation can be completed when the vehicle is driving forward or reverse. The traffic cone machine vision positioning system uses a convolutional neural network algorithm, takes traffic cones as the object, and makes a self-made traffic cone position data set. The Yolov5 network is used as a deep convolutional neural network for training, and the best training weights are selected. The traffic cones are recognized and positioned with this weight, so that the camera can dynamically obtain the cone position and contour information at any time during the movement of the cone automatic retracting machine.

[0086] The electrical control system is based on a programmable controller and is equipped with operation panels, machine vision systems, servo drives, sensor probes and other modules. Figure 4 As shown in the figure, the entire control system is divided into the equipment layer, control layer and management layer.

[0087] The equipment layer includes various sensors, photoelectric switches, connecting rod motors, push-pull rod motors, workbench lifting motors, running / retracting motors, various motor drivers, etc. The lifting motor is used to control the left and right swing of the rotating mechanical arm; the push-pull rod motor controls the piston rod of the electric push-pull rod to move forward or backward, and controls the scissor-type mechanical clamping or releasing the tightening cone; the workbench lifting motor is used to control and adjust the height of the scissor-type mechanical arm from the ground; the running / retracting motor is used to place and store the workbench.

[0088] The control layer uses a programmable controller as the central controller, which plays the role of cone collection and placement action logic control management, machine vision information collection and processing, and execution of the whole machine coordination control strategy.

[0089] The management layer includes the operation panel and the machine vision part. The operation panel is equipped with buttons and switches such as start and stop, run / retract, rise / fall, collect / release cones, left arm / right arm, left light / right light, etc. When collecting cones, the machine vision captures the cone signal and transmits it to the central controller and automatically grabs it. The double-arm balance bar rotates to the photoelectric switch and stops rotating, completing the cone collection; when placing cones, the cones are placed in the robot, and the double-arm balance bar rotates to the photoelectric switch and stops rotating, completing the cone placement.

[0090] The working process of the cone automatic retracting and unreeling machine is as follows: Figure 5As shown. The cone automatic retractor is mounted on both sides of the pickup truck compartment. The operator can select the operation direction and mode through the buttons on the operation panel such as start / stop, run / retract, rise / fall, retract / release, left arm / right arm, left light / right light, etc. The operation direction is left and right, and one of the two scissor-hand manipulators can be used to retract and release the cones; the operation mode is divided into placing cones and collecting cones.

[0091] The deployment operation process is as follows: when the rotating robotic arm moves to the upper limit photoelectric switch, the piston rod of the electric push-pull rod extends forward, the scissor-type robotic arm opens, and the operator places the traffic cone in the scissor-type robotic arm.

[0092] The infrared sensor under the scissor-type manipulator sends a feedback signal to the programmable controller, the piston rod of the electric push-pull rod retracts, and the scissor-type manipulator clamps the cone.

[0093] The rotating mechanical arm moves downward, and when the rotating mechanical arm moves to the lower limit photoelectric switch, the lower limit photoelectric switch sends a feedback signal to the programmable controller, and the scissor-type manipulator releases the cone, and the touch rod straightens the cone and places it on the ground. Then, the rotating mechanical arm moves upward, and the reciprocating cycle is completed to complete the cone placement operation.

[0094] The collection operation process is as follows: when the rotating robotic arm descends to the lower limit photoelectric switch, the machine vision camera begins to automatically track and dynamically measure the contour dimensions of the cone. Once the cone enters the angle between the scissor-type robotic arms, the feeler rod captures and straightens the cone, and the sensor under the fixture sends a feedback signal to the programmable controller.

[0095] The programmable controller will use the servo drive to dynamically adjust the robot's outreach height, lifting height and electronic push rod angle according to information such as the cone's position and contour. The original upright cones on the road will be automatically introduced into the cones in different positions and postures through the double-arm guide adjustment mechanism. The electric push-pull rod piston rod will retract, the fixture will clamp the cone, and the robot arm will move upward.

[0096] When the robotic arm moves to the upper limit photoelectric switch, the robotic arm stops moving upward, the piston rod of the electric push-pull rod extends forward, the clamp opens, and the operator collects the cones and places them in the car.

[0097] The rotating robotic arm then moves downward in a reciprocating cycle, controlling the rotating robotic arm to grab the cones and transport them up and down and turn them around, so as to complete the collection of cones of different specifications and sizes. This not only enhances the automation level of the cone automatic retracting and unretracting machine, but also improves work efficiency and the driver's driving safety factor, as well as the universal collection of cones of different specifications and sizes.

[0098] Embodiment 2: As a further improved solution or parallel solution or optional independent solution, the hardware of the traffic cone machine vision positioning system consists of a real-time imaging camera, a Raspberry Pi microcomputer and a lighting fill light system, and the software uses a convolutional neural network algorithm. Deep learning is performed with traffic cones as the target, and a weight file is generated through training. The weight file with the best effect is selected and deployed on the Raspberry Pi microcomputer; after the device is turned on, the camera supporting the Raspberry Pi can be called for real-time imaging. After a single recovery or deployment is completed, the camera starts to capture images and identify and locate the position of the traffic cone, and transmits the position information to the PLC control unit. The control system sends a signal to move the fixture to 2 / 3 of the height of the traffic cone. When the infrared sensor in the middle of the fixture acts on the feedback signal, the electric push rod acts on the fixture to clamp it.

[0099] The substantial technical effects and implementation process of the technical solution hereof are as follows:

[0100] The system components for identifying and locating traffic cones are as follows: Figure 6 As shown in the figure, the hardware of the traffic cone machine vision positioning system consists of a real-time imaging camera, a Raspberry Pi microcomputer, and a lighting system. The software uses a convolutional neural network algorithm. It performs deep learning with traffic cones as the target and generates a weight file through training.

[0101] Embodiment 3: As a further improved scheme or a parallel scheme or an optional independent scheme, since the camera imaging is affected by factors such as lighting, physical fill light is usually required under the above extreme conditions to ensure the clarity of the camera imaging. Therefore, a lighting fill light system is set up, and a fill light system switch is set on the operation panel. Workers can control the operation of the fill light system through simple operations; the fill light system is light fill light.

[0102] The weight file used for traffic cone recognition and positioning is generated by Yolov5 convolutional neural network; Yolov5 convolutional neural network uses convolutional neural network to perform and complete the task of target detection and classification; the network structure consists of four parts: Input, Backbone, Neck, and Head. Mosaic, Copy paste, Random affine, MixUp, and Cutout are used at the Input input to enhance the data set, thereby improving the diversity of the data set, the robustness of the model, and the generalization ability; the Focus network in the Backbone module slices the input image to obtain the feature layer, and then passes it to CSPDarknet to obtain three different effective feature layers through convolution, normalization, and SiLu activation function operations; the three effective feature layers obtained from Backbone are passed to the Neck module and upsampled through the feature pyramid FPN: stacked and feature extracted with the other two effective feature layers, and downsampled through the path aggregation network PAN after upsampling, and finally the prediction result is obtained, and the weight file is generated; finally, the weight file with the best recognition effect is selected and burned to the Raspberry On the Pi microcomputer, the algorithm is used to call the camera and weight file to achieve real-time detection and positioning of traffic cones.

[0103] Embodiment 4: As a further improved solution or parallel solution or optional independent solution, the method is implemented based on a double-arm traffic cone automatic retracting device, and the double-arm traffic cone automatic retracting device has the following structure: a fixed double-arm traffic cone automatic retracting device; the fixed double-arm traffic cone automatic retracting device has the following structure:

[0104] It comprises a basic bracket 15, a lifting bracket 8 is arranged on the vertical track of the basic bracket 15, two groups of scissor-type manipulators 4 are arranged on the lifting bracket 8, and a thread block is also arranged on the lifting bracket 8, the thread block and the lifting screw 10 are threadedly connected, and the lifting screw 10 is a workbench lifting motor 11; the lifting bracket 8 can be lifted and lowered as a whole;

[0105] A collecting and fixing frame 13 is arranged on the basic support 15; the collecting and fixing frame 13 can accommodate multiple groups of overlapping cone structures;

[0106] The basic bracket 15 is provided with a fixed extrusion structure 14 capable of fixing the basic bracket on the carriage; the fixed extrusion structure 14 rotates forward to squeeze the basic bracket 15 on the carriage;

[0107] A camera is fixed on the basic bracket 15 for acquiring image data;

[0108] A plurality of position sensors and / or distance sensors are arranged on the basic support 15 , and the position sensors and / or distance sensors can obtain the height and position of the scissor-type manipulator 4 ; at the same time, the position sensors and / or distance sensors can obtain the position of the cone on the ground.

[0109] The substantial technical effects and implementation process of the technical solution here are as follows: Figure 1 , Figure 2 , Figure 3 ; The structure is a lifting scissor-type manipulator 4, and the schematic diagram of the manipulator fixture structure is as follows Figure 3 As shown. The rear of the scissor-type manipulator is connected to an electric push-pull rod. The movement of the piston rod of the electric push-pull rod can drive the clamp to move, and the scissor-type manipulator clamps the cone. When the piston rod of the electric push-pull rod moves forward, the scissor-type manipulator opens to release the cone. The locking nut is used to fix the clamp. The ultrasonic sensor is used to measure the distance between the manipulator and the cone. When a certain distance is reached, the feeler rod will automatically capture and collect the cone.

[0110] Product structure diagram Figure 1 As shown. The handle is easy to move and hang on both sides of the vehicle. The entire workbench is L-structured. The control box integrates a programmable controller, a motor driver, and a machine vision circuit. The table is equipped with buttons such as start and stop, run / retract, rise / descend, retract / release, left arm / right arm, and left light / right light to analyze the control instructions.

[0111] The scissor-type manipulator is connected to an electric push-pull rod, the piston rod of which drives the scissor hand to move. The scissor hand can be retracted to clamp or place cones, so as to complete the collection and placement of cones of different specifications and sizes in the front, rear, left and right directions by using the vehicle to travel in different directions, and integrate the collected cones in the car. The collection and placement machine is arranged with a base and rollers for towing the equipment.

[0112] The base support 15 may be arranged on a removable base 30 .

[0113] Embodiment 5: As a further improved solution or parallel solution or optional independent solution, the method is implemented based on a double-arm traffic cone automatic retracting device, and the double-arm traffic cone automatic retracting device has the following structure: a retractable double-arm traffic cone automatic retracting device;

[0114] It comprises a basic bracket 15, a lifting bracket 8 is arranged on the vertical track of the basic bracket 15, two groups of scissor-type manipulators 4 are arranged on the lifting bracket 8, and a thread block is also arranged on the lifting bracket 8, the thread block and the lifting screw 10 are threadedly connected, and the lifting screw 10 is a workbench lifting motor 11; the lifting bracket 8 can be lifted and lowered as a whole;

[0115] A telescopic structure 22 is fixed on the lifting bracket 8, the upper part of the telescopic structure 22 is hinged on the upper plane 23 of the flip structure, and the bottom of the telescopic structure 22 is hinged on the flip bracket 25, and the flip bracket 25 is located on the lifting bracket 8; two groups of scissor-type manipulators 4 are located on the flip bracket 25; the flip bracket 25 can also rotate around the rotating shaft 27 of the overall structure of the fixture;

[0116] The bracket on which the handle 1 is located can rotate around the flip bracket rotation axis 26;

[0117] A camera is fixed on the basic bracket 15 for acquiring image data;

[0118] A plurality of position sensors and / or distance sensors are arranged on the basic support 15 , and the position sensors and / or distance sensors can obtain the height and position of the scissor-type manipulator 4 ; at the same time, the position sensors and / or distance sensors can obtain the position of the cone on the ground.

[0119] The substantial technical effects and implementation process of the technical solution here are as follows: Figure 8 , Fig. 9 , Fig.10 , Fig.11 The motor box is equipped with multiple motors. The rotation of the rotary robot arm can drive the rocker arm to swing left and right, and the two scissor-type manipulators work alternately.

[0120] The machine vision camera automatically tracks and dynamically measures the outer contour dimensions of the cone. Once the cone enters the angle between the manipulator clamps, the cone signal is transmitted to the control system. The control system uses the servo drive to dynamically adjust the manipulator's outreach height, lifting height, and electronic push rod angle based on the cone position, direction, and contour information. The original upright cones on the road are automatically introduced into the cones through the double-arm guide adjustment mechanism in different positions and postures, and the rotating manipulator is controlled to grab the cones and transport them up and down and turn them to complete the collection of cones of different specifications and sizes, thereby enhancing the automation level of the cone automatic retractor while improving work efficiency and the safety factor of the driver's driving.

[0121] The rear view structure diagram of the cone automatic retracting and unretracting machine is as follows Figure 2 When running,

[0122] The height of the manipulator can be controlled. The lifting motor controls the lifting bracket through the screw and adjusts the manipulator to a suitable height through the lifting guide groove. The operator can manually control the working height through the button, and the touch rod is used to capture and straighten the cone.

[0123] Embodiment 6: As a further improved solution or parallel solution or optional independent solution, the position sensor and / or distance sensor is any one of an infrared position sensor and an ultrasonic sensor. The substantial technical effect of the technical solution here and its implementation process are as follows: Similar sensors are within the protection scope of this patent.

[0124] Embodiment 7: As a further improved solution or parallel solution or optional independent solution, a rotating mechanical arm 28 is arranged in the middle of the flip bracket 25, and a power part capable of driving the rotating mechanical arm 28 to rotate is arranged inside the flip bracket 25, and two sets of scissor-type manipulators 4 are fixed to the rotating mechanical arm 28. The substantial technical effect and implementation process of the technical solution here are as follows: the rotating mechanical arm 28 is used for the two scissor-type manipulators 4 to work alternately.

[0125] Embodiment 8: As a further improved solution or parallel solution or optional independent solution, the scissor-type manipulator 4 is arranged with an electric push-pull rod 16, and a hinge rod 20 is hinged on the telescopic shaft of the electric push-pull rod 16, and the hinge rod 20 is hinged to two sets of clamps 18; the electric push-pull rod 16 can drive the two sets of clamps 18 to clamp the cone. The substantial technical effect and implementation process of the technical solution here are as follows: it can realize clamping, and similar manipulators are within the protection scope of this patent.

[0126] Embodiment 9: As a further improved solution or a parallel solution or an optional independent solution, a touch rod 9 is also arranged on the basic bracket.

[0127] Innovatively, each of the above effects exists independently, and a set of structures can be used to combine the above results.

[0128] It should be noted that the multiple solutions provided by this patent include the basic solution itself, which are independent of each other and do not restrict each other, but they can also be combined with each other without conflict to achieve multiple effects together.

[0129] The basic principle, main features and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the protection required.

Claims

1. The automatic retracting and releasing method of double-arm traffic cones based on machine vision is characterized in that: The following steps are included: The machine vision camera identifies the cone through the visual sensing system and image analysis; Automatically track and dynamically measure the contour dimensions of the cone; Once the cone enters the angle between the scissor-type manipulators, the feeler catches and straightens the cone, and the sensor under the fixture sends a feedback signal to the programmable controller; The programmable controller will use the servo drive to dynamically adjust the manipulator's outreach height, lifting height, and electronic push rod opening angle according to the cone's position and contour information. The original upright cone on the road will be automatically introduced into the cone through the double-arm guide adjustment mechanism in different positions and postures. The electric push-pull rod piston rod will retract, the fixture will clamp the cone, and the manipulator will move upward. When the robot arm moves to the upper limit photoelectric switch, the robot arm stops moving upward, the piston rod of the electric push-pull rod extends forward, the clamp opens, and the operator collects the cones and places them in the car; The rotating robotic arm then moves downward in a reciprocating cycle, controlling the rotating robotic arm to grab the cones and transport them up and down and turn them around, so as to complete the collection of cones of different specifications and sizes. This not only enhances the automation level of the cone automatic retracting and unretracting machine, but also improves work efficiency and the safety factor of the driver.

2. The method for automatically releasing and retracting double-arm traffic cones based on machine vision as claimed in claim 1, characterized in that: The hardware of the traffic cone machine vision positioning system consists of a real-time imaging camera, a Raspberry Pi microcomputer, and a lighting fill light system. The software uses a convolutional neural network algorithm. Deep learning is performed on the traffic cone as the target, and a weight file is generated through training. The weight file with the best effect is selected and deployed on the Raspberry Pi microcomputer. After the device is turned on, the camera supporting the Raspberry Pi can be called for real-time imaging. After a single recovery or deployment is completed, the camera starts to capture images and identify and locate the position of the traffic cone, and transmits the position information to the PLC control unit. The control system sends a signal to move the fixture to 2 / 3 of the height of the traffic cone. When the infrared sensor in the middle of the fixture acts on the feedback signal, the electric push rod acts on the fixture to clamp it.

3. The method for automatically releasing and retracting double-arm traffic cones based on machine vision as claimed in claim 2, characterized in that: Since camera imaging is affected by factors such as lighting, physical fill light is usually required under the above-mentioned extreme conditions to ensure the clarity of camera imaging. Therefore, a lighting fill light system is set up, and a fill light system switch is set on the operation panel. Workers can control the operation of the fill light system through simple operations; the fill light system is light fill light.

4. The method for automatically releasing and retracting double-arm traffic cones based on machine vision as claimed in claim 2, characterized in that: The weight file used for traffic cone recognition and positioning is generated by the Yolov5 convolutional neural network; the Yolov5 convolutional neural network uses a convolutional neural network to perform and complete the task of target detection and classification; the network structure consists of four parts: Input, Backbone, Neck, and Head. Mosaic, Copy paste, and Random are used at the Input input. The data set is enhanced by using methods such as affine transformation, MixUp and Cutout to improve the diversity of the data set, the robustness and generalization ability of the model; the Focus network in the Backbone module slices the input image to obtain the feature layer, which is then passed to the CSPDarknet to obtain three different effective feature layers through convolution, normalization and SiLu activation function operations; the three effective feature layers obtained from the Backbone are passed to the Neck module and upsampled through the feature pyramid FPN: stacked and feature extracted with the other two effective feature layers, and downsampled through the path aggregation network PAN after the upsampling is completed, and finally the prediction result is obtained and the weight file is generated; finally, the weight file with the best recognition effect is selected and burned to the RaspberryPi microcomputer, and the camera and weight file are called by the algorithm to realize real-time detection and positioning of traffic cones.

5. The automatic retracting and releasing method of double-arm traffic cones based on machine vision as claimed in claim 1, characterized in that: The method is implemented based on a double-arm traffic cone automatic retracting device, and the structure of the double-arm traffic cone automatic retracting device is as follows: it is a fixed double-arm traffic cone automatic retracting device; the structure of the fixed double-arm traffic cone automatic retracting device is as follows: The invention comprises a basic support (15), a lifting support (8) is arranged on the vertical track of the basic support (15), two groups of scissor-type manipulators (4) are arranged on the lifting support (8), a thread block is also arranged on the lifting support (8), the thread block and the lifting screw (10) are threadedly connected, and the lifting screw (10) is a workbench lifting motor (11); the lifting support (8) can be lifted and lowered as a whole; A collecting and fixing frame (13) is arranged on the basic support (15); the collecting and fixing frame (13) can accommodate multiple groups of overlapping cone cylinder structures; A fixed extrusion structure (14) capable of fixing the basic bracket on the carriage is arranged on the basic bracket (15); the fixed extrusion structure (14) rotates forward to squeeze the basic bracket (15) on the carriage; A camera is fixed on the basic bracket (15) for acquiring image data; A plurality of position sensors and / or distance sensors are arranged on the basic support (15), and the position sensors and / or distance sensors can obtain the height and position of the scissor-type manipulator (4); at the same time, the position sensors and / or distance sensors can obtain the position of the cone on the ground.

6. The method for automatically releasing and retracting double-arm traffic cones based on machine vision as claimed in claim 1, characterized in that: The method is implemented based on a double-arm traffic cone automatic retracting and releasing device, and the double-arm traffic cone automatic retracting and releasing device has the following structure: a retractable double-arm traffic cone automatic retracting and releasing device; The invention comprises a basic support (15), a lifting support (8) is arranged on the vertical track of the basic support (15), two groups of scissor-type manipulators (4) are arranged on the lifting support (8), a thread block is also arranged on the lifting support (8), the thread block and the lifting screw (10) are threadedly connected, and the lifting screw (10) is a workbench lifting motor (11); the lifting support (8) can be lifted and lowered as a whole; A telescopic structure (22) is fixed on the lifting bracket (8); the upper part of the telescopic structure (22) is hinged on the upper plane (23) of the flip structure; the lower part of the telescopic structure (22) is hinged on the flip bracket (25); the flip bracket (25) is located on the lifting bracket (8); two groups of scissor-type manipulators (4) are located on the flip bracket (25); the flip bracket (25) can also rotate around the rotating shaft (27) of the overall structure of the clamp; The bracket on which the handle (1) is located can rotate around the flip bracket rotation axis (26); A camera is fixed on the basic bracket (15) for acquiring image data; A plurality of position sensors and / or distance sensors are arranged on the basic support (15), and the position sensors and / or distance sensors can obtain the height and position of the scissor-type manipulator (4); at the same time, the position sensors and / or distance sensors can obtain the position of the cone on the ground.

7. The method for automatically releasing and retracting double-arm traffic cones based on machine vision as claimed in claim 6, characterized in that: The position sensor and / or distance sensor is any one of an infrared position sensor and an ultrasonic sensor.

8. The method for automatically releasing and retracting double-arm traffic cones based on machine vision as claimed in claim 6, characterized in that: A rotating mechanical arm (28) is arranged in the middle of the flip bracket (25), a power part capable of driving the rotating mechanical arm (28) to rotate is arranged inside the flip bracket (25), and two groups of scissor-type mechanical arms (4) are fixed to the rotating mechanical arm (28).

9. The method for automatically releasing and retracting double-arm traffic cones based on machine vision as claimed in claim 6, characterized in that: The scissor-type manipulator (4) is provided with an electric push-pull rod (16), a hinge rod (20) is hinged on the telescopic shaft of the electric push-pull rod (16), and the hinge rod (20) is hinged to two groups of clamps (18); the electric push-pull rod (16) can drive the two groups of clamps (18) to clamp the cone.

10. The method for automatically releasing and retracting double-arm traffic cones based on machine vision as claimed in claim 6, characterized in that: A feeler rod (9) is also arranged on the basic support.

Images