A man-machine collaborative dynamic visual information intelligent adaptation device

Abstract

The application discloses a kind of man-machine collaborative dynamic visual information intelligent adaptation device, belong to man-machine collaborative dynamic visual information technical field, it includes box, lifting rotating mechanism, visual camera, production frame;The lifting rotating mechanism is installed on box.In the application, by the rotation of driving shaft of first belt pulley, transmission belt and second belt pulley driven by stepper motor, cooperate with the sliding constraint of slide bar and strip slot, so that visual camera can stably perform horizontal rotation action, while electric push rod drives slide bar to lift, drives top plate and visual camera to carry out vertical direction position adjustment, to realize the all-around dynamic visual tracking of mobile object of different height, different motion trajectory, to expand the effective monitoring range of visual sensor, can also adaptively adjust capture angle according to object motion state, ensure the continuity and accuracy of visual information acquisition, provide reliable high-quality visual data support for man-machine collaborative sorting.

Description

Technical Field

[0001] This invention belongs to the field of human-machine collaborative dynamic visual information technology, specifically a human-machine collaborative dynamic visual information intelligent adaptation device. Background Technology

[0002] Human-machine collaborative dynamic visual information intelligent adaptation devices belong to the field of industrial automation visual inspection, involving the technology of real-time dynamic capture and tracking of moving objects during sorting using visual sensors. In human-machine collaborative sorting operations, the vision system, as the "eyes" of the robot's end effector, undertakes the key tasks of real-time acquisition of visual information, identification of target objects, prediction of motion trajectories, and guidance of grasping operations. With the rapid development of intelligent manufacturing and flexible production lines, intelligent adaptation technology based on dynamic visual information has become a core support for achieving high-speed and accurate sorting. Through the linkage control of visual cameras and mechanical structures, it can effectively expand the monitoring range and improve the continuity and accuracy of visual information acquisition.

[0003] However, existing visual inspection devices have significant shortcomings in practical applications. On the one hand, traditional visual cameras mostly adopt fixed installation structures, and their viewing angle and height cannot be adaptively adjusted according to the object's motion state. This results in blind spots in visual tracking of moving objects, making it difficult to cover targets at different heights and with complex motion trajectories, and limiting the effective monitoring range of visual sensors. On the other hand, existing vision systems lack deep collaboration with mechanical actuators, resulting in delays in information transmission between target detection, dynamic tracking, and motion prediction. This makes it impossible to perform real-time position compensation for high-speed moving objects, leading to large dynamic errors during the grasping process and low efficiency in real-time processing of visual information. Consequently, it is difficult to meet the stringent requirements for the continuity and accuracy of visual data in high-speed human-machine collaborative sorting scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a human-machine collaborative dynamic visual information intelligent adaptation device to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a human-machine collaborative dynamic visual information intelligent adaptation device, comprising a box, a lifting and rotating mechanism, a visual camera, and a production rack;

[0006] The lifting and rotating mechanism is installed on the box and is used to drive the vision camera to dynamically capture the moving objects being sorted in real time.

[0007] The lifting and rotating mechanism includes a support plate, a stepper motor, a first pulley, a transmission belt, a second pulley, a drive shaft, a first slide rod, a top plate, an electric push rod, and a second slide rod.

[0008] The support plate is fixed to the housing, the stepper motor is fixed to the housing and also fixed to the support plate, the first pulley is fixed to the output end of the stepper motor, the transmission belt is meshed with the first pulley and the second pulley is meshed with the end of the transmission belt away from the first pulley, the first slide rod is fixed to the drive shaft and also slides in the groove inside the second pulley, and the top plate is fixed to the end of the drive shaft away from the second pulley, for driving the vision camera to rotate, and the vision camera is fixed to the drive shaft;

[0009] The electric push rod is fixed to the box body and also fixed to the support plate. The second slide rod is fixed to the output end of the electric push rod and is slidably connected to the box body. The end of the second slide rod away from the electric push rod is fixed to the top plate and is used to drive the visual camera to move up and down.

[0010] As a further preferred embodiment of this technical solution: both the drive shaft and the first slide rod are slidably connected to the housing, and both the drive shaft and the first slide rod are rotatably connected to the housing.

[0011] As a further preferred embodiment of this technical solution: a grooved rod is fixedly connected to the box body, and a movable block is slidably connected inside the production rack. A clamping mechanism is installed inside the grooved rod for mounting the box body onto the production rack. The clamping mechanism includes a micro motor, a bidirectional threaded rod, a threaded block, and an insert plate. The micro motor is fixedly connected to the grooved rod, and the bidirectional threaded rod is fixedly connected to the output end of the micro motor. The end of the bidirectional threaded rod away from the micro motor is rotatably connected to the grooved rod. The threaded block is threadedly connected to the bidirectional threaded rod and slidably connected inside the grooved rod. The insert plate is fixedly connected to the threaded block, and a slot is provided inside the movable block. The insert plate is slidably connected inside the slot.

[0012] As a further preferred embodiment of this technical solution: the production rack is provided with a guide groove, and the guide groove limits and guides the sliding of the movable block through an internally slidably connected guide block, while the end of the guide block away from the guide groove is fixed to the movable block;

[0013] As a further preferred embodiment of this technical solution: an alarm is fixedly connected to the box body, and the alarm is electrically connected to the visual camera;

[0014] As a further preferred embodiment of this technical solution: the vision camera is embedded with a dynamic vision sensor, which is used to control the opening and closing of the lifting and rotating mechanism to capture the moving objects in real time during sorting.

[0015] As a further preferred embodiment of this technical solution: the visual camera also embeds an image acquisition module, a target detection module, a dynamic tracking module, a motion prediction and compensation module, and a control execution module;

[0016] The image acquisition module is configured to acquire video streams containing the moving object in real time;

[0017] The target detection module is connected to the image acquisition module and is configured to perform real-time target detection on each frame of the video stream based on a deep learning algorithm in order to identify and locate the objects to be sorted.

[0018] The dynamic tracking module is connected to the target detection module and is configured to dynamically lock the identified object between consecutive frames based on a visual feature tracking algorithm and combined with motion prediction results, thereby generating the motion trajectory of the object.

[0019] The motion prediction and compensation module, connected to the dynamic tracking module, is configured to estimate the future position of the object using a predictive filtering algorithm based on the object's historical motion state, and calculate the dynamic error compensation amount during the grasping process based on visual servoing technology.

[0020] The control execution module, connected to the motion prediction and compensation module, is configured to generate control commands based on the future position and dynamic error compensation amount, driving the robot end effector to grasp and sort moving objects.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] 1. In this invention, a stepper motor drives the drive shaft to rotate via a first pulley, a transmission belt, and a second pulley. Combined with the sliding constraint between the slide bar and the strip groove, the vision camera can stably perform horizontal rotation. Simultaneously, an electric push rod drives the slide bar to rise and fall, causing the top plate and vision camera to adjust their vertical positions. This enables omnidirectional dynamic visual tracking of moving objects at different heights and with different trajectories, thereby expanding the effective monitoring range of the vision sensor. Furthermore, it can adaptively adjust the capture angle according to the object's motion state, ensuring the continuity and accuracy of visual information acquisition and providing reliable, high-quality visual data support for human-machine collaborative sorting.

[0023] 2. In this invention, the clamping mechanism and the production rack's adaptable structure enable rapid installation and stable positioning of the vision inspection module, facilitating flexible deployment in different production line scenarios. A micro motor drives a bidirectional threaded rod to rotate, causing the threaded block to insert or retract the insert plate into the slot within the movable block. Combined with the sliding limit of the guide groove and guide block, the relative position between the vision camera and the sorting line is precisely adjustable. When the vision system detects an anomaly or tracking failure, an alarm can promptly issue a warning, ensuring the safety of human-machine collaborative operations. Overall, this device integrates dynamic vision capture, intelligent motion compensation, and convenient mechanical adaptation functions, effectively reducing the frequency of manual intervention and improving the real-time processing efficiency of visual information in high-speed sorting scenarios. Attached Figure Description

[0024] Figure 1 This is a perspective view of a human-machine collaborative dynamic visual information intelligent adaptation device according to the present invention;

[0025] Figure 2 This is a partial exploded view of the structure of a human-machine collaborative dynamic visual information intelligent adaptation device according to the present invention;

[0026] Figure 3 This is a partial structural cross-sectional view of a human-machine collaborative dynamic visual information intelligent adaptation device according to the present invention;

[0027] Figure 4 for Figure 2 Enlarged view of point A in the middle;

[0028] Figure 5 for Figure 2 Enlarged view of point B in the middle.

[0029] Legend: 1. Box body; 2. Lifting and rotating mechanism; 21. Support plate; 22. Stepper motor; 23. First pulley; 24. Transmission belt; 25. Second pulley; 26. Drive shaft; 27. First slide bar; 28. Top plate; 29. ​​Electric push rod; 210. Second slide bar; 3. Vision camera; 4. Groove rod; 5. Production rack; 6. Movable block; 7. Clamping mechanism; 71. Micro motor; 72. Bidirectional threaded rod; 73. Threaded block; 74. Insert plate; 8. Slot; 9. Guide groove; 10. Guide block; 11. Alarm. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example

[0032] Please see Figures 1-5 As shown, the present invention provides a technical solution: a human-machine collaborative dynamic visual information intelligent adaptation device, including a box 1, a lifting and rotating mechanism 2, a visual camera 3, and a production rack 5;

[0033] The lifting and rotating mechanism 2 is installed on the box body 1 and is used to drive the vision camera 3 to dynamically capture the moving objects in the sorting process in real time.

[0034] The lifting and rotating mechanism 2 includes a support plate 21, a stepper motor 22, a first pulley 23, a transmission belt 24, a second pulley 25, a drive shaft 26, a first slide bar 27, a top plate 28, an electric push rod 29, and a second slide bar 210.

[0035] The support plate 21 is fixed to the box body 1, the stepper motor 22 is fixed to the box body 1, and the stepper motor 22 is also fixed to the support plate 21. The first pulley 23 is fixed to the output end of the stepper motor 22. The transmission belt 24 is engaged with the first pulley 23, and the second pulley 25 is engaged with the end of the transmission belt 24 away from the first pulley 23. The first slide rod 27 is fixed to the drive shaft 26, and the first slide rod 27 is slidably connected to the strip groove inside the second pulley 25. The top plate 28 is fixed to the end of the drive shaft 26 away from the second pulley 25, and is used to drive the vision camera 3 to rotate. The vision camera 3 is fixed to the drive shaft 26.

[0036] The electric push rod 29 is fixed to the box body 1 and the support plate 21. The second slide rod 210 is fixed to the output end of the electric push rod 29 and is slidably connected to the box body 1. The end of the second slide rod 210 away from the electric push rod 29 is fixed to the top plate 28 and is used to drive the visual camera 3 to move up and down.

[0037] Both the drive shaft 26 and the first slide rod 27 are slidably connected to the housing 1, and both the drive shaft 26 and the first slide rod 27 are rotatably connected to the housing 1.

[0038] Furthermore: The box body 1 is made of aluminum alloy and is fixedly installed above the sorting station. The lifting and rotating mechanism 2 is installed inside the box body 1. The support plate 21 is vertically fixed to the inner bottom surface of the box body 1 by bolts. The stepper motor 22 is also fixed to the inner bottom surface of the box body 1 by bolts. Its output axis passes through the through hole on the support plate 21 and is fixed to the first pulley 23 by a coupling. At the same time, the housing of the electric push rod 29 is fixed to the bottom surface of the box body 1, and its second slide rod 210 passes downward through the top of the box body 1 and is fixedly connected to the top plate 28. When the electric push rod 29 extends or retracts, it drives the top plate 28, the drive shaft 26 and the vision camera 3 to lift and lower as a whole, thereby realizing dynamic visual capture of objects moving at different heights.

[0039] In this embodiment, specifically: a grooved rod 4 is fixedly connected to the box body 1, and a movable block 6 is slidably connected inside the production rack 5. A clamping mechanism 7 is installed inside the grooved rod 4 for mounting the box body 1 onto the production rack 5. The clamping mechanism 7 includes a micro motor 71, a bidirectional threaded rod 72, a threaded block 73, and an insert plate 74. The micro motor 71 is fixedly connected to the grooved rod 4, and the bidirectional threaded rod 72 is fixedly connected to the output end of the micro motor 71. The end of the bidirectional threaded rod 72 away from the micro motor 71 is rotatably connected to the grooved rod 4. The threaded block 73 is threadedly connected to the bidirectional threaded rod 72 and slidably connected inside the grooved rod 4. The insert plate 74 is fixedly connected to the threaded block 73. Meanwhile, a slot 8 is provided inside the movable block 6, and the insert plate 74 is slidably connected inside the slot 8.

[0040] Furthermore, both the threaded block 73 and the insert plate 74 are provided in two sets. The two sets of threaded blocks 73 are located on the left and right turns of the bidirectional threaded rod 72, respectively. When the micro motor 71 rotates in the forward direction, the bidirectional threaded rod 72 drives the two threaded blocks 73 to move towards the middle, thereby driving the insert plate 74 to insert into the slot 8 of the movable block 6, thus locking the box 1 onto the production frame 5. Reverse rotation will loosen it, which is convenient for quick disassembly or position adjustment.

[0041] In this embodiment, specifically: the production rack 5 is provided with a guide groove 9, and the guide groove 9 limits and guides the sliding of the movable block 6 through the guide block 10 which is internally slidably connected. At the same time, the end of the guide block 10 away from the guide groove 9 is fixed to the movable block 6.

[0042] Furthermore, when the movable block 6 slides along the production rack 5, the cooperation between the guide block 10 and the guide groove 9 not only restricts the lateral swing of the movable block 6, but also ensures the straightness of its movement, thereby ensuring that the vision camera 3 is always aligned with the center of the sorting area and avoiding visual information distortion caused by installation offset.

[0043] In this embodiment, specifically: an alarm 11 is fixedly connected to the box 1, and the alarm 11 is electrically connected to the visual camera 3.

[0044] Furthermore, the alarm 11 is electrically connected to the microcontroller inside the vision camera 3 via a shielded cable. When the vision camera 3 fails to detect the target object for three or more consecutive frames, or when the target tracking error exceeds a preset threshold, the microcontroller outputs a high-level signal to drive the alarm 11 to emit a red flashing light and a buzzer sound, reminding the operator to check the vision system or adjust the sorting parameters in a timely manner.

[0045] In this embodiment, specifically: the vision camera 3 is embedded with a dynamic vision sensor, which is used to control the opening and closing of the lifting and rotating mechanism 2 to capture objects in the sorting process in real time.

[0046] Furthermore, the dynamic vision sensor integrated inside the vision camera 3 adopts an event-based dynamic vision sensor (DVS) chip with microsecond-level time resolution. The sensor is directly connected to the drive circuit that controls the lifting and rotating mechanism 2. When the dynamic vision sensor detects that an object is moving in the sorting area, it asynchronously outputs an event stream to trigger the start signal of the stepper motor 22 and the electric push rod 29, so that the vision camera 3 automatically enters the tracking mode. When the object moves out of the sorting area or stops moving for more than 2 seconds, the sensor outputs an idle signal to control the lifting and rotating mechanism 2 to return to the standby position, thereby realizing vision-triggered intelligent start and stop control.

[0047] In this embodiment, specifically: the visual camera 3 also embeds an image acquisition module, a target detection module, a dynamic tracking module, a motion prediction and compensation module, and a control execution module;

[0048] The image acquisition module is configured to acquire video streams containing the moving object in real time;

[0049] The target detection module is connected to the image acquisition module and is configured to perform real-time target detection on each frame of the video stream based on a deep learning algorithm in order to identify and locate the objects to be sorted.

[0050] The dynamic tracking module is connected to the target detection module and is configured to dynamically lock the identified object between consecutive frames based on a visual feature tracking algorithm and combined with motion prediction results, thereby generating the motion trajectory of the object.

[0051] The motion prediction and compensation module, connected to the dynamic tracking module, is configured to estimate the future position of the object using a predictive filtering algorithm based on the object's historical motion state, and calculate the dynamic error compensation amount during the grasping process based on visual servoing technology.

[0052] The control execution module, connected to the motion prediction and compensation module, is configured to generate control commands based on the future position and dynamic error compensation amount, driving the robot end effector to grasp and sort moving objects.

[0053] Furthermore, in addition to the dynamic vision sensor, the visual camera 3 integrates an embedded AI processing module, specifically including: an image acquisition module using an OV5640 global shutter CMOS sensor to acquire images of moving objects at 120 frames per second; an object detection module running a lightweight YOLOv5s deep learning model to perform object detection on each frame on an STM32H7 series microcontroller, outputting object bounding boxes and categories; a dynamic tracking module using a kernel correlation filter (KCF) algorithm combined with a Kalman filter to perform ID assignment and trajectory prediction for the same object across consecutive frames, generating the object's motion trajectory; a motion prediction and compensation module using an unscented Kalman filter (UKF) to predict the object's position within the next 0.2 seconds and calculating the dynamic offset of the robot's gripping point based on a visual servoing method to compensate for errors caused by conveyor belt speed and object slippage; and a control execution module converting the future position and compensation amount into Modbus. RTU commands are sent to the robot controller via RS485 bus, driving the robot's end effector pneumatic gripper to accurately grasp and sort moving objects using a "predict-correction" method. The entire processing cycle does not exceed 10 milliseconds, meeting the real-time requirements of high-speed sorting.

[0054] Working principle or structural principle: First, the device is installed on the movable block 6 of the production rack 5 via the clamping mechanism 7 on the box 1. The micro motor 71 drives the bidirectional threaded rod 72 to rotate, causing the threaded block 73 to drive the insert plate 74 to insert into the slot 8 of the movable block 6 for quick fixation. Then, the dynamic vision sensor embedded in the vision camera 3 detects moving objects in real time. Once a target is detected, the lifting and rotating mechanism 2 is automatically activated. The stepper motor 22 drives the first pulley 23 to rotate via the transmission belt 24, which in turn drives the second pulley 25 to rotate. This causes the drive shaft 26 and its slide bar 27 to rotate under the drive of the top plate 28. At the same time, the electric push rod 29 drives the slide bar 210 to slide and lift on the box 1, thereby driving the vision camera 3 to rotate at multiple angles and adjust its height, realizing real-time dynamic capture of moving objects in the sorting process. The image acquisition module in the vision camera 3 captures images of moving objects in real time. The video stream is processed by a target detection module that performs real-time target detection frame by frame based on a deep learning algorithm to identify and locate objects to be sorted. The dynamic tracking module uses a visual feature tracking algorithm combined with motion prediction results to continuously lock the identified objects between frames and generate motion trajectories. The motion prediction and compensation module uses a predictive filtering algorithm to estimate the future position of the object based on its historical motion state and calculates the dynamic error compensation amount during the grasping process based on visual servoing technology. Finally, the control execution module generates control commands based on the future position and the dynamic error compensation amount to drive the robot's end effector to accurately grasp and sort the moving objects. When the vision system detects an abnormality or malfunction, the alarm 11 will issue an alarm in time. At the same time, the guide groove 9 and guide block 10 on the production rack 5 cooperate to limit and guide the sliding of the movable block 6, ensuring the stable operation and accurate positioning of the vision device on the sorting line.

[0055] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A human-machine collaborative dynamic visual information intelligent adaptation device, characterized in that, Includes box body (1), lifting and rotating mechanism (2), vision camera (3), production rack (5); The lifting and rotating mechanism (2) is installed on the box body (1) and is used to drive the vision camera (3) to capture the moving objects in real time. The lifting and rotating mechanism (2) includes a support plate (21), a stepper motor (22), a first pulley (23), a transmission belt (24), a second pulley (25), a drive shaft (26), a first slide bar (27), a top plate (28), an electric push rod (29), and a second slide bar (210). The support plate (21) is fixed to the box body (1), the stepper motor (22) is fixed to the box body (1), and the stepper motor (22) is fixed to the support plate (21) through. The first pulley (23) is fixed to the output end of the stepper motor (22). The transmission belt (24) is engaged with the first pulley (23), and the second pulley (25) is engaged with the end of the transmission belt (24) away from the first pulley (23). The first slide rod (27) is fixed to the drive shaft (26), and the first slide rod (27) is slidably connected to the strip groove inside the second pulley (25). The top plate (28) is fixed to the end of the drive shaft (26) away from the second pulley (25) to drive the visual camera (3) to rotate. The visual camera (3) is fixed to the drive shaft (26). The electric push rod (29) is fixed to the box body (1) and the electric push rod (29) is fixed to the support plate (21). The second slide rod (210) is fixed to the output end of the electric push rod (29) and slides through the box body (1). The end of the second slide rod (210) away from the electric push rod (29) is fixed to the top plate (28) to drive the visual camera (3) to rise and fall.

2. The human-machine collaborative dynamic visual information intelligent adaptation device according to claim 1, characterized in that: The drive shaft (26) and the first slide rod (27) are both slidably connected to the box body (1), and the drive shaft (26) and the first slide rod (27) are both rotatably connected to the box body (1).

3. The human-machine collaborative dynamic visual information intelligent adaptation device according to claim 2, characterized in that: The box body (1) is fixedly connected to a groove rod (4), and a movable block (6) is slidably connected inside the production rack (5). A clamping mechanism (7) is installed inside the groove rod (4) for mounting the box body (1) on the production rack (5). The clamping mechanism (7) includes a micro motor (71), a bidirectional threaded rod (72), a threaded block (73), and a insert plate (74). The micro motor (71) is fixedly connected to the groove rod (4), and the bidirectional threaded rod (72) is fixedly connected to the output end of the micro motor (71). The end of the bidirectional threaded rod (72) away from the micro motor (71) is rotatably connected to the groove rod (4). The threaded block (73) is threadedly connected to the bidirectional threaded rod (72), and the threaded block (73) is slidably connected inside the groove rod (4). The insert plate (74) is fixedly connected to the threaded block (73), and a slot (8) is provided inside the movable block (6). The insert plate (74) is slidably connected inside the slot (8).

4. The human-machine collaborative dynamic visual information intelligent adaptation device according to claim 1, characterized in that: The production rack (5) is provided with a guide groove (9). The guide groove (9) limits and guides the sliding of the movable block (6) through the internally slidably connected guide block (10). At the same time, the end of the guide block (10) away from the guide groove (9) is fixed to the movable block (6).

5. The human-machine collaborative dynamic visual information intelligent adaptation device according to claim 1, characterized in that: An alarm (11) is fixedly connected to the box (1), and the alarm (11) is electrically connected to the visual camera (3).

6. The human-machine collaborative dynamic visual information intelligent adaptation device according to claim 1, characterized in that: The vision camera (3) is embedded with a dynamic vision sensor, which is used to control the opening and closing of the lifting and rotating mechanism (2) to capture objects in the sorting process in real time.

7. The human-machine collaborative dynamic visual information intelligent adaptation device according to claim 6, characterized in that: The visual camera (3) also includes an image acquisition module, a target detection module, a dynamic tracking module, a motion prediction and compensation module, and a control execution module. The image acquisition module is configured to acquire video streams containing the moving object in real time; The target detection module is connected to the image acquisition module and is configured to perform real-time target detection on each frame of the video stream based on a deep learning algorithm in order to identify and locate the objects to be sorted. The dynamic tracking module is connected to the target detection module and is configured to dynamically lock the identified object between consecutive frames based on a visual feature tracking algorithm and combined with motion prediction results, thereby generating the motion trajectory of the object. The motion prediction and compensation module, connected to the dynamic tracking module, is configured to estimate the future position of the object using a predictive filtering algorithm based on the object's historical motion state, and calculate the dynamic error compensation amount during the grasping process based on visual servoing technology. The control execution module, connected to the motion prediction and compensation module, is configured to generate control commands based on the future position and dynamic error compensation amount, driving the robot end effector to grasp and sort moving objects.

Images