A control method for a robot vision system that automatically aims at a target

Abstract

A control method for a robot vision system that automatically aims at a target, belonging to the technical fields of computer vision and embedded development. First, calibrate the industrial camera to obtain the internal parameter matrix and distortion parameters of the camera, measure the size of the object to obtain the coordinates of the object in the world coordinate system, use the PnP algorithm to obtain the values of the yaw axis, pitch axis, and distance, obtain the coordinates of the target in the gyroscope coordinate system through the coordinates of the target in the camera coordinate system, and superimpose the movement of the robot itself with the movement of the target, then a physical model can be constructed for predictive strikes. The present invention solves the problem of target recognition and locking for a robot facing a high-speed moving target, and performs motion tracking and motion prediction on the target. The industrial camera of the present invention has an image resolution of 640*480, the armor plate recognition frame rate can reach about 200fps, and can reach 280fps after introducing ROI, and the digital recognition accuracy rate can reach 98%. Using a calibration board for testing, the distance error calculated by angle resolution within 5m is within 6%, and the angle error is within 5%.

Description

Technical Field

[0001] The present invention belongs to the technical fields of computer vision and embedded development, and particularly relates to a control method for a robot vision system for automatically aiming at a target. Background Art

[0002] Visual tracking technology is an important topic in the field of computer vision and has important research significance; and it has broad application prospects in many aspects such as military guidance, video surveillance, robot vision navigation, human-computer interaction, and medical diagnosis. However, the current mainstream visual tracking technologies face problems such as occlusion, deformation, and motion blur, and it is difficult to meet the requirements of complex scenarios in practical applications. For example, occlusion of the target will cause the loss of the target, and deformation and motion blur will cause the inability to track the target, etc. The present invention can solve problems such as partial occlusion of the target, image blur and trailing caused by high-speed movement of the target by expanding the Kalman filter algorithm and improving the object motion positioning algorithm from PnP 3D to 2D point pairs, and performing real-time ROI (Region of Interest) image region-of-interest processing according to the target position. Summary of the Invention

[0003] In order to solve the above existing problems, the present invention proposes: a control method for a robot vision system for automatically aiming at a target, based on computer vision technology, enabling a robot to identify and lock a target moving at high speed through a monocular industrial camera, and performing motion tracking and motion prediction on the target, including the following steps:

[0004] S1. First perform ROI processing on the collected pictures;

[0005] S2. Extract the contours of the pictures after preliminary processing, perform preliminary screening on all the contours iteratively, screen out rectangles with an aspect ratio greater than 1.4 and less than 2.1, and let these rectangles enter the dynamic array of suspected center rectangles, and then screen out rectangles with too small an area and remove small noise points in the binary region;

[0006] S3. Iterate on the suspected center rectangles obtained in the previous step and perform screening according to the area, aspect ratio, and SVM;

[0007] S4. Iterate on the rectangle set screened out in step S2 again. At this time, group these rectangles according to nesting and various morphological features of the rectangles. Each group is a structure, which is a set of arms that have been struck and to be struck. Each structure includes three members: an armor plate rectangle, an arm rectangle, and the number of inlays;

[0008] S5. After selecting the final recognized rectangle, scale the recognized center to obtain the effective rectangle of the next frame, and perform ROI restoration on the final rectangle and the center coordinates; if the center of the circle is not recognized in the current frame, continuously expand the ROI area for searching until it expands to the entire picture;

[0009] S6. During the calculation process, the rotation direction is first determined according to the change in the angle between the center of the target rectangle and the center of the circle. After the rotation direction is known, the preset angle is calculated. For the rotating target, the real-time angular velocity is first obtained, and a sine function is fitted for the first 40 frames. Then, the integral is calculated according to the delay time as the preset angle. After the preset angle is obtained, the four points of the target rectangle are translated accordingly and put into the PNP calculation for calculation. Finally, the data is sent to the lower computer.

[0010] S7, after receiving the data, the lower computer processes the data and passes the data through the proportional, integral and differential control algorithm PID, inputs the deviation err, sets the value of the adjustment proportion kp, integral Ti and differential Td, and calculates the value through the PID calculation formula

[0011]

[0012] Calculate the difference U(t) between itself and the target, add the difference to the target angle of the gimbal itself, use the summed value as the expected value, and use PID calculation again to get the angle that the gimbal needs to rotate to follow the target, so as to ultimately achieve following the target.

[0013] Furthermore, in step S1, a rectangular range preset in the previous frame is reasonably scaled to intercept the ROI of the image, different HSV threshold processing is performed according to different colors, and a (5,5) convolution is performed to close the image to complete the binarization processing.

[0014] Furthermore, the system is first run to initialize the industrial camera, and the exposure, image resolution, image format, and image channel are set through the camera SDK;

[0015] Create threads through the C++thread library. Create three threads in total: the processing thread processes the image, the communication thread is responsible for the communication and interaction between the upper computer and the lower computer, and the main thread is the core operation process of the system;

[0016] In the main thread, the camera is calibrated by the camera's internal and external parameters, and the collected image is transmitted to the processing thread. In the processing thread, the image is preprocessed first, and color extraction is performed to detect the red / blue light bar. Morphological processing is performed on the color extraction binary image for image noise reduction and light bar area closure. By judging the position information between the two light bars: the angle difference, the misalignment angle, the light bar length difference ratio and the X, Y direction projection difference ratio, it is determined whether it is a suitable target, and then all the armor plates judged to be suitable are placed in the pre-selected target array vector;

[0017] Weighted sum is performed on the above-mentioned target information to obtain the best strike armor plate as the final target armor.

[0018] Further, angle calculation: First, calibrate the industrial camera to obtain the internal parameter matrix and distortion parameters of the camera, measure the size of the object to obtain the coordinates of the object in the world coordinate system, and use the PnP algorithm to obtain the values of the yaw axis, pitch axis, and distance.

[0019] Further, target prediction: Obtain the coordinates of the target in the gyroscope coordinate system through the coordinates of the target in the camera coordinate system. The origin of the gyroscope coordinate system follows the movement of the robot itself. At this time, considering the origin as stationary, superimpose the movement of the robot itself and the movement of the target to construct a physical model for predictive strike.

[0020] Further, data processing: Data processing must be carried out during the predictive strike process to counter data errors. After obtaining the coordinates in the gyroscope coordinate system, first perform data rejection and interpolation, that is, reject the obviously incorrect data and use the previous correct data for interpolation. After the preliminary processing, perform Kalman filtering on the data to ensure the smoothness and correctness of the data, and then calculate the target motion state.

[0021] The beneficial effects of the present invention are as follows: When the industrial camera has an image resolution of 640*480, the recognition frame rate of the armor plate can reach about 200fps, and can reach 280fps after introducing ROI. The digital recognition accuracy rate can reach 98%. Using a calibration plate for testing, the distance error calculated by the angle calculation within 5m is within 6%, and the angle error is within 5%. Brief Description of the Drawings

[0022] Figure 1 is the overall framework flowchart of the present invention;

[0023] Figure 2 is the recognition and tracking flowchart of the present invention;

[0024] Figure 3 is the color recognition algorithm diagram of the present invention;

[0025] Figure 4 is the morphological operation algorithm diagram of the present invention;

[0026] Figure 5 is the attitude calculation algorithm diagram of the present invention;

[0027] Figure 6 is the target prediction algorithm diagram of the present invention;

[0028] Figure 7 is the three-dimensional structure schematic diagram of the present invention;

[0029] Figure 8Side view of the present invention;

[0030] Figure 9 Bottom view of the present invention;

[0031] Figure 10 Schematic diagram of the wheel set structure of the present invention;

[0032] Figure 11 Partial schematic diagram of the wheel set of the present invention;

[0033] Figure 12 Schematic diagram of the structure of the cartridge ejecting device of the present invention;

[0034] Figure 13 Rear view of the cartridge ejecting device of the present invention;

[0035] Figure 14 Schematic diagram of the structure of the cartridge magazine of the present invention;

[0036] Figure 15 Barrel sectional view of the cartridge ejecting device of the present invention;

[0037] Figure 16 Schematic diagram of the structure of the friction wheel of the present invention;

[0038] Figure 17 Partial schematic diagram of the pitch axis pitching of the pan-tilt of the present invention;

[0039] Figure 18 Partial schematic diagram of the 360-degree rotation of the yaw axis of the pan-tilt of the present invention;

[0040] Figure 19 Partial structural diagram of the 360-degree rotation of the yaw axis of the pan-tilt of the present invention;

[0041] Figure 20 Front view of the pan-tilt of the present invention;

[0042] Figure 21 Right view of the pan-tilt of the present invention;

[0043] Figure 22 Top view of the pan-tilt of the present invention. Detailed implementation manners

[0044] Embodiment 1

[0045] A control method for a robot vision system for automatically aiming at a target, based on computer vision methods, enabling a robot to identify and lock a target moving at high speed through a monocular industrial camera, and performing motion tracking and motion prediction on the target, including the following steps: S1. First perform ROI processing on the collected pictures;

[0046] S2. Extract the contours of the preliminarily processed images, iteratively perform preliminary screening on all the contours, filter out rectangles with an aspect ratio greater than 1.4 and less than 2.1, and let these rectangles enter the dynamic array of suspected center rectangles. Then, filter out rectangles with too small an area and remove small noise points in the binary region;

[0047] S3. Iterate through the suspected center rectangles obtained in the previous step and perform screening based on area, aspect ratio, and SVM;

[0048] S4. Iterate through the rectangle set screened in step S2 again. At this time, group these rectangles according to nesting and various morphological features of the rectangles. Each group is a structure, which is a set of armored plates and cantilevers to be struck. Each structure includes three members: armored plate rectangle, cantilever rectangle, and the number of inlays;

[0049] S5. After selecting the final recognized rectangle, scale according to the recognized center to obtain the effective rectangle of the next frame, and perform ROI restoration on the final rectangle and the center coordinates; if the center is not recognized in the current frame, continuously expand the ROI area for searching until it expands to the entire image;

[0050] S6. During the solution process, first judge the rotation direction according to the angle change of the line connecting the center of the target rectangle and the center of the circle. After knowing the rotation direction, solve the angle that needs to be preset. For the rotating target, first obtain the real-time angular velocity, and perform sine function fitting on the first 40 frames. Then, solve the integral according to the delay time as the preset angle. After obtaining the preset angle, translate the four points of the target rectangle accordingly and put them into the PNP solution for calculation. Finally, send the obtained data to the lower computer;

[0051] S7. After the lower computer receives the data, process the data, and input the deviation err through the proportional, integral, and derivative control algorithm PID. Set the values of the adjustment ratio kp, integral Ti, and derivative Td. Calculate the difference U(t) between itself and the target through the PID calculation formula

[0052]

[0053] Calculate the difference between itself and the target, add the difference to the target angle of the pan-tilt itself, and use the added value as the expected value to calculate the angle that the pan-tilt needs to rotate to follow the target through PID again. Finally, achieve the follow-up of the target.

[0054] Among them, in step S1, perform reasonable scaling according to a preset rectangular range in the previous frame to intercept the ROI of the image, perform different hsv threshold processing according to different colors, and perform closing operation on the image through a (5,5) convolution to complete the binarization process.

[0055] Among them, first run the system to initialize the industrial camera, and set the exposure, image resolution, image format, and image channels through the camera's SDK;

[0056] Create threads through the C++ thread library, a total of three threads are created: the processing thread processes the images, the communication thread is responsible for the communication interaction between the host computer and the lower computer, and the main thread is the core operation process of the system;

[0057] Calibrate the camera in the main thread through the internal and external parameters of the camera, collect the images and transmit them to the processing thread. In the processing thread, first preprocess the images, perform color extraction to detect red / blue light bars, perform morphological processing on the color-extracted binary images for image noise reduction and closing of the light bar areas, and determine the position information between the two light bars: the size of the angle difference, the size of the misalignment angle, the ratio of the light bar length difference, and the ratio of the projection differences in the X and Y directions, so as to distinguish whether it is a suitable target, and then put all the armor plates judged to be suitable into the preselected target array vector;

[0058] Perform weighted summation on the above-mentioned various target information to obtain the best strike armor plate as the final target armor.

[0059] Among them, angle calculation: First, calibrate the industrial camera to obtain the internal parameter matrix and distortion parameters of the camera, measure the size of the object to obtain the coordinates of the object in the world coordinate system, and obtain the values of the yaw axis, pitch axis, and distance through the PnP algorithm.

[0060] Among them, target prediction: Obtain the coordinates of the target in the gyroscope coordinate system through the coordinates of the target in the camera coordinate system. The origin of the gyroscope coordinate system follows the movement of its own robot. At this time, consider the origin stationary, and superimpose the movement of its own robot and the movement of the target together to build a physical model for predictive strike.

[0061] Among them, data processing: Data processing must be carried out during the predictive strike process to counter data errors. After obtaining the coordinates in the gyroscope coordinate system, first perform data rejection and interpolation, that is, reject the obviously incorrect data and use the previous correct data for interpolation. After the preliminary processing is completed, perform Kalman filtering on the data to ensure the smoothness and correctness of the data, and then calculate the target motion state.

[0062] Embodiment 2

[0063] The present invention is based on computer vision technology, and through a monocular industrial camera, solves the problem of target recognition and locking for a robot facing a high-speed moving target, and performs motion tracking and motion prediction on the target.

[0064] The specific technical solution of the present invention:

[0065] 1. As attachedFigure 2 As shown in Figure 2 , the collected images are first subjected to ROI processing. The ROI of the image is intercepted by reasonably scaling according to a preset rectangular range in the previous frame. Then, different hsv threshold processing is performed according to different colors, and a closing operation is performed on the image by convolution of (5, 5) to complete the binarization processing.

[0066] 2. Extract the contours of the preliminarily processed images, and perform preliminary screening on all the contours iteratively. Rectangles with an aspect ratio greater than 1.4 and less than 2.1 are screened out, and these rectangles are put into the dynamic array of suspected center rectangles. Then, rectangles with too small an area are screened out to remove small noise points in the binarized area.

[0067] 3. Iterate through the suspected center rectangles obtained in the previous step, and screen them according to area, aspect ratio, and SVM. However, according to the actual effect, the SVM effect is not very ideal. Therefore, before screening according to SVM, all rectangles that meet the morphological requirements are put into the dynamic array. After one round of screening, if the center is not found, the above rectangles are screened again, and the rectangle that meets the morphological screening conditions with the center rectangle in the previous frame is selected as the center rectangle.

[0068] 4. Iterate through the rectangle set screened out in step 2 again. At this time, group these rectangles according to nesting and various morphological features of the rectangles. Each group is a structure, which is actually a set of arms that have been hit and to be hit. Each structure includes three members: the armor plate rectangle, the arm rectangle, and the number of inlays. Such grouping operations facilitate subsequent processing and maintenance. After grouping, distinguish the arms to be hit and the arms that have been hit according to the number of inlays in each structure. The number of inlays in the arms to be hit is significantly less than that of the arms that have been hit. Therefore, the structure to be hit is obtained, and the armor plate to be hit is the member inside.

[0069] 5. After selecting the final recognized rectangle, scale the recognized center to obtain the effective rectangle for the next frame, and perform ROI restoration on the final rectangle and the center coordinates. If the center is not recognized in the current frame, continuously expand the ROI area for searching until it expands to the entire image.

[0070] 6. During the solution process, first judge the rotation direction according to the change in the angle of the line connecting the center of the target rectangle and the center. After knowing the rotation direction, solve the preset angle. For the rotating target, first obtain the real-time angular velocity, and perform sine function fitting on the first 40 frames. Then, solve the integral according to the delay time as the preset angle. After obtaining the preset angle, translate the four points of the target rectangle accordingly and put them into the PNP solution for solution. Finally, the obtained data is sent to the lower computer.

[0071] 7. After receiving the data, the lower computer processes the data and calculates the difference between itself and the target through PID. The difference is added to the target angle of the gimbal itself, and the summed value is used as the expected value to calculate the angle that the gimbal needs to rotate to follow the target again through PID, thus achieving the tracking of the target.

[0072] As attached Figure 1 As shown, first run the system to initialize the industrial camera, and set the exposure, image resolution, image format, image channel, etc. through the camera SDK.

[0073] Threads are created through the C++thread library. A total of three threads are created: the processing thread processes the image, the communication thread is responsible for communication and interaction between the upper computer and the lower computer, and the main thread is the core operation process of the system.

[0074] In the main thread, the camera is calibrated using the camera's internal and external parameters, and the captured image is transmitted to the processing thread. Figure 3 As shown, the image is preprocessed in the processing thread first. In order to detect the red / blue light bar, color extraction is required. The present invention uses the channel subtraction method to extract the color of the target. The principle is that under low exposure (3000-5000), the B channel value of the blue light bar area is much higher than the R channel value. Using the B channel to subtract the R channel and then binarize, the blue light bar area can be extracted, and vice versa. In addition, as shown in the attached Figure 4 As shown, morphological processing is performed on the color extraction binary image to reduce image noise and close the light bar area.

[0075] As attached Figure 6 As shown, by judging the position information between the two light bars: the angle difference, the misalignment angle, the light bar length difference ratio and the X, Y direction projection difference ratio, it is possible to distinguish whether it is a suitable target, and then put all the armor plates judged to be suitable into the pre-selected target array vector. At the same time, in order to eliminate the wrong targets caused by "free light bars", a function is written through detection to detect and delete wrong targets.

[0076] The above target information is weighted and summed to obtain the best strike armor plate as the final target armor.

[0077] Angle calculation: First, calibrate the industrial camera to obtain the camera's intrinsic matrix and distortion parameters. Then measure the size of the object to obtain the object's coordinates in the world coordinate system. Figure 5 As shown, the values ​​of the yaw axis, pitch axis and distance are obtained through the PnP algorithm.

[0078] Target Prediction: Regarding the predictive strike as a kinematic problem requires a reference system. However, the camera coordinate system moves along with the camera, and the coordinate system also changes when following the target, making it difficult to make predictions in the camera coordinate system. Therefore, the concept of the gyroscope coordinate system is introduced. A gyroscope is an attitude sensor, usually fixed at a position on the pan-tilt head away from the vibration source. The function of the gyroscope is to feedback the current angles of the pan-tilt head (Yaw axis, Pitch axis, Roll axis angles) (taking the Yaw axis as an example, the Yaw axis angle has a zero point, which is relatively fixed and does not change in the case of ignoring the gyroscope zero drift. This zero point can be analogized to the south pole of a compass, which always points in one direction no matter how it is moved (in fact, the Yaw axis data of the gyroscope is obtained through the magnetometer). In this way, there is a reference - regardless of the attitude of the pan-tilt head, as long as the camera coordinate system is rotated in the opposite direction according to the angle data of the Yaw axis and Pitch axis, the same coordinate system can be obtained, and this coordinate system is the gyroscope coordinate system. Since the coordinate transformation can be carried out, the coordinate transformation is very easy, and the coordinates of the target in the gyroscope coordinate system can be obtained through the coordinates of the target in the camera coordinate system. The origin of the gyroscope coordinate system follows the movement of the robot itself. At this time, regarding the origin as stationary and superimposing the movement of the robot itself and the movement of the target, a physical model can be constructed for predictive strike.

[0079] Data Processing: During the process of predictive strike, data processing must be carried out to counter data errors. After obtaining the coordinates in the gyroscope coordinate system, a wave of simple data rejection and interpolation is first carried out. That is, the obviously incorrect data is rejected and the previous correct data is used for interpolation. After the preliminary processing is completed, the data is subjected to Kalman filtering to ensure the smoothness and correctness of the data. Then, the calculation of the target motion state is carried out, which can ensure the smoothness of the pan-tilt head prediction.

[0080] Embodiment 3

[0081] As Figures 7 - 9As shown in the figure, it includes a chassis, a pan-tilt, an adaptive wheel set, and a bullet feeding device. The chassis includes short aluminum squares 2 and long aluminum squares 8. The short aluminum square 2 is connected to the long aluminum square 8 through short aluminum square connectors 1 and aluminum square right-angle connecting plates 5. Four adaptive wheel sets are fixedly connected at the internal corners of the chassis aluminum frame. The adaptive wheel sets are fixed on the short aluminum square 2 inside the chassis through wheel set connecting milling parts 30. Two support plates 10 and a top cover plate 13 are respectively installed in four directions of the chassis aluminum frame. A TB47 battery 9 is arranged on the top cover plate 13. A light bar module 14 and a robot determination system 12 are respectively arranged on the support plates 10 on the front and rear sides of the vehicle body. Protection plates 7 are arranged under the top cover plates 13 on both sides of the vehicle body. A 6020 mounting plate 37 is fixedly arranged in the middle of the symmetric chassis aluminum frame on the left and right. The second 6020 motor 210 is installed on the 6020 mounting plate 37, and a field perception module 38 is installed under the chassis aluminum frame. The pan-tilt is connected to the chassis through a 3D printed pan-tilt connector 25. A calculation device 26 is installed at a hole position on one side of the 3D printed pan-tilt connector 25. Two pan-tilt connecting plates 16 are fixedly connected to both sides of the calculation device 26. An image transmission module 20, a central control board 18, an industrial camera 23, and an infrared aiming device 29 are respectively and sequentially fixedly arranged on the front ends of the pan-tilt connecting plates 16 from top to bottom through copper columns 22. A bullet feeding device 28 is fixed at the end of the pan-tilt connecting plate 16 through a mortise and tenon structure. A first 6020 motor 21 is installed on the outer side of the middle of the pan-tilt connecting plate 16;

[0082] As Figures 20 - 22 shown in the figure, it includes a chassis, a pan-tilt, and an adaptive wheel set. A 6020 mounting plate 37 is fixedly arranged in the middle of the symmetric chassis aluminum frame on the left and right. The first 6020 motor 21 is installed on the 6020 mounting plate 37, and a field perception module 38 is installed under the chassis aluminum frame. The pan-tilt is connected to the chassis through a 3D printed pan-tilt connector 25. The chassis supports the pan-tilt through an upper extension plate 72 and upper extension side plates 74. A calculation device is installed at a hole position on one side of the 3D printed pan-tilt connector 25. Two pan-tilt connecting plates 16 are fixedly connected to both sides of the calculation device. An image transmission module 20, a central control board 18, an industrial camera, and an infrared aiming device 29 are respectively and sequentially fixedly arranged on the front ends of the pan-tilt connecting plates 16 from top to bottom through copper columns 22. The first 6020 motor 21 is installed on the outer side of the middle of the pan-tilt connecting plate 16. The inner side of the middle of the pan-tilt connecting plate 16 is connected to the outer side of the pan-tilt connecting plate 16 through a central internal milling part 48. The first 6020 motor 21 is connected to the central internal milling part 48 through a long straight milling part 47. One end of the long straight milling part 47 is connected to the outer hole position of the first 6020 motor 21 through a limit plate 46, and the other end of the long straight milling part 47 is connected to the central internal milling part 48. The chassis includes short aluminum squares 2 and long aluminum squares 8. The short aluminum square 2 is connected to the long aluminum square 8 through short aluminum square connectors 1 and aluminum square right-angle connecting plates 5. Four adaptive wheel sets are fixedly connected at the internal corners of the chassis aluminum frame.

[0083] The adaptive wheel set is fixed on the short aluminum square 2 inside the chassis.

[0084] Two support plates 10 and a top cover plate 13 are respectively installed in four directions of the chassis aluminum frame. The position of the TB47 battery 9 is set on the top cover plate 13. A light bar module 14 and a robot determination system 12 are respectively set on the support plates 10 on the front and rear of the vehicle body.

[0085] A protection plate 7 is set under the top cover plates 13 on both sides of the vehicle body.

[0086] A partition support plate 73 is set on the central control board 18, and a main control protection plate 19 is set on the partition support plate 73.

[0087] A friction wheel carrier plate 45 is set on the pan-tilt connection plate 16. Two friction wheels are set on the friction wheel carrier plate 45. Two 3508 motors without a reduction box 49 are set on the pan-tilt connection plate 16, and the two 3508 motors without a reduction box 49 are respectively connected to the two friction wheels.

[0088] It further includes a magazine 75. The servo motor 17 is installed on the servo motor mounting part 171 and connected to the magazine cover plate 43 to form a closed magazine 75. The magazine 75 is connected to the pan-tilt connection plate 16 through a finished connecting piece 40. A servo motor fixing piece 41 is set on the magazine limiting plate 42, and the servo motor fixing piece 41 is connected to the 2006 DC brushless reduction motor 27.

[0089] As Figure 17 shown, the inner side of the middle part of the pan-tilt connection plate 16 is connected to the outer side of the pan-tilt connection plate 16 through a central internal milling part 48. The first 6020 motor 21 is connected to the central internal milling part 48 through a long straight milling part 47. One end of the long straight milling part 47 is connected to the outer hole position of the first 6020 motor 21 through a limiting plate 46, and the other end of the long straight milling part 47 is connected to the central internal milling part 48;

[0090] The first 6020 motor 21 drives the central internal milling part 48 to change the pitch angle of the pan-tilt pitch axis.

[0091] As Figures 18 - 19 shown, the epoxy board 67 includes an inner ring milling part 68 and an outer ring milling part 69. The upper part of the second 6020 motor 210 is connected to the inner ring milling part 68 fixed on the 3D printing pan-tilt connection piece 25 through a chassis connection piece 71. The inner ring milling part 68 and the outer ring milling part 69 clamp the rolling bearing 66, and the outer ring milling part 69 is fixedly connected to the chassis connection piece 71.

[0092] The second 6020 motor 210 drives the 3D printing pan-tilt connection piece 25, the inner ring milling part 68, and the chassis connection piece 71 to rotate, so as to realize the 360-degree rotation of the pan-tilt yaw axis.

[0093] Among them, asFigures 12 - 15 The described bullet feeding device 28 includes a dial 35. A central perforation 61 is provided on the dial 35. The 2006 DC brushless reduction motor 27 is fixed on the central perforation 61. The rotating shaft of the 2006 DC brushless reduction motor 27 is connected to the central axis 63 of the fork 34. A number of bullet holes 64 are evenly arranged on the fork 34. A notch limiting plate 36 is fixedly arranged on the dial 35. A number of through holes 62 are arranged around the central perforation 61.

[0094] Among them, the bullet feeding device 28 is connected to the magazine limiting plate 42 through a finished connecting piece 40. A servo motor fixing piece 41 is provided on the magazine limiting plate 42. The servo motor 17 is installed on the fixing piece 41 and connected to the magazine cover 43 to form a closed magazine.

[0095] The bullets are stored in the magazine. The servo motor 17 drives the opening and closing of the magazine cover 43. The function of the through holes 2 on the dial 35 is ventilation and heat dissipation. The 2006 DC brushless reduction motor 27 drives the central axis 3 of the fork 34 to drive the bullets to rotate in the dial 35. When the bullets rotate to the notch limiting plate 36, the bullets are ejected from the notch limiting plate 36. The annular tangent bullet feeding uses the principle of perpendicular tangents to solve the problem of bullet jamming. The installation position of the bullet feeding device can be adjusted, reducing the overall mass of the firing mechanism.

[0096] Among them, as Figure 16 shown, the pan tilt connecting plate 16 is connected to the pan tilt carrier plate 65. Two 3508 gearless motors 49 and a friction wheel carrier plate 45 are arranged on the pan tilt carrier plate 65. Two friction wheels 44 are arranged on the friction wheel carrier plate 45. The two 3508 gearless motors 49 are respectively connected to the two friction wheels 44. The notch limiting plate 36 is connected to the barrel 24 through a bullet chain 15. The bullets are movably connected to the two friction wheels 44.

[0097] After the bullets are ejected from the notch limiting plate 36, the bullets pass through the bullet chain 15 and pass through the middle of the two friction wheels 44. The two 3508 gearless motors 49 respectively drive the two friction wheels 44. The two friction wheels 44 play a role in accelerating the rapid rotation of the bullets, which can increase the firing speed of the passing bullets.

[0098] Among them, as Figures 10 - 11 shown, the adaptive wheel set includes Mecanum wheels 6. The Mecanum wheels 6 are connected and locked to the motor shaft of the 3508 DC brushless reduction motor 31 through the Mecanum wheel outer mounting plate 3 and the self-made Mecanum wheel fastener 39. The self-made Mecanum wheel fastener 39 is hinged to the wheel set connecting milling part 30 through the milling part connecting plate 32. The self-made Mecanum wheel fastener 39 on the Mecanum wheels 6 is connected to the motor 31 through a giant bearing 33. The self-made Mecanum wheel fastener 39 is fixedly connected to the wheel set connecting milling part 30 through a shock absorber 11.

[0099] The 3508 DC brushless reduction motor 31 drives the Mecanum wheel 6 to rotate, and the cooperation among the milling part connecting plate 32, the wheel set connecting the milling part 30, and the self-made Mecanum wheel fastener 39 realizes the shock absorption function of the shock absorber 11. In the present invention, the self-made Mecanum wheel fastener 39 is connected to the 3508 DC brushless reduction motor 31. By means of the self-made Mecanum wheel fastener 39, it can ensure that the Mecanum wheel is firmly connected to the 3508 DC brushless reduction motor 31, maximizing the role of the motor. Moreover, the cost of the self-made Mecanum wheel fastener is relatively low, correspondingly reducing the manufacturing cost.

[0100] Among them, anti-collision wheels 4 are arranged on the aluminum square right-angle connecting plates 5 at the four corners of the chassis. The anti-collision wheels 4 play a role in preventing the vehicle body from being damaged due to impact.

[0101] Among them, a main control protection plate 19 is arranged on the central control board 18. The main control protection plate 19 plays a role in protecting the central control board 18.

[0102] The functions that the present invention can achieve are as follows:

[0103] 1. The pitch axis of the pan-tilt changes the angle of pitch;

[0104] 2. The yaw axis of the pan-tilt rotates 360 degrees;

[0105] 3. The ammunition feeding magazine can feed ammunition;

[0106] 4. The camera and the video transmitter can transmit images;

[0107] 5. Two friction wheels increase the projectile launch speed;

[0108] 6. The pan-tilt can visually identify and follow the movement of an object;

[0109] 7. The chassis can move omnidirectionally through the wheel set;

[0110] 8. The chassis pan-tilt can lock and rotate;

[0111] 9. The angle of the pan-tilt can be fixed during the self-rotation of the chassis.

[0112] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and its concept of the present invention, makes equivalent substitutions or changes, and all should be covered within the protection scope of the present invention.

Claims

1. A control method for a robot vision system that automatically aims at a target, characterized in that, Based on computer vision methods, a robot uses a monocular industrial camera to identify and lock a target moving at high speed, and perform motion tracking and motion prediction on the target, including the following steps: S1. First, perform Region of Interest (ROI) processing on the captured images; S2. Extract the contours of the preliminarily processed images, and iteratively perform preliminary screening on all the contours. Rectangles with an aspect ratio greater than 1.4 and less than 2.1 are screened out, and these rectangles are put into a dynamic array of suspected center rectangles. Then, rectangles with too small an area are screened out, and small noise points in the binary region are removed; S3. Iterate through the suspected center rectangles obtained in the previous step, and screen them according to area, aspect ratio, and Support Vector Machine (SVM); S4. Iterate through the set of rectangles screened out in step S2 again. At this time, group these rectangles according to nesting and various morphological features of the rectangles. Each group is a structure, which is a set of arms that have been hit and to be hit. Each structure includes three members: armor plate rectangle, arm rectangle, and the number of inlays; S5. After selecting the final recognized rectangle, scale the recognized center to obtain the effective rectangle of the next frame, and perform ROI restoration on the final rectangle and the center coordinates; if the center is not recognized in the current frame, continuously expand the ROI area for searching until it expands to the entire image; S6. During the solution process, first judge the rotation direction according to the change in the angle of the line connecting the center of the target rectangle and the center of the circle. After knowing the rotation direction, solve the angle to be preset. For a rotating target, first obtain the real-time angular velocity, and perform sine function fitting on the first 40 frames. Then, solve the integral according to the delay time as the preset angle. After obtaining the preset angle, translate the four points of the target rectangle accordingly and put them into the Perspective-n-Point (PNP) solution for calculation. Finally, send the obtained data to the lower computer; S7. After the lower computer receives the data, process the data, and input the deviation err through the Proportional-Integral-Derivative (PID) control algorithm, set the values of the proportional gain kp, integral time Ti, and derivative time Td, and calculate the difference U(t) between itself and the target through the PID calculation formula Calculate the sum of the difference and the target angle of the pan-tilt itself, and use the sum value as the expected value to calculate the angle required for the pan-tilt to follow the target through PID again, and finally achieve the following of the target.

2. The control method for a robot vision system that automatically aims at a target according to claim 1, characterized in that, In step S1, perform reasonable scaling according to a preset rectangular range in the previous frame to intercept the ROI of the image, perform HSV threshold processing in different color spaces according to different colors, and perform closing operation on the image through a (5, 5) convolution to complete the binary processing.

3. The control method for a robot vision system that automatically aims at a target according to claim 1, characterized in that, First, run the system, initialize the industrial camera, and set the exposure, image resolution, image format, and image channels through the Software Development Kit (SDK) of the camera; Create threads through the C++ thread library. A total of three threads are created: the processing thread processes the images, the communication thread is responsible for the communication interaction between the upper computer and the lower computer, and the main thread is the core operation process of the system; Calibrate the camera through the internal and external parameters of the camera in the main thread, collect images and transmit them to the processing thread. In the processing thread, first preprocess the images, perform color extraction to detect red / blue light bars, and perform morphological processing on the color-extracted binary images for image noise reduction and closing of the light bar areas. By judging the position information between the two light bars: the size of the angle difference, the size of the misalignment angle, the ratio of the light bar length difference, and the ratio of the projection differences in the X and Y directions, determine whether it is a suitable target, and then put all the judged suitable armor plates into the preselected target array vector; Perform a weighted sum of the above target information to obtain the best strike armor plate as the final target armor.

4. The control method for a robot vision system that automatically aims at a target according to claim 3, characterized in that, In step S6, angle calculation: First, calibrate the industrial camera to obtain the internal parameter matrix and distortion parameters of the camera, measure the size of the object to obtain the coordinates of the object in the world coordinate system, and use the PnP algorithm for object motion positioning from 3D to 2D point pairs to obtain the values of the yaw axis, pitch axis, and distance.

5. The control method for a robot vision system that automatically aims at a target according to claim 4, characterized in that, In step S5, target prediction: Obtain the coordinates of the target in the gyroscope coordinate system through the coordinates of the target in the camera coordinate system. The origin of the gyroscope coordinate system follows the movement of its own robot. At this time, consider the origin stationary, and superimpose the movement of its own robot and the movement of the target to construct a physical model for predictive strikes.

6. The control method for a robot vision system that automatically aims at a target according to claim 5, characterized in that, In step S7, data processing: Data processing must be carried out during the predictive strike process to counter data errors. After obtaining the coordinates in the gyroscope coordinate system, first perform data rejection and interpolation, that is, reject the obviously incorrect data and use the previous correct data for interpolation. After the preliminary processing is completed, perform Kalman filtering on the data to ensure the smoothness and correctness of the data, and then calculate the target motion state.

Images