A pan-tilt receiving and adjusting system based on vision control

Abstract

The present invention discloses a pan-tilt receiving and adjustment system based on visual control, comprising: a machine vision device, a multi-axis pan-tilt, a pan-tilt positioning system, a visual analysis control system, and a shipping control system; the multi-axis pan-tilt is connected to the pan-tilt positioning system, and the movement trajectory of the multi-axis pan-tilt is determined based on the pan-tilt positioning system; the machine vision device is set on a multi-axis platform, and the visual analysis control system detects whether the multi-axis platform has reached a preset position and whether the shipped product is correct based on the visual analysis control system. If the detection is correct, a shipping instruction is sent to the shipping control system, and the shipping control system controls the movement of products in the corresponding cargo lanes, so that the products are moved onto the multi-axis pan-tilt. Using machine vision to perform pan-tilt receiving control eliminates the problems that would exist with completely mechanical control, and this solution can improve the accuracy and safety of pan-tilt receiving and adjustment.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic vending machines, and in particular to a pan-tilt platform receiving and adjusting system based on vision control. Background Art

[0002] Vending machines already use a lifting platform to receive goods. However, existing lifting platforms have the following problems: the platform is raised and lowered entirely by a mechanical design, making the equipment bulky and one component in the mechanical equipment damaged, directly resulting in the platform being unable to fully and normally rise or operate. Summary of the Invention

[0003] The present invention provides a pan-tilt cargo receiving and adjusting system based on vision control to solve the above-mentioned problems existing in the prior art.

[0004] The present invention provides a pan-tilt receiving and adjusting system based on visual control, comprising: a machine vision device, a multi-axis pan-tilt head, a pan-tilt head positioning system, a visual analysis control system, and a shipment control system; the multi-axis pan-tilt head is connected to the pan-tilt head positioning system, and the movement trajectory of the multi-axis pan-tilt head is determined based on the pan-tilt head positioning system; the machine vision device is set on the multi-axis platform, and based on the visual analysis control system, it detects whether the multi-axis platform has reached a preset position and whether the shipped product is correct. If the detection is correct, a shipment instruction is sent to the shipment control system, and the shipment control system controls the movement of products in the corresponding cargo channel to move the products to the multi-axis pan-tilt head.

[0005] Preferably, the machine vision device includes an industrial camera and a camera bracket; the industrial camera is rotatably fixed to the camera bracket;

[0006] The industrial camera is set on a rotatable base, which is fixed on the camera bracket. The rotating end of the rotatable base rotates 360 degrees relative to the rotating fulcrum. The initial position of the industrial camera is a first shooting angle. The first shooting angle is the angle for shooting products in the aisle. The image shot at the first shooting angle is transmitted to the visual analysis control system. When the visual analysis control system analyzes that the shot product is the correct product, the visual analysis control system sends an instruction to the rotation control end of the rotatable base to control the rotation end to rotate a preset angle. The working camera follows the rotation and then switches to a second shooting angle. The second shooting angle is for shooting the gap between the aisle entrance and the multi-axis pan-tilt head. The image shot at the second shooting angle is transmitted to the visual analysis control system. The visual analysis control system analyzes whether the multi-axis pan-tilt head has reached the preset position. If it has reached the preset position, the shipping instruction is sent to the shipping control system.

[0007] Preferably, after the image captured by the second shooting angle is transmitted to the visual analysis control system, the visual analysis control system sends instructions to the rotation control end again to control the rotation of the rotation end so that the industrial camera reaches a third shooting angle, and transmits the image captured by the third shooting angle to the visual analysis control system, and adjusts the shooting angle in a cycle until the visual analysis control system determines that a three-dimensional model can be constructed based on the received image. The visual analysis control system stops sending instructions to the rotation control end, and the visual analysis control system constructs a three-dimensional model based on the received image, and determines whether the multi-axis gimbal has reached the preset position based on the three-dimensional model.

[0008] Preferably, if the visual analysis control system detects that the multi-axis platform has not reached the preset position, the visual analysis control system sends a fine-tuning instruction to the pan-tilt positioning system, and the fine-tuning instruction includes: the current detection position coordinates, the preset position coordinates and the moving path; the pan-tilt positioning system fine-tunes the position of the multi-axis pan-tilt according to the fine-tuning instruction; when the multi-axis pan-tilt moves to the preset position coordinates according to the moving path, the pan-tilt positioning system sends a detection instruction to the visual analysis control system, and the visual analysis control system controls the industrial camera in the machine vision equipment to capture the fine-tuned image, and transmits the image to the visual analysis control system again. If the visual analysis control system detects that the multi-axis platform has reached the preset position, the fine-tuning action of the multi-axis platform is stopped. If the visual analysis control system detects that the multi-axis platform has not reached the preset position, it continues to send fine-tuning instructions to the pan-tilt positioning system and repeats the fine-tuning action until the visual analysis control system detects that the multi-axis platform has reached the preset position.

[0009] Preferably, the multi-axis pan-tilt platform includes a receiving platform, a multi-axis driving device and a driving control device, wherein the multi-axis driving device is connected to the receiving platform, and the driving control device is connected to the multi-axis driving device to control the six-axis movement of the multi-axis driving device in four horizontal directions and two vertical directions, and the multi-axis driving device drives the movement of the receiving platform;

[0010] The drive control device is connected to the pan-tilt positioning system and forms control instructions based on instructions from the pan-tilt positioning system for controlling the multi-axis drive device.

[0011] Preferably, if the visual analysis and control system detects that the shipped product is incorrect, the position of the aisle reached by the multi-axis pan-tilt stage is incorrect, and the visual analysis and control system sends a product error instruction to the pan-tilt stage positioning system. The pan-tilt stage positioning system transmits the product error instruction to the multi-axis pan-tilt stage, and the multi-axis pan-tilt stage automatically moves horizontally according to the product error instruction to change to an adjacent aisle; an industrial camera in a machine vision device on the multi-axis pan-tilt stage is used to capture images on an adjacent aisle, and the captured images are transmitted to the visual analysis and control system. The visual analysis and control system obtains the user's order information from the order platform, analyzes the product information corresponding to the user's order, performs image analysis on the captured images, obtains real-time product information, compares the real-time product information with the product information corresponding to the order, and determines whether it is the product ordered by the user. If so, a shipping instruction is sent to the shipping control system. If not, the product error instruction is continued to be sent to the pan-tilt stage positioning system, and the multi-axis pan-tilt stage continues to move horizontally until the correct shipping product is found.

[0012] Preferably, the multi-axis pan-tilt platform is provided with a stabilizing device and a mode setting module; the stabilizing device includes a fixing clamp and a gripper; the mode setting module is connected to the visual analysis and control system; the fixing clamp is a clamp with a spring to fix the product on the multi-axis pan-tilt platform, and the gripper fixes the product on the multi-axis pan-tilt platform through a machine gripper;

[0013] When the product moves onto the multi-axis pan-tilt platform, the industrial camera in the machine vision device rotates to photograph the product on the multi-axis pan-tilt platform, and transmits the photographed image to the visual analysis and control system. The visual analysis and control system performs image analysis on the photographed image to obtain the position coordinates, shape parameters, and status information of the product on the multi-axis pan-tilt platform. The shape parameters include: length, width, and height in the horizontal direction; the status information includes: product upright state, product tilted state, and product overturned state.

[0014] The visual analysis and control system sends the position coordinates, shape parameters and status information of the product on the multi-axis pan-tilt platform to the mode setting module. The mode setting module selects the stabilization method required for the product based on the received information, determines the stabilization method based on the selection result, selects and controls the stabilization device corresponding to the stabilization method, and uses the selected stabilization device to stabilize the product on the multi-axis pan-tilt platform.

[0015] Preferably, the machine vision equipment, multi-axis pan-tilt platform, pan-tilt platform positioning system, visual analysis control system, and delivery control system are arranged inside the vending machine;

[0016] The multi-axis pan-tilt platform is provided with a pan-tilt speed adjustment mode, and the speed mode is displayed on the user order interface. If the user selects the fast mode, the pan-tilt platform positioning system will enter the fast mode. The pan-tilt platform positioning system sets the moving trajectory of the shortest path to reach the cargo entrance. When the visual analysis control system detects whether the multi-axis platform has reached the preset position correctly, if the position deviation is within the set error range, no fine-tuning is performed, and the visual analysis control system directly sends a shipping instruction to the shipping control system.

[0017] Preferably, it further comprises a position error analysis system, the position error analysis system being connected to the pan-tilt positioning system and the visual analysis control system to perform error analysis and error correction on the moving coordinates of the machine vision device and the multi-axis pan-tilt platform;

[0018] The position error analysis system comprises:

[0019] The center of the image captured by the industrial camera in the machine vision device is set as the camera origin, half of the sliding travel of the multi-axis pan-tilt platform in the horizontal x direction and the vertical y direction is set as the origin format of the pan-tilt platform return to zero, and the center point of the origin format is set as the pan-tilt platform origin;

[0020] Move the multi-axis gimbal to a cargo channel entrance. The point where the center of the cargo channel entrance coincides with the center of the multi-axis gimbal is the center of the marker circle. The center of the marker circle is the absolute origin of the world coordinate system after the multi-axis gimbal returns to zero. After returning to zero, the straight line passing through the absolute origin and parallel to the x-direction is the x-axis of the world coordinate system. After returning to zero, the straight line passing through the absolute origin and parallel to the y-direction is the y-axis of the world coordinate system.

[0021] Pass the marker circle on the multi-axis gimbal through the shooting format of the industrial camera to perform image acquisition, identify the pixel coordinates of the center of the marker circle, perform difference calculation between the camera origin and the pixel coordinates to obtain the directional pixel distance coordinates, and use the affine transformation matrix to move the multi-axis gimbal from the camera origin to the shooting format and sum the moving coordinates. The summed result is the coordinate of the center of the camera format in the world coordinate system;

[0022] Based on the coordinates of the camera frame center in the world coordinate system, determine the offset between the camera coordinate system and the multi-axis gimbal coordinate system. Based on the offset and the affine transformation matrix, calculate and determine the world coordinates of any point within the camera shooting frame.

[0023] Based on the world coordinates, the preset position and movement trajectory of the multi-axis gimbal are determined, as well as the rotation angle and position of the industrial camera in the machine vision equipment.

[0024] Preferably, the visual analysis control system includes an image preprocessing module, which corrects the distortion of the image to obtain a distortion-corrected image;

[0025] The image preprocessing module is specifically configured to extract coordinate information of a dot array of an image, determine a marker dot on the image based on the coordinate information, set the marker dot as dot number one, and sort and number the identified dots from left to right and from top to bottom based on the distances and angles between the center coordinates of different dots;

[0026] The sorted and numbered dots are formed into a marker dot set, and the mapping relationship between all the marker dots in the marker dot set and the standard marker dots is determined. Based on the mapping relationship, the difference between the coordinates of each theoretical marker dot and the corresponding collected marker dot coordinates is determined and determined as the error compensation value. The error compensation value is subjected to least squares fitting of the error compensation surface to determine the error compensation surface equation. The compensation parameters required for correcting each dot are calculated based on the error compensation surface method, and the error is corrected based on the compensation parameters to obtain a distortion-corrected image.

[0027] Compared with the prior art, the present invention has the following advantages:

[0028] The present invention provides a pan-tilt receiving and adjustment system based on visual control, comprising: a machine vision device, a multi-axis pan-tilt, a pan-tilt positioning system, a visual analysis control system, and a shipping control system; the multi-axis pan-tilt is connected to the pan-tilt positioning system, and the movement trajectory of the multi-axis pan-tilt is determined based on the pan-tilt positioning system; the machine vision device is mounted on a multi-axis platform, and the visual analysis control system detects whether the multi-axis platform has reached a preset position and whether the shipped product is correct based on the multi-axis platform. If the detection is correct, a shipping instruction is sent to the shipping control system, and the shipping control system controls the movement of products in the corresponding aisle, so that the products are moved onto the multi-axis pan-tilt. Using machine vision to control pan-tilt receiving eliminates the problems associated with purely mechanical control and improves the accuracy and safety of pan-tilt receiving and adjustment.

[0029] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0030] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0032] Figure 1Schematic diagram of the structure of a pan-tilt receiving and adjusting system based on vision control in an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of the structure of a machine vision device in an embodiment of the present invention;

[0034] Figure 3 Schematic diagram of the structure of a multi-axis pan-tilt head in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0036] The embodiment of the present invention provides a pan-tilt receiving and adjusting system based on visual control, please refer to Figure 1 The system consists of the following parts:

[0037] Machine vision equipment, multi-axis pan-tilt head, pan-tilt head positioning system, visual analysis control system, shipment control system; the multi-axis pan-tilt head is connected to the pan-tilt head positioning system, and the movement trajectory of the multi-axis pan-tilt head is determined based on the pan-tilt head positioning system; the machine vision equipment is set on the multi-axis platform, and based on the visual analysis control system, it detects whether the multi-axis platform has reached the preset position and whether the shipped product is correct. If the detection is correct, a shipment instruction is sent to the shipment control system, and the shipment control system controls the movement of products in the corresponding cargo channel to move the products to the multi-axis pan-tilt head.

[0038] The working principle of the above technical solution is: the solution adopted in this embodiment includes: machine vision equipment, a multi-axis pan-tilt head, a pan-tilt head positioning system, a visual analysis control system, and a shipment control system; the multi-axis pan-tilt head is connected to the pan-tilt head positioning system, and the movement trajectory of the multi-axis pan-tilt head is determined based on the pan-tilt head positioning system; the machine vision equipment is set on the multi-axis platform, and based on the visual analysis control system, it detects whether the multi-axis platform has reached the preset position and whether the shipped product is correct. If the detection is correct, a shipment instruction is sent to the shipment control system, and the shipment control system controls the movement of products in the corresponding cargo channel to move the products to the multi-axis pan-tilt head.

[0039] The beneficial effects of the above technical solution are: the machine vision equipment, multi-axis pan-tilt head, pan-tilt head positioning system, visual analysis control system, and shipment control system provided in this embodiment are adopted; the multi-axis pan-tilt head is connected to the pan-tilt head positioning system, and the movement trajectory of the multi-axis pan-tilt head is determined based on the pan-tilt head positioning system; the machine vision equipment is set on the multi-axis platform, and based on the visual analysis control system, it detects whether the multi-axis platform has reached the preset position and whether the shipped product is correct. If the detection is correct, a shipment instruction is sent to the shipment control system, and the shipment control system controls the movement of products in the corresponding cargo channel to move the products to the multi-axis pan-tilt head.

[0040] Based on the principle of machine vision control, machine vision analysis is performed on the cargo lanes and shipped products, and the shipping time or shipping procedure is determined according to the analysis results. The existing technology relies entirely on mechanical mechanisms to design the control method for shipping control and pan-tilt movement. This embodiment reduces the degree of dependence on mechanical equipment control, and avoids the problem of all control failures caused by problems with a certain device in the mechanical equipment. For example, in this embodiment, if there is a problem with the machine vision equipment, it can be replaced alone without affecting the function and parameter setting of the multi-axis pan-tilt. Similarly, if there is a problem with the multi-axis pan-tilt, it can be replaced alone. The existing technology uses the mechanical principle of shipping control and pan-tilt movement that are closely linked. Once a problem occurs in a certain component, all of them need to be replaced or disassembled. In short, the solution provided by the embodiment of the present application not only uses machine vision analysis to improve the accuracy of control, but also avoids problems caused by complete mechanical coordination.

[0041] In another embodiment, please refer to Figure 2 , the machine vision device includes an industrial camera and a camera bracket; the industrial camera is rotatably fixed to the camera bracket;

[0042] The industrial camera is set on a rotatable base, which is fixed on the camera bracket. The rotating end of the rotatable base rotates 360 degrees relative to the rotating fulcrum. The initial position of the industrial camera is a first shooting angle. The first shooting angle is the angle for shooting products in the aisle. The image shot at the first shooting angle is transmitted to the visual analysis control system. When the visual analysis control system analyzes that the shot product is the correct product, the visual analysis control system sends an instruction to the rotation control end of the rotatable base to control the rotation end to rotate a preset angle. The working camera follows the rotation and then switches to a second shooting angle. The second shooting angle is for shooting the gap between the aisle entrance and the multi-axis pan-tilt head. The image shot at the second shooting angle is transmitted to the visual analysis control system. The visual analysis control system analyzes whether the multi-axis pan-tilt head has reached the preset position. If it has reached the preset position, the shipping instruction is sent to the shipping control system.

[0043] The working principle of the above technical solution is as follows: the solution adopted in this embodiment is that the machine vision device includes an industrial camera and a camera bracket; the industrial camera is fixed to the camera bracket in a rotatable manner;

[0044] The industrial camera is set on a rotatable base, which is fixed on the camera bracket. The rotating end of the rotatable base rotates 360 degrees relative to the rotating fulcrum. The initial position of the industrial camera is a first shooting angle. The first shooting angle is the angle for shooting products in the aisle. The image shot at the first shooting angle is transmitted to the visual analysis control system. When the visual analysis control system analyzes that the shot product is the correct product, the visual analysis control system sends an instruction to the rotation control end of the rotatable base to control the rotation end to rotate a preset angle. The working camera follows the rotation and then switches to a second shooting angle. The second shooting angle is for shooting the gap between the aisle entrance and the multi-axis pan-tilt head. The image shot at the second shooting angle is transmitted to the visual analysis control system. The visual analysis control system analyzes whether the multi-axis pan-tilt head has reached the preset position. If it has reached the preset position, the shipping instruction is sent to the shipping control system.

[0045] The beneficial effects of the above technical solution are as follows: the machine vision device according to the solution provided in this embodiment includes an industrial camera and a camera bracket; the industrial camera is fixed to the camera bracket in a rotatable manner;

[0046] The industrial camera is mounted on a rotatable base, which is fixed to the camera bracket. The rotating end of the rotatable base rotates 360 degrees relative to the rotation fulcrum. The initial position of the industrial camera is a first shooting angle, which is the angle at which the products in the aisle are photographed. The image captured at the first shooting angle is transmitted to the visual analysis control system. When the visual analysis control system determines that the photographed product is the correct product, the visual analysis control system sends a command to the rotation control end of the rotatable base, controlling the rotation end to rotate a preset angle. The working camera rotates accordingly and then switches to a second shooting angle. The second shooting angle is the gap between the aisle entrance and the multi-axis pan-tilt head. The image captured at the second shooting angle is transmitted to the visual analysis control system. The visual analysis control system analyzes whether the multi-axis pan-tilt head has reached the preset position. If so, it sends a shipping instruction to the shipping control system. The combination of the industrial camera and the camera bracket allows the industrial camera to capture images at any angle. In this embodiment, images at different angles are required. Therefore, the same set of equipment is used to complete image acquisition at different shooting angles, saving cost and space.

[0047] In another embodiment, after the image captured by the second shooting angle is transmitted to the visual analysis control system, the visual analysis control system sends instructions to the rotation control end again to control the rotation of the rotation end so that the industrial camera reaches a third shooting angle, and transmits the image captured by the third shooting angle to the visual analysis control system, and adjusts the shooting angle in a loop until the visual analysis control system determines that a three-dimensional model can be constructed based on the received image. The visual analysis control system stops sending instructions to the rotation control end, and the visual analysis control system constructs a three-dimensional model based on the received image, and determines whether the multi-axis pan-tilt head has reached the preset position based on the three-dimensional model.

[0048] The working principle of the above technical solution is: the solution adopted in this embodiment is that after the image captured by the second shooting angle is transmitted to the visual analysis control system, the visual analysis control system sends instructions to the rotation control end again to control the rotation of the rotation end to make the industrial camera reach the third shooting angle, and transmit the image captured by the third shooting angle to the visual analysis control system, and adjust the shooting angle in a cycle until the visual analysis control system determines that a three-dimensional model can be constructed based on the received image. The visual analysis control system stops sending instructions to the rotation control end, and the visual analysis control system constructs a three-dimensional model based on the received image, and determines whether the multi-axis gimbal has reached the preset position based on the three-dimensional model.

[0049] The beneficial effect of the above technical solution is as follows: when the image captured by the second shooting angle is transmitted to the visual analysis control system using the solution provided by this embodiment, the visual analysis control system sends instructions to the rotary control end again to control the rotation of the rotary end so that the industrial camera reaches the third shooting angle, and transmits the image captured by the third shooting angle to the visual analysis control system. The shooting angle is adjusted cyclically until the visual analysis control system determines that a three-dimensional model can be constructed based on the received image. The visual analysis control system stops sending instructions to the rotary control end, and the visual analysis control system constructs a three-dimensional model based on the received image, and determines whether the multi-axis pan-tilt head has reached the preset position based on the three-dimensional model. By constructing a three-dimensional model, the position of the multi-axis pan-tilt head can be monitored more accurately, avoiding the problem of the multi-axis pan-tilt head being higher or lower than the cargo aisle entrance, causing shipping jams or obstacles.

[0050] In another embodiment, if the visual analysis control system detects that the multi-axis platform has not reached the preset position, the visual analysis control system sends a fine-tuning instruction to the pan-tilt positioning system, and the fine-tuning instruction includes: the current detection position coordinates, the preset position coordinates and the moving path; the pan-tilt positioning system fine-tunes the position of the multi-axis pan-tilt according to the fine-tuning instruction; when the multi-axis pan-tilt moves to the preset position coordinates according to the moving path, the pan-tilt positioning system sends a detection instruction to the visual analysis control system, and the visual analysis control system controls the industrial camera in the machine vision equipment to capture the fine-tuned image and transmits the image to the visual analysis control system again. If the visual analysis control system detects that the multi-axis platform has reached the preset position, the fine-tuning action of the multi-axis platform is stopped. If the visual analysis control system detects that the multi-axis platform has not reached the preset position, it continues to send fine-tuning instructions to the pan-tilt positioning system and repeats the fine-tuning action until the visual analysis control system detects that the multi-axis platform has reached the preset position.

[0051] The working principle of the above technical solution is: the solution adopted in this embodiment is that if the visual analysis and control system detects that the multi-axis platform has not reached the preset position, the visual analysis and control system sends a fine-tuning instruction to the pan-tilt positioning system, and the fine-tuning instruction includes: the current detection position coordinates, the preset position coordinates and the moving path; the pan-tilt positioning system fine-tunes the position of the multi-axis pan-tilt according to the fine-tuning instruction; when the multi-axis pan-tilt moves to the preset position coordinates according to the moving path, the pan-tilt positioning system sends a detection instruction to the visual analysis and control system, and the visual analysis and control system controls the industrial camera in the machine vision equipment to capture the fine-tuned image and transmits the image to the visual analysis and control system again. If the visual analysis and control system detects that the multi-axis platform has reached the preset position, the fine-tuning action of the multi-axis platform is stopped. If the visual analysis and control system detects that the multi-axis platform has not reached the preset position, it continues to send fine-tuning instructions to the pan-tilt positioning system and repeats the fine-tuning action until the visual analysis and control system detects that the multi-axis platform has reached the preset position.

[0052] The beneficial effect of the above technical solution is as follows: if the visual analysis control system detects that the multi-axis platform has not reached the preset position, the visual analysis control system sends a fine-tuning instruction to the pan-tilt positioning system, and the fine-tuning instruction includes: the current detection position coordinates, the preset position coordinates and the movement path; the pan-tilt positioning system fine-tunes the position of the multi-axis pan-tilt according to the fine-tuning instruction; when the multi-axis pan-tilt moves to the preset position coordinates according to the movement path, the pan-tilt positioning system sends a detection instruction to the visual analysis control system, and the visual analysis control system controls the industrial camera in the machine vision device to capture the fine-tuned image and transmits the image again to the visual analysis control system. If the visual analysis control system detects that the multi-axis platform has reached the preset position, the fine-tuning action of the multi-axis platform is stopped. If the visual analysis control system detects that the multi-axis platform has not reached the preset position, it continues to send fine-tuning instructions to the pan-tilt positioning system and repeats the fine-tuning action until the visual analysis control system detects that the multi-axis platform has reached the preset position. By setting fine-tuning, the distance difference between the multi-axis pan-tilt and the cargo aisle is more accurately shortened, ensuring that small products are not missed when shipped.

[0053] In another embodiment, please refer to Figure 3 The multi-axis pan-tilt platform includes a receiving platform, a multi-axis driving device and a driving control device. The multi-axis driving device is connected to the receiving platform. The driving control device is connected to the multi-axis driving device to control the six-axis movement of the multi-axis driving device in four horizontal directions and two vertical directions. The multi-axis driving device drives the movement of the receiving platform.

[0054] The drive control device is connected to the pan-tilt positioning system and forms control instructions based on instructions from the pan-tilt positioning system for controlling the multi-axis drive device.

[0055] The working principle of the above technical solution is as follows: the solution adopted in this embodiment is that the multi-axis pan-tilt platform includes a receiving platform, a multi-axis drive device and a drive control device, the multi-axis drive device is connected to the receiving platform, the drive control device is connected to the multi-axis drive device, controls the six-axis movement of the multi-axis drive device in four horizontal directions and two vertical directions, and the multi-axis drive device drives the movement of the receiving platform;

[0056] The drive control device is connected to the pan-tilt positioning system and forms control instructions based on instructions from the pan-tilt positioning system for controlling the multi-axis drive device.

[0057] The beneficial effects of the above technical solution are as follows: the multi-axis pan-tilt platform provided by this embodiment includes a receiving platform, a multi-axis drive device and a drive control device, the multi-axis drive device is connected to the receiving platform, the drive control device is connected to the multi-axis drive device, controls the six-axis movement of the multi-axis drive device in four horizontal directions and two vertical directions, and the multi-axis drive device drives the movement of the receiving platform;

[0058] The drive control device is connected to the pan-tilt positioning system and generates control instructions based on the pan-tilt positioning system's instructions for controlling the multi-axis drive device. Multi-axis drive control ensures six degrees of freedom of movement, which can better complement the precision advantages of machine vision technology.

[0059] In another embodiment, if the visual analysis and control system detects that the shipped product is incorrect, the position of the aisle reached by the multi-axis pan-tilt stage is incorrect, and the visual analysis and control system sends a product error instruction to the pan-tilt stage positioning system. The pan-tilt stage positioning system transmits the product error instruction to the multi-axis pan-tilt stage, and the multi-axis pan-tilt stage automatically moves horizontally according to the product error instruction to change to an adjacent aisle; an industrial camera in a machine vision device on the multi-axis pan-tilt stage is used to capture images on an adjacent aisle, and the captured images are transmitted to the visual analysis and control system. The visual analysis and control system obtains the user's order information from the order platform, analyzes the product information corresponding to the user's order, performs image analysis on the captured images, obtains real-time product information, compares the real-time product information with the product information corresponding to the order, and determines whether it is the product ordered by the user. If so, a shipping instruction is sent to the shipping control system. If not, the product error instruction is continued to be sent to the pan-tilt stage positioning system, and the multi-axis pan-tilt stage continues to move horizontally until the correct shipping product is found.

[0060] The working principle of the above technical solution is: the solution adopted in this embodiment is that if the visual analysis and control system detects that the shipped product is incorrect, the position of the cargo aisle reached by the multi-axis pan-tilt stage is incorrect, the visual analysis and control system sends a product error instruction to the pan-tilt stage positioning system, and the pan-tilt stage positioning system transmits the product error instruction to the multi-axis pan-tilt stage. The multi-axis pan-tilt stage automatically moves horizontally according to the product error instruction to change to the adjacent cargo aisle; the industrial camera in the machine vision device on the multi-axis pan-tilt stage is used to capture images on the adjacent cargo aisle, and the captured images are transmitted to the visual analysis and control system. The visual analysis and control system obtains the user's order information from the order platform, analyzes the product information corresponding to the user's order, performs image analysis on the captured images, obtains real-time product information, compares the real-time product information with the product information corresponding to the order, and determines whether it is the product ordered by the user. If so, a shipping instruction is sent to the shipping control system. If not, the product error instruction is continued to be sent to the pan-tilt stage positioning system, and the multi-axis pan-tilt stage continues to move horizontally until the correct shipping product is found.

[0061] The beneficial effects of the above technical solution are as follows: if the visual analysis and control system detects that the shipped product is incorrect, the position of the cargo aisle reached by the multi-axis pan-tilt stage is incorrect when adopting the solution provided by this embodiment, the visual analysis and control system sends a product error instruction to the pan-tilt stage positioning system, and the pan-tilt stage positioning system transmits the product error instruction to the multi-axis pan-tilt stage. The multi-axis pan-tilt stage automatically moves horizontally according to the product error instruction to change to the adjacent cargo aisle; the industrial camera in the machine vision equipment on the multi-axis pan-tilt stage is used to capture images on the adjacent cargo aisle, and the captured images are transmitted to the visual analysis and control system. The visual analysis and control system obtains the user's order information from the order platform, analyzes the product information corresponding to the user's order, performs image analysis on the captured images, obtains real-time product information, compares the real-time product information with the product information corresponding to the order, and determines whether it is the product ordered by the user. If so, a shipping instruction is sent to the shipping control system. If not, the product error instruction is continued to be sent to the pan-tilt stage positioning system, and the multi-axis pan-tilt stage continues to move horizontally until the correct shipping product is found. If a bug occurs in the order system or any system, causing the product corresponding to the aisle to which the multi-axis gimbal moves to be different from the product required by the user, the control system of the multi-axis gimbal will automatically control the multi-axis gimbal to traverse each aisle in the horizontal direction, and then collect product information for each aisle based on machine vision technology to ensure that the shipped products are consistent with the user's ordered products, thereby ensuring a high user experience.

[0062] In another embodiment, the multi-axis gimbal is provided with a stabilizing device and a mode setting module; the stabilizing device includes a fixing clamp and a gripper; the mode setting module is connected to the visual analysis control system; the fixing clamp is a clamp with a spring to fix the product to the multi-axis gimbal, and the gripper fixes the product to the multi-axis gimbal via a machine gripper;

[0063] When the product moves onto the multi-axis pan-tilt platform, the industrial camera in the machine vision device rotates to photograph the product on the multi-axis pan-tilt platform, and transmits the photographed image to the visual analysis and control system. The visual analysis and control system performs image analysis on the photographed image to obtain the position coordinates, shape parameters, and status information of the product on the multi-axis pan-tilt platform. The shape parameters include: length, width, and height in the horizontal direction; the status information includes: product upright state, product tilted state, and product overturned state.

[0064] The visual analysis and control system sends the position coordinates, shape parameters and status information of the product on the multi-axis pan-tilt platform to the mode setting module. The mode setting module selects the stabilization method required for the product based on the received information, determines the stabilization method based on the selection result, selects and controls the stabilization device corresponding to the stabilization method, and uses the selected stabilization device to stabilize the product on the multi-axis pan-tilt platform.

[0065] The working principle of the above technical solution is as follows: the solution adopted in this embodiment is that the multi-axis pan-tilt platform is provided with a stabilization device and a mode setting module; the stabilization device includes a fixed clamp and a gripper; the mode setting module is connected to the visual analysis control system; the fixed clamp is a clamp with a spring to fix the product to the multi-axis pan-tilt platform, and the gripper fixes the product to the multi-axis pan-tilt platform through a machine gripper;

[0066] When the product moves onto the multi-axis pan-tilt platform, the industrial camera in the machine vision device rotates to photograph the product on the multi-axis pan-tilt platform, and transmits the photographed image to the visual analysis and control system. The visual analysis and control system performs image analysis on the photographed image to obtain the position coordinates, shape parameters, and status information of the product on the multi-axis pan-tilt platform. The shape parameters include: length, width, and height in the horizontal direction; the status information includes: product upright state, product tilted state, and product overturned state.

[0067] The visual analysis and control system sends the position coordinates, shape parameters and status information of the product on the multi-axis pan-tilt platform to the mode setting module. The mode setting module selects the stabilization method required for the product based on the received information, determines the stabilization method based on the selection result, selects and controls the stabilization device corresponding to the stabilization method, and uses the selected stabilization device to stabilize the product on the multi-axis pan-tilt platform.

[0068] The beneficial effects of the above technical solution are as follows: the multi-axis pan-tilt platform provided by the solution of this embodiment is provided with a stabilization device and a mode setting module; the stabilization device includes a fixed clamp and a gripper; the mode setting module is connected to the visual analysis control system; the fixed clamp is a clamp with a spring to fix the product on the multi-axis pan-tilt platform, and the gripper fixes the product on the multi-axis pan-tilt platform through a machine gripper;

[0069] When the product moves onto the multi-axis pan-tilt platform, the industrial camera in the machine vision device rotates to photograph the product on the multi-axis pan-tilt platform, and transmits the photographed image to the visual analysis and control system. The visual analysis and control system performs image analysis on the photographed image to obtain the position coordinates, shape parameters, and status information of the product on the multi-axis pan-tilt platform. The shape parameters include: length, width, and height in the horizontal direction; the status information includes: product upright state, product tilted state, and product overturned state.

[0070] The visual analysis and control system transmits the product's position coordinates, shape parameters, and status information on the multi-axis platform to the mode setting module. Based on this information, the mode setting module selects the desired stabilization method for the product, determines the stabilization method based on the selected result, and then selects and controls the stabilization device corresponding to the stabilization method. The selected stabilization device is then used to stabilize the product on the multi-axis platform. Holding the product in a fixed manner rather than in a frame ensures product stability during shipment and avoids the negative impact of collisions on carbonated beverages or fragile products.

[0071] In another embodiment, the machine vision device, multi-axis pan-tilt head, pan-tilt head positioning system, visual analysis control system, and delivery control system are arranged inside the vending machine;

[0072] The multi-axis pan-tilt platform is provided with a pan-tilt speed adjustment mode, and the speed mode is displayed on the user order interface. If the user selects the fast mode, the pan-tilt platform positioning system will enter the fast mode. The pan-tilt platform positioning system sets the moving trajectory of the shortest path to reach the cargo entrance. When the visual analysis control system detects whether the multi-axis platform has reached the preset position correctly, if the position deviation is within the set error range, no fine-tuning is performed, and the visual analysis control system directly sends a shipping instruction to the shipping control system.

[0073] The working principle of the above technical solution is as follows: the solution adopted in this embodiment is that the machine vision device, multi-axis pan-tilt head, pan-tilt head positioning system, visual analysis control system, and delivery control system are arranged inside the vending machine;

[0074] The multi-axis pan-tilt platform is provided with a pan-tilt speed adjustment mode, and the speed mode is displayed on the user order interface. If the user selects the fast mode, the pan-tilt platform positioning system will enter the fast mode. The pan-tilt platform positioning system sets the moving trajectory of the shortest path to reach the cargo entrance. When the visual analysis control system detects whether the multi-axis platform has reached the preset position correctly, if the position deviation is within the set error range, no fine-tuning is performed, and the visual analysis control system directly sends a shipping instruction to the shipping control system.

[0075] The beneficial effects of the above technical solution are as follows: the machine vision device, multi-axis pan-tilt head, pan-tilt head positioning system, visual analysis control system, and delivery control system described in the solution provided by this embodiment are arranged inside the vending machine;

[0076] The multi-axis gimbal is equipped with a gimbal speed adjustment mode, and the speed mode is displayed on the user order interface. If the user selects the fast mode, the gimbal positioning system will enter the fast mode. The gimbal positioning system sets the shortest path movement trajectory to reach the cargo entrance. Based on the visual analysis control system, it detects whether the multi-axis platform has reached the preset position correctly. If the position deviation is within the set error range, no fine-tuning is performed, and the visual analysis control system directly sends the shipping instruction to the shipping control system. The setting of the fast channel ensures that when the user is in urgent need, saving time is the first factor, ignoring other factors that affect the shipping speed, and ensuring the user's first need.

[0077] In another embodiment, a position error analysis system is further included, wherein the position error analysis system is connected to the pan-tilt positioning system and the visual analysis control system to perform error analysis and error correction on the moving coordinates of the machine vision device and the multi-axis pan-tilt position;

[0078] The position error analysis system comprises:

[0079] The center of the image captured by the industrial camera in the machine vision device is set as the camera origin, half of the sliding travel of the multi-axis pan-tilt platform in the horizontal x direction and the vertical y direction is set as the origin format of the pan-tilt platform return to zero, and the center point of the origin format is set as the pan-tilt platform origin;

[0080] Move the multi-axis gimbal to a cargo channel entrance. The point where the center of the cargo channel entrance coincides with the center of the multi-axis gimbal is the center of the marker circle. The center of the marker circle is the absolute origin of the world coordinate system after the multi-axis gimbal returns to zero. After returning to zero, the straight line passing through the absolute origin and parallel to the x-direction is the x-axis of the world coordinate system. After returning to zero, the straight line passing through the absolute origin and parallel to the y-direction is the y-axis of the world coordinate system.

[0081] Pass the marker circle on the multi-axis gimbal through the shooting format of the industrial camera to perform image acquisition, identify the pixel coordinates of the center of the marker circle, perform difference calculation between the camera origin and the pixel coordinates to obtain the directional pixel distance coordinates, and use the affine transformation matrix to move the multi-axis gimbal from the camera origin to the shooting format and sum the moving coordinates. The summed result is the coordinate of the center of the camera format in the world coordinate system;

[0082] Based on the coordinates of the camera frame center in the world coordinate system, determine the offset between the camera coordinate system and the multi-axis gimbal coordinate system. Based on the offset and the affine transformation matrix, calculate and determine the world coordinates of any point within the camera shooting frame.

[0083] Based on the world coordinates, the preset position and movement trajectory of the multi-axis gimbal are determined, as well as the rotation angle and position of the industrial camera in the machine vision equipment.

[0084] The working principle of the above technical solution is as follows: the solution adopted in this embodiment further includes a position error analysis system, which is connected to the pan-tilt positioning system and the visual analysis control system to perform error analysis and error correction on the moving coordinates of the machine vision device and the multi-axis pan-tilt platform;

[0085] The position error analysis system comprises:

[0086] The center of the image captured by the industrial camera in the machine vision device is set as the camera origin, half of the sliding travel of the multi-axis pan-tilt platform in the horizontal x direction and the vertical y direction is set as the origin format of the pan-tilt platform return to zero, and the center point of the origin format is set as the pan-tilt platform origin;

[0087] Move the multi-axis gimbal to a cargo channel entrance. The point where the center of the cargo channel entrance coincides with the center of the multi-axis gimbal is the center of the marker circle. The center of the marker circle is the absolute origin of the world coordinate system after the multi-axis gimbal returns to zero. After returning to zero, the straight line passing through the absolute origin and parallel to the x-direction is the x-axis of the world coordinate system. After returning to zero, the straight line passing through the absolute origin and parallel to the y-direction is the y-axis of the world coordinate system.

[0088] Pass the marker circle on the multi-axis gimbal through the shooting format of the industrial camera to perform image acquisition, identify the pixel coordinates of the center of the marker circle, perform difference calculation between the camera origin and the pixel coordinates to obtain the directional pixel distance coordinates, and use the affine transformation matrix to move the multi-axis gimbal from the camera origin to the shooting format and sum the moving coordinates. The summed result is the coordinate of the center of the camera format in the world coordinate system;

[0089] Based on the coordinates of the camera frame center in the world coordinate system, determine the offset between the camera coordinate system and the multi-axis gimbal coordinate system. Based on the offset and the affine transformation matrix, calculate and determine the world coordinates of any point within the camera shooting frame.

[0090] Based on the world coordinates, the preset position and movement trajectory of the multi-axis gimbal are determined, as well as the rotation angle and position of the industrial camera in the machine vision equipment.

[0091] The beneficial effects of the above technical solution are as follows: the solution provided by this embodiment also includes a position error analysis system, which is connected to the pan-tilt positioning system and the visual analysis control system to perform error analysis and error correction on the moving coordinates of the machine vision device and the multi-axis pan-tilt platform;

[0092] The position error analysis system comprises:

[0093] The center of the image captured by the industrial camera in the machine vision device is set as the camera origin, half of the sliding travel of the multi-axis pan-tilt platform in the horizontal x direction and the vertical y direction is set as the origin format of the pan-tilt platform return to zero, and the center point of the origin format is set as the pan-tilt platform origin;

[0094] Move the multi-axis gimbal to a cargo channel entrance. The point where the center of the cargo channel entrance coincides with the center of the multi-axis gimbal is the center of the marker circle. The center of the marker circle is the absolute origin of the world coordinate system after the multi-axis gimbal returns to zero. After returning to zero, the straight line passing through the absolute origin and parallel to the x-direction is the x-axis of the world coordinate system. After returning to zero, the straight line passing through the absolute origin and parallel to the y-direction is the y-axis of the world coordinate system.

[0095] Pass the marker circle on the multi-axis gimbal through the shooting format of the industrial camera to perform image acquisition, identify the pixel coordinates of the center of the marker circle, perform difference calculation between the camera origin and the pixel coordinates to obtain the directional pixel distance coordinates, and use the affine transformation matrix to move the multi-axis gimbal from the camera origin to the shooting format and sum the moving coordinates. The summed result is the coordinate of the center of the camera format in the world coordinate system;

[0096] Based on the coordinates of the camera frame center in the world coordinate system, determine the offset between the camera coordinate system and the multi-axis gimbal coordinate system. Based on the offset and the affine transformation matrix, calculate and determine the world coordinates of any point within the camera shooting frame.

[0097] Based on world coordinates, the preset position and movement trajectory of the multi-axis gimbal are determined, as well as the rotation angle and position of the industrial camera in the machine vision equipment. By unifying the world coordinate system, the position coordinates of the industrial camera and the multi-axis gimbal are easily controlled.

[0098] In another embodiment, the visual analysis control system includes an image preprocessing module, which corrects the distortion of the image to obtain a distortion-corrected image;

[0099] The image preprocessing module is specifically configured to extract coordinate information of a dot array of an image, determine a marker dot on the image based on the coordinate information, set the marker dot as dot number one, and sort and number the identified dots from left to right and from top to bottom based on the distances and angles between the center coordinates of different dots;

[0100] The sorted and numbered dots are formed into a marker dot set, and the mapping relationship between all the marker dots in the marker dot set and the standard marker dots is determined. Based on the mapping relationship, the difference between the coordinates of each theoretical marker dot and the corresponding collected marker dot coordinates is determined and determined as the error compensation value. The error compensation value is subjected to least squares fitting of the error compensation surface to determine the error compensation surface equation. The compensation parameters required for correcting each dot are calculated based on the error compensation surface method, and the error is corrected based on the compensation parameters to obtain a distortion-corrected image.

[0101] The working principle of the above technical solution is as follows: the solution adopted in this embodiment is that the visual analysis control system includes an image preprocessing module, and the image preprocessing module corrects the distortion of the image to obtain a distortion-corrected image;

[0102] The image preprocessing module is specifically configured to extract coordinate information of a dot array of an image, determine a marker dot on the image based on the coordinate information, set the marker dot as dot number one, and sort and number the identified dots from left to right and from top to bottom based on the distances and angles between the center coordinates of different dots;

[0103] The sorted and numbered dots are formed into a marker dot set, and the mapping relationship between all the marker dots in the marker dot set and the standard marker dots is determined. Based on the mapping relationship, the difference between the coordinates of each theoretical marker dot and the corresponding collected marker dot coordinates is determined and determined as the error compensation value. The error compensation value is subjected to least squares fitting of the error compensation surface to determine the error compensation surface equation. The compensation parameters required for correcting each dot are calculated based on the error compensation surface method, and the error is corrected based on the compensation parameters to obtain a distortion-corrected image.

[0104] The beneficial effects of the above technical solution are as follows: the visual analysis control system according to the solution provided in this embodiment includes an image preprocessing module, which corrects the distortion of the image to obtain a distortion-corrected image;

[0105] The image preprocessing module is specifically configured to extract coordinate information of a dot array of an image, determine a marker dot on the image based on the coordinate information, set the marker dot as dot number one, and sort and number the identified dots from left to right and from top to bottom based on the distances and angles between the center coordinates of different dots;

[0106] The sorted and numbered dots are formed into a marker dot set. The mapping relationship between all the marker dots in the marker dot set and the standard marker dots is determined. Based on the mapping relationship, the difference between the coordinates of each theoretical marker dot and the corresponding collected marker dot coordinates is determined and used as the error compensation value. The error compensation value is then fitted to an error compensation surface using a least squares method to determine the error compensation surface equation. The compensation parameters required to correct each dot are calculated based on the error compensation surface. The errors are corrected based on the compensation parameters to obtain a distortion-corrected image. Distortion correction ensures the accuracy of image processing and improves the precision of image-based control.

[0107] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A pan-tilt receiving and adjusting system based on vision control, characterized in that: include: A machine vision device, a multi-axis pan-tilt head, a pan-tilt head positioning system, a visual analysis control system, and a shipping control system; the multi-axis pan-tilt head is connected to the pan-tilt head positioning system, and the movement trajectory of the multi-axis pan-tilt head is determined based on the pan-tilt head positioning system; the machine vision device is set on the multi-axis pan-tilt head, and based on the visual analysis control system, it detects whether the multi-axis pan-tilt head has reached a preset position and whether the shipped product is correct. If the detection is correct, a shipping instruction is sent to the shipping control system, and the shipping control system controls the movement of products in the corresponding cargo lanes, so that the products are moved to the multi-axis pan-tilt head; The machine vision device includes an industrial camera and a camera bracket; the industrial camera is rotatably fixed to the camera bracket; The industrial camera is set on a rotatable base, which is fixed on the camera bracket. The rotating end of the rotatable base rotates 360 degrees relative to the rotating fulcrum. The initial position of the industrial camera is a first shooting angle. The first shooting angle is the angle for shooting products in the aisle. The image shot at the first shooting angle is transmitted to the visual analysis control system. When the visual analysis control system analyzes that the shot product is the correct product, the visual analysis control system sends an instruction to the rotation control end of the rotatable base to control the rotation end to rotate a preset angle. The working camera follows the rotation and then switches to a second shooting angle. The second shooting angle is for shooting the gap between the aisle entrance and the multi-axis pan-tilt head. The image shot at the second shooting angle is transmitted to the visual analysis control system. The visual analysis control system analyzes whether the multi-axis pan-tilt head has reached the preset position. If it has reached the preset position, the shipping instruction is sent to the shipping control system.

2. The pan-tilt receiving and adjusting system based on vision control according to claim 1 is characterized in that: After the image captured by the second shooting angle is transmitted to the visual analysis control system, the visual analysis control system sends instructions to the rotation control end again to control the rotation of the rotation end to make the industrial camera reach the third shooting angle, and transmits the image captured by the third shooting angle to the visual analysis control system, and adjusts the shooting angle in a cycle until the visual analysis control system determines that a three-dimensional model can be constructed based on the received image. The visual analysis control system stops sending instructions to the rotation control end, and the visual analysis control system constructs a three-dimensional model based on the received image, and determines whether the multi-axis gimbal has reached the preset position based on the three-dimensional model.

3. The pan-tilt receiving and adjusting system based on vision control according to claim 1 is characterized in that: If the visual analysis control system detects that the multi-axis gimbal has not reached the preset position, the visual analysis control system sends a fine-tuning instruction to the gimbal positioning system, and the fine-tuning instruction includes: the current detection position coordinates, the preset position coordinates and the moving path; the gimbal positioning system fine-tunes the position of the multi-axis gimbal according to the fine-tuning instruction; when the multi-axis gimbal moves to the preset position coordinates according to the moving path, the gimbal positioning system sends a detection instruction to the visual analysis control system, and the visual analysis control system controls the industrial camera in the machine vision equipment to capture the fine-tuned image and transmits the image to the visual analysis control system again. If the visual analysis control system detects that the multi-axis gimbal has reached the preset position, the fine-tuning action of the multi-axis gimbal is stopped. If the visual analysis control system detects that the multi-axis gimbal has not reached the preset position, it continues to send fine-tuning instructions to the gimbal positioning system and repeats the fine-tuning action until the visual analysis control system detects that the multi-axis gimbal has reached the preset position.

4. The visual control-based pan-tilt receiving and adjusting system according to claim 1 is characterized in that: The multi-axis pan-tilt platform includes a receiving platform, a multi-axis driving device and a driving control device, wherein the multi-axis driving device is connected to the receiving platform, and the driving control device is connected to the multi-axis driving device to control the six-axis movement of the multi-axis driving device in four horizontal directions and two vertical directions, and the multi-axis driving device drives the movement of the receiving platform; The drive control device is connected to the pan-tilt positioning system and forms control instructions based on instructions from the pan-tilt positioning system for controlling the multi-axis drive device.

5. The visual control-based pan-tilt receiving and adjusting system according to claim 1 is characterized in that: If the visual analysis and control system detects that the shipped product is incorrect, and the position of the cargo aisle reached by the multi-axis pan-tilt head is incorrect, the visual analysis and control system sends a product error instruction to the pan-tilt head positioning system, and the pan-tilt head positioning system transmits the product error instruction to the multi-axis pan-tilt head, and the multi-axis pan-tilt head automatically moves horizontally according to the product error instruction to change to an adjacent cargo aisle; The industrial camera in the machine vision equipment on the multi-axis pan-tilt head is used to capture images on adjacent aisles, and the captured images are transmitted to the visual analysis and control system. The visual analysis and control system obtains the user's order information from the order platform, analyzes the product information corresponding to the user's order, performs image analysis on the captured images, obtains real-time product information, compares the real-time product information with the product information corresponding to the order, and determines whether it is the product ordered by the user. If so, a shipping instruction is sent to the shipment control system. If not, a product error instruction is continued to be sent to the pan-tilt head positioning system, and the multi-axis pan-tilt head continues to move horizontally until the correct shipment product is found.

6. The visual control-based pan-tilt receiving and adjusting system according to claim 1 is characterized in that: The multi-axis gimbal is provided with a stabilizing device and a mode setting module; the stabilizing device includes a fixing clamp and a gripper; the mode setting module is connected to the visual analysis control system; the fixing clamp is a clamp with a spring to fix the product to the multi-axis gimbal, and the gripper fixes the product to the multi-axis gimbal via a machine gripper; When the product moves onto the multi-axis pan-tilt platform, the industrial camera in the machine vision device rotates to photograph the product on the multi-axis pan-tilt platform, and transmits the photographed image to the visual analysis and control system. The visual analysis and control system performs image analysis on the photographed image to obtain the position coordinates, shape parameters, and status information of the product on the multi-axis pan-tilt platform. The shape parameters include: length, width, and height in the horizontal direction; the status information includes: product upright state, product tilted state, and product overturned state. The visual analysis and control system sends the position coordinates, shape parameters and status information of the product on the multi-axis pan-tilt platform to the mode setting module. The mode setting module selects the stabilization method required for the product based on the received information, determines the stabilization method based on the selection result, selects and controls the stabilization device corresponding to the stabilization method, and uses the selected stabilization device to stabilize the product on the multi-axis pan-tilt platform.

7. The pan-tilt receiving and adjusting system based on vision control according to claim 1 is characterized in that: The machine vision equipment, multi-axis pan-tilt head, pan-tilt head positioning system, visual analysis control system, and delivery control system are arranged inside the vending machine; The multi-axis pan-tilt head is provided with a pan-tilt head speed adjustment mode, and the speed mode is displayed on the user order interface. If the user selects the fast mode, the pan-tilt head positioning system will enter the fast mode. The pan-tilt head positioning system sets the moving trajectory of the shortest path to reach the cargo entrance. When the visual analysis control system detects whether the multi-axis pan-tilt head has reached the preset position correctly, if the position deviation is within the set error range, no fine-tuning is performed, and the visual analysis control system directly sends a shipping instruction to the shipping control system.

8. The pan-tilt receiving and adjusting system based on vision control according to claim 1 is characterized in that: It also includes a position error analysis system, which is connected to the pan-tilt positioning system and the visual analysis control system to perform error analysis and error correction on the moving coordinates of the machine vision device and the multi-axis pan-tilt platform; The position error analysis system comprises: The center of the image captured by the industrial camera in the machine vision device is set as the camera origin, half of the sliding travel of the multi-axis pan-tilt platform in the horizontal x direction and the vertical y direction is set as the origin format of the pan-tilt platform return to zero, and the center point of the origin format is set as the pan-tilt platform origin; Move the multi-axis gimbal to a cargo channel entrance. The point where the center of the cargo channel entrance coincides with the center of the multi-axis gimbal is the center of the marker circle. The center of the marker circle is the absolute origin of the world coordinate system after the multi-axis gimbal returns to zero. After returning to zero, the straight line passing through the absolute origin and parallel to the x-direction is the x-axis of the world coordinate system. After returning to zero, the straight line passing through the absolute origin and parallel to the y-direction is the y-axis of the world coordinate system. Pass the marker circle on the multi-axis gimbal through the shooting format of the industrial camera to perform image acquisition, identify the pixel coordinates of the center of the marker circle, perform difference calculation between the camera origin and the pixel coordinates to obtain the directional pixel distance coordinates, and use the affine transformation matrix to move the multi-axis gimbal from the camera origin to the shooting format and sum the moving coordinates. The summed result is the coordinate of the center of the camera format in the world coordinate system; Based on the coordinates of the camera frame center in the world coordinate system, determine the offset between the camera coordinate system and the multi-axis gimbal coordinate system. Based on the offset and the affine transformation matrix, calculate and determine the world coordinates of any point within the camera shooting frame. Based on the world coordinates, the preset position and movement trajectory of the multi-axis gimbal are determined, as well as the rotation angle and position of the industrial camera in the machine vision equipment.

9. The pan-tilt receiving and adjusting system based on vision control according to claim 1 is characterized in that: The visual analysis control system includes an image preprocessing module, which corrects the distortion of the image to obtain a distortion-corrected image; The image preprocessing module is specifically configured to extract coordinate information of a dot array of an image, determine a marker dot on the image based on the coordinate information, set the marker dot as dot number one, and sort and number the identified dots from left to right and from top to bottom based on the distances and angles between the center coordinates of different dots; The sorted and numbered dots are formed into a marker dot set, and the mapping relationship between all the marker dots in the marker dot set and the standard marker dots is determined. Based on the mapping relationship, the difference between the coordinates of each theoretical marker dot and the corresponding collected marker dot coordinates is determined and determined as the error compensation value. The error compensation value is subjected to least squares fitting of the error compensation surface to determine the error compensation surface equation. The compensation parameters required for correcting each dot are calculated based on the error compensation surface method, and the error is corrected based on the compensation parameters to obtain a distortion-corrected image.

Images