Material box car hopper motor and camera calibration method and system, storage medium

Abstract

The application discloses a kind of material box car hopper motor and camera calibration method and system, storage medium, wherein method steps include: visual reference mark is set on the vehicle body structural member of material box car, establishes hopper coordinate system and reference coordinate system;Adjust hopper mechanism, so that hopper coordinate system and reference coordinate system parallel coplanar and preset coordinate axis opposite collinear, calibrate hopper motor zero position;Visual reference mark is identified by identification camera, obtain the initial pose of identification camera in reference coordinate system, and its coordinate component in the plane of reference coordinate system;Control hopper mechanism to drive identification camera to scan visual reference mark, obtain a plurality of groups of identification camera in the sampling pose of reference coordinate system, to fit to obtain trajectory parameter;According to trajectory parameter and coordinate component, calculate the offset distance of hopper coordinate system origin in reference coordinate system, establish the pose transformation relationship of reference coordinate system relative to hopper coordinate system and calibrate camera external parameter pose in combination with initial pose.

Description

Technical Field

[0001] This invention relates to multi-sensor calibration technology for mobile robots, and more particularly to a method, system, and storage medium for joint calibration of the zero position of the hopper rotary motor and the external parameters of the onboard recognition camera of a hopper cart. Background Technology

[0002] A box-type transport robot (also known as a box-type handling robot) is a type of mobile robot widely used in warehousing and logistics scenarios. It achieves automated loading and unloading of goods through a rotating hopper mechanism. (Reference) Figure 1 , Figure 2 As shown, a hopper truck is typically equipped with two sets of recognition cameras: a rear recognition camera (usually a 3D camera) to identify the position of the goods to be retrieved, and a front recognition camera (usually a 2D camera) to identify the position of the rack stacks, thereby controlling the hopper telescopic mechanism to complete precise picking and placing operations. In addition, the hopper truck can also navigate directly to a fixed location and orientation without relying on visual recognition, and perform picking and placing operations by controlling the hopper telescopic mechanism.

[0003] In both operating modes described above, the extrinsic parameter calibration accuracy of the recognition camera directly determines the accuracy of visual recognition for picking up and placing goods, while the zero-position accuracy of the hopper rotary motor directly affects the picking and placing accuracy in non-recognition mode. Therefore, the camera extrinsic parameter calibration accuracy and the motor zero-position accuracy are two key parameters for the hopper cart to achieve high-precision operation.

[0004] Currently, existing technologies have proposed calibration schemes for the external parameter calibration of bin truck recognition cameras. For example, patent CN118628587A discloses a method for correcting the loading and unloading deviation of a bin handling robot. This method involves manually adjusting the calibration plate to the camera's field of view, and then manually measuring the position parameters of the calibration tool in the pallet coordinate system to complete the calibration.

[0005] However, such existing technical solutions still have some obvious drawbacks. On the one hand, they require operators to manually place the calibration plate, adjust its position and orientation to be within the camera's field of view, and manually read the three-dimensional coordinates of the calibration plate in the pallet coordinate system using external measuring tools. The entire process is cumbersome and time-consuming, and manual measurement inevitably introduces large errors, making it difficult to meet the consistency and accuracy requirements of mass production. On the other hand, this method only calibrates the camera's extrinsic parameters and does not involve the calibration of the zero position of the hopper rotary motor. If a step-by-step calibration method is adopted, not only will the operation process be lengthy, but the overall calibration accuracy will also be reduced due to the accumulation of errors. At the same time, since the front camera is installed at the front of the hopper and its field of view faces the material rack, it is extremely inconvenient to manually arrange the calibration fixture in the narrow passage in front.

[0006] It is evident that existing hopper vehicle calibration technologies are inadequate in terms of ease of operation, calibration accuracy, parameter coverage, and field applicability. Therefore, there is an urgent need in this field for a calibration solution that does not require external calibration fixtures, can be completed on the vehicle itself, and can simultaneously calibrate the zero position of the hopper rotary motor and the external parameters of the recognition camera. Summary of the Invention

[0007] Therefore, the main objective of this invention is to provide a calibration method and system for the hopper motor and camera of a material box vehicle, as well as a storage medium, to solve the technical problems mentioned in the background art.

[0008] To achieve the above objectives, according to one aspect of the present invention, a method for calibrating a hopper motor and camera in a hopper cart is provided, the steps of which include:

[0009] Visual reference marks are set on the body structure of the hopper car, and a hopper coordinate system and a reference coordinate system are established that are fixed to the hopper mechanism and the visual reference marks respectively, and have preset relative position constraints in the theoretical installation state.

[0010] Adjust the hopper mechanism so that the hopper coordinate system is parallel and coplanar with the reference coordinate system and the preset coordinate axes are facing each other and collinear, in order to calibrate the zero position of the hopper motor;

[0011] When the hopper motor is in the zero position, the visual reference mark is identified by the recognition camera, the initial pose of the recognition camera in the reference coordinate system is obtained, and the coordinate components of the initial pose in two orthogonal directions in the reference coordinate system plane are recorded.

[0012] The hopper mechanism is controlled to rotate, which drives the recognition camera to recognize visual reference marks from multiple angles. Multiple sets of sampling poses of the recognition camera in the reference coordinate system are obtained, and the trajectory parameters of the recognition camera rotating around the origin of the hopper coordinate system are fitted.

[0013] Based on the trajectory parameters and coordinate components, the offset distance of the origin of the hopper coordinate system in the reference coordinate system is calculated, the pose transformation relationship between the reference coordinate system and the hopper coordinate system is established, and the extrinsic pose of the recognition camera in the hopper coordinate system is calculated based on the pose transformation relationship and the initial pose.

[0014] In a possible preferred embodiment, the visual reference marker is any one of a fixed-size triplet, a checkerboard pattern, or a dotted grid.

[0015] In a possible preferred embodiment, the recognition camera includes at least one of a 3D rear recognition camera mounted at the rear of the hopper mechanism or a 2D front recognition camera mounted at the front of the hopper mechanism.

[0016] In a possible preferred embodiment, the vehicle body structural component is the crossbeam of the bin truck, and the visual reference mark is located at the middle position of the crossbeam.

[0017] In a possible preferred embodiment, the hopper coordinate system is defined on the bottom plane of the tray of the hopper mechanism, with the origin being the hopper rotation center, the hopper rising direction being the positive Z-axis direction, and the hopper opening direction being the positive X-axis direction; the reference coordinate system is defined at the center of the visual reference mark, with the hopper rising direction being the positive Z-axis direction, the X-axis pointing towards the hopper mechanism, and the Y-axis determined by the right-hand coordinate system.

[0018] In a possible preferred embodiment, the preset relative position constraint includes: the origin of the hopper coordinate system is located on the X-axis of the reference coordinate system in the theoretical installation state.

[0019] In a possible preferred embodiment, the pre-set coordinate axes being collinear and facing each other means that the X-axis of the hopper coordinate system and the reference coordinate system are collinear and facing each other.

[0020] In a possible preferred embodiment, the trajectory parameter is the radius R of the circular trajectory; the offset distance D is determined according to the formula... The calculations show that X0 and Y0 are the coordinate components of the initial pose in two orthogonal directions within the XOY plane of the reference coordinate system.

[0021] To achieve the above objectives, corresponding to the above method example, according to another aspect of the present invention, a bin cart calibration system is also provided, comprising:

[0022] A visual reference mark is set at the middle position of the crossbeam of the hopper cart;

[0023] The control unit is communicatively connected to the hopper motor and the recognition camera, and is used to adjust the pose of the hopper mechanism and trigger the recognition camera to perform image acquisition on the visual reference mark;

[0024] The processing unit is communicatively connected to the recognition camera and the control unit, and is used to receive the image acquisition results and perform pose calculation and trajectory fitting.

[0025] A storage unit for storing a program of calibration method steps as described in any of the above examples;

[0026] The control unit and the processing unit are configured to perform collaboratively via the calibration program: In the zero-position calibration state of the hopper motor, where the hopper coordinate system and the reference coordinate system are adjusted to be parallel and coplanar, with corresponding coordinate axes collinear and facing each other, the initial pose of the recognition camera relative to the visual reference mark is acquired; the hopper mechanism is controlled to rotate and scan within a preset range to acquire multiple sets of sampled poses; the trajectory parameters of the recognition camera rotating around the origin of the hopper coordinate system are obtained by fitting the multiple sets of sampled poses; and the pose transformation relationship between the hopper coordinate system and the reference coordinate system is determined based on the trajectory parameters and the initial pose, so as to jointly solve the zero position of the hopper motor and the extrinsic pose of the recognition camera in the hopper coordinate system.

[0027] To achieve the above objectives, in accordance with the above method examples, according to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the calibration method for the hopper motor and camera of the hopper car as described above.

[0028] The hopper motor and camera calibration method, system, and storage medium provided by this invention cleverly utilize the structural characteristics of the hopper car, directly setting visual reference marks on the car's own structural components (such as crossbeams). Using these inherent structural components as calibration references replaces the cumbersome steps of relying on external calibration fixtures and manual operation in existing technologies, avoiding human-introduced installation errors. It particularly solves the problem of limited space for front camera calibration, significantly reducing the difficulty of calibration operations and improving production line efficiency and after-sales maintenance convenience. Furthermore, this invention is the first to integrate hopper motor zero-position calibration and recognition camera extrinsic parameter calibration into a single technical process, simultaneously completing the calibration of two key parameters in a single operation sequence. Compared to the step-by-step independent calibration method in existing technologies, this invention avoids the accumulation of errors caused by coordinate system transformation, inconsistent references, and other factors during step-by-step calibration, improving overall calibration accuracy. Simultaneously, completing both calibrations in a single operation significantly shortens calibration time and reduces labor costs. Attached Figure Description

[0029] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0030] Figures 1 to 3 The calibration method and system of the present invention are illustrated in the material box cart structure to which they are applicable, wherein... Figure 1 , Figure 3 The example shown has a triplet code set at the crossbeam of the hopper cart;

[0031] Figure 4This is a schematic diagram illustrating the relative positional constraint relationship between the hopper coordinate system and the reference coordinate system under the theoretical installation state in the calibration method and system of the present invention.

[0032] Figure 5 This is a schematic diagram of the calibration method steps of the present invention;

[0033] Figure 6 This is a schematic diagram illustrating the pose calculation relationship of the identification camera in the hopper coordinate system within the calibration method and system of the present invention.

[0034] Figure 7 This is a schematic diagram illustrating the layout of a laser testing scheme in the calibration method and system of the present invention, used to determine whether the hopper coordinate system and the reference coordinate system are aligned.

[0035] Figure 8 This is a schematic diagram of the calibration system structure of the present invention.

[0036] Explanation of reference numerals in the attached figures

[0037] 1. Hopper, 2. Robot mobile chassis, 3. Crossbeam, 4. Front recognition camera, 5. Rear recognition camera, 6. Hopper mechanism, 7. Laser emitter, 8. Triple code, 9. Reference coordinate system O, 10. Hopper coordinate system W. Detailed Implementation

[0038] To enable those skilled in the art to better understand the technical solutions of the present invention, the specific technical solutions of the present invention will be clearly and completely described below in conjunction with embodiments, so as to help those skilled in the art further understand the present invention. Obviously, the embodiments described in this application are merely some embodiments of the present invention, and not all embodiments. It should be noted that, for those skilled in the art, the embodiments and features in the embodiments of this application can be combined with each other without departing from the concept of the present invention and without conflict. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the disclosure and protection scope of the present invention.

[0039] Furthermore, the terms "first," "second," "S100," "S200," etc., used in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such features can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those described herein. At the same time, the stages described in each step are not necessarily to be implemented in the same step; it should be understood that the implementation order of the contents of each step stage can be adjusted and interchanged without violating the inventive concept, so that embodiments of the invention described herein can be implemented in orders other than those described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. Unless otherwise expressly specified and limited, the terms "set," "arrange," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; a mechanical connection or an electrical connection; a direct connection or an indirect connection through an intermediate medium; or a connection within two elements. Those skilled in the art can understand the specific meaning of the above terms in this case based on the specific circumstances and in conjunction with existing technology.

[0040] To achieve calibration without external tooling, the calibration can be completed within the vehicle itself, and it can simultaneously calibrate the zero position of the hopper rotary motor and the extrinsic parameters of the recognition camera. For example... Figures 1 to 6 As shown, the present invention provides a calibration method for a hopper motor and camera of a hopper cart, wherein the hopper cart includes a cart body, a hopper mechanism driven to rotate by a hopper motor, and a recognition camera mounted on the hopper mechanism. The example steps include:

[0041] Step S100: Visual reference marks are set on the body structure of the hopper car, and a hopper coordinate system and a reference coordinate system are established that are fixed to the hopper mechanism and the visual reference marks respectively, and have preset relative position constraints in the theoretical installation state.

[0042] Specifically, such as Figure 1 , Figure 3 As shown, the present invention ingeniously sets the visual reference mark directly on the structural components of the material box truck itself (in the example, on the crossbeam), using the inherent structural components of the truck body as the calibration reference. Therefore, it can replace the calibration scheme that relies on external calibration fixtures in the prior art, effectively avoiding the cumbersome steps of manual operation and the problem of human-introduced installation errors.

[0043] Meanwhile, considering that the rear recognition camera used when identifying the material bins on the material bin cart is usually a 3D camera, while the front recognition camera used when identifying the triple barcodes on the material rack is usually a 2D camera, in order to ensure that the calibration method of the present invention is applicable to both the front and rear recognition cameras, in this example, the visual reference mark is preferably a triple barcode, which is affixed to the middle position of the crossbeam at a fixed size. In addition, in other optional embodiments, the visual reference mark can also be selected as a checkerboard grid or dotted grid of fixed size. The present invention is not limited to this and can be implemented according to the actual situation.

[0044] Among them, such as Figure 4 As shown, in this example, the hopper coordinate system is defined on the bottom plane of the tray of the hopper mechanism, with the origin being the hopper rotation center. The upward direction of the hopper is the positive Z-axis, the opening direction of the hopper is the positive X-axis, and the Y-axis is determined by the right-hand coordinate system. The reference coordinate system is defined at the center of the visual reference mark, with the upward direction of the hopper being the positive Z-axis, the X-axis pointing towards the hopper mechanism, and the Y-axis determined by the right-hand coordinate system.

[0045] Based on the above definition, during the mass production stage of the hopper truck, the triple clamp can be pre-installed in the middle of the crossbeam. Mechanical drawings are used to ensure that the theoretical position of the hopper's rotation center is on the X-axis of the reference coordinate system, thus establishing a relative positional constraint between the hopper coordinate system and the reference coordinate system. Therefore, in this example, it can be assumed that after installation, the center of the hopper coordinate system is on the X-axis of the reference coordinate system, with a lateral deviation of 0. For example... Figure 4 As shown, since the zero position of the hopper motor is not calibrated, the X-axis of the hopper coordinate system and the reference coordinate system are not parallel when the zero position of the hopper motor is not calibrated. The next step is to calibrate the zero position of the hopper motor by adjusting the position of the hopper.

[0046] Step S200: Adjust the hopper mechanism so that the hopper coordinate system is parallel and coplanar with the reference coordinate system and the preset coordinate axes are facing each other and collinear, in order to calibrate the zero position of the hopper motor.

[0047] Specifically, such as Figure 4 As shown, the hopper mechanism is controlled to rise until the XOY planes of the hopper coordinate system and the reference coordinate system are on the same plane, and then the hopper mechanism is controlled to rotate until the hopper coordinate system is parallel to the reference coordinate system. At this time, since the X-axis of the hopper coordinate system and the reference coordinate system are collinear and opposite in direction, the current position of the hopper motor is the zero position of the hopper motor.

[0048] like Figure 7As shown, this embodiment provides a method for determining whether the hopper has risen to a specified level position and is in the zero position: "Front midpoint" and "Rear midpoint" are marked on the front and rear parts of the bottom plane of the hopper, respectively, both located on the X-axis of the hopper coordinate system. A laser emitter shines from the rear midpoint towards the front midpoint, projecting the beam onto the reference coordinate system. When the Z-axis coordinate of the beam is 0 in the reference coordinate system, it indicates that the hopper is level with the crossbeam; when the Y-axis coordinate of the beam is 0 in the reference coordinate system, it indicates that the hopper is aligned.

[0049] In step S300, with the hopper motor in the zero position, the visual reference mark is identified by the recognition camera, the initial pose of the recognition camera in the reference coordinate system is obtained, and the coordinate components of the initial pose in two orthogonal directions in the reference coordinate system plane are recorded.

[0050] Specifically, since the X-axis of the hopper coordinate system and the reference coordinate system are collinear and opposite in direction, identification at the zero position of the hopper motor can obtain the pose of the three-part code in the camera coordinate system. The initial pose of the camera under the three-part code can be obtained by inverting the matrix. And record the initial pose coordinate components X0 and Y0 on the X and Y axes in the reference coordinate system plane.

[0051] Formula 1

[0052] Since the technology for obtaining pose information based on image information recognition codes is already quite mature and is a known technology, it will not be elaborated on in this example. You can refer to existing technologies for implementation.

[0053] Since the final requirement is to determine the pose of the recognition camera in the hopper coordinate system. Based on the relationship between the hopper coordinate system and the reference coordinate system, we can obtain:

[0054] Formula 2

[0055] In Formula 2, this example uses the pose of the reference coordinate system O in the hopper coordinate system W. The relative relationship between O and W is analyzed below. Since the Y-axis component of the hopper coordinate system W under the reference coordinate system O is 0, according to step S200, the hopper and the triplet are aligned at this point, and the Z-axis component of the hopper coordinate system W under the reference coordinate system O is also 0. However, the X-axis component of the hopper coordinate system W under the reference coordinate system O is unknown, and since the origin of the hopper coordinate system W is defined as the hopper's rotation center, it cannot be directly measured. Therefore, the following steps are required to obtain... The X-axis component (i.e., the offset distance D of the origin of the hopper coordinate system in the X-axis direction of the reference coordinate system).

[0056] Step S400 controls the hopper mechanism to rotate, driving the recognition camera to recognize the visual reference mark from multiple angles, and obtains multiple sets of sampling poses of the recognition camera in the reference coordinate system, so as to fit the trajectory parameters of the recognition camera rotating around the origin of the hopper coordinate system.

[0057] For example, such as Figure 6 As shown, the rotation range of the hopper motor is first set to [-α, α], so that... The sampling step size is used for identification sampling. The identification result obtained directly from the i-th identification sampling is the pose of the tri-code in the camera coordinate system. By inverting the matrix according to Formula 1, the pose of the recognition camera under the three-part code can be obtained. Assume the results of n samplings are as follows: As the hopper rotates, the recognition camera rotates around the center of the hopper. Therefore, a circle can be fitted based on the position information of the sampling results, thus obtaining the radius R of the rotation trajectory of the recognition camera around the center of the hopper. Since there are many algorithms for calculating the radius of a circle by fitting a circle based on its position in space, such as the least squares method, existing technologies can be referenced, and will not be elaborated here.

[0058] Step S500 calculates the offset distance of the origin of the hopper coordinate system in the reference coordinate system based on the trajectory parameters and coordinate components, establishes the pose transformation relationship between the reference coordinate system and the hopper coordinate system, and calculates the extrinsic pose of the recognition camera in the hopper coordinate system based on the pose transformation relationship and the initial pose.

[0059] Specifically, since Formula 1 has already determined the initial pose of the recognition camera in the reference coordinate system when the hopper motor is in the zero position, And its coordinate components X0 and Y0 in two orthogonal directions within the reference coordinate system XOY plane. Therefore, according to Figure 6 The geometric relationships shown can be used to calculate the offset distance D of the hopper coordinate system W in the X-axis direction of the reference coordinate system O:

[0060] Formula 3

[0061] Once D is determined, the relative poses of W and O can be completely determined:

[0062] Translation component: The coordinates of the center of the triplet (origin O) in the hopper coordinate system W are (D, 0, 0). This is because the line connecting the two origins is along the X-axis, and the distance between them is D.

[0063] Rotational component: Since the X-axis of W is parallel to the X-axis of O at the zero position (the hopper opening is directly opposite to the triplet), and the Z-axis of both coordinate systems are in the same direction (both are upward), according to the right-hand coordinate system rule, the orientation of O relative to W is a rotation of 180° around the Z-axis.

[0064] therefore ,when Once determined, the camera's pose in the reference coordinate system is identified by combining it with the obtained zero-position state. Through the coordinate transformation chain (i.e., Equation 2):

[0065]

[0066] The pose of the recognition camera in the hopper coordinate system can then be determined. This refers to the final identification of the camera's extrinsic parameters. Furthermore, the above example method can be used for both the front and rear cameras of the material handling vehicle.

[0067] Through the above design, this invention defines the zero position of the hopper motor (the motor position when the hopper opening is directly opposite the three-column code and the X-axis of the two coordinate systems are parallel and coplanar and collinear with each other), and integrates the motor zero position calibration and the camera extrinsic parameter calibration into the same process. The former serves as the starting procedure for the latter calibration. The calibration of the two parameters can be completed simultaneously through one sequential operation, avoiding the accumulation of errors in step-by-step calibration.

[0068] Experimental Example

[0069] - Initial pose acquisition in zero position state

[0070] Since the X-axis of the hopper coordinate system and the reference coordinate system coincide, assuming the current position of the hopper motor is... Then the zero position of the hopper motor is This value will be used as the zero position value of the motor when controlling the hopper later.

[0071] The camera identifies the position of the three-part code under the camera's view. Corresponding coordinates: for:

[0072]

[0073] According to formula (1), the pose of the recognition camera in the reference coordinate system can be obtained. The corresponding coordinates are:

[0074]

[0075] - Rotational scanning and sampling

[0076] The rotation range of the hopper motor is set as follows: ,by The sampling step size is used for identification sampling. The table below shows the sampling results after 20 samplings. After converting the results to the camera's coordinates in the reference coordinate system, only the X-axis and Y-axis coordinates are retained:

[0077]

[0078] Trajectory Fitting

[0079] Based on the coordinates of the above 20 sets of sampled poses in the XOY plane of the reference coordinate system, the least squares method is used to fit a circle, and the radius R of the circular trajectory of the recognition camera rotating around the origin of the hopper coordinate system is approximately: R=0.309079m.

[0080] - Jointly solve camera extrinsic parameters

[0081] According to Formula 1, the initial pose of the camera in the reference coordinate system when the hopper motor is in the zero position can be obtained. Corresponding coordinates:

[0082]

[0083] Its position on the X-axis is then: Its position on the Y-axis is .according to Figure 6 The geometric relationships and formula 3 shown can be used to obtain the position D of the hopper coordinate system W in the X-axis direction of the reference coordinate system O:

[0084]

[0085] according to Figure 6 As shown, when the hopper is in the motor zero position, because the X-axis of the reference coordinate system is defined as "towards the hopper," while the X-axis of the hopper coordinate system points towards the triplet, the two coordinate systems have opposite "orientations." Therefore, the third component of the Euler angle of rotation is 180°. The pose of the reference coordinate system in the hopper coordinate system. The corresponding coordinates are: (D, 0, 0, 0, 0, 180°). The camera's pose in the hopper coordinate system can be obtained according to Formula 2. The corresponding coordinates are: .

[0086] Therefore, this invention designs a method that controls the hopper motor to rotate and scan the three-part code at a known step size. This allows the trajectory of the recognition camera in the reference coordinate system to be fitted as a circle. The center of the circle is then used to deduce the rotation center of the hopper, thereby jointly solving for the zero position of the hopper motor and the extrinsic parameters of the camera. This method does not rely on manual alignment accuracy, and its residual error is automatically compensated by the circle fitting, improving the overall calibration accuracy.

[0087] On the other hand, corresponding to the above-mentioned calibration methods for the hopper motor and camera of the material box car, such as Figure 8 As shown, the present invention also provides a bin cart calibration system, the bin cart including a vehicle body, a hopper mechanism driven to rotate by a hopper motor, and an identification camera mounted on the hopper mechanism. The bin cart calibration system includes:

[0088] A visual reference mark is set at the middle position of the crossbeam of the hopper cart;

[0089] The control unit is communicatively connected to the hopper motor and the recognition camera, and is used to adjust the pose of the hopper mechanism and trigger the recognition camera to perform image acquisition on the visual reference mark;

[0090] The processing unit is communicatively connected to the recognition camera and the control unit, and is used to receive the image acquisition results and perform pose calculation and trajectory fitting.

[0091] A storage unit for storing the program of the calibration method steps as described in the example above;

[0092] The control unit and the processing unit are configured to perform collaboratively via the calibration program: In the zero-position calibration state of the hopper motor, where the hopper coordinate system and the reference coordinate system are adjusted to be parallel and coplanar, with corresponding coordinate axes collinear and facing each other, the initial pose of the recognition camera relative to the visual reference mark is acquired; the hopper mechanism is controlled to rotate and scan within a preset range to acquire multiple sets of sampled poses; the trajectory parameters of the recognition camera rotating around the origin of the hopper coordinate system are obtained by fitting the multiple sets of sampled poses; and the pose transformation relationship between the hopper coordinate system and the reference coordinate system is determined based on the trajectory parameters and the initial pose, so as to jointly solve the zero position of the hopper motor and the extrinsic pose of the recognition camera in the hopper coordinate system.

[0093] On the other hand, corresponding to the above-mentioned calibration method for the hopper motor and camera of the hopper cart, the present invention also provides a computer-readable storage medium storing a computer program thereon, wherein when the computer program is executed by a processor, it implements the calibration method for the hopper motor and camera of the hopper cart as described in any of the above examples.

[0094] In summary, the calibration method and system for the hopper motor and camera, as well as the storage medium provided by this invention, have the following significant advantages compared to existing technologies:

[0095] 1. Eliminates the need for external calibration fixtures, significantly reducing operational complexity.

[0096] This invention directly sets visual reference marks on the structural components of the material box cart itself (such as crossbeams), using the inherent structural components of the cart as the calibration reference. This replaces the cumbersome steps of manually placing external calibration plates / tooling in existing technologies. Operators no longer need to carry, transport, or manage dedicated calibration tooling, and it particularly solves the problem of limited space for front camera calibration (the front camera's field of view faces the material rack, making it extremely inconvenient to arrange external tooling in narrow storage aisles). This significantly reduces the difficulty of calibration operations, improves production line efficiency, and enhances the convenience of after-sales on-site maintenance.

[0097] 2. Joint calibration of motor zero position and camera extrinsic parameters avoids the accumulation of step-by-step errors.

[0098] This invention is the first to integrate the zero-position calibration of the hopper motor and the extrinsic parameter calibration of the recognition camera into a single technical process, simultaneously completing the calibration of two key parameters in a single operation. Compared to the step-by-step independent calibration method in existing technologies, this invention avoids the accumulation of errors caused by factors such as coordinate system transformation and inconsistent references during step-by-step calibration, thus improving the overall calibration accuracy. Furthermore, since both calibrations can be completed in a single operation, it significantly shortens calibration time and reduces labor costs.

[0099] 3. The rotation center is indirectly solved by using rotational scanning and trajectory fitting, avoiding the difficulties of direct measurement.

[0100] The hopper's rotation center (i.e., the origin of the hopper coordinate system) is located inside the vehicle body and is a physical parameter that cannot be directly observed. This invention controls the hopper mechanism to rotate and scan within a preset range, acquiring multiple sets of camera pose data and fitting their rotational trajectories. Using geometric relationships, it indirectly infers the offset distance of the hopper coordinate system origin relative to the reference coordinate system. This method avoids the technical difficulties of directly measuring the rotation center, decomposing the three-dimensional unknown pose problem into a planar trajectory fitting problem. The method has complete logic, and its solution accuracy is significantly higher than that of manual measurement.

[0101] 4. Triple-constraint progressive solution improves calibration accuracy and reliability.

[0102] This invention utilizes three constraints—mechanical design guarantee (the origin of the hopper coordinate system is located on the X-axis of the reference coordinate system, Y=0), height alignment control (Z=0), and rotation trajectory fitting (calculating the offset distance D in the X direction)—to progressively decompose and solve the three-dimensional unknown pose problem of the camera's extrinsic parameters. Each constraint is independent and verifiable, forming a complete calibration accuracy assurance system, resulting in highly repeatable and reliable calibration results.

[0103] 5. It does not rely on manual alignment accuracy; circle fitting automatically compensates for residual errors.

[0104] In step S200, although the hopper needs to be adjusted to a parallel and coplanar state, the present invention can automatically compensate for any residual errors that may exist in the manual adjustment in step S200 through the rotational scanning and circle fitting in the subsequent step S400. The circle fitting algorithm determines the rotation center based on the optimal estimate of multiple sets of sampled data, which is robust to the accuracy fluctuations of a single adjustment and reduces the dependence on the operator's skills.

[0105] 6. A single method is applicable to both 2D and 3D cameras, offering strong engineering versatility.

[0106] This invention uses a fixed-size triplet code as a visual reference mark, which can be recognized simultaneously by both a 2D camera (front recognition camera) and a 3D camera (rear recognition camera). Therefore, the same calibration method and system can be applied to both the front and rear recognition cameras of the material box vehicle without modification, exhibiting good engineering versatility and compatibility, and reducing the types of calibration stations required on the production line.

[0107] 7. The reference standard is included and its accuracy is guaranteed by the factory, avoiding installation errors introduced by human error.

[0108] This invention uses a vehicle body structural component (crossbeam) as the calibration benchmark. The machining accuracy of the crossbeam is guaranteed by the factory mechanical drawings, and the installation position of the visual benchmark mark is fixed by mechanical design constraints. Compared with the existing technology where external tooling needs to be rearranged and repositioned for each calibration, the benchmark accuracy of this invention is not affected by manual placement operations, and the consistency and traceability of the calibration results are better guaranteed.

[0109] 8. Can be repeatedly executed on the production line and at the customer's site, with user-friendly after-sales maintenance.

[0110] Since this invention requires no external calibration fixtures, and the calibration process is entirely completed by the vehicle's own structure, it can be repeatedly executed in mass production lines, customer sites, or after-sales maintenance scenarios. After the material box vehicle has undergone long-term operation, component replacement, or collision recovery, the calibration process can be quickly re-executed to restore the accuracy of picking and placing goods, significantly improving the maintainability and lifecycle management capabilities of the equipment.

[0111] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The present invention is limited only by the claims and their full scope and equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

[0112] Those skilled in the art will understand that, besides implementing the system, apparatus, unit, and its modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and its modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0113] Furthermore, all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0114] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. A method for calibrating a hopper motor and camera in a material bin cart, comprising the following steps: Visual reference marks are set on the body structure of the hopper car, and a hopper coordinate system and a reference coordinate system are established that are fixed to the hopper mechanism and the visual reference marks respectively, and have preset relative position constraints in the theoretical installation state. The preset relative position constraints include: the origin of the hopper coordinate system is located on the X-axis of the reference coordinate system in the theoretical installation state; Adjust the hopper mechanism so that the hopper coordinate system is parallel and coplanar with the reference coordinate system and the preset coordinate axes are collinear and facing each other, in order to calibrate the zero position of the hopper motor; wherein the preset coordinate axes are collinear and facing each other means that the X-axis of the hopper coordinate system and the reference coordinate system are collinear and facing each other. When the hopper motor is in the zero position, the visual reference mark is identified by the recognition camera, the initial pose of the recognition camera in the reference coordinate system is obtained, and the coordinate components of the initial pose in two orthogonal directions in the reference coordinate system plane are recorded. The hopper mechanism is controlled to rotate, which drives the recognition camera to recognize visual reference marks from multiple angles. Multiple sets of sampling poses of the recognition camera in the reference coordinate system are obtained, and the trajectory parameters of the recognition camera rotating around the origin of the hopper coordinate system are fitted. Based on the trajectory parameters and coordinate components, the offset distance of the origin of the hopper coordinate system in the reference coordinate system is calculated, the pose transformation relationship between the reference coordinate system and the hopper coordinate system is established, and the extrinsic pose of the recognition camera in the hopper coordinate system is calculated based on the pose transformation relationship and the initial pose. The vehicle body structural component is the crossbeam of the hopper vehicle, and the visual reference mark is set at the middle position of the crossbeam; the hopper coordinate system is defined on the bottom plane of the tray of the hopper mechanism, with the origin being the hopper rotation center, the upward direction of the hopper being the positive Z-axis, and the opening direction of the hopper being the positive X-axis; the reference coordinate system is defined at the center position of the visual reference mark, with the upward direction of the hopper being the positive Z-axis, the X-axis pointing towards the hopper mechanism, and the Y-axis determined by the right-hand coordinate system; The trajectory parameter is the radius R of the circular trajectory; the offset distance D is based on the formula... The calculation yielded, where , These are the coordinate components of the initial pose in two orthogonal directions within the XOY plane of the reference coordinate system.

2. The calibration method according to claim 1, wherein the visual reference mark is any one of a fixed-size triplet, a checkerboard, or a dotted grid.

3. The calibration method according to claim 1, wherein the recognition camera includes at least one of a 3D rear recognition camera mounted at the rear of the hopper mechanism or a 2D front recognition camera mounted at the front of the hopper mechanism.

4. A hopper car calibration system, controlled by the hopper car hopper motor and camera calibration method as described in any one of claims 1 to 3, comprising: A visual reference mark is set at the middle position of the crossbeam of the hopper cart; The control unit is communicatively connected to the hopper motor and the recognition camera, and is used to adjust the pose of the hopper mechanism and trigger the recognition camera to perform image acquisition on the visual reference mark; The processing unit is communicatively connected to the recognition camera and the control unit, and is used to receive the image acquisition results and perform pose calculation and trajectory fitting. A storage unit for storing a program of the calibration method steps as described in any one of claims 1 to 3; The control unit and the processing unit are configured by the program to perform collaboratively: when the hopper mechanism is adjusted to a zero-position calibration state where the hopper coordinate system and the reference coordinate system are parallel and coplanar, and the corresponding coordinate axes are collinear and facing each other, the initial pose of the recognition camera relative to the visual reference mark is obtained; the hopper mechanism is controlled to rotate and scan within a preset range to obtain multiple sets of sampled poses; the trajectory parameters of the recognition camera rotating around the origin of the hopper coordinate system are obtained by fitting the multiple sets of sampled poses; the pose transformation relationship between the hopper coordinate system and the reference coordinate system is determined based on the trajectory parameters and the initial pose, so as to jointly solve the zero position of the hopper motor and the extrinsic pose of the recognition camera in the hopper coordinate system.

5. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the calibration method for the hopper motor and camera of a hopper cart as described in any one of claims 1-3.

Images