Sensor position calibration method and device of automatic mobile equipment, equipment, medium

Abstract

This disclosure relates to a sensor position calibration method, apparatus, device, and medium for automated mobile devices. The method includes acquiring sensor data, which includes odometer data from a wheeled odometer, first QR code data from a first barcode reader, and second QR code data from a second barcode reader. Position calibration is performed based on the sensor data to obtain the actual positions of the wheeled odometer, the first barcode reader, and the second barcode reader. This disclosure, by processing the acquired QR code data and odometer data, can simultaneously determine the actual positions of both the barcode reader and the wheeled odometer, providing a basis for subsequently determining their respective offset information.

Description

Technical Field

[0001] This disclosure relates to the field of calibration technology, and in particular to a sensor position calibration method, apparatus, device, and medium for automatic mobile equipment. Background Technology

[0002] Automated mobile equipment utilizes various types of sensors, each with a different installation location. As the usage time of the automated mobile equipment increases, the installation positions of these sensors may shift to varying degrees. Failure to calibrate these sensors in a timely manner will directly lead to inaccurate positioning and other problems during subsequent use of the automated mobile equipment, affecting its usability. Therefore, calibrating the positions of sensors in automated mobile equipment is of great importance. Summary of the Invention

[0003] In view of this, this disclosure proposes a sensor position calibration method, apparatus, device, and medium for an automated mobile device capable of simultaneously calibrating multiple sensors.

[0004] According to one aspect of this disclosure, a sensor position calibration method for an automated mobile device is provided, the method comprising:

[0005] Acquire sensor data, which includes odometer data from a wheel odometer, first QR code data from a first barcode reader, and second QR code data from a second barcode reader;

[0006] Position calibration is performed based on the sensor data to obtain the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader.

[0007] In one possible implementation, the position calibration based on the sensor data includes:

[0008] A target function is constructed based on the sensor data, and constraints are set.

[0009] Under the given constraints, the objective function is optimized to obtain the optimization result;

[0010] The actual positions of the wheel odometer, the first barcode reader, and the second barcode reader are determined based on the optimization results.

[0011] In one possible implementation, the construction objective function includes:

[0012] A target function is constructed based on a first sub-function, a second sub-function, and a third sub-function. The first sub-function is related to the first QR code data and the initial position of the first barcode reader. The second sub-function is related to the second QR code data and the initial position of the second barcode reader. The third sub-function is related to the odometer data and the initial position of the wheel odometer.

[0013] In one possible implementation, the constraints are related to the first sub-function, the second sub-function, and the third sub-function.

[0014] In one possible implementation, the constraints include at least one of the following:

[0015] The size relationship between the first sub-function and the second sub-function after scaling based on the first coefficient;

[0016] The size relationship between the first and third sub-functions after scaling based on the second coefficient;

[0017] The weighted sum of the first sub-function, the second sub-function, and the third sub-function is within a preset numerical range, and the weighting coefficients and the preset numerical range are related to the first coefficient and the second coefficient.

[0018] In one possible implementation, determining the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader based on the optimization result includes:

[0019] By combining the optimization results with the deviation information, the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader are determined.

[0020] In one possible implementation, the method further includes:

[0021] Based on the actual and initial positions of the wheel odometer, the first barcode reader, and the second barcode reader, the offset information of each of the wheel odometer, the first barcode reader, and the second barcode reader is determined.

[0022] In one possible implementation, prior to the location calibration based on the sensor data, the method further includes:

[0023] The sensor data is preprocessed, including deleting invalid data from the sensor data and / or performing a timestamp alignment operation on the sensor data.

[0024] According to another aspect of this disclosure, a sensor position calibration device for an automated mobile device is provided, the device comprising:

[0025] The data acquisition unit is configured to acquire sensor data, which includes odometer data from a wheel odometer, first QR code data from a first barcode reader, and second QR code data from a second barcode reader.

[0026] The data processing unit is configured to perform position calibration based on the sensor data to obtain the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader.

[0027] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the sensor position calibration method of the above-described automatic mobile device when executing the instructions stored in the memory.

[0028] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the sensor position calibration method of the above-described automatic mobile device.

[0029] By processing the acquired QR code data and odometer data, the actual positions of the reader and the wheel odometer can be determined simultaneously, providing a basis for subsequently determining their respective offset information.

[0030] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0031] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0032] Figure 1 A first flowchart of a sensor position calibration method for an automated mobile device according to an embodiment of the present disclosure is shown.

[0033] Figure 2 A first schematic diagram of an automated mobile device according to an embodiment of the present disclosure is shown.

[0034] Figure 3 A second schematic diagram of an automated mobile device according to an embodiment of the present disclosure is shown.

[0035] Figure 4 A schematic diagram of an automatic mobile device performing sensor position calibration according to an embodiment of the present disclosure is shown.

[0036] Figure 5A schematic diagram of a two-wheel differential model according to an embodiment of the present disclosure is shown.

[0037] Figure 6 A schematic diagram of an automated mobile device according to an embodiment of the present disclosure is shown when it is transporting goods.

[0038] Figure 7 A schematic diagram of a sensor position calibration device for an automated mobile device according to an embodiment of the present disclosure is shown.

[0039] Figure 8 A schematic diagram of a sensor position calibration system for an automated mobile device according to an embodiment of the present disclosure is shown.

[0040] Figure 9 A block diagram of a sensor position calibration device for an automated mobile device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0041] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0042] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0043] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0044] To facilitate understanding of the technical solutions provided by the embodiments of this disclosure by those skilled in the art, the technical environment for implementing the technical solutions will be described below.

[0045] Robots and other automated mobile devices utilize various types of sensors, such as barcode readers and wheeled odometers. Each sensor is installed in a different location. During operation, these devices may collide with obstacles, encounter bumps due to poor road conditions, or experience positional shifts due to unstable mechanical structures. This directly leads to inaccurate positioning in subsequent use, potentially preventing the device from achieving its intended goals and reducing user experience. For example, consider the odometer on a robot. The robot's positioning in the world coordinate system involves odometer data. Over time, external factors may cause odometer errors. If the odometer isn't calibrated periodically, when controlling the robot to a designated location—for instance, a 5m leftward movement might be insufficient—the odometer error could cause the robot to move only 3.9m to the left before stopping, resulting in inaccurate positioning. Therefore, timely sensor calibration is crucial for obtaining more accurate sensor positions. In addition, the position calibration results based on single sensor data have large errors and cannot achieve ideal calibration results.

[0046] Based on practical technical needs similar to those described above, the sensor position calibration method for automatic mobile devices provided in this disclosure can process the acquired QR code data and odometer data to obtain the actual positions of the barcode reader and the wheeled odometer, which is efficient and accurate.

[0047] This disclosure provides a sensor position calibration method for automated mobile devices, such as... Figure 1 The diagram shows a first flowchart of a sensor position calibration method for an automated mobile device according to an embodiment of this disclosure. This method can be executed by the automated mobile device's own processor or by an external device (e.g., a terminal device, a server, etc.). The sensor position calibration method for an automated mobile device may include:

[0048] Acquire sensor data, which includes odometer data from a wheel odometer, first QR code data from a first barcode reader, and second QR code data from a second barcode reader;

[0049] Position calibration is performed based on the sensor data to obtain the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader.

[0050] The automated mobile device can be a robot, an AGV (Automated Guided Vehicle), etc. Sensor data can be acquired by the automated mobile device's data acquisition module through an I / O interface.

[0051] The first QR code data may include the displacement of the first reader relative to the x-axis, y-axis, and z-axis of the world coordinate system, i.e., the position information of the center of the first reader in the world coordinate system. The second QR code data may include the displacement of the second reader relative to the x-axis, y-axis, and z-axis of the world coordinate system, i.e., the position information of the center of the second reader in the world coordinate system. The odometer data may include the displacement of the center of the automated mobile device relative to the x-axis, y-axis, and z-axis of the wheeled odometer coordinate system, i.e., the position information of the center of the automated mobile device in the wheeled odometer coordinate system.

[0052] The first and second barcode readers can be any type of barcode reader used to read barcodes or QR codes by scanning or other means.

[0053] Figure 2 A first schematic diagram of an automated mobile device, i.e., an AGV, provided according to an embodiment of the present disclosure is shown. Figure 3 A second schematic diagram of an automated mobile device, i.e., an AGV, provided according to an embodiment of the present disclosure is shown. Figure 4 A schematic diagram of an automated mobile device performing sensor position calibration according to an embodiment of this disclosure is shown. Now, let's... Figure 2 , Figure 3 The AGV shown and Figure 4 The calibration environment shown is used as an example for illustration:

[0054] The AGV has a first barcode reader 101 on its upper side and a second barcode reader 102 on its lower side. The first barcode reader 101 works in conjunction with a first QR code 21 affixed to the bottom of the shelf 3, and the second barcode reader 102 works in conjunction with a second QR code 22 affixed to the ground below the shelf 3. This ensures that when the AGV moves under the shelf 3, both the first barcode reader 101 and the second barcode reader 102 can read the first QR code 21 and the second barcode reader 102, respectively. The AGV has a left drive wheel 103 on its left side and a right drive wheel 104 on its right side.

[0055] After the first barcode reader 101, the second barcode reader 102, the left drive wheel 103, and the right drive wheel 104 are installed, their initial positions relative to the AGV coordinate system (i.e., dx1, dy1, dth1, dx2, dy2, dth2, rL, rR, wheelDist as described below) are determined. The AGV coordinate system can be determined with the AGV center as the origin, the front of the AGV as the positive x-axis, the left side of the AGV as the positive y-axis, and the top of the AGV as the positive z-axis. Alternatively, the center of the axes of the left drive wheel 103 and the right drive wheel 104 during the first power-on can be used as the AGV center, which remains unchanged during subsequent AGV operation. Another option is to determine the odometer coordinate system with this AGV center as the origin, the front of the AGV as the positive x-axis, the left side of the AGV as the positive y-axis, and the top of the AGV as the positive z-axis.

[0056] When the AGV moves to the bottom of the shelf 3, the first barcode reader 101 reads the first QR code 21 and parses it to obtain the first QR code data; the second barcode reader 102 reads the second QR code 22 and parses it to obtain the second QR code data; and the odometer data is obtained based on the data from the wheel odometer.

[0057] The parsing process for the first QR code 21 and the second QR code 22 is similar, and the QR code parsing may include the following process:

[0058] The acquired image is segmented. One possible implementation is an adaptive thresholding method. Region features are generally more stable than those of a single pixel. Adaptive thresholding primarily finds a reasonable threshold value for the average grayscale value within the pixel's neighborhood for segmentation. For example, the acquired image is divided into 4x4 grid blocks, and the maximum and minimum grayscale values ​​of each block are calculated. A 3-neighborhood maximum filter and a 3-neighborhood minimum filter are then applied to the calculated maximum and minimum grayscale values ​​of all blocks. The average of the filtered maximum and minimum grayscale values ​​(maximum grayscale value + minimum grayscale value) / 2 is used as the threshold for the segmented region. The block is then segmented based on this threshold, resulting in a segmented binary image. By segmenting the image into blocks, random noise interference is reduced, robustness is increased, and computational efficiency is improved.

[0059] Contour finding. In one possible implementation, contours that may form a QR code can be found in the segmented binary image. Algorithms such as union-find can be used to find connected components, where each connected component has a unique ID.

[0060] Finding Quadrilaterals. In one possible implementation, after obtaining the contours, each contour can be segmented to generate a convex quadrilateral with the minimum residual, serving as a candidate for the QR code location. For regular rectangles such as squares, finding the four vertices of the square is relatively easy. However, the actual QR codes obtained are often not regular rectangles; they may be deformed and / or undergo affine transformations, making the search for quadrilateral vertices more challenging. In one possible implementation, the unordered contour points can be sorted according to their angles relative to the center. With the sorted contour points, points within a certain range from the center point are selected sequentially for line fitting, iterating through the indices and calculating the total error of each line. A low-pass filter is applied to the total error to increase robustness, and then the corner indices corresponding to the four lines with the largest total error are selected as the four pre-selected corner points. Then, lines are fitted between these four corner points, and the corner points of the four obtained lines are used as the vertices of the QR code. In this method, when fitting a straight line between corner points, the true gradient edge line is found as much as possible. Points on the line are sampled, and the point with the largest gradient on the line's normal vector is calculated and used as the final point for line fitting. This yields more accurate corner coordinates. This method effectively addresses issues such as blurry or skewed QR codes caused by fast shooting speeds or undesirable shooting positions. Even when acquiring QR codes that are not regular rectangles, it can still efficiently and accurately determine the quadrilaterals within them.

[0061] Filtering out abnormal quadrilaterals. In one possible implementation, among the numerous quadrilaterals found, obviously abnormal quadrilaterals can be filtered out based on the actual geometric characteristics of the QR code. These actual geometric characteristics may include the quadrilateral's side length ratio falling within a certain range and the included angle between two adjacent sides being close to 90 degrees. Using these commonly used actual geometric characteristics for anomaly processing can filter out ideal QR codes.

[0062] Homography transformation. Since the quadrilaterals found are highly likely to have undergone affine transformations, it's difficult to directly find a regular square. Homography transformation is used to project the quadrilateral image into a square that perfectly conforms to the theoretical model. In one possible implementation, for the transformed image P' and the original image P (the actual image), where the homogeneous coordinates of a point in image P' are (u2, v2, 1) and the homogeneous coordinates of a point in image P are (u1, v1, 1), for each corresponding point pair:

[0063]

[0064] Where s is the scale factor, and H is a 3×3 homography matrix. Because of the scale factor s, the last element of H can be normalized to 1, therefore the homography matrix H has 8 degrees of freedom. Expanding the above equation yields:

[0065]

[0066] You can set h 33 =1, or by adding constraints to make the matrix modulus 1, and further transforming it to eliminate the scale factor s, and treating the elements of H as a column vector, we can obtain the following equation:

[0067]

[0068] The homography matrix H, representing 8 degrees of freedom, requires at least 4 pairs of points to calculate. In real-world applications, these point pairs often contain noise, such as pixel-level deviations or even mismatches. Therefore, choosing far more than 4 pairs can reduce errors and yield more accurate results. Methods for solving H include direct linearization to derive the homography matrix H from the equations, and alternatively, singular value decomposition, the LM algorithm, or other methods. After determining the homography matrix H, it is used to perform homography transformation on the selected QR codes.

[0069] Decoding. In one possible implementation, the QR code after homography is a square with alternating light and dark color blocks. The entire square is composed of multiple rows and columns of color blocks. Each row of color blocks is a string of binary codes, and multiple rows of color blocks are multiple strings of binary codes. These binary codes can include data such as the displacement of the QR code along the x-axis, the displacement along the y-axis, the deflection angle along the z-axis in the reader's coordinate system, and the QR code's ID. The reader's coordinate system can be determined by taking the center of the reader as the origin, the front of the AGV as the positive x-axis direction, the left side of the AGV as the positive y-axis direction, and the top of the AGV as the positive z-axis direction. Once the ID of the QR code is read, the displacement of the QR code along the x-axis, y-axis, and z-axis in the world coordinate system can be obtained (i.e., the position information of the QR code in the world coordinate system, which is known and fixed when the QR code is set). By combining this with the displacement of the QR code along the x-axis, y-axis, and z-axis in the reader's coordinate system, the displacement of the reader along the x-axis, y-axis, and z-axis in the world coordinate system can be obtained.

[0070] Using the above QR code parsing process, the first QR code data and the second QR code data can be obtained. The first QR code data may include the displacement of the first reader 101 relative to the x-axis, the displacement of the y-axis, and the deflection angle of the z-axis of the world coordinate system. The second QR code data may include the displacement of the second reader 102 relative to the x-axis, the displacement of the y-axis, and the deflection angle of the z-axis of the world coordinate system.

[0071] Before explaining the process of acquiring odometer data, let's take the AGV mentioned above as an example to illustrate the wheeled odometer and its coordinate system: The wheeled odometer is generated based on the data from the wheels, which can include wheel spacing, wheel radius, and the number of wheel rotations. The initial position of the wheeled odometer relative to the AGV coordinate system can be represented by the distance between the left drive wheel 103 and the right drive wheel 104, the radius of the left drive wheel 103, and the radius of the right drive wheel 104. As mentioned above, when the left drive wheel 103 and the right drive wheel 104 are installed, the initial position of the wheeled odometer relative to the AGV coordinate system is determined. Therefore, the wheeled odometer coordinate system can be defined with the initial position of the wheeled odometer as the origin, the front of the AGV as the positive x-axis, the left side of the AGV as the positive y-axis, and the top of the AGV as the positive z-axis.

[0072] In one possible implementation, the odometer data can be obtained based on the motion data of the left drive wheel 103 and the right drive wheel 104:

[0073] Within one acquisition cycle, the AGV's data acquisition module collects the actual number of rotations of the left drive wheel 103 and the right drive wheel 104. Combined with the radii of the left drive wheel 103 and the right drive wheel 104 (which can be the radii of the left drive wheel 103 and the right drive wheel 104 when the AGV is installed), the speeds of the left drive wheel 103 and the right drive wheel 104 can be calculated and substituted into... Figure 5 The dual-wheel differential model shown:

[0074]

[0075] Where, ν c Let ω be the speed of the AGV center. c ν is the angular velocity of the AGV center; d is the distance between the left drive wheel 103 and the right drive wheel 104; r The speed of the right drive wheel 104 is ν. l The speed of the left drive wheel 103.

[0076] Based on the determined velocity and angular velocity of the AGV center, the displacement of the AGV center relative to the wheel odometer coordinate system along the x-axis, y-axis, and z-axis can be obtained, which is the odometer data.

[0077] In one example, prior to the location calibration based on the sensor data, the following may also be included:

[0078] The sensor data is preprocessed, including deleting invalid data from the sensor data and / or performing a timestamp alignment operation on the sensor data.

[0079] To remove invalid data from sensor data, methods for determining whether sensor data includes invalid data can include checking if the data exceeds upper or lower limits. For example, if the displacement along the x-axis exceeds the outer shell of the automated mobile device, which clearly exceeds the maximum displacement value, then it can be deleted. Alternatively, it can check if the change in data between two adjacent frames exceeds a threshold. For example, if the z-axis deflection angles in two adjacent frames are 3° and 60°, and the change is 57°, exceeding a predetermined threshold such as 5°, then it can be deleted. Another method is to check if the data contains too much duplicate data, which can be deleted to reduce the amount of subsequent calculations.

[0080] For timestamp alignment, the timestamps of sensor data from different sources may differ. For example, on a certain day, the first QR code data was acquired between 11:40 and 11:48, the second QR code data between 11:38 and 11:46, and the odometer data between 11:43 and 11:51. In this case, the first QR code data, the second QR code data, and the odometer data acquired between 11:43 and 11:46 will be retained, while the data acquired at other times will be deleted, ensuring that all acquired sensor data are from the same time period.

[0081] Still using Figure 2 , 3 The AGV shown and Figure 4 The preprocessing process is illustrated using the calibration environment shown below as an example:

[0082] After acquiring the sensor data, the first QR code data is placed in buffer queue q1, the second QR code data in buffer queue q2, and the odometer data in buffer queue q3. The AGV's data acquisition module can perform the following preprocessing on q1, q2, and q3:

[0083] v_qr1 = valid(q1|q1,q2,q3);

[0084] v_qr2 = valid(q2|q1,q2,q3);

[0085] v_odom=valid(q3|q1,q2,q3).

[0086] That is, invalid data is deleted from the first QR code data, the second QR code data, and the odometer data respectively, resulting in the deleted first QR code data v_qr1, second QR code data v_qr2, and odometer data v_odom.

[0087] Based on v_qr1, v_qr2, and v_odom, the following preprocessing can be performed:

[0088] syn_qr1=ts(v_qr1|v_qr1,v_qr2,v_odom);

[0089] syn_qr2=ts(v_qr2|v_qr1,v_qr2,v_odom);

[0090] syn_odom=ts(v_odom|v_qr1,v_qr2,v_odom).

[0091] This involves performing timestamp alignment on v_qr1, v_qr2, and v_odom to obtain the aligned first QR code data syn_qr1, the second QR code data syn_qr2, and the odometer data syn_odom.

[0092] By using the above preprocessing methods, outliers in the queue can be removed, providing a higher-quality data source for subsequent optimization calculations and reducing errors.

[0093] In one example, the location calibration based on the sensor data may include:

[0094] A target function is constructed based on the sensor data, and constraints are set.

[0095] Under the given constraints, the objective function is optimized to obtain the optimization result;

[0096] The actual positions of the wheel odometer, the first barcode reader, and the second barcode reader are determined based on the optimization results.

[0097] Specifically, an objective function can be constructed based on the preprocessed sensor data to minimize errors.

[0098] In one possible implementation, constructing the objective function may include:

[0099] A target function is constructed based on a first sub-function, a second sub-function, and a third sub-function. The first sub-function is related to the first QR code data and the initial position of the first barcode reader. The second sub-function is related to the second QR code data and the initial position of the second barcode reader. The third sub-function is related to the odometer data and the initial position of the wheel odometer.

[0100] In one possible implementation, the constraint conditions can be related to the first sub-function, the second sub-function, and the third sub-function, and can also be related to some basic constraint parameters such as the wheel radius being a positive number and the wheel spacing being a positive number.

[0101] The constraints may include at least one of the following:

[0102] The size relationship between the first sub-function and the second sub-function after scaling based on the first coefficient;

[0103] The size relationship between the first and third sub-functions after scaling based on the second coefficient;

[0104] The weighted sum of the first sub-function, the second sub-function, and the third sub-function is within a preset numerical range, and the weighting coefficients and the preset numerical range are related to the first coefficient and the second coefficient.

[0105] Still using Figure 2 , 3 The AGV shown and Figure 4 The optimization calculation process is illustrated using the calibration environment shown below as an example:

[0106] Construct the objective function: fun = cost_fun(input). fun can be constructed based on the first sub-function fun1, the second sub-function fun2, and the third sub-function fun3.

[0107] The first sub-function fun1 can be related to the first QR code data and the initial position of the first reader 101. That is:

[0108] fun1 = qr_cost_fun(syn_qr1, dx1, dy1, dth1). Here, fun1 is the first sub-function, syn_qr1 is the preprocessed first QR code data, dx1, dy1, and dth1 are the positions of the first barcode reader 101 relative to the AGV coordinate system; that is, dx1 is the displacement of the first barcode reader 101 relative to the x-axis of the AGV coordinate system, dy1 is the displacement of the first barcode reader 101 relative to the y-axis of the AGV coordinate system, and dth1 is the deflection angle of the first barcode reader 101 relative to the z-axis of the AGV coordinate system. At the start of optimization, the initial values ​​of dx1, dy1, and dth1 can be set to the initial positions of the first barcode reader 101 relative to the AGV coordinate system as described above.

[0109] The second sub-function, fun2, can be related to the second QR code data and the initial position of the second reader 102. That is:

[0110] fun2 = qr_cost_fun(syn_qr2, dx2, dy2, dth2). Here, fun2 is the second sub-function, syn_qr2 is the preprocessed second QR code data, dx2, dy2, and dth2 are the positions of the second barcode reader 102 relative to the AGV coordinate system. Specifically, dx2 is the displacement of the second barcode reader 102 relative to the x-axis of the AGV coordinate system, dy2 is the displacement of the second barcode reader 102 relative to the y-axis of the AGV coordinate system, and dth2 is the deflection angle of the second barcode reader 102 relative to the z-axis of the AGV coordinate system. At the start of optimization, the initial values ​​of dx2, dy2, and dth2 can be set to the initial positions of the second barcode reader 102 relative to the AGV coordinate system as described above.

[0111] The third sub-function, fun3, can be related to the odometer data and the initial position of the wheel odometer. That is:

[0112] fun3 = odom_cost_fun(syn_odom, rL, rR, wheelDist). Here, fun3 is the third sub-function, syn_odom is the preprocessed odometer data, and rL, rR, and wheelDist are the positions of the wheel odometer relative to the AGV coordinate system. Specifically, rL is the radius of the left drive wheel 103, rR is the radius of the right drive wheel 104, and wheelDist is the wheel spacing between the left drive wheel 103 and the right drive wheel 104. At the start of optimization, the initial values ​​of rL, rR, and wheelDist can be set to the initial positions of the wheel odometer relative to the AGV coordinate system as described above.

[0113] In addition, different objective functions can be designed for different models of AGVs and other automated mobile equipment.

[0114] Set constraints: limits(dx1,dy1,dth1,dx2,dy2,dth2,rL,rR,wheelDist). limits can be related to the first, second, and third sub-functions mentioned above, as well as some basic constraint parameters; that is, constraints can include:

[0115] The relationship between the first sub-function fun1 and the second sub-function fun2 after scaling based on the first coefficient α is, for example, limit1: fun1 / α≤fun2≤α*fun1, where the first coefficient α is any non-zero real number.

[0116] The relationship between the first sub-function fun1 and the third sub-function fun3 after scaling based on the second coefficient β is, for example, limit2: fun1 / β≤fun3≤β*fun1, where the second coefficient β is any non-zero real number.

[0117] The weighted sum of the first sub-function fun1, the second sub-function fun3, and the third sub-function fun3 is within a preset numerical range. The weighting coefficients and the preset numerical range are related to the first coefficient α and the second coefficient β. For example, limit3: ε≤α*(fun1+fun2)+β*fun3≤(α+β) / ε, where the first coefficient α and the second coefficient β are any non-zero real numbers, and ε is any real number greater than 0.

[0118] Constraints may also include:

[0119] limit4: rL>0, rR>0, wheelDist>0, meaning the radius rL of the left drive wheel 103 is a positive number, the radius rR of the right drive wheel 104 is a positive number, and the wheel spacing wheelDist between the left drive wheel 103 and the right drive wheel 104 is a positive number.

[0120] limit5: 0≤dth1≤3.14159, 0≤dth2≤3.14159, that is, the deflection angle dth1 of the first code reader 101 relative to the z-axis of the AGV coordinate system is within π, and the deflection angle dth2 of the second code reader 102 relative to the z-axis of the AGV coordinate system is within π.

[0121] Under the above constraints, the objective function is optimized to obtain the optimization result. Based on the optimization result, the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader are determined. That is, the minimum value of the objective function min fun = cost_fun(fun1,fun2,fun3) that satisfies the above constraints is obtained through optimization. The dx1, dy1, dth1, dx2, dy2, dth2, rL, rR, and wheelDist corresponding to min fun are taken as the results of this optimization calculation. That is, the actual positions of the first barcode reader 101, the second barcode reader 102, and the wheel odometer relative to the AGV coordinate system are obtained through this optimization calculation.

[0122] By combining and optimizing multiple data sources (i.e., the first QR code data, the second QR code data, and the odometer data) to obtain the actual position of each sensor, not only can multiple sensors be calibrated simultaneously, which is convenient and fast, but also the error is smaller and the actual position is more accurate compared to the result obtained by relying on a single data source.

[0123] In one possible implementation, determining the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader based on the optimization result may include:

[0124] By combining the optimization results with the deviation information, the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader are determined.

[0125] Still using Figure 2 , 3 The AGV shown and Figure 4 Taking the calibration environment shown as an example:

[0126] After the above optimization calculation, an optimization result is obtained. The corresponding deviation is then calculated from this result.

[0127]

[0128] Where X = (dx1, dy1, dth1, dx2, dy2, dth2, rL, rR, wheelDist) T Cov represents the covariance matrix of the input data source, which can be the preprocessed input data syn_qr1, syn_qr2, or syn_odom.

[0129] By acquiring sensor data multiple times through the first barcode reader 101, the second barcode reader 102, and the wheel odometer, and repeating the calculations multiple times, a series (n) of optimized calculation results and their corresponding deviation information are obtained. Based on these optimized calculation results and deviation information, the following operations can be determined:

[0130]

[0131] Among them, X i The values ​​(dx1, dy1, dth1, dx2, dy2, dth2, rL, rR, wheelDist) are obtained from the i-th optimization calculation. T , L(X i X is the value obtained from the i-th optimization calculation. i The magnitude of the deviation, where L_final is the final optimization calculation result.

[0132] The final optimization calculation result L_final determined using the above method is used as the final calibration result, which is the actual position of the first code reader 101, the second code reader 102, and the wheel odometer relative to the AGV coordinate system.

[0133] By combining the optimization calculation results with the covariance matrix method described above, a more robust solution can be obtained. The determined sensor position calibration results are more accurate, leading to more precise actions performed based on these sensors in subsequent practical applications of automated mobile devices, such as more accurate positioning of the automated mobile device.

[0134] In one example, the sensor position calibration method for an automated mobile device may further include: determining the offset information of each of the wheel odometer, the first barcode reader, and the second barcode reader based on their respective actual and initial positions.

[0135] Still using Figure 2 , 3 The AGV shown and Figure 4 Taking the calibration environment shown as an example:

[0136] Based on the initial and actual positions of the first barcode reader 101, the second barcode reader 102, and the wheeled odometer relative to the AGV coordinate system, the offset information of each barcode reader 101, the second barcode reader 102, and the wheeled odometer relative to their initial positions is determined. In subsequent AGV applications, for example... Figure 6 The diagram shown illustrates an embodiment of this disclosure of an automated moving vehicle (AGV) transporting goods. When an instruction is sent to the AGV to move goods from point A to point B, the AGV moves to point A. When the second barcode reader 102 scans the third QR code 23 below the shelf 3 at point A, combined with the previously determined sensor offset information, the AGV determines that it has reached point A. It then uses its lifting device to lift the shelf 3. Based on the QR code information laid on the ground, the AGV determines its next direction of movement. Each time the second barcode reader 102 scans a QR code, the AGV knows its current location and determines its next move. This continues until the second barcode reader 102 scans the fourth QR code 24 at point B. The AGV then knows it has reached the target location B and can control the lifting device to lower the shelf 3, thus completing the goods transport. Throughout this process, by combining the previously calibrated sensor positions during AGV positioning, its own position can be more accurately determined, thus better completing the transport task.

[0137] This disclosure also proposes a sensor position calibration device for an automated mobile device, such as... Figure 7 The diagram shown is a schematic of a sensor position calibration device for an automatic mobile device according to an embodiment of the present disclosure. The device includes a data acquisition unit 100 and a data processing unit 200.

[0138] The data acquisition unit 100 is configured to acquire sensor data, including odometer data from a wheeled odometer, first QR code data from a first barcode reader, and second QR code data from a second barcode reader. The data processing unit 200 is configured to perform position calibration based on the sensor data to obtain the actual positions of the wheeled odometer, the first barcode reader, and the second barcode reader.

[0139] The idea behind this sensor position calibration device embodiment is the same as the working process of the sensor position calibration method in the above embodiment. All the contents of the above sensor position calibration method embodiment are incorporated into this sensor position calibration device embodiment by means of full reference, and will not be repeated.

[0140] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0141] This disclosure also proposes a sensor position calibration system for automated mobile devices, such as... Figure 8 The diagram shown is a schematic of a sensor position calibration system for an automatic mobile device according to an embodiment of the present disclosure. The system includes a data acquisition module, an optimization calculation module, and a result analysis module.

[0142] The data acquisition module is configured to acquire sensor data, including odometer data from the wheel odometer, first QR code data from the first barcode reader, and second QR code data from the second barcode reader. The optimization calculation module is configured to perform optimization calculations based on the sensor data to obtain the sensor's position calibration result. The result analysis module is configured to analyze the position calibration result obtained from the optimization calculation to determine the final actual positions of the wheel odometer, the first barcode reader, and the second barcode reader.

[0143] The idea behind this sensor position calibration system embodiment is the same as the working process of the sensor position calibration method described above. All contents of the above-described sensor position calibration method embodiment are incorporated into this sensor position calibration system embodiment by means of full reference, and will not be repeated here.

[0144] This disclosure also proposes an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described sensor position calibration method when executing the instructions stored in the memory.

[0145] This disclosure also proposes a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the aforementioned sensor position calibration method. The computer-readable storage medium can be volatile or non-volatile.

[0146] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described sensor position calibration method.

[0147] Figure 9 This is a block diagram illustrating a sensor position calibration device 1900 for an automated mobile device according to an exemplary embodiment. For example, the device 1900 may be provided as a server or terminal device. (Refer to...) Figure 9 The device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the aforementioned sensor position calibration method.

[0148] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output (I / O) interface 1958. Device 1900 can operate on an operating system stored in memory 1932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0149] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to complete the sensor position calibration method described above.

[0150] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0151] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0152] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0153] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0154] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0155] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0156] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0157] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0158] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A sensor position calibration method for an automated mobile device, characterized in that, The method includes: Acquire sensor data, which includes odometer data from a wheel odometer, first QR code data from a first barcode reader, and second QR code data from a second barcode reader. The first QR code data is obtained by parsing the first QR code read by the first barcode reader, and the second QR code data is obtained by parsing the second QR code read by the second barcode reader. Position calibration is performed based on the sensor data to obtain the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader. The actual positions are determined based on an objective function and constraints. The objective function is constructed based on a first sub-function, a second sub-function, and a third sub-function. The first sub-function is related to the initial positions of the first QR code data and the first barcode reader. The second sub-function is related to the initial positions of the second QR code data and the second barcode reader. The third sub-function is related to the odometer data and the initial positions of the wheel odometer. The constraints include the weighted sum of the first, second, and third sub-functions being within a preset numerical range.

2. The method according to claim 1, characterized in that, The location calibration based on the sensor data includes: A target function is constructed based on the sensor data, and constraints are set. Under the given constraints, the objective function is optimized to obtain the optimization result; The actual positions of the wheel odometer, the first barcode reader, and the second barcode reader are determined based on the optimization results.

3. The method according to claim 1, characterized in that, The constraints are related to the first sub-function, the second sub-function, and the third sub-function.

4. The method according to claim 3, characterized in that, The constraints include at least one of the following: The size relationship between the first sub-function and the second sub-function after scaling based on the first coefficient; The size relationship between the first and third sub-functions after scaling based on the second coefficient; The weighting coefficients of the weighted sum of the first sub-function, the second sub-function, and the third sub-function, as well as the preset numerical range, are related to the first coefficient and the second coefficient.

5. The method according to claim 2, characterized in that, Determining the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader based on the optimization results includes: By combining the optimization results with the deviation information, the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader are determined.

6. The method according to claim 1, characterized in that, The method further includes: Based on the actual and initial positions of the wheel odometer, the first barcode reader, and the second barcode reader, the offset information of each of the wheel odometer, the first barcode reader, and the second barcode reader is determined.

7. The method according to claim 1, characterized in that, Before the location calibration based on the sensor data, the method further includes: The sensor data is preprocessed, including deleting invalid data from the sensor data and / or performing a timestamp alignment operation on the sensor data.

8. A sensor position calibration device for an automated mobile device, characterized in that, The device includes: The data acquisition unit is configured to acquire sensor data, which includes odometer data from a wheel odometer, first QR code data from a first barcode reader, and second QR code data from a second barcode reader. The first QR code data is obtained by parsing the first QR code read by the first barcode reader, and the second QR code data is obtained by parsing the second QR code read by the second barcode reader. A data processing unit is configured to perform position calibration based on the sensor data to obtain the actual positions of the wheel odometer, the first barcode reader, and the second barcode reader. The actual positions are determined based on an objective function and constraints. The objective function is constructed based on a first sub-function, a second sub-function, and a third sub-function. The first sub-function is related to the initial positions of the first QR code data and the first barcode reader. The second sub-function is related to the initial positions of the second QR code data and the second barcode reader. The third sub-function is related to the odometer data and the initial positions of the wheel odometer. The constraints include the weighted sum of the first, second, and third sub-functions being within a preset numerical range.

9. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; The processor is configured to implement the sensor position calibration method of any one of claims 1 to 7 when executing instructions stored in the memory.

10. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the sensor position calibration method of any one of claims 1 to 7 for an automatic mobile device.

Images