A method for splicing and correcting express delivery barcodes

Abstract

The present invention relates to the field of image stitching, and specifically discloses a method for stitching and correcting express barcodes. Multiple cameras are used to obtain images of the express items with codes to be read. The obtaining process is as follows: 1) Calibrate the internal and external parameters of the cameras in advance to obtain the transformation relationship between multiple cameras; 2) Obtain the imaging images and perform preprocessing. Use the morphological method to find and judge whether there is barcode information at the edges of the stitching parts of each imaging image; 3) If not, obtain the roughly matched stitching image, read the barcode information and output it; 4) If there is barcode information, perform fine-matched stitching and then read the barcode information and output it. The method of the present invention replaces the use of position detection instruments, thereby reducing the investment in equipment costs. For those that require barcode stitching, fine-matched stitching is performed. This method achieves stitching at a relatively accurate position, the stitched barcode is relatively complete, can be accurately and effectively read, and both the stitching and reading accuracies are relatively high.

Description

Technical Field

[0001] The present invention relates to the field of image stitching, and particularly to a stitching and correction method for express barcode images. Background Art

[0002] With the rapid development of the logistics industry and the acceleration of the improvement of logistics operations and their automation level, the barcodes on express delivery forms have evolved from being read by the initial laser scanners to being automatically read by industrial cameras. Currently, in the express delivery industry, laser scanners or industrial cameras are generally used to read barcodes. An ordinary laser scanner can only read one barcode at a time, and the operator needs to align the scanner with the barcode, resulting in low work efficiency. To solve this problem, it was later improved to use industrial cameras, which can read multiple barcodes at a time, greatly improving the work efficiency of barcode reading. However, when the area where barcodes are distributed on multiple express deliveries within the field of view of an industrial camera is large, an industrial camera with ordinary pixels cannot capture clearly and improvements are needed. For example, using an industrial camera with high pixels will greatly increase the cost of the equipment; or using multiple ordinary cameras to synchronously read barcodes, but currently, using multiple ordinary cameras to synchronously read barcodes requires the simultaneous use of position detection instruments to achieve barcode combination, increasing the setting of detection instruments and also greatly increasing the cost of the equipment. Summary of the Invention

[0003] The purpose of the present invention is to provide a stitching and correction method for express barcodes that uses multiple ordinary cameras to obtain images for stitching calculation and processing to solve the problems existing in the above-mentioned prior art.

[0004] To achieve the above purpose, the technical solution of the present invention is: a stitching and correction method for express barcodes, which uses multiple cameras to obtain images of the express deliveries to be coded. The method is characterized in that the process is as follows:

[0005] 1), Calibrate the internal and external parameters of the cameras in advance to obtain the imaging image transformation relationship between two cameras, including the transformation matrix between the cameras;

[0006] 2), During operation, collect images through each camera to obtain imaging images, preprocess each imaging image, and use morphological methods to find the positions of barcode information on each imaging image;

[0007] 3), Complete the rough matching stitching between each imaging image by feature point matching to obtain a rough matching stitching image; Determine whether there is barcode information at the edge of the stitching part of the rough matching stitching image. If not, enter 3.1), if so, enter 4);

[0008] 3.1), If there is no barcode information at the edge of the stitching part of each imaging image, directly obtain the stitching image according to the rough matching stitching image, read the barcode information on the stitching image and output it;

[0009] 4). If the edge of the splicing part of each imaging image contains barcode information, perform precise matching splicing; the precise matching splicing method is as follows:

[0010] 4.1). Calculate the barcode coordinate value in the camera coordinate system. The actual width value of the barcode is a known value. According to the following formula, the barcode coordinate value in the camera coordinate system can be obtained.

[0011] In the formula

[0012] d is the width value of the barcode on the imaging image.

[0013] D is the actual width value of the barcode on the imaging image.

[0014] f is the normalized focal length of the camera.

[0015] z is the Z-axis coordinate value of the plane where the barcode is located in the coordinate system of Camera 1.

[0016] 4.2). Convert multiple images to the same image coordinate system and calculate the conversion through the following two formulas.

[0017]

[0018] In the formula,

[0019] (x, y) is the coordinate value in the image pixel coordinates.

[0020] (X, Y.Z) T is the coordinate value in the camera coordinate system.

[0021] M is the internal parameter of the camera obtained by calibration in step 1).

[0022] is the coordinate value in the coordinate system of Camera 1 calculated correspondingly.

[0023] M RT is the transformation matrix between the coordinate systems of Camera 1 and Camera 2 obtained by calibration in step 1).

[0024] is the coordinate value in the coordinate system of Camera 2 calculated correspondingly.

[0025] 4.3). For the imaging image of the barcode position located in step 2), use the canny operator to detect the edge information of the barcode, and perform translational micro-adjustment according to the edge coincidence of the common area. The translational micro-adjustment first obtains the length of the barcode on the imaging image where the barcode exists on the edge to be spliced on the imaging image through the following judgment. The formula is

[0026] In the formula,

[0027] l is the length of the barcode existing on the edge of the part to be stitched on the imaging image,

[0028] L is the actual length value of the barcode,

[0029] z is the Z-axis coordinate value of the plane where the barcode is located in the coordinate system of Camera 1,

[0030] f is the normalized focal length of the camera;

[0031] Then, the horizontal translation amount and vertical translation amount of the right part of the barcode to be stitched are obtained through the following formula for translational fine adjustment to complete barcode stitching.

[0032] In the formula

[0033] kk is the slope of the straight line where the left part of the barcode is located at the edge of the imaging image,

[0034] l is the length straight line of the barcode existing on the edge of the part to be stitched on the imaging image,

[0035] Δd is the offset value of the barcode edge in the barcode width direction,

[0036] Δl is the offset value of the barcode edge in the barcode length direction,

[0037] Δx is the horizontal translation amount of the right part of the barcode,

[0038] Δy is the vertical translation amount of the right part of the barcode;

[0039] 4.4) Determine whether the barcode is tilted according to the quadrilateral boundary of the stitched barcode in step 4.3). If it is tilted, find the corresponding perspective transformation matrix to transform the tilted quadrilateral barcode into a rectangle to obtain a precisely matched stitched image, read the barcode information and output it.

[0040] By adopting the above technical solutions, the beneficial effects of the present invention are as follows: When multiple ordinary cameras are used to acquire images, through the above-mentioned splicing and calibration methods of the present invention, for the situation where the same barcode is within the common field of view of two adjacent cameras and each of the two cameras can only collect partial barcode images, precise matching splicing can be performed to accurately splice and restore the barcode, which is conducive to accurately completing barcode reading, and solves the problems mentioned in the above prior art that when using ordinary cameras, a single camera cannot clearly capture the barcode and multiple cameras need to rely on position detection instruments to achieve barcode combination. The present invention solves the problem that when using multiple ordinary cameras, it is necessary to rely on position detection instruments through image splicing algorithm processing, replaces the use of position detection instruments, and thus also reduces the investment in equipment costs. In the method of the present invention, in the precise matching splicing, the splicing of the corresponding barcodes is first performed, and then the translation adjustment is carried out, that is, the calibration after the splicing of the corresponding barcodes before, to achieve precise matching splicing, which is conducive to providing accurate and effective barcode reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 FIG. is a flowchart of a method for splicing and calibrating an express barcode according to the present invention;

[0042] Figure 2 FIG. is a splicing schematic diagram of a method for splicing and calibrating an express barcode according to the present invention;

[0043] Figure 2a FIG. is a schematic diagram of the position and shape of barcode splicing in a method for splicing and calibrating an express barcode according to the present invention;

[0044] Figure 2b is Figure 2a a border schematic diagram of barcode splicing;

[0045] Figure 3a FIG. is another schematic diagram of the position and shape of barcode splicing in a method for splicing and calibrating an express barcode according to the present invention;

[0046] Figure 3b is Figure 3a a border schematic diagram of barcode splicing. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] In order to further explain the technical solutions of the present invention, the present invention will be elaborated in detail below through specific embodiments.

[0048] A method for splicing and correcting express barcodes disclosed by the present invention is applicable to a barcode scanning device that uses multiple cameras (especially ordinary cameras) to acquire images of express items with barcodes to be read and read barcodes. In this embodiment, the setting and use of two cameras are taken as an example for detailed description. In fact, for more cameras, the processing of pairwise images is also involved in splicing. Therefore, the method for setting more cameras is actually the same as that for the two cameras in this embodiment. Those skilled in the art can clearly understand how to apply the method through this embodiment. The method of the present invention will be described in detail below.

[0049] A method for splicing and correcting express barcodes of the present invention has the following steps in the process, as Figure 1 shown

[0050] 1) Install two cameras at the correct corresponding positions after the production of the barcode scanning device. Before starting the barcode scanning work, calibrate the internal and external parameters of the cameras in advance. The Zhang Zhengyou checkerboard calibration method can be used to obtain the internal and external parameters of the images. Place the same calibration board in the common area of the two cameras, and the two cameras respectively collect checkerboard images, and respectively obtain the rotation and translation matrices from the cameras to the checkerboard coordinate system, and obtain the imaging image transformation relationship between multiple cameras, including the transformation matrix between the cameras. This calibration method is a prior art, and no more detailed description of the calibration process will be given here.

[0051] 2) When the barcode scanning device starts the barcode scanning work, obtain imaging images by respectively collecting images through the two cameras, and preprocess each imaging image. The preprocessing is a technology often used in existing image processing, such as performing gray-scale transformation, denoising, etc. on the imaging image, removing the information in the imaging image that affects barcode recognition and barcode reading, which is beneficial to improving the accuracy of the subsequent precise matching splicing and precise reading required;

[0052] After preprocessing the imaging images, use the morphological method (a well-known method) to find the positions of barcode information on each imaging image and judge whether there is barcode information at the splicing part edges of each imaging image, that is, find the positions of barcode information on the imaging images of the two cameras.

[0053] 3) Complete the rough matching splicing between each imaging image by using feature point matching (a well-known matching method, which will not be described in detail here), obtain the rough matching splicing image, and judge whether there is barcode information at the corresponding edges of the corresponding splicing parts of the rough matching splicing image, that is, judge whether the two cameras respectively collect partial barcode information of the same barcode. If not, enter 3.1); if so, enter 4);

[0054] 3.1), if the barcode information does not exist at the edge of the splicing part of the roughly matched splicing image in the judgment result of step 3), directly obtain the splicing image according to the roughly matched splicing image, read the barcode information on the splicing image and output it. That is, in this case of this step, high requirements for the splicing result of the imaging images of the two cameras are not needed;

[0055] 4), if the barcode information exists at the edge of the splicing part of each imaging image in the judgment result of step 2), perform fine matching splicing; the fine matching splicing method is as follows,

[0056] 4.1), calculate the Z-axis coordinate value of the barcode in the camera coordinate system. Here, it should be noted that usually, the express deliveries that batch scan and read barcodes are of the same type of express, that is, from the same express company. Then the actual width value of the barcode is a known value. Of course, if they are not of the same type, the corresponding information data can also be obtained in advance, and then the Z-axis coordinate value of the barcode in the camera coordinate system can be obtained according to the following formula,

[0057] In the formula

[0058] d is the width value of the barcode on the imaging image,

[0059] D is the actual width value of the barcode on the imaging image,

[0060] f is the normalized focal length of the camera,

[0061] z is the Z-axis coordinate value of the plane where the barcode is located in the coordinate system of Camera 1;

[0062] 4.2), convert multiple images to the same image coordinate system, and calculate the conversion through the following two formulas,

[0063]

[0064] In the formula,

[0065] (x, y) is the coordinate value in the image pixel coordinate,

[0066] (X, Y, Z) T is the coordinate value in the camera coordinate system,

[0067] M is the camera internal parameter calibrated in step 1),

[0068] is the corresponding calculated coordinate value in the coordinate system of Camera 1,

[0069] M RT is the transformation matrix between the coordinate system of Camera 1 and Camera 2 calibrated in step 1),

[0070] Are the coordinate values in the coordinate system of camera 2 corresponding to the calculation,

[0071] 4.3), Detect the edge information of the barcodes in the imaging images of the barcode positions located in step 2) using the canny operator (a well-known detection method, not described in detail here), and perform translational fine-tuning based on the edge coincidence of the common area. For example, Figure 2a and Figure 3a show two splicing form situations of barcode splicing at the barcode positions to be spliced on the two imaging images found in step 2), Figure 2b and Figure 3b are Figure 2a and Figure 3a The corresponding schematic diagrams of the barcode image borders. The straight lines where the upper and lower edges are located should coincide after splicing the barcodes on the left and right parts. It is necessary to perform a certain translation on one of the imaging images to be completely aligned and spliced. For example, Figure 2 shown. Therefore, translational fine-tuning in this step is still required. In this step, translational fine-tuning first calculates the length of the barcode existing on the edge of the part to be spliced on the imaging image by the following method. The formula is,

[0072] In the formula,

[0073] l is the length of the barcode existing on the edge of the part to be spliced on the imaging image. Here, the length is not the actual length of the barcode or the length on the image, but the partial length of the barcode,

[0074] L is the actual length value of the barcode,

[0075] z is the Z-axis coordinate value of the plane where the barcode is located in the coordinate system of camera 1,

[0076] f is the normalized focal length of the camera;

[0077] Then, the horizontal translation amount and vertical translation amount of the right part of the barcode to be spliced are calculated through the following formula for translational fine-tuning to complete barcode splicing.

[0078] In the formula

[0079] k is the slope of the straight line where the left part of the barcode is located on the edge of the imaging image ( Figure 2 Line 1 in

[0080] l is the length of the barcode existing on the edge of the part to be spliced on the imaging image ( Figure 2 Line 1 in

[0081] Δd is the offset value of the barcode edge in the barcode width direction,

[0082] Δl is the offset value of the barcode edge in the barcode length direction,

[0083] Δx is the horizontal translation amount of the right part of the bar code,

[0084] and Δy is the vertical translation amount of the right part of the bar code;

[0085] 4.4), Determine whether the bar code of the quadrilateral boundary of the spliced bar code is inclined according to step 4.3). If it is inclined, find the corresponding perspective transformation matrix to transform the inclined quadrilateral bar code into a rectangle, obtain the accurately matched spliced image, read the bar code information and output it.

[0086] Through the above steps, the splicing and correction method of the present invention is completed. It can be seen from the above steps that due to the setting of the common working field of multiple cameras, the combined shooting field of view is large enough to cover all express deliveries, and the cameras can be set at a shooting position where the shooting is clear. Therefore, for the situation where there is no case where both cameras can only collect partial bar code images, the two cameras can perform simple image splicing, read the bar codes respectively, and no more arithmetic processing is required. For the situation where both cameras can only collect partial bar code images, after the image splicing of the camera coordinate conversion relationship, micro-adjustment correction is also performed to achieve a more accurate position splicing. In this way, the spliced bar code is relatively complete, can be accurately and effectively read, and both the splicing and reading accuracy are relatively high, thus achieving the purpose and effect of the present invention.

[0087] The above embodiments and drawings do not limit the product form and style of the present invention. Any appropriate changes or modifications made by those of ordinary skill in the art shall be regarded as not departing from the patent scope of the present invention.

Claims

1. A method for splicing and correcting express barcodes, which uses multiple cameras to obtain images of the express items with barcodes to be read, characterized in that, the method process is as follows: 1). Calibrate the internal and external parameters of the cameras in advance to obtain the imaging image transformation relationship between the two cameras, including the transformation matrix between the cameras; 2). During operation, collect images through each camera to obtain imaging images, preprocess each imaging image, and use morphological methods to find the positions of barcode information on each imaging image; 3). Complete the rough matching and splicing between each imaging image by feature point matching to obtain a rough matching and splicing image; judge whether there is barcode information at the edge of the splicing part of the rough matching and splicing image. If not, enter 3.1), if so, enter 4); 3.1) If there is no barcode information at the edge of the splicing part of each imaging image, directly obtain the splicing image according to the rough matching and splicing image, read the barcode information on the splicing image and output it; 4). If there is barcode information at the edge of the splicing part of each imaging image, perform fine matching and splicing; the fine matching and splicing method is as follows, 4.1). Calculate the barcode coordinate values in the camera coordinate system. The actual width value of the barcode is a known value. According to the following formula, the barcode coordinate values in the camera coordinate system can be obtained. In the formula is the width value of the barcode on the imaging image, is the actual width value of the barcode on the imaging image, is the normalized focal length of the camera, is the Z-axis coordinate value of the plane where the barcode is located in the coordinate system of Camera 1; 4.2). Convert multiple images to the same image coordinate system, and calculate the conversion through the following two formulas, , , in the formula, is the coordinate value in the image pixel coordinates, is the coordinate value in the camera coordinate system, is the camera internal parameter obtained by calibration in step 1), is the coordinate value in the camera 1 coordinate system obtained by corresponding calculation, is the transformation matrix between the camera 1 coordinate system and the camera 2 coordinate system obtained by calibration in step 1), is the coordinate value in the camera 2 coordinate system obtained by corresponding calculation, 4.3), Detect the edge information of the barcode in the imaging image of the barcode position located in step 2) using the canny operator, and perform translational fine-tuning according to the edge coincidence of the common area. The translational fine-tuning first obtains the length of the barcode on the imaging image where the barcode exists on the edge of the part to be spliced on the imaging image through the following formula, , In the formula, is the length of the barcode existing on the edge of the part to be spliced on the imaging image, is the actual length value of the barcode, is the Z-axis coordinate value of the plane where the barcode is located in the coordinate system of camera 1, is the normalized focal length of the camera; and then obtain the horizontal translation amount and vertical translation amount of the right side of the barcode to be spliced through the following formula for translational micro-adjustment to complete barcode splicing, , , in the formula, is the slope of the straight line where the left part of the barcode is located at the edge of the imaging image, is the length straight line of the barcode existing on the edge of the part to be spliced on the imaging image, is the offset value of the barcode edge in the barcode width direction, is the offset value of the barcode edge in the barcode length direction, is the horizontal translation amount of the right part of the barcode, is the numerical translation amount of the right part of the barcode; 4.4). Judge whether the barcode is tilted according to the quadrilateral boundary of the spliced barcode in step 4.3). If it is tilted, find the corresponding perspective transformation matrix to transform the tilted quadrilateral barcode into a rectangle to obtain a fine matching and splicing image, read the barcode information and output it.

Images