A cylindrical correction and weighted fusion-based vial image stitching method

Abstract

The application provides a penicillin bottle image splicing method based on cylindrical correction and weighted fusion, and relates to the field of image processing. The penicillin bottle image splicing method based on cylindrical correction and weighted fusion comprises the following steps: collecting a cylindrical label image; respectively performing distortion correction on the cylindrical label image from horizontal and vertical directions to obtain an approximately planar label image; roughly estimating an image overlapping area, then performing image registration on the estimated overlapping area; performing feature extraction on the overlapping area image, performing coarse feature matching, removing mis-matching points, and finally realizing feature matching of the image according to the sorting of the ratio of the Euclidean distance of the nearest neighbor and the second nearest neighbor of the feature points; and performing image fusion on the image after feature matching. The method solves the problem of the traditional fade-in and fade-out fusion algorithm, which adopts linear weighted fusion, and the splicing line after image fusion is obvious, and ghosting phenomenon is prone to occur.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for stitching images of vials based on cylinder correction and weighted fusion. Background Technology

[0002] Intravenous infusion drugs are a crucial component of medical infusions. In practice, medical staff must strictly adhere to hospital regulations, rigorously implement verification procedures, and follow aseptic techniques. To improve efficiency, machines are gradually replacing manual dispensing, and machine vision is increasingly being applied to drug bottle information management to reduce the workload of medical staff. The first step in managing drug bottle label information is acquiring complete label images. Due to the curved shape of the drug bottle and limitations of camera shooting angles, it is impossible to obtain a complete label image in a single shot. Therefore, images taken from different angles are typically stitched together to obtain a complete label image. Currently, the main method for acquiring curved labels is through linear scan cameras, which requires high-end hardware and is costly. Therefore, this paper proposes a bottle image stitching method based on cylindrical correction and weighted fusion. This method allows for the acquisition of complete curved label unfolded images using ordinary industrial cameras, laying the foundation for the next step of drug bottle information identification and management. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] To address the shortcomings of existing technologies, this invention provides a bottle image stitching method based on cylindrical correction and weighted fusion, which solves the problem of traditional fade-in / fade-out fusion algorithms, which use linear weighted fusion, resulting in obvious stitching lines after image fusion and are also prone to ghosting.

[0005] (II) Technical Solution

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] Firstly, a method for stitching images from vials based on cylinder correction and weighted fusion is provided, including:

[0008] Acquire images of cylindrical labels;

[0009] Distortion correction is performed on the cylindrical label image in both the horizontal and vertical directions to obtain an approximately planar label image;

[0010] The overlapping regions of the images are roughly estimated, and then image registration is performed on the estimated overlapping regions;

[0011] Feature extraction is performed on the overlapping region image, coarse feature matching is carried out, mismatched points are removed, and the feature points are sorted according to the Euclidean distance ratio between the nearest and second nearest neighbors to finally achieve image feature matching.

[0012] Image fusion is performed on the images after feature matching.

[0013] Preferably, the acquisition of cylindrical label images specifically involves acquiring cylindrical label images from four directions using four cameras.

[0014] Preferably, the distortion correction of the cylindrical label image in both the horizontal and vertical directions to obtain an approximately planar label image specifically includes:

[0015] Let R be the radius of the input cylindrical label, and x be the x-coordinate of the label image pixels. Then, it can be obtained through... Obtain the coordinates x' of the horizontally corrected image pixels;

[0016] Let f be the distance between the camera lens and the image plane, d1 be the distance between the image plane and the object plane, center_x be the horizontal center value of the image, and x and h be the coordinates of each point in the horizontal direction of the image. b The coordinates are the vertical coordinates of the image; through Obtain the ordinate h of the corrected image f .

[0017] Preferably, the step of roughly estimating the overlapping regions of the images and then performing image registration on the estimated overlapping regions specifically includes:

[0018] Let the two input images be I1(x,y) and I2(x,y), then:

[0019] I1(x,y)=I2(x-Δx,y-Δy)

[0020] Performing a Fourier transform yields:

[0021]

[0022] in, and It is the Fourier transform of I1(x,y) and I2(x,y);

[0023] Seek complex conjugate And obtained and Cross power spectrum

[0024] Define the normalized power spectrum.

[0025] Taking the inverse Fourier transform of the normalized power spectrum, we obtain the two-dimensional impulse function, δ(x-Δx,y-Δy)=F -1 [e -j2π(uΔx+vΔy) The horizontal and vertical coordinates obtained when the impulse function reaches its maximum value are the horizontal displacements of the image, which are the overlapping regions Δx and Δy of the image.

[0026] Preferably, the step of extracting features from the overlapping region image, performing coarse feature matching, removing mismatched points, and sorting the features based on the Euclidean distance ratio between the nearest and second nearest neighbors of each feature point to finally achieve image feature matching specifically involves:

[0027] The SURF algorithm is used to extract features from the overlapping region image, the FLANN fast matching algorithm is used for coarse feature matching, the random sampling consistency algorithm is used to remove mismatched points, and the feature points are sorted according to the Euclidean distance ratio between the nearest and second nearest neighbors to finally achieve image feature matching.

[0028] Preferably, the image fusion of the feature-matched images specifically includes:

[0029] The first row of the overlapping area image is regarded as a stitching line, and each pixel value is used as the intensity value. The column number of the image is the point where the stitching line is located.

[0030] Starting from the first row, calculate the intensity value of the current pixel and compare it with the intensity values ​​of the pixels at the left, center, and right positions in the next row. The column number of the pixel with the lowest intensity value is the point where the stitching line is located.

[0031] E g =[S x ·(I1(x,y)-I2(x,y))] 2 +[S y ·(I1(x,y)-I2(x,y))] 2

[0032] in, The difference in intensity values ​​between pixels is E(x,y) = E c (x,y) 2 +E g (x,y);

[0033] Repeat the above operation to find the line with the smallest strength value as the best splicing line, and calculate the left and right boundaries L and R of the outer rectangle of the best splicing line.

[0034] Perform the following operations on two images f1(x,y) and f2(x,y):

[0035]

[0036] The weight values ​​ω1 and ω2 are: ω2=1-ω1.

[0037] Secondly, a bottle image stitching system based on cylinder correction and weighted fusion is provided, including:

[0038] The image acquisition module is used to acquire images of cylindrical labels;

[0039] The image correction module is used to correct the distortion of the cylindrical label image from the horizontal and vertical directions respectively, so as to obtain an approximately planar label image;

[0040] The image registration module is used to roughly estimate the overlapping regions of the images, and then perform image registration on the estimated overlapping regions;

[0041] The feature extraction module is used to extract features from the overlapping region image;

[0042] The feature matching module is used to perform coarse feature matching, remove mismatched points, and sort the feature points according to the Euclidean distance ratio between the nearest and second nearest neighbors to finally achieve feature matching of the image.

[0043] The image fusion module is used to perform image fusion on images after feature matching.

[0044] Thirdly, a computer-readable storage medium is provided for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any of the methods described.

[0045] Fourthly, a computing device is provided, comprising:

[0046] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described.

[0047] (III) Beneficial Effects

[0048] This invention presents a bottle image stitching method based on cylinder correction and weighted fusion, solving the problems of traditional fade-in / fade-out fusion algorithms, which use linear weighted fusion and result in obvious stitching lines and ghosting. The cylinder correction algorithm unfolds the cylindrical labels into a planar image, facilitating subsequent stitching. A phase correlation algorithm extracts overlapping areas, significantly improving image stitching speed and shortening matching time. Finally, combining an optimal stitching line algorithm and nonlinear weighted fusion improves image fusion quality, meeting the requirements for stitching cylindrical label images. Attached Figure Description

[0049] Figure 1 This is a flowchart of the method of the present invention;

[0050] Figure 2 This is a schematic diagram of image acquisition in an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0052] Example

[0053] like Figure 1 As shown, this embodiment of the invention provides a bottle image stitching method based on cylinder correction and weighted fusion, including:

[0054] Acquire images of cylindrical labels;

[0055] Distortion correction is performed on the cylindrical label image in both the horizontal and vertical directions to obtain an approximately planar label image;

[0056] The overlapping regions of the images are roughly estimated, and then image registration is performed on the estimated overlapping regions;

[0057] Feature extraction is performed on the overlapping region image, coarse feature matching is carried out, mismatched points are removed, and the feature points are sorted according to the Euclidean distance ratio between the nearest and second nearest neighbors to finally achieve image feature matching.

[0058] Image fusion is performed on the images after feature matching.

[0059] For details, please refer to Figure 2 Four cameras were used to capture images of the cylindrical label from four different directions.

[0060] A cylindrical correction algorithm is used to correct the bending distortion of the label image.

[0061] (1-1) Let R be the radius of the input cylindrical label and x be the x-coordinate of the label image pixel. Then, it can be obtained by... Obtain the coordinates x' of the horizontally corrected image pixels.

[0062] (1-2) Let f be the distance between the camera lens and the image plane, d1 be the distance between the image plane and the object plane, center_x be the horizontal center value of the image, and x be the coordinates of each point in the horizontal direction of the image. b These are the coordinates of the image's vertical direction. Then, it can be obtained through... Obtain the ordinate h of the corrected image f .

[0063] Step 002. Before feature extraction, a phase correlation algorithm is used to roughly estimate the overlapping regions of the image, and then image registration is performed on the estimated overlapping regions.

[0064] (2-1) Let the two input images be I1(x,y) and I2(x,y). Since the two images to be stitched have a certain translation relationship, I1(x,y) = I2(x-Δx,y-Δy). Performing a Fourier transform on them, we get... in, and It is the Fourier transform of I1(x,y) and I2(x,y).

[0065] (2-2) is obtained complex conjugate And obtained and Cross power spectrum

[0066] (2-3) Define the normalized power spectrum.

[0067] (2-4) Obtain the inverse Fourier transform of the normalized power spectrum to obtain the two-dimensional impulse function, δ(x-Δx,y-Δy)=F -1 [e -j2π(uΔx+vΔy) The x and y coordinates obtained when the impulse function reaches its maximum value represent the horizontal displacement of the image, which is the overlapping region of the image, x, y.

[0068] The SURF algorithm is used to extract features from overlapping regions of the image; the FLANN fast matching algorithm is used for coarse feature matching; mismatched points are removed using the Random Sample Consensus Algorithm (RANSAC); the feature points are sorted according to the Euclidean distance ratio between their nearest and second nearest neighbors to achieve final feature matching; and an improved optimal stitching line algorithm is used for image fusion.

[0069] Image fusion is performed using an improved optimal stitching line algorithm:

[0070] (7-1) First, the first row of the overlapping area image is regarded as a splicing line, and each pixel value is used as the intensity value. The column number of the image is the point where the splicing line is located.

[0071] (7-2) Starting from the first row, calculate the intensity value of the current pixel and compare it with the intensity values ​​of the pixels at the left, center, and right positions in the next row. The column number of the pixel with the lowest intensity value is the location of the splicing line, also known as the extension direction of the splicing line. Where E c E represents the difference in color values. g E g The difference is the structural value, and

[0072] E g =[S x ·(I1(x,y)-I2(x,y))] 2 +[S y ·(I1(x,y)-I2(x,y))] 2 ,in Thus, the difference in intensity values ​​between pixels is obtained as E(x,y)=E c (x,y) 2 +E g (x,y).

[0073] (7-3) Repeat the above operation, and count the line with the smallest strength value as the best splicing line, and find the left and right boundaries L and R of the outer rectangle of the best splicing line.

[0074] (7-4) Perform the following operations on the two images f1(x,y) and f2(x,y) The weight values ​​ω1 and ω2 are: ω2=1-ω1.

[0075] In the above embodiments, the improved nonlinear weighted fusion can effectively reduce grayscale differences, eliminate stitching marks, and make the stitched image smoother, more natural, and with better imaging quality.

[0076] Simultaneously, image stitching was performed using the SURF algorithm and the method provided in the embodiments. The stitching results are shown in Tables 1, 2, and 3 below. The resolution of images img1 and img2 is 924 pixels x 1171 pixels, and the image format is JPG.

[0077] Table 1 Comparison of image stitching speeds for different algorithms

[0078]

[0079] As can be seen from Table 1, although the number of feature points extracted by this algorithm is reduced, the number of correct matching pairs is not significantly reduced. Moreover, the algorithm in this paper saves nearly 51.62% of the time in the image registration stage.

[0080] Table 2 Image fusion performance indicators of different methods

[0081]

[0082]

[0083] Table 3 Comparison of Image Fusion Effects

[0084]

[0085] Table 2 selects image fusion quality evaluation metrics such as Information Entropy (IE), Average Gradient (AG), Peak Signal Noise Ratio (PSNR), Root Mean Square Error (RMSE), Cross Entropy (CE), and Mutual Information (MI) to evaluate the algorithm, and compares it with direct fusion algorithms and traditional gradual-in / gradual-out fusion algorithms. To comprehensively assess the merits of the algorithms, Table 3 uses a gray-level correlation-based comprehensive evaluation method to apply multiple quantization criteria to the three algorithms, with equal weights assigned during calculation. It can be seen that compared with the other two algorithms, this algorithm significantly improves image fusion quality.

[0086] Another embodiment of the present invention provides a vial image stitching system based on cylinder correction and weighted fusion, comprising:

[0087] The image acquisition module is used to acquire images of cylindrical labels;

[0088] The image correction module is used to correct the distortion of the cylindrical label image from the horizontal and vertical directions respectively, so as to obtain an approximately planar label image;

[0089] The image registration module is used to roughly estimate the overlapping regions of the images, and then perform image registration on the estimated overlapping regions;

[0090] The feature extraction module is used to extract features from overlapping regions of the image.

[0091] The feature matching module is used to perform coarse feature matching, remove mismatched points, and sort the feature points according to the Euclidean distance ratio between the nearest and second nearest neighbors to finally achieve feature matching of the image.

[0092] The image fusion module is used to perform image fusion on images after feature matching.

[0093] Embodiments of this application may be provided as methods or computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application may be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0094] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0095] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0096] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0097] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for stitching images from vials based on cylinder correction and weighted fusion, characterized in that, include: Acquire images of cylindrical labels; Distortion correction is performed on the cylindrical label image in both the horizontal and vertical directions to obtain an approximately planar label image; The overlapping regions of the images are roughly estimated, and then image registration is performed on the estimated overlapping regions; Feature extraction is performed on the overlapping region image, coarse feature matching is carried out, mismatched points are removed, and the feature points are sorted according to the Euclidean distance ratio between the nearest and second nearest neighbors to finally achieve feature matching of the image. Image fusion is performed on the feature-matched images; The image fusion process after feature matching specifically includes: The first row of the overlapping area image is regarded as a stitching line, and each pixel value is used as the intensity value. The column number of the image is the point where the stitching line is located. Starting from the first row, calculate the intensity value of the current pixel and compare it with the intensity values ​​of the pixels at the left, center, and right positions in the next row. The column number of the pixel with the lowest intensity value is the point where the stitching line is located. in, , ; The difference in intensity values ​​between pixels is obtained. ; Repeat the above operation to find the line with the smallest strength value as the best splicing line, and calculate the left and right boundaries L and R of the outer rectangle of the best splicing line. For two images , Perform the following calculations: Among them, the weight value and for: , .

2. The method for stitching images in vials based on cylinder correction and weighted fusion according to claim 1, characterized in that: The acquisition of cylindrical label images specifically involves using four cameras to capture images of the cylindrical labels from four different directions.

3. The method for stitching images in vials based on cylinder correction and weighted fusion according to claim 2, characterized in that: The distortion correction of the cylindrical label image in both the horizontal and vertical directions to obtain an approximately planar label image specifically includes: Let R be the radius of the input cylindrical label, and x be the x-coordinate of the label image pixels. Then, it can be obtained through... Obtain the coordinates of the horizontally corrected image pixels. ; Let f be the distance between the camera lens and the image plane, d1 be the distance between the image plane and the object plane, and the horizontal center value of the image be d1. The coordinates of each point in the horizontal direction of the image are x. The coordinates are the vertical coordinates of the image; through The ordinate of the corrected image is obtained. .

4. The method for stitching images in vials based on cylinder correction and weighted fusion according to claim 3, characterized in that: The process of roughly estimating the overlapping regions of the images and then performing image registration on the estimated overlapping regions specifically includes: Let the two input images be... , ,but: Performing a Fourier transform yields: in, and yes and Fourier transform; Seeking complex conjugate and obtained and Cross power spectrum ; Define the normalized power spectrum. ; By taking the inverse Fourier transform of the normalized power spectrum, the two-dimensional impulse function is obtained. The x and y coordinates obtained when the impulse function reaches its maximum value represent the horizontal displacement of the image, which is the overlapping region of the image. , .

5. The method for stitching images in vials based on cylinder correction and weighted fusion according to claim 4, characterized in that: The process of extracting features from overlapping regions of the image, performing coarse feature matching, removing mismatched points, and sorting the features based on the Euclidean distance ratio between their nearest and second nearest neighbors to achieve final image feature matching is as follows: The SURF algorithm is used to extract features from the overlapping region image, the FLANN fast matching algorithm is used for coarse feature matching, the random sampling consistency algorithm is used to remove mismatched points, and the feature points are sorted according to the Euclidean distance ratio between the nearest and second nearest neighbors to finally achieve image feature matching.

6. A system for performing the vial image stitching method based on cylinder correction and weighted fusion as described in any one of claims 1-5, characterized in that, include: The image acquisition module is used to acquire images of cylindrical labels; The image correction module is used to correct the distortion of the cylindrical label image from the horizontal and vertical directions respectively, so as to obtain an approximately planar label image; The image registration module is used to roughly estimate the overlapping regions of the images, and then perform image registration on the estimated overlapping regions; The feature extraction module is used to extract features from overlapping regions of the image. The feature matching module is used to perform coarse feature matching, remove mismatched points, and sort the feature points according to the Euclidean distance ratio between the nearest and second nearest neighbors to finally achieve feature matching of the image. The image fusion module is used to perform image fusion on images after feature matching.

7. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods according to claims 1-5.

8. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods according to claims 1-5.

Images