Large-format tile imaging method and system based on multi-line scan camera image stitching

Abstract

The present invention relates to a large-format tile imaging method based on multi-line scan camera image stitching, comprising the following steps: after an imaging system is determined, multiple cameras in the imaging system are calibrated to obtain a homography matrix adapted to the hardware of the imaging system; the type of target tile is determined, and the complexity of the target tile texture is determined according to the determined type; the target tile image is obtained, and if the complexity of the target tile texture is a simple texture tile, the target tile image is stitched using a simple texture image, and if the complexity of the target tile texture is a complex texture tile, the target tile image is stitched using a complex texture image. The present invention corrects simple texture tiles through the corrected homography matrix to achieve basic large-format image stitching. Real-time image stitching performed in the case of complex texture tiles can alleviate the impact of changes in homography matrix parameters caused by vibration during the operation of the tile imaging platform.

Description

Technical Field

[0001] The present invention relates to the field of robot visual positioning technology, and in particular to a large-format ceramic tile imaging method and system based on multi-line scanning camera image splicing. Background Art

[0002] When inspecting tiles, how to image them is a problem that must be solved. In today's market, imaging is often performed using a single camera. In this way, if the field of view is the same, it is more economical to use multiple low-resolution cameras to shoot the same format than to use a single higher-resolution camera. The use of images obtained by multiple cameras for stitching can overcome the disadvantage that a single camera cannot obtain a larger inspection field of view, and real-time image stitching performed in the case of tiles with complex textures can alleviate the impact of changes in homography matrix parameters caused by vibration during the operation of the tile imaging platform.

[0003] However, the image stitching technology in this regard is not perfect in the current market. The current market needs a large-format tile imaging method based on multi-line scan camera image stitching. Summary of the invention

[0004] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a large-format tile imaging method and system based on multi-line scan camera image stitching.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] Specifically, a large-format tile imaging method based on multi-line scan camera image stitching is proposed, including the following:

[0007] After the imaging system is determined, multiple cameras in the imaging system are calibrated to obtain a homography matrix adapted to the hardware of the imaging system;

[0008] Determine the type of the target tile, and determine the complexity of the texture of the target tile according to the determined type;

[0009] Get the target tile image. If the complexity of the target tile texture is simple texture tile, use simple texture image stitching for the target tile image. If the complexity of the target tile texture is complex texture tile, use complex texture image stitching for the target tile image.

[0010] Further, specifically, calibrating multiple cameras in the imaging system to obtain a homography matrix adapted to the hardware of the imaging system includes:

[0011] Obtain the result of processing the camera through the black and white calibration plate during manual calibration, and calculate the intrinsic and extrinsic parameters of the camera based on the result to obtain the intrinsic and extrinsic parameter matrix of each camera to complete the calibration;

[0012] At the intersection of the fields of view of any two cameras, the feature points of the correction plate are obtained by the SIFT algorithm to obtain multiple feature points, and the homography matrix *M between multiple cameras is calculated based on the multiple feature points 1,2 ,M 2,3 ,…M n-1,n +, where n is the number of cameras used.

[0013] Further, specifically, the type of the target tile is determined, and the complexity of the target tile texture is determined according to the determined type, including:

[0014] Obtaining the tile type selected by the user from a pre-established database as the type of the target tile;

[0015] Recalling tile images in the interaction area corresponding to the two cameras pre-stored in the database;

[0016] Extracting feature points from the tile image to obtain the number of feature points and calculating the histogram variance of the tile image;

[0017] When the number of feature points is greater than a preset threshold T1 and the histogram variance is greater than T2, the target tile is determined to be a complex texture tile, otherwise the target tile is determined to be a simple texture tile.

[0018] Further, specifically, the simple texture image stitching process includes the following,

[0019] Multiple cameras jointly capture multiple tile images, assuming that there are n images;

[0020] Based on multiple tile images through the homography matrix *M 1,2 ,M 2,3 ,…M n-1,n + Stitching, output a single stitched high-resolution tile image.

[0021] Further, specifically, the complex texture image stitching process includes the following,

[0022] Multiple cameras jointly capture multiple tile images, assuming that there are n images, where I1(x, y) and I2(x, y) are two images with overlapping areas, and (x0, y0) is the translation change between the images;

[0023] Perform the following calculations,

[0024]

[0025] The cross power spectrum of two images is defined as follows:

[0026]

[0027] in is the complex conjugate of F1(u,v),

[0028] Then perform inverse Fourier transform on the above equation to get:

[0029]

[0030] Find the maximum value of G max , and obtain G max The corresponding position is the translation amount x0, y0 between the images.

[0031] Similarly, convert the (x, y) coordinates of the two images into polar coordinates (ρ, θ), and perform the above calculation process again to find the corresponding rotation coefficients.

[0032] Finally, by determining the translation amount and rotation coefficient, the image I2(x, y) can be projected to the image coordinates of I1(x, y), completing the stitching of the two images with overlapping areas.

[0033] In this way, all images with overlapping areas are stitched together, and finally a single stitched high-resolution tile image is output.

[0034] The present invention also proposes a large-format tile imaging system based on multi-line scan camera image stitching, comprising:

[0035] A tile optical detection platform, the tile optical detection platform comprising a conveying device for conveying target tiles via a conveyor belt;

[0036] An imaging system, the imaging system comprising a light source and a plurality of cameras, the imaging system being used to acquire a target tile image, and the plurality of cameras being installed in parallel on the same straight line, and the fields of view between every two cameras having an overlapping portion;

[0037] A calibration module, used for calibrating a plurality of cameras in the imaging system to obtain a homography matrix adapted to the hardware of the imaging system after the imaging system is determined;

[0038] A complexity calculation module, used to determine the type of the target tile, and determine the complexity of the target tile texture according to the determined type;

[0039] The image stitching module is used to obtain the target tile image. If the complexity of the target tile texture is a simple texture tile, the target tile image is stitched using a simple texture image. If the complexity of the target tile texture is a complex texture tile, the target tile image is stitched using a complex texture image.

[0040] Furthermore, the system also includes:

[0041] A pre-established database is established based on tile types and tile images in the interaction area corresponding to two cameras. When the user selects a tile type as the type of the target tile, the tile image in the interaction area corresponding to the two cameras pre-stored in the database is returned.

[0042] The present invention also proposes a computer-readable storage medium, which stores a computer program, and is characterized in that when the computer program is executed by a processor, it implements the steps of any of the above-mentioned large-format tile imaging methods based on multi-line scan camera image stitching.

[0043] The beneficial effects of the present invention are:

[0044] The present invention is suitable for the scene of fusion of tile imaging of multiple line scan cameras by judging the texture of the tile, and executing the corresponding algorithms according to the judgment results to perform multi-image stitching, a simple texture tile image stitching method and a complex texture tile image stitching fast method, so as to fuse the tile images taken by multiple low-resolution cameras into a complete high-resolution tile. Under the condition of the same field of view, it is more economical to use multiple low-resolution cameras to shoot the same format than to use a single higher-resolution camera to shoot. Simple texture tiles are corrected by the corrected homography matrix to achieve basic large-format image stitching. Real-time image stitching performed in the case of complex texture tiles can alleviate the impact of changes in homography matrix parameters caused by vibration during the operation of the tile imaging platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and other features of the present disclosure will become more apparent by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0046] Figure 1 It is a flow chart of a large-format ceramic tile imaging method based on multi-line scan camera image stitching of the present invention;

[0047] Figure 2 It is a schematic diagram of a simple texture tile image stitching method of a large-format tile imaging method based on multi-line scan camera image stitching of the present invention;

[0048] Figure 3 The figure shows a principle diagram of a complex texture tile image stitching method of a large-format tile imaging method based on multi-line scan camera image stitching of the present invention;

[0049] Figure 4 Shown is a schematic diagram of the layout of a tile optical detection platform of a large-format tile imaging system based on multi-line scan camera image stitching of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0051] Reference Figure 1 , Figure 2 as well as Figure 3 , Embodiment 1, the present invention proposes a large-format tile imaging method based on multi-line scan camera image stitching, comprising the following:

[0052] After the imaging system is determined, multiple cameras in the imaging system are calibrated to obtain a homography matrix adapted to the hardware of the imaging system;

[0053] Determine the type of the target tile, and determine the complexity of the texture of the target tile according to the determined type;

[0054] Get the target tile image. If the complexity of the target tile texture is simple texture tile, use simple texture image stitching for the target tile image. If the complexity of the target tile texture is complex texture tile, use complex texture image stitching for the target tile image.

[0055] The solution in this embodiment is suitable for the scene of fusion of tile imaging of multiple line scan cameras. By judging the texture of the tile, appropriate algorithms are respectively executed to perform multi-image stitching, simple texture tile image stitching method and complex texture tile image stitching fast method, so as to fuse the tile images taken by multiple low-resolution cameras into a complete high-resolution tile. This solves the problem that a single line scan camera cannot image a larger field of view at this stage and the high cost of using a single high-resolution camera.

[0056] As a preferred embodiment of the present invention, specifically, multiple cameras in an imaging system are calibrated to obtain a homography matrix adapted to the hardware of the imaging system, including:

[0057] Obtain the result of processing the camera through the black and white calibration plate during manual calibration, and calculate the intrinsic and extrinsic parameters of the camera based on the result to obtain the intrinsic and extrinsic parameter matrix of each camera to complete the calibration;

[0058] At the intersection of the fields of view of any two cameras, the feature points of the correction plate are obtained by the SIFT algorithm to obtain multiple feature points, and the homography matrix *M between multiple cameras is calculated based on the multiple feature points 1,2 ,M 2,3 ,…M n-1,n +, where n is the number of cameras used.

[0059] In this preferred embodiment, the system calibration part is the manual calibration performed by the platform during the debugging stage. First, the camera's internal and external parameters are calculated through the black and white calibration plate to obtain the internal and external parameter matrix. After the calibration is completed, the feature points of the calibration plate are obtained by the SIFT algorithm at the intersection of the fields of view of the two cameras, and the homography matrix *M between multiple cameras is calculated. 1,2 ,M 2,3 ,…M n-1,n +, where n is the number of cameras used.

[0060] As a preferred embodiment of the present invention, specifically, determining the type of the target tile, and determining the complexity of the target tile texture according to the determined type, including:

[0061] Obtaining the tile type selected by the user from a pre-established database as the type of the target tile;

[0062] Recalling tile images in the interaction area corresponding to the two cameras pre-stored in the database;

[0063] Extracting feature points from the tile image to obtain the number of feature points and calculating the histogram variance of the tile image;

[0064] When the number of feature points is greater than a preset threshold T1 and the histogram variance is greater than T2, the target tile is determined to be a complex texture tile, otherwise the target tile is determined to be a simple texture tile.

[0065] As a preferred embodiment of the present invention, specifically, the simple texture image stitching process includes the following:

[0066] Multiple cameras jointly capture multiple tile images, assuming that there are n images;

[0067] Based on multiple tile images through the homography matrix *M 1,2 ,M 2,3 ,…M n-1,n + Stitching, output a single stitched high-resolution tile image.

[0068] As a preferred embodiment of the present invention, specifically, the complex texture image stitching process includes the following:

[0069] Multiple cameras jointly capture multiple tile images, assuming that there are n images, where I1(x, y) and I2(x, y) are two images with overlapping areas, and (x0, y0) is the translation change between the images;

[0070] Perform the following calculations,

[0071]

[0072] The cross power spectrum of two images is defined as follows:

[0073]

[0074] in is the complex conjugate of F1(u,v),

[0075] Then perform inverse Fourier transform on the above equation to get:

[0076]

[0077] Find the maximum value of G max , and obtain G max The corresponding position is the translation amount x0, y0 between the images.

[0078] Similarly, convert the (x, y) coordinates of the two images into polar coordinates (ρ, θ), and perform the above calculation process again to find the corresponding rotation coefficients.

[0079] Finally, by determining the translation amount and rotation coefficient, the image I2(x, y) can be projected to the image coordinates of I1(x, y), completing the stitching of the two images with overlapping areas.

[0080] In this way, all images with overlapping areas are stitched together, and finally a single stitched high-resolution tile image is output.

[0081] Reference Figure 4 The present invention also proposes a large-format tile imaging system based on multi-line scan camera image stitching, comprising:

[0082] A tile optical detection platform, the tile optical detection platform comprising a conveying device for conveying target tiles via a conveyor belt;

[0083] An imaging system, the imaging system comprising a light source and a plurality of cameras, the imaging system being used to acquire a target tile image, and the plurality of cameras being installed in parallel on the same straight line, and the fields of view between every two cameras having an overlapping portion;

[0084] A calibration module, used for calibrating a plurality of cameras in the imaging system to obtain a homography matrix adapted to the hardware of the imaging system after the imaging system is determined;

[0085] A complexity calculation module, used to determine the type of the target tile, and determine the complexity of the target tile texture according to the determined type;

[0086] The image stitching module is used to obtain the target tile image. If the complexity of the target tile texture is a simple texture tile, the target tile image is stitched using a simple texture image. If the complexity of the target tile texture is a complex texture tile, the target tile image is stitched using a complex texture image.

[0087] In this embodiment, the shooting position of the line scan camera and the irradiation position of the light source are approximately coincident with the position of the dotted line in the center of the figure. Taking the first camera as an example, the field of view of camera 1 is L1, and the interactive field of view of camera 1 and camera 2 is L 1,2 When the tile reaches the red dotted line position, n cameras jointly shoot to obtain n tile images.

[0088] As a preferred embodiment of the present invention, the system further includes:

[0089] A pre-established database is established based on tile types and tile images in the interaction area corresponding to two cameras. When the user selects a tile type as the type of the target tile, the tile image in the interaction area corresponding to the two cameras pre-stored in the database is returned.

[0090] The present invention also proposes a computer-readable storage medium, which stores a computer program, and is characterized in that when the computer program is executed by a processor, it implements the steps of any of the above-mentioned large-format tile imaging methods based on multi-line scan camera image stitching.

[0091] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

[0092] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.

[0093] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0094] Although the description of the present invention has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively covering the intended scope of the present invention. In addition, the above description of the present invention is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen may still represent equivalent changes to the present invention.

[0095] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.

Claims

1. A large-format tile imaging method based on multi-line scan camera image stitching, characterized in that: Includes the following: After the imaging system is determined, multiple cameras in the imaging system are calibrated to obtain a homography matrix adapted to the hardware of the imaging system; Determine the type of the target tile, and determine the complexity of the texture of the target tile according to the determined type; Obtain a target tile image. If the complexity of the target tile texture is a simple texture tile, simple texture image stitching is used for the target tile image. If the complexity of the target tile texture is a complex texture tile, complex texture image stitching is used for the target tile image. Specifically, multiple cameras in the imaging system are calibrated to obtain a homography matrix adapted to the hardware of the imaging system, including: Obtain the result of processing the camera through the black and white calibration plate during manual calibration, and calculate the intrinsic and extrinsic parameters of the camera based on the result to obtain the intrinsic and extrinsic parameter matrix of each camera to complete the calibration; At the intersection of the fields of view of any two cameras, the feature points of the correction plate are obtained by the SIFT algorithm to obtain multiple feature points, and the homography matrix between multiple cameras is calculated based on the multiple feature points. , where n is the number of cameras used; Specifically, the simple texture image stitching process includes the following: Multiple cameras jointly capture multiple tile images, assuming that there are n images; Based on multiple tile images through the homography matrix Perform stitching and output a single stitched high-resolution tile image; Specifically, the complex texture image stitching process includes the following: Multiple cameras take multiple tile images together, assuming there are n images, where , are two images with overlapping areas, is the translation change between images; Perform the following calculations, ； The cross power spectrum of two images is defined as follows: in yes The complex conjugate of Then perform inverse Fourier transform on the above equation to get: ； Find the maximum value of G , and obtain The corresponding position is the translation between the images , Similarly, the two images Convert coordinates to polar coordinates , execute the above calculation process again to find the corresponding rotation coefficient, Finally, by determining the translation amount and rotation coefficient, the image can be Project to The image coordinates of the two images with overlapping areas are stitched together. In this way, all images with overlapping areas are stitched together, and finally a single stitched high-resolution tile image is output.

2. The large-format ceramic tile imaging method based on multi-line scan camera image stitching according to claim 1 is characterized in that: Specifically, the type of the target tile is determined, and the complexity of the target tile texture is determined according to the determined type, including: Obtaining the tile type selected by the user from a pre-established database as the type of the target tile; Recalling tile images in the interaction area corresponding to the two cameras pre-stored in the database; Extracting feature points from the tile image to obtain the number of feature points and calculating the histogram variance of the tile image; When the number of feature points is greater than a preset threshold T1 and the histogram variance is greater than T2, the target tile is determined to be a complex texture tile, otherwise the target tile is determined to be a simple texture tile.

3. A large-format tile imaging system based on multi-line scan camera image stitching, characterized in that: The steps of applying the method described in any one of claims 1 to 2 above include: A tile optical detection platform, the tile optical detection platform comprising a conveying device for conveying target tiles via a conveyor belt; An imaging system, the imaging system comprising a light source and a plurality of cameras, the imaging system being used to acquire a target tile image, and the plurality of cameras being installed in parallel on the same straight line, and the fields of view between every two cameras having an overlapping portion; A calibration module, used for calibrating a plurality of cameras in the imaging system to obtain a homography matrix adapted to the hardware of the imaging system after the imaging system is determined; A complexity calculation module, used to determine the type of the target tile, and determine the complexity of the target tile texture according to the determined type; The image stitching module is used to obtain the target tile image. If the complexity of the target tile texture is a simple texture tile, the target tile image is stitched using a simple texture image. If the complexity of the target tile texture is a complex texture tile, the target tile image is stitched using a complex texture image.

4. The large-format tile imaging system based on multi-line scan camera image stitching according to claim 3 is characterized in that: The system further comprises, A pre-established database is established based on tile types and tile images in the interaction area corresponding to two cameras. When the user selects a tile type as the type of the target tile, the tile image in the interaction area corresponding to the two cameras pre-stored in the database is returned.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images