Real-time feedback image stitching system and method based on active optical beacon

Abstract

This invention discloses a real-time feedback image stitching system and method based on active optical beacons, belonging to the field of image stitching and real-time calibration correction technology. It includes an imaging acquisition module, an active optical beacon module, a workpiece conveying module, and an image processing and closed-loop control module. By adding active optical beacons to form reference light spot markers to replace natural features, the imaging acquisition module and the image processing and closed-loop control module perform real-time detection and closed-loop feedback dynamic compensation for system offset. This fundamentally solves the technical problem of achieving feature matching and accurate stitching of large workpieces with no or weak texture in harsh imaging environments. It effectively resists camera thermal drift, conveyor belt vibration, and frame micro-deformation, ensuring stable stitching accuracy over long periods. The real-time feedback image stitching system and method based on active optical beacons of this invention have low production line modification and deployment costs, simple matrix operation logic, fast execution speed, and are adaptable to various conveyor belt industrial production lines.

Description

Technical Field

[0001] This invention belongs to the field of image stitching and real-time calibration and correction technology, specifically involving a real-time feedback image stitching system and method based on active optical beacons. It is applicable to panoramic imaging stitching and accuracy compensation scenarios for large workpieces (such as metal plates, large shells, precision castings, and sheet metal parts) with no texture or weak texture in industrial production line single-camera combined with conveyor belt moving mode. Background Technology

[0002] When performing panoramic visual inspection on large workpieces on industrial production lines, the limited field of view of a single camera typically necessitates a fixed single camera and a conveyor belt to transport the workpiece. Multiple local images are acquired at different workstations and then stitched together to generate a complete panoramic image. This approach is low-cost and flexible in deployment, making it the mainstream solution for inspecting large-size workpieces. However, this solution suffers from two major technical drawbacks in practical applications: First, the workpieces to be inspected are often made of textureless materials such as metal and plastic, lacking natural feature points on their surfaces. This makes it impossible to accurately stitch sequential images using traditional feature matching algorithms such as SIFT, SURF, and ORB, as conventional stitching algorithms are prone to misalignment and ghosting issues. Second, traditional image stitching relies on offline calibration using a checkerboard pattern to obtain the initial transformation matrix. The mechanical vibrations of the conveyor belt during long-term operation, camera lens thermal drift, and slight deformation of the frame can cause the calibration parameters to continuously shift, leading to the rapid invalidation of the initial transformation matrix. Furthermore, offline calibration requires downtime, which cannot meet the real-time correction requirements of continuous production lines and makes it difficult to guarantee long-term inspection accuracy.

[0003] In existing technologies, correction schemes for splicing drift mostly rely on repeatedly setting up calibration boards, laser trackers, and other equipment, which are not only costly and time-consuming due to downtime, but also cannot achieve closed-loop real-time compensation. Simple spot positioning technology is only used for static single-point positioning and does not combine with the sequential splicing scenario of conveyor belt moving parts to build a real-time correction mechanism for the transformation matrix, thus failing to solve the problem of long-term stable and high-precision splicing of textureless workpieces. Summary of the Invention

[0004] In response to one or more of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a real-time feedback image stitching system and method based on active optical beacons. It forms a reference spot mark by deploying fixed active optical beacons, compares the centroid offset of the spot in real time, and dynamically corrects the initial calibration matrix in a closed loop, so as to achieve non-stop, high-precision, long-term stable panoramic stitching of textureless workpieces.

[0005] To achieve the above objectives, one aspect of the present invention provides a real-time feedback image stitching system based on active optical beacons, comprising: An active optical beacon module is used to project an optical beacon spot onto the data acquisition station; The workpiece conveying module is used to move the workpiece and sequentially send different local areas of the workpiece into the acquisition station. The imaging acquisition module is used to acquire images from the acquisition station, including images of local areas of the workpiece and light spot images. The image processing and closed-loop control module is communicatively connected to the imaging acquisition module. After each local area of ​​the workpiece is sent to the acquisition station, the imaging acquisition module sequentially acquires images of the acquisition station and transmits them to the image processing and closed-loop control module in real time. The image processing and closed-loop control module extracts the real-time centroid coordinates of the light spot in each frame of the image in real time, calculates the real-time compensation matrix based on the deviation between the real-time centroid coordinates and the initial reference coordinates, and uses the real-time compensation matrix to correct the initial homography transformation matrix of the local image of that frame, thereby obtaining the real-time stitching matrix. The real-time stitching matrix of each local area image is then used to stitch the images together to obtain a stitched panoramic image of the workpiece.

[0006] As a further improvement of the present invention, a reference background plate is provided below the workpiece conveying module. The reference background plate is perpendicular to the optical axis of the camera of the imaging acquisition module. When the workpiece has an irregular shape, the active optical beacon module projects a light spot onto the reference background plate to provide a stable coordinate reference.

[0007] As a further improvement of the present invention, the workpiece conveying module includes a conveyor belt, a drive motor, and a station positioning sensor; the conveyor belt is driven by the drive motor to convey the workpiece at a uniform speed, and the workpiece positioning sensor is used to detect the position of the workpiece and trigger the imaging acquisition module to acquire images at the corresponding acquisition station after the local area of ​​the workpiece is in place.

[0008] As a further improvement of the present invention, the imaging acquisition module includes a camera, and the active optical beacon module includes multiple fixed-point light sources and a filter component; the multiple fixed-point light sources are arranged circumferentially around the outer periphery of the camera, and the optical axis of the fixed-point light sources is parallel to the optical axis of the camera; the filter component is coaxially arranged with the optical axis of the fixed-point light sources and is used to filter ambient light interference in the non-beacon band of the light sources.

[0009] As a further improvement of the present invention, the image processing and closed-loop control module uses the least squares method to fit the affine transformation relationship between the real-time centroid coordinates and the initial spot reference coordinates when calculating the real-time compensation matrix.

[0010] As a further improvement of the present invention, the image processing and closed-loop control module uses a gray-scale centroid method combined with a Gaussian surface fitting algorithm to perform sub-pixel-level centroid positioning of the light spot when extracting the real-time centroid coordinates of the light spot.

[0011] As a further improvement of the present invention, when the image processing and closed-loop control module stitches local images based on the real-time stitching matrix, it adopts a gradient-in and gradient-out weighted seamless fusion algorithm to perform image fusion processing.

[0012] As a further improvement of the present invention, an alarm module is also included. The alarm module is communicatively connected to the image processing and closed-loop control module. When the offset of the real-time centroid coordinates of the light spot exceeds the matrix compensation range, the image processing and closed-loop control module triggers the alarm module to issue an alarm.

[0013] Another aspect of the present invention provides a real-time feedback image stitching method based on active optical beacons, used in the aforementioned real-time feedback image stitching system based on active optical beacons, comprising the following steps: S1: Obtain the initial homography transformation matrix corresponding to each local region of the workpiece, and obtain the initial centroid coordinate set of the light spot corresponding to each local region; S2: The workpiece conveying module drives the workpiece to move horizontally, and the imaging acquisition module acquires images from the acquisition station. Each frame of the image simultaneously acquires images of the local area of ​​the workpiece and the optical beacon spot image. S3: Extract the real-time centroid coordinates of the current frame spot, compare the real-time centroid coordinate set with the initial centroid coordinate set point by point, and calculate the spot coordinate offset. S4: Solve the real-time compensation matrix based on the spot coordinate offset, and perform matrix multiplication between the real-time compensation matrix and the initial transformation matrix to obtain the optimized real-time stitching matrix. S5: Repeat steps S2 to S5 until the real-time stitching matrix corresponding to all local area images of the workpiece is obtained; use the real-time stitching matrix of each local area image to stitch the images and output the stitched panoramic image of the workpiece.

[0014] As a further improvement of the present invention, in step S3, a coordinate deviation threshold is preset for the offset of the light spot coordinates. When the offset of the light spot is within the threshold, step S5 is executed directly; when the offset exceeds the threshold, steps S4 and S5 are executed sequentially.

[0015] The aforementioned improved technical features can be combined with each other as long as they do not conflict with each other.

[0016] In summary, the beneficial effects of the above-described technical solutions conceived by this invention compared with the prior art include: (1) The real-time feedback image stitching system and method based on active optical beacons of the present invention, by adding active optical beacons to form reference spot marks to replace natural features, and by using the imaging acquisition module and the image processing and closed-loop control module to perform real-time detection and closed-loop feedback dynamic compensation of system offset, fundamentally solves the technical problem that it is difficult to achieve feature matching and accurate stitching of large workpieces with no texture or weak texture in harsh imaging environments, effectively resists camera thermal drift, conveyor belt vibration and frame micro deformation, and maintains stable stitching accuracy over a long period of time.

[0017] (2) The real-time feedback image stitching system and method based on active optical beacons of the present invention only requires a single camera and multiple light sources, without the need for high-priced equipment such as multi-camera groups and laser trackers, which greatly reduces the cost of production line transformation and deployment; and the matrix operation logic is simple, the execution speed is fast, and it is fully compatible with various conveyor belt industrial production lines. There is no need to modify the original production line structure, the installation and debugging are simple, and it can be quickly deployed and applied without affecting production efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a simplified structural diagram of the real-time feedback image stitching system based on active optical beacons in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the real-time feedback image stitching method based on active optical beacons in an embodiment of the present invention. In all the accompanying drawings, the same reference numerals denote the same technical features, specifically: 1. Imaging acquisition module; 2. Active optical beacon module; 3. Image processing and closed-loop control module; 4. Workpiece conveying module; 5. Workpiece; 501. Local region one; 502. Local region two; 503. Local region three; 504. Local region four. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0022] Example: Please see Figures 1-2 The real-time feedback image stitching system based on active optical beacons in a preferred embodiment of the present invention includes an imaging acquisition module 1, an active optical beacon module 2, a workpiece conveying module 4, and an image processing and closed-loop control module 3. The active optical beacon module 2 projects an optical beacon spot onto the acquisition station, the workpiece conveying module 4 moves the workpiece, and each local area of ​​the workpiece is sequentially sent to the acquisition station. The imaging acquisition module acquires the image of the acquisition station, and the image processing and closed-loop control module 3 compensates and stitches the images of each local area of ​​the workpiece 5 to obtain a stitched panoramic image of the workpiece 5.

[0023] Specifically, in the preferred embodiment, the active optical beacon module 2 includes multiple fixed-point light sources, which act as active optical beacons to project light spots onto the acquisition station, so that the local area image of each workpiece acquired by the imaging acquisition module 1 contains light spots, thereby forming a fixed reference mark array that is not affected by the texture of the workpiece 5.

[0024] Preferably, the active optical beacon module 2 further includes a filter assembly, which is coaxially arranged with the optical axis of the fixed-point light source to filter ambient light interference in the non-beacon band of the active optical beacon. Preferably, the fixed-point light source is an infrared narrowband LED and is used in conjunction with a narrowband filter.

[0025] Furthermore, in the preferred embodiment, the workpiece conveying module 4 is used to move the workpiece 5 so that each local area of ​​the workpiece 5 is sequentially sent into the acquisition station through the movement of the workpiece 5, and the local images of the workpiece 5 are acquired in an orderly manner in conjunction with the camera.

[0026] Specifically, the workpiece conveying module 4 includes a conveyor belt, a drive motor, and a station positioning sensor. The conveyor belt is driven by the drive motor to convey the workpiece 5 at a constant speed. The workpiece positioning sensor is used to detect the position of the workpiece 5 and triggers the imaging acquisition module 1 to acquire images of the acquisition station after the local area of ​​the workpiece is in place.

[0027] Furthermore, in the preferred embodiment, the imaging acquisition module 1 includes a camera rigidly fixed to the frame, the frame being located above the conveyor belt, and the camera preferably being a high-definition industrial camera. The camera's field of view serves as the acquisition station, capable of partially covering the surface of the workpiece 5, so that during the conveying process of the workpiece 5, multiple local area images of the workpiece are sequentially acquired, and the multiple local area images are stitched together to completely cover the panoramic area of ​​the workpiece 5.

[0028] In a specific embodiment of the present invention, four fixed-point light sources are uniformly arranged in the circumferential direction at the front end of the camera lens. The optical axes of the four fixed-point light sources are parallel to the optical axis of the camera, and the light spots are constantly projected onto the non-detection area of ​​the workpiece 5 surface.

[0029] Preferably, a reference background plate is provided below the workpiece conveying module 4. The reference background plate is set perpendicular to the optical axis of the camera so that when the workpiece 5 has an irregular shape or a non-planar surface, the light spot is projected onto the background plate, so that the active optical beacon is not affected by the characteristics of the workpiece 5, thereby providing a stable coordinate reference. The reference background plate is preferably hard, light-colored, and has a flat, matte surface.

[0030] Preferably, the imaging acquisition module 1 is configured to take into account the uniform translation feature of the conveyor belt and set the camera acquisition delay parameters to eliminate the influence of the motion trail of the workpiece 5 on the light spot imaging and ensure that the light spot imaging at each station is clear and distortion-free.

[0031] like Figure 1 As shown, the camera is fixed above the conveyor belt with a fixed field of view, serving as the acquisition station. A large workpiece 5 is placed on the conveyor belt and divided into four local areas according to the conveyor belt's transport direction: local area 1 501, local area 2 502, local area 3 503, and local area 4 504. The images of workpiece 5 in adjacent local areas overlap. When each local area moves directly below the camera's field of view, the workpiece positioning sensor triggers the camera to acquire images. By stitching together the local images of workpiece 5 in the four local areas, a panoramic image of workpiece 5 is obtained.

[0032] Furthermore, in the preferred embodiment, the image processing and closed-loop control module 3 is communicatively connected to the imaging acquisition module 1; after each local area of ​​the workpiece 5 is sent to the acquisition station, the imaging acquisition module 1 sequentially acquires images of the acquisition station and transmits the images to the image processing and closed-loop control module 3 in real time; the image processing and closed-loop control module 3 extracts the real-time centroid coordinates of the light spot in each frame of the image in real time, and calculates the real-time compensation matrix based on the deviation between the real-time centroid coordinates and the initial reference coordinates, and uses the real-time compensation matrix to correct the initial homography transformation matrix of the local image of the frame, thereby obtaining the real-time stitching matrix, and uses the real-time stitching matrix of each local area image to perform image stitching, and outputs the stitched panoramic image of the workpiece.

[0033] Preferably, when extracting the real-time centroid coordinates of the light spot, the image processing and closed-loop control module 3 uses the gray-scale centroid method combined with the Gaussian surface fitting algorithm to perform sub-pixel-level centroid positioning of the light spot, with a positioning error ≤0.1 pixels, providing high-precision data support for solving the real-time compensation matrix.

[0034] Preferably, the image processing and closed-loop control module 3 adopts beacon spot anti-interference optimization for the corresponding image spot. Specifically, it removes the interference of workpiece 5 reflection, dust and environmental stray light through adaptive threshold separation and morphological opening operation, and combines narrowband filter to filter background light to ensure that the spot outline is complete and clear and improve the reliability of centroid extraction.

[0035] Preferably, the image processing and closed-loop control module 3 uses the least squares method to fit the affine transformation relationship between the real-time centroid coordinates and the initial spot reference coordinates to obtain the real-time compensation matrix.

[0036] Preferably, the image processing and closed-loop control module 3 has a preset coordinate deviation threshold for the deviation of the light spot coordinates. The real-time stitching matrix update and correction operation is only performed when the offset between the real-time centroid coordinates of the light spot and the initial reference coordinates exceeds the threshold, so as to avoid erroneous corrections caused by environmental noise and minor jitter, and ensure the stability of system operation.

[0037] Preferably, when the image processing and closed-loop control module 3 stitches local images based on the real-time stitching matrix, it adopts a gradient-in / gradual-out weighted seamless fusion algorithm to perform fusion processing on the images, so as to eliminate image stitching line marks and improve the visual consistency and flatness of the panoramic image.

[0038] Preferably, it also includes an alarm module, which is communicatively connected to the image processing and closed-loop control module 3. When the offset of the real-time centroid of the light spot exceeds the matrix compensation range, the image processing and closed-loop control module 3 triggers an alarm to indicate equipment abnormalities such as frame deformation and camera loosening, thereby avoiding invalid stitching and false detection.

[0039] Furthermore, the present invention also provides a real-time feedback image stitching method based on active optical beacons, for use in the aforementioned real-time feedback image stitching system based on active optical beacons, such as... Figure 2 As shown, the steps include: S1: Offline initial calibration: The checkerboard calibration method is used to perform offline calibration on multiple acquisition stations to obtain the initial homography transformation matrix corresponding to each acquisition station and complete the coarse stitching of the sequence images; Optical beacon spot images are acquired simultaneously, and the initial centroid coordinate set of the spot corresponding to each station is obtained through the sub-pixel centroid extraction algorithm. S2: Online Sequence Image Acquisition: During normal operation of the production line, the workpiece conveying module drives the workpiece to move at a constant speed, and the imaging acquisition module acquires images of the acquisition station. Each frame of the image synchronously acquires the local area image of the workpiece and the optical beacon spot. S3: Spot coordinate offset comparison: Extract the real-time centroid coordinates of the spot in the current frame in real time, compare the real-time centroid coordinate set with the initial centroid coordinate set point by point, and calculate the spot coordinate offset. S4: Real-time compensation matrix solution: Based on the spot coordinate offset, the affine transformation relationship is fitted using the least squares method to solve the real-time compensation matrix that characterizes camera drift and conveyor belt jitter error; the real-time compensation matrix is ​​multiplied by the initial homography transformation matrix to obtain the optimized real-time stitching matrix; S5: Closed-loop iterative correction and stitching: Repeat steps S2 to S5 according to the set acquisition frame rate or workstation switching frequency until the real-time stitching matrix corresponding to all local area images of the workpiece is obtained; replace the initial homography transformation matrix of each local area image with the corresponding real-time stitching matrix, use the real-time stitching matrix of each local area image to perform image stitching, output the stitched panoramic image of the workpiece, completely eliminate the stitching error caused by working condition drift, realize real-time closed-loop compensation without stopping the machine, and ensure the long-term stability of the panoramic stitching accuracy based on matrix transformation.

[0040] Preferably, such as Figure 2 As shown in the figure, in step S3, the corresponding spot coordinate offset is preset with a coordinate deviation threshold. When the offset is within the threshold, step S5 is executed directly. Only when the offset exceeds the threshold, steps S4 and S5 are executed sequentially.

[0041] In a specific embodiment of the present invention, taking conveyor belt panoramic inspection of large, textureless automotive sheet metal workpieces as an example, the technical solution is described in detail: The imaging acquisition module 1 uses a global shutter industrial camera with a resolution of 1280×1024 and a frame rate of 30fps; the lens focal length is 16mm and the working distance is 500mm. The active optical beacon module is equipped with four infrared LED light sources (850nm) coaxially fixed to the front of the camera lens, with a spot diameter of approximately 5-8 pixels, and is fitted with a narrow-band filter to suppress ambient light and workpiece reflections. The image processing and closed-loop control module uses an industrial computer as its carrier. Spot centroid extraction employs the sub-pixel grayscale centroid method and Gaussian surface fitting; the transformation model uses homography; and the real-time compensation frequency is one image update per workstation. The workpiece conveying module has a conveying speed of 100mm / s and an effective travel of 1200mm; the workpiece is divided into four continuous local acquisition areas along the conveying direction, each area covering a workpiece length of 300mm; the images from the four workstations are stitched together to cover the entire 1200mm length.

[0042] S1: Offline initial calibration; S11: Initial homography transformation matrix H0 calibration; The single camera was calibrated offline at four acquisition stations using a checkerboard calibration board, yielding the initial homography transformation matrix H0 for each station. Taking local region 501 as an example, the initial matrix H0... 01 for:

[0043] The calculations for the other three local regions are similar, yielding their respective H values. 02 H 03 H 04 .

[0044] S12: Acquisition of optical beacon reference coordinates; After calibration, images of four LED spots were acquired, and the reference coordinate set P0 was obtained using sub-pixel centroid extraction. Taking local region 501 as an example, the reference centroid coordinates of the four spots are:

[0045] The above coordinates are stored in the system, and the calculation of the other three local areas is done in the same way, forming a long-term reference coordinate library.

[0046] S2: Online sequential image acquisition; During production line operation, the conveyor belt moves the workpiece at a constant speed of 100mm / s, and the camera sequentially captures images of the acquisition station. Each frame of the image contains 4 light spots.

[0047] S3: Comparison of light spot coordinate offset; S31: Taking a certain frame of local region 501 as an example, the real-time spot coordinates P1 are extracted:

[0048] S32: Calculate coordinate offset; Using P0 as the reference, calculate the offset of each light spot:

[0049] Based on the data above, it can be seen that the overall pattern is translation, slight rotation, and scaling, which is consistent with the typical error patterns of camera thermal drift and conveyor belt jitter. The calculations for the other three local areas are similar.

[0050] S4: Solving for the real-time compensation matrix H1; Using 4 pairs of points P0 P1, the real-time compensation homography matrix H is solved using the least squares method. 11 In this example, the error mainly consists of overall translation and slight scaling. The real-time compensation matrix is ​​obtained by solving for:

[0051] The physical meaning of the above matrix is ​​an overall offset of approximately 1.29 pixels in the x-direction, an overall offset of approximately 0.94 pixels in the y-direction, a scaling factor of approximately 1.0005, and essentially no deformation. The calculations for the other three local regions are similar, yielding their respective H values. 12 H 13 H 14 .

[0052] S5: Closed-loop iterative correction and splicing; The initial matrix H 01 With compensation matrix H 11 Multiply, substitute the values, and after matrix multiplication, we obtain the optimized real-time concatenation matrix H1:

[0053] This matrix H1 is directly used for panoramic stitching of the current frame image, replacing the original H. 01 This completes one closed-loop compensation. The calculations for the remaining three local regions are performed similarly, yielding their respective H2, H3, and H4 values.

[0054] The images of the four local regions were stitched together using the corrected matrices H1, H2, H3, and H4, and the stitching lines were eliminated by a weighted fusion method with fade-in and fade-out to obtain a stitched image after closed-loop compensation.

[0055] This embodiment can run continuously for a long time. Compared with traditional methods that only perform initial checkerboard calibration, the stitching drift can reach 2 to 5 pixels. However, the present invention provides real-time feedback compensation, and the stitching error prediction can be stably ≤0.2 pixels, corresponding to a physical accuracy of ≤0.08mm, which fully meets the high-precision visual inspection requirements of automotive sheet metal, large plates, precision housings, etc.

[0056] The present invention relates to a real-time feedback image stitching system and method based on active optical beacons. Through an active optical beacon module, LED active optical beacons are fixedly arranged on a ring bracket to form a reference light spot mark. The centroid offset of the light spot is compared in real time, and the initial calibration matrix is ​​dynamically corrected in a closed loop, so as to achieve non-stop, high-precision, long-term stable panoramic stitching of textureless workpieces.

[0057] This invention uses active optical beacons to replace natural image features and dynamically compensates for system offset through real-time detection and closed-loop feedback. It fundamentally solves the technical problem of feature matching and accurate stitching of large workpieces with no or weak texture in harsh imaging environments. The method has extremely low hardware cost and lightweight and efficient algorithm, and does not require repeated calibration. It has broad industrial application prospects in aerospace, large component manufacturing and other fields.

[0058] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0059] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0060] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0061] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0062] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0063] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0064] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0065] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0066] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0067] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A real-time feedback image stitching system based on active optical beacons, characterized in that, include: An active optical beacon module is used to project an optical beacon spot onto the data acquisition station; The workpiece conveying module is used to move the workpiece and sequentially send different local areas of the workpiece into the acquisition station. The imaging acquisition module is used to acquire images from the acquisition station, including images of local areas of the workpiece and light spot images. An image processing and closed-loop control module is communicatively connected to the imaging acquisition module. After each local area of ​​the workpiece is sent to the acquisition station, the imaging acquisition module sequentially acquires images from the acquisition station and transmits the images to the image processing and closed-loop control module in real time. The image processing and closed-loop control module extracts the real-time centroid coordinates of the light spot in each frame of the image in real time, calculates a real-time compensation matrix based on the deviation between the real-time centroid coordinates and the initial reference coordinates, and uses the real-time compensation matrix to correct the initial homography transformation matrix of the local image of that frame, thereby obtaining a real-time stitching matrix. The real-time stitching matrix of each local area image is then used to stitch the images together and output a stitched panoramic image of the workpiece.

2. The real-time feedback image stitching system based on active optical beacons according to claim 1, characterized in that, A reference background plate is provided below the workpiece conveying module. The reference background plate is perpendicular to the optical axis of the camera of the imaging acquisition module. When the workpiece has an irregular shape, the active optical beacon module projects a light spot onto the reference background plate to provide a stable coordinate reference.

3. The real-time feedback image stitching system based on active optical beacons according to claim 1, characterized in that, The workpiece conveying module includes a conveyor belt, a drive motor, and a workstation positioning sensor. The conveyor belt is driven by the drive motor to convey the workpiece at a constant speed. The workpiece positioning sensor is used to detect the position of the workpiece and triggers the imaging acquisition module to acquire images of the acquisition station after the local area of ​​the workpiece is in place.

4. The real-time feedback image stitching system based on active optical beacons according to claim 1, characterized in that, The imaging acquisition module includes a camera, and the active optical beacon module includes multiple fixed-point light sources and a filter assembly. The multiple fixed-point light sources are arranged circumferentially around the outer periphery of the camera, and the optical axes of the fixed-point light sources are parallel to the optical axis of the camera. The filter assembly is coaxially arranged with the optical axis of the fixed-point light sources and is used to filter ambient light interference in the non-beacon band of the light sources.

5. The real-time feedback image stitching system based on active optical beacons according to any one of claims 1 to 4, characterized in that, When calculating the real-time compensation matrix, the image processing and closed-loop control module uses the least squares method to fit the affine transformation relationship between the real-time centroid coordinates and the initial spot reference coordinates.

6. The real-time feedback image stitching system based on active optical beacons according to any one of claims 1 to 4, characterized in that, When extracting the real-time centroid coordinates of the light spot, the image processing and closed-loop control module uses a gray-scale centroid method combined with a Gaussian surface fitting algorithm to perform sub-pixel-level centroid positioning of the light spot.

7. The real-time feedback image stitching system based on active optical beacons according to any one of claims 1 to 4, characterized in that, When the image processing and closed-loop control module stitches local images based on the real-time stitching matrix, it uses a fade-in / fade-out weighted seamless fusion algorithm to perform image fusion processing.

8. The real-time feedback image stitching system based on active optical beacons according to any one of claims 1 to 4, characterized in that, It also includes an alarm module, which is communicatively connected to the image processing and closed-loop control module. When the offset of the real-time centroid coordinates of the light spot exceeds the matrix compensation range, the image processing and closed-loop control module triggers an alarm from the alarm module.

9. A real-time feedback image stitching method based on active optical beacons, used in the real-time feedback image stitching system based on active optical beacons according to any one of claims 1 to 8, characterized in that, Includes the following steps: S1: Obtain the initial homography transformation matrix corresponding to each local region of the workpiece, and obtain the initial centroid coordinate set of the light spot corresponding to each local region; S2: The workpiece conveying module drives the workpiece to move horizontally, and the imaging acquisition module acquires images from the acquisition station. Each frame of the image simultaneously acquires images of the local area of ​​the workpiece and the optical beacon spot image. S3: Extract the real-time centroid coordinates of the current frame spot, compare the real-time centroid coordinate set with the initial centroid coordinate set point by point, and calculate the spot coordinate offset. S4: Solve the real-time compensation matrix based on the spot coordinate offset, and perform matrix multiplication between the real-time compensation matrix and the initial transformation matrix to obtain the optimized real-time stitching matrix. S5: Repeat steps S2 to S5 until the real-time stitching matrix corresponding to all local area images of the workpiece is obtained; use the real-time stitching matrix of each local area image to stitch the images and output the stitched panoramic image of the workpiece.

10. The real-time feedback image stitching method based on active optical beacons according to claim 9, characterized in that, In step S3, a coordinate deviation threshold is preset for the offset of the light spot coordinates. When the offset of the light spot is within the threshold, step S5 is executed directly; when the offset exceeds the threshold, steps S4 and S5 are executed sequentially.

Images