A stitching imaging fast calibration method and system

Abstract

This invention relates to the field of image calibration technology, and more particularly to a fast calibration method and system for stitched imaging. This method constructs two features—a perturbation coupling index and a tracking stability index—to characterize the image from two dimensions: external perturbation input and internal feature stability, respectively. This solves the problem of insufficient robustness caused by existing methods relying solely on corner / depth models. Simultaneously, this method ensures comparability and stability across different scenarios; avoids the problem of scattered and difficult-to-unify judgments among multiple indicators in traditional methods; and improves the accuracy of calibration reliability assessment.

Description

Technical Field

[0001] This invention relates to the field of image calibration technology, and in particular to a rapid calibration method and system for stitched imaging. Background Technology

[0002] In scenarios such as urban emergency management centers, rail transit control halls, and smart mine monitoring rooms, there are numerous ultra-large-sized splicing display terminals (such as LCD, DLP, and LED), which consist of multiple display units spliced ​​together to form a large screen. These splicing screens need to accurately visualize real-time information from multiple sources, including cameras, radar, SCADA, and GIS systems. However, due to limitations such as camera angle, optical distortion, splicing seam deviation, and installation errors, multiple images are prone to misalignment, ghosting, breakage, and blurring in spliced ​​displays. In this scenario, there is a contradiction between the "requirement of high-order feature fitting for image geometric consistency" and the "extremely short calibration time window under environmental disturbances," making it difficult for existing calibration methods to balance accuracy, stability, and real-time performance.

[0003] The shortcomings of existing technologies are as follows: Based on the checkerboard calibration method, it cannot run online, is extremely sensitive to changes in lighting, and is time-consuming; based on fast corner matching, the point matching stability is poor, and corners drift with lighting / edges; based on deep learning geometric estimation, it lacks interpretability, has poor robustness, is highly dependent on training, and is difficult to adapt to device changes. Furthermore, existing technologies have not yet utilized the tracking stability feature. Summary of the Invention

[0004] The main objective of this invention is to provide a rapid calibration method for stitched imaging, and further to provide a rapid calibration system for stitched imaging capable of running and implementing the above method, effectively solving the problems mentioned in the background art.

[0005] The technical solution of the present invention is as follows:

[0006] Firstly, a rapid calibration method for stitched imaging is proposed, which includes the following steps:

[0007] S1. Acquire a set of image frames using an industrial camera at a set frame rate, and calculate the mean gray level and standard deviation of gray level of each image frame in the set of image frames to obtain the gray level change rate of each image frame and output the gray level statistics of each image frame.

[0008] S2. For a single image frame in the image frame set, the ORB feature extraction algorithm is applied to obtain the feature point set, the number of feature points and the feature point response intensity. The feature point sets of a single image frame and the previous image frame are extracted respectively. Feature matching is performed and the number of matching pairs is obtained. The feature point matching rate of a single image frame is calculated based on the number of matching pairs and the number of feature points in a single image frame.

[0009] S3. Based on the average gray level, gray level change rate, and feature point matching rate of a single image frame, calculate the perturbation coupling index. At the same time, based on the number of feature points and the feature point response intensity of a single image frame, calculate the tracking stability index.

[0010] S4. Based on the perturbation coupling index and the tracking stability index, calculate the image calibration stability index, and determine whether the current stitching calibration is reliable based on the image calibration stability index. If and only if the image calibration stability index is less than or equal to the calibration state boundary threshold, maintain the current stitching state; otherwise, trigger fast recalibration.

[0011] A further improvement of the present invention is that step S1 includes the following specific steps:

[0012] S11. Acquire a set of image frames using an industrial camera at a set frame rate;

[0013] S12. Perform grayscale conversion on the image frame set, extract the mean grayscale value and standard deviation of the grayscale value of a single image frame in the image frame set, and form an image frame pair with the previous image frame. Based on the mean grayscale value of the single image frame and the previous image frame, obtain the grayscale change rate of the single image frame. The mean grayscale value, standard deviation of the grayscale value, and grayscale change rate of the single image frame together constitute the grayscale statistics of the single image frame.

[0014] S13. Repeat step S12 until all individual image frames in the image frame set have been traversed.

[0015] A further improvement of the present invention is that the feature point matching rate of a single image frame in S2 is calculated as follows:

[0016] ;

[0017] in, This represents the feature point matching rate of a single image frame t. This represents the number of feature points in a single image frame t. This represents the number of matching pairs between a single image frame t and the previous image frame t-1.

[0018] A further improvement of the present invention is that the method for obtaining the number of matching pairs in S2 is as follows: extract the feature point set of a single image frame, obtain the ORB binary descriptor of each feature point in the feature point set, and simultaneously obtain the ORB binary descriptor of each feature point in the feature point set of the previous image frame; calculate the descriptor distance between the ORB binary descriptor of a single feature point in the feature point set of a single image frame and the ORB binary descriptors of all feature points in the feature point set of the previous image frame, and when the descriptor distance is less than the distance threshold, a match is successful and a matching pair is formed; traverse all feature points in the feature point set of a single image frame to obtain the number of matching pairs.

[0019] A further improvement of this invention is that the calculation formula for the disturbance coupling index in S3 is as follows:

[0020] ;

[0021] in, This represents the perturbation coupling index of a single image frame t. This represents the average grayscale value of a single image frame t. This represents the rate of change in image grayscale for a single image frame t.

[0022] A further improvement of this invention is that the formula for calculating the tracking stability index in S3 is as follows:

[0023] ;

[0024] in, This represents the tracking stability index for a single image frame t. This represents the response intensity of feature point i on a single image frame t. This represents the response intensity of feature point i on a single image frame t-1.

[0025] A further improvement of this invention is that the calculation formula for the image calibration stability index in step S4 is:

[0026] ;

[0027] in, The image calibration stability index represents a single image frame t.

[0028] Secondly, a rapid calibration system for stitched imaging is proposed, which includes: an image acquisition module, a feature extraction module, a feature calculation module, and a state division module.

[0029] The image acquisition module is used to acquire a set of image frames using an industrial camera at a set frame rate, and to calculate the mean gray level and standard deviation of the gray level of a single image frame in the set of image frames, further obtaining the gray level change rate of a single image frame, and outputting the gray level statistics of a single image frame.

[0030] The feature extraction module is used to apply the ORB feature extraction algorithm to obtain the feature point set, the number of feature points and the feature point response intensity for a single image frame in the image frame set. It extracts the feature point set of a single image frame and the previous image frame respectively, performs feature matching and obtains the number of matching pairs, and calculates the feature point matching rate of a single image frame based on the number of matching pairs and the number of feature points of a single image frame.

[0031] The feature calculation module is used to calculate the perturbation coupling index based on the average gray level, gray level change rate, and feature point matching rate of a single image frame, and to calculate the tracking stability index based on the number of feature points and the feature point response intensity of a single image frame.

[0032] The state division module is used to calculate the image calibration stability index based on the perturbation coupling index and the tracking stability index, and to determine whether the current stitching calibration is reliable based on the image calibration stability index. If and only if the image calibration stability index is less than or equal to the calibration state boundary threshold, the current stitching state is maintained; otherwise, fast recalibration is triggered.

[0033] The technical effects of this invention are as follows:

[0034] A rapid calibration method for stitched imaging is proposed. This method constructs two features—a perturbation coupling index and a tracking stability index—to characterize the image from two dimensions: external perturbation input and internal feature stability, respectively. This solves the problem of insufficient robustness caused by existing methods relying solely on corner / depth models. Simultaneously, this method ensures comparability and stability across different scenarios; avoids the problem of scattered and difficult-to-unify judgments among multiple indicators in traditional methods; and improves the accuracy of calibration reliability assessment. Attached Figure Description

[0035] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0036] Figure 1 This is a flowchart illustrating a rapid calibration method for stitched imaging according to Embodiment 1 of the present invention;

[0037] Figure 2 This is a schematic diagram of a rapid calibration system for stitched imaging according to Embodiment 2 of the present invention. Detailed Implementation

[0038] Example 1

[0039] This embodiment constructs a rapid calibration method for stitched imaging. This method features two characteristics: a perturbation coupling index and a tracking stability index. These characteristics are used to characterize the image from two dimensions: external perturbation input and internal feature stability, respectively. This solves the problem of insufficient robustness caused by existing methods relying solely on corner / depth models. Simultaneously, this method ensures comparability and stability across different scenarios; avoids the problem of scattered and difficult-to-unify judgments among multiple indicators in traditional methods; and improves the accuracy of calibration reliability assessment.

[0040] A fast calibration method for stitched imaging, such as Figure 1 As shown, the specific steps include the following:

[0041] S1. Acquire a set of image frames using an industrial camera at a set frame rate, and calculate the mean gray level and standard deviation of gray level of each image frame in the set of image frames to obtain the gray level change rate of each image frame and output the gray level statistics of each image frame.

[0042] S2. For a single image frame in the image frame set, the ORB feature extraction algorithm is applied to obtain the feature point set, the number of feature points and the feature point response intensity. The feature point sets of a single image frame and the previous image frame are extracted respectively. Feature matching is performed and the number of matching pairs is obtained. The feature point matching rate of a single image frame is calculated based on the number of matching pairs and the number of feature points in a single image frame.

[0043] S3. Based on the average gray level, gray level change rate, and feature point matching rate of a single image frame, calculate the perturbation coupling index. At the same time, based on the number of feature points and the feature point response intensity of a single image frame, calculate the tracking stability index.

[0044] S4. Based on the perturbation coupling index and the tracking stability index, calculate the image calibration stability index, and determine whether the current stitching calibration is reliable based on the image calibration stability index. If and only if the image calibration stability index is less than or equal to the calibration state boundary threshold, maintain the current stitching state; otherwise, trigger fast recalibration.

[0045] In this embodiment, step S1 includes the following specific steps:

[0046] S11. Acquire a set of image frames using an industrial camera at a set frame rate;

[0047] S12. Perform grayscale conversion on the image frame set, extract the mean grayscale value and standard deviation of the grayscale value of a single image frame in the image frame set, and form an image frame pair with the previous image frame. Based on the mean grayscale value of the single image frame and the previous image frame, obtain the grayscale change rate of the single image frame. The mean grayscale value, standard deviation of the grayscale value, and grayscale change rate of the single image frame together constitute the grayscale statistics of the single image frame.

[0048] S13. Repeat step S12 until all individual image frames in the image frame set have been traversed.

[0049] In this embodiment, the feature point matching rate of a single image frame in S2 is calculated as follows:

[0050] ;

[0051] in, This represents the feature point matching rate of a single image frame t. This represents the number of feature points in a single image frame t. This represents the number of matching pairs between a single image frame t and the previous image frame t-1.

[0052] In this embodiment, the method for obtaining the number of matching pairs in S2 is as follows: extract the feature point set of a single image frame, obtain the ORB binary descriptor of each feature point in the feature point set, and simultaneously obtain the ORB binary descriptor of each feature point in the feature point set of the previous image frame; calculate the descriptor distance between the ORB binary descriptor of a single feature point in the feature point set of a single image frame and the ORB binary descriptors of all feature points in the feature point set of the previous image frame, and when the descriptor distance is less than the distance threshold, a match is successful and a matching pair is formed; traverse all feature points in the feature point set of a single image frame to obtain the number of matching pairs.

[0053] In this embodiment, the formula for calculating the disturbance coupling index in S3 is:

[0054] ;

[0055] in, This represents the perturbation coupling index of a single image frame t. This represents the average grayscale value of a single image frame t. This represents the rate of change in image grayscale for a single image frame t.

[0056] In this embodiment, the formula for calculating the tracking stability index in step S3 is as follows:

[0057] ;

[0058] in, This represents the tracking stability index for a single image frame t. This represents the response intensity of feature point i on a single image frame t. This represents the response intensity of feature point i on a single image frame t-1.

[0059] In this embodiment, the formula for calculating the image calibration stability index in step S4 is:

[0060] ;

[0061] in, The image calibration stability index represents a single image frame t.

[0062] Example 2

[0063] This embodiment proposes a rapid calibration system for stitched imaging, such as... Figure 2 As shown, it includes: an image acquisition module, a feature extraction module, a feature calculation module, and a state division module;

[0064] The image acquisition module is used to acquire a set of image frames using an industrial camera at a set frame rate, and to calculate the mean gray level and standard deviation of the gray level of a single image frame in the set of image frames, further obtaining the gray level change rate of a single image frame, and outputting the gray level statistics of a single image frame.

[0065] The feature extraction module is used to apply the ORB feature extraction algorithm to obtain the feature point set, the number of feature points and the feature point response intensity for a single image frame in the image frame set. It extracts the feature point set of a single image frame and the previous image frame respectively, performs feature matching and obtains the number of matching pairs, and calculates the feature point matching rate of a single image frame based on the number of matching pairs and the number of feature points of a single image frame.

[0066] The feature calculation module is used to calculate the perturbation coupling index based on the average gray level, gray level change rate, and feature point matching rate of a single image frame, and to calculate the tracking stability index based on the number of feature points and the feature point response intensity of a single image frame.

[0067] The state division module is used to calculate the image calibration stability index based on the perturbation coupling index and the tracking stability index, and to determine whether the current stitching calibration is reliable based on the image calibration stability index. If and only if the image calibration stability index is less than or equal to the calibration state boundary threshold, the current stitching state is maintained; otherwise, fast recalibration is triggered.

[0068] The parameters and steps for implementing the corresponding functions of each unit module in the stitched imaging rapid calibration system of the present invention described above can be referred to the parameters and steps in the embodiment of the stitched imaging rapid calibration method in Embodiment 1 above.

[0069] Example 3

[0070] This embodiment provides an electronic device, including a processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the above-described rapid calibration method for stitched imaging by calling the computer program stored in the memory.

[0071] The electronic device can vary considerably depending on its configuration or performance. It may include one or more Central Processing Units (CPUs) and one or more memories, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the rapid calibration method for stitched imaging provided in the above-described embodiment. The electronic device may also include other components for implementing its functions; for example, it may have wired or wireless network interfaces and input / output interfaces for data input and output. Details will not be elaborated upon in this embodiment.

[0072] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this disclosure can be embodied in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0073] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0074] This invention is described with reference to flowchart illustrations and block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and block diagrams, as well as combinations of blocks in the flowchart illustrations and block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and boxes Figure 1 The steps of the function specified in one or more boxes.

[0076] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A rapid calibration method for stitched imaging, characterized in that: The method comprises the following specific steps: S1, acquiring an image frame set by an industrial camera at a set frame rate, and calculating the image gray mean value and the image gray standard deviation of a single image frame in the image frame set, and further obtaining the image gray change rate of the single image frame, and outputting the image gray statistics of the single image frame; S2, for a single image frame in the image frame set, applying an ORB feature extraction algorithm to obtain a feature point set, a feature point number and a feature point response intensity, extracting the feature point set of the single image frame and the feature point set of a previous image frame respectively, performing feature matching and obtaining a matching pair number, and calculating a feature point matching rate of the single image frame based on the matching pair number and the feature point number of the single image frame; S3, calculating a disturbance coupling index based on the image gray mean value, the image gray change rate and the feature point matching rate of the single image frame, and calculating a tracking stability index based on the feature point number and the feature point response intensity of the single image frame; S4, calculating an image calibration stability index based on the disturbance coupling index and the tracking stability index, and judging whether the current calibration is reliable based on the image calibration stability index, and maintaining the current splicing state only when the image calibration stability index is less than or equal to a calibration state division threshold; otherwise, triggering a rapid recalibration; The S1 comprises the following specific steps: S11, acquiring an image frame set by an industrial camera at a set frame rate; S12, performing gray conversion on the image frame set, extracting the image gray mean value and the image gray standard deviation of a single image frame in the image frame set, and forming an image frame pair by the single image frame and a previous image frame, and obtaining the image gray change rate of the single image frame based on the image gray mean value of the single image frame and the previous image frame, wherein the image gray mean value, the image gray standard deviation and the image gray change rate of the single image frame together constitute the image gray statistics of the single image frame; S13, repeating step S12 until all single image frames in the image frame set are traversed; The calculation formula of the disturbance coupling index in S3 is: ； wherein, denotes the perturbation coupling index of a single image frame t, denotes the image gray mean value of a single image frame t, denotes the image gray rate of change of a single image frame t, The calculation formula of the tracking stability index in S3 is: ； wherein, denotes the tracking stability index for a single image frame t, denotes the response strength of feature point i on a single image frame t, denotes the response strength of feature point i on a single image frame t-1; The calculation formula of the image calibration stability index in S4 is: ； wherein, denotes the image calibration stability index for a single image frame t.

2. The method of claim 1, wherein: The calculation manner of the feature point matching rate of the single image frame in S2 is: ； wherein, represents a feature point matching rate of a single image frame t, represents a number of feature points of a single image frame t, represents a number of matching pairs of a single image frame t and a previous image frame t-1.

3. The method of claim 2, wherein: The acquisition manner of the matching pair number in S2 is: extracting the feature point set of the single image frame, obtaining the ORB binary descriptor of each feature point in the feature point set, and simultaneously obtaining the ORB binary descriptor of each feature point in the feature point set of the previous image frame; performing descriptor distance calculation on the ORB binary descriptor of a single feature point in the feature point set of the single image frame and the ORB binary descriptors of all feature points in the feature point set of the previous image frame, and successfully matching and forming a matching pair when the descriptor distance is less than a distance threshold; and obtaining the matching pair number by traversing all feature points in the feature point set of the single image frame.

4. A stitching imaging fast calibration system based on the stitching imaging fast calibration method of any one of claims 1-3, characterized in that, The system comprises an image acquisition module, a feature extraction module, a feature calculation module and a state division module. The image acquisition module is configured to acquire a set of image frames at a set frame rate by using an industrial camera, and calculate an image gray mean value and an image gray standard deviation of a single image frame in the set of image frames, further obtain an image gray variation rate of the single image frame, and output image gray statistics of the single image frame; The feature extraction module is configured to, for a single image frame in the set of image frames, apply an ORB feature extraction algorithm to obtain a feature point set, a feature point number, and a feature point response intensity, extract the feature point set of the single image frame and a previous image frame respectively, perform feature matching and obtain a matching pair number, and calculate a feature point matching rate of the single image frame based on the matching pair number and the feature point number of the single image frame; The feature calculation module is configured to calculate a disturbance coupling index based on the image gray mean value, the image gray variation rate, and the feature point matching rate of the single image frame, and calculate a tracking stability index based on the feature point number and the feature point response intensity of the single image frame; The state division module is configured to calculate an image calibration stability index based on the disturbance coupling index and the tracking stability index, and determine whether a current calibration is reliable based on the image calibration stability index. When and only when the image calibration stability index is less than or equal to a calibration state boundary threshold, the current stitching state is maintained. Otherwise, a rapid recalibration is triggered.

Images