A method and related apparatus for seamless stitching of four-channel video images

Abstract

This application pertains to a video image stitching method. Addressing the technical problem that most existing field-of-view stitching methods cannot be used to stitch ultra-high bandwidth video images, and that the stitched images cannot be directly used for optical precision measurement, this application provides a seamless stitching method and related apparatus for four-channel video images. It synchronously acquires each frame of video image from four video input channels, along with the corresponding optical precision measurement equipment parameters. Each frame is then sequentially processed using histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction, and size cropping, yielding four intermediate processing results. These four intermediate processing results are then stitched together according to the target format. This overcomes the contradiction between large field-of-view coverage and high-resolution imaging.

Description

Technical Field

[0001] This application pertains to a video image stitching method, specifically a method and apparatus for seamless stitching of four-channel video images. Background Technology

[0002] Optoelectronic precision measurement equipment uses optical means to photograph and measure the target or group of targets, and obtain various parameters of the target or group of targets in space. During the measurement process, a motion mechanism is often required to drive the optical lens to track and photograph the moving target.

[0003] Optoelectronic precision measurement equipment usually needs to balance large field of view coverage and high-resolution imaging, but these two points are inherently contradictory. Using field of view stitching is an effective way to balance large field of view coverage and high-resolution imaging in general scenarios.

[0004] In existing field-of-view stitching methods, such as the Chinese invention patent with publication number CN107895344A, a video stitching device and method are disclosed. This method uses a planar sensor to sense the movement trajectory of the video stitching device, quickly determining the unified coordinates of the images to be stitched. It then determines the overlapping area based on the movement trajectory of the video stitching device and the position of image units, achieving image fusion, simplifying the stitching process, and reducing computational requirements. Another example is the Chinese invention patent with publication number CN118691466A, which discloses a video image stitching method, device, electronic device, and storage medium. This method uses cross-camera tracking of moving targets to determine the target image set and detection box in the video segment. It adjusts the stitching position based on the size change value of the target detection box and uses optical flow to confirm the optimal adjustment position, achieving high-precision stitching of the video segment. However, the images stitched by these existing technologies cannot be directly used for optical precision measurement and cannot be used to stitch ultra-high bandwidth video images. Summary of the Invention

[0005] This application addresses the technical problem that most existing field-of-view stitching methods cannot be used to stitch ultra-high bandwidth video images, and the stitched images cannot be directly used for optical precision measurement. It provides a seamless stitching method and related apparatus for four-channel video images.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, this application proposes a method for seamless stitching of four-channel video images, including:

[0008] Simultaneously acquire each frame of video image from four video inputs, as well as the corresponding optical precision measurement equipment parameters for each frame of video image;

[0009] Based on the parameters of the corresponding optical precision measurement equipment, histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction and size cropping are performed on each frame of video image in sequence to obtain four intermediate processing results; wherein, the spatial affine transformation method includes: first determining the linear relationship between the image plane coordinate system and the millimeter unit, and then using the projection transformation principle under the pinhole model to determine the relationship between the camera coordinate system and the image plane coordinate system.

[0010] The four intermediate processing results are concatenated according to the target format to obtain the concatenated result.

[0011] Furthermore, determining the linear relationship between the image plane coordinate system and the millimeter unit includes:

[0012]

[0013] in, Represents the x-coordinate of a pixel in the image plane coordinate system. Represents the ordinate of a pixel in the image plane coordinate system. Indicates each pixel in Physical dimensions Indicates each pixel in Physical dimensions Indicates the x-coordinate of the projection center. Indicates the ordinate of the projection center. The abscissa represents the physical coordinates in a planar coordinate system. The ordinate represents the physical coordinates in a planar coordinate system. Indicates the horizontal direction of the detector. This indicates the vertical direction of the detector.

[0014] Furthermore, the method for determining the relationship between the camera coordinate system and the image plane coordinate system using the projection transformation principle under the pinhole model includes:

[0015] Determine the camera coordinates corresponding to the physical coordinates in the image plane coordinate system:

[0016]

[0017] in, Indicates the camera's focal length. This represents the X-axis coordinate of the projection point in the image plane coordinate system. This represents the Y-axis coordinate of the projection point in the image plane coordinate system. This represents the Z-axis coordinate of the projection point in the image plane coordinate system. This represents the angle between the camera coordinate system's Z-axis and the image plane;

[0018] Based on the physical coordinates in the image plane coordinate system corresponding to the coordinates in the camera coordinate system, the corresponding matrix form is obtained:

[0019]

[0020] Based on the linear relationship between the image plane coordinate system and the millimeter unit, and the corresponding matrix form, the relationship between the camera coordinate system and the image plane coordinate system is obtained:

[0021] .

[0022] Furthermore, the method of sequentially performing histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction, and size cropping on each frame of video image includes:

[0023] Histogram statistics, spatial affine transformation and distortion correction are performed on each frame of video image in sequence to obtain the correction results of the video images corresponding to the four video streams, which are recorded as the first correction result to the fourth correction result.

[0024] The grayscale values ​​of the overlapping area between the first and second correction results are statistically analyzed to obtain the first statistical result, and then the size is trimmed.

[0025] Based on the first statistical result, grayscale consistency correction is performed on the second correction result to obtain the first grayscale correction result; the grayscale of the overlapping area between the first grayscale correction result and the fourth correction result is statistically analyzed to obtain the second statistical result, and then the size is trimmed.

[0026] Based on the first statistical result, grayscale consistency correction is performed on the third correction result to obtain the second grayscale correction result; the grayscale of the overlapping area between the second grayscale correction result and the fourth correction result is statistically analyzed to obtain the third statistical result, and then the size is trimmed.

[0027] Combining the second and third statistical results, the fourth correction result is subjected to grayscale consistency correction to obtain the third grayscale correction result, and then the third grayscale correction result is sized and trimmed.

[0028] Furthermore, after concatenating the four intermediate processing results according to the target format, image enhancement is also included.

[0029] Furthermore, after image enhancement, the process also includes:

[0030] The splicing results are matched with the information from the optical precision measurement equipment to complete the measurement information matching.

[0031] Furthermore, the information of the optical precision measuring device includes absolute time, lens focal length, lens aperture, platform orientation, and target position information.

[0032] Secondly, this application proposes a four-channel video image seamless stitching system, comprising:

[0033] The data module is used to synchronously acquire each frame of video image from the four video inputs, as well as the optical precision measurement equipment parameters corresponding to each frame of video image;

[0034] The processing module is used to combine the parameters of the corresponding optical precision measurement equipment to perform histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction and size cropping on each frame of video image in sequence, and obtain four intermediate processing results. The spatial affine transformation method includes: first determining the linear relationship between the image plane coordinate system and the millimeter unit, and then determining the relationship between the camera coordinate system and the image plane coordinate system by using the projection transformation principle under the pinhole model.

[0035] The splicing module is used to splice the four intermediate processing results according to the target format to obtain the spliced ​​result.

[0036] Thirdly, this application proposes an electronic device, including: a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the above-described seamless stitching method for four-channel video images.

[0037] Fourthly, this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described seamless stitching method for four video images.

[0038] Compared with the prior art, this application has the following beneficial effects:

[0039] This application proposes a seamless stitching method for four-channel video images. It simultaneously acquires each frame of video image from four input channels, along with the corresponding optical precision measurement equipment parameters. Each frame is then sequentially processed using histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction, and size cropping, resulting in four intermediate processing results. These four intermediate results are then stitched together according to the target format. The spatial affine transformation method involves first determining the linear relationship between the image plane coordinate system and millimeter units, and then using the pinhole projection transformation principle to determine the relationship between the camera coordinate system and the image plane coordinate system. The pinhole projection transformation principle includes a nonlinear distortion model. This application has sufficient versatility and can serve as a general real-time seamless stitching method for video images in the field of optical precision measurement. In practical applications, with the aid of a processor and combined with frame-by-frame processing, it can process ultra-high bandwidth videos with a single-channel resolution of up to 12k×8k in real time, achieving a stitching accuracy of less than or equal to 2 pixels.

[0040] This application also proposes a four-channel video image seamless stitching system, an electronic device, and a computer-readable storage medium, which possess all the advantages of the aforementioned four-channel video image seamless stitching method. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating a method for seamless stitching of four video images according to this application.

[0043] Figure 2 This is a schematic diagram illustrating an implementation method for histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction, and size trimming in an embodiment of this application.

[0044] Figure 3 This is a schematic diagram of the three-coordinate system relationship of the camera in the embodiments of this application;

[0045] Figure 4 This is a schematic diagram of a four-channel video image seamless splicing system according to this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0047] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0048] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. The terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0049] In the field of precision optical measurement, technological development faces constraints imposed by the inherent characteristics of optical systems. These devices continuously track and photograph moving targets using optical lenses to acquire key parameters such as spatial position and morphological features. They are then widely used in scenarios requiring micrometer-level precision measurements, such as semiconductor wafer inspection, spacecraft rendezvous and docking monitoring, and dynamic observation of biological cells. The core challenge lies in resolving the contradiction between "large field-of-view coverage" and "high-resolution imaging." This contradiction is particularly pronounced in dynamic tracking measurements; for example, when monitoring the trajectory of a high-speed aircraft, it is necessary to capture the entire trajectory while simultaneously ensuring image clarity at every moment.

[0050] While existing technologies can achieve basic stitching of multiple images, they are designed for display-level applications such as security monitoring, and stitching errors are typically at the pixel level, far from meeting the sub-pixel level requirements for precision measurement. Furthermore, dynamic response capabilities are severely limited. Some existing technologies improve stitching accuracy in specific scenarios through intelligent recognition, but the high algorithm complexity leads to a significant increase in processing latency, resulting in noticeable spatiotemporal misalignment in high-speed motion scenarios. Especially when the data volume of a single video stream exceeds 4Gbps, existing devices commonly experience frame synchronization loss and data packet corruption, directly causing measurement distortion.

[0051] Currently, there are attempts to overcome these limitations through hardware stacking. However, these solutions typically cannot solve the problem of optical axis consistency calibration for multi-sensor arrays. Even minor installation errors can lead to measurement discontinuities at the seams, and they also cannot meet the real-time tracking requirements of high-speed targets. This directly restricts the application depth of optoelectronic precision measurement systems in cutting-edge fields such as advanced manufacturing and space exploration, making it urgent to reconstruct the field-of-view stitching technology system from the underlying architecture.

[0052] Based on the above, this application proposes a method and related apparatus for seamless stitching of four-channel video images. The following is a detailed description of this application in conjunction with embodiments and accompanying drawings.

[0053] like Figure 1 The diagram shown is a flowchart illustrating one method for seamless stitching of four video images according to this application, which may include:

[0054] S101 synchronously acquires each frame of video image from the four video inputs, as well as the optical precision measurement equipment parameters corresponding to each frame of video image.

[0055] When simultaneously acquiring each frame of video images from four video input channels, real-time acquisition of the video stream can be performed using a high-speed FPGA (Field-Programmable Gate Array) interface chip, and a timestamp mechanism can be used to record and synchronize data transmission time. In practical applications, for acquiring parameters of optical precision measurement equipment, parameter information from the equipment can be read in real time using a network interface. Synchronous acquisition ensures strict spatiotemporal synchronization of multiple video images, providing a foundation for subsequent image processing, ensuring that image stitching is not affected by time deviations, and enhancing the overall system stability and accuracy.

[0056] In other embodiments of this application, besides FPGA, dedicated integrated circuit chips can also be used in conjunction with high-speed caches to achieve synchronous acquisition and data reading of multiple video frame images. Additionally, time synchronization can also be achieved using GPS (Global Positioning System) signals to ensure higher precision timestamps.

[0057] S102, combined with the parameters of the corresponding optical precision measurement equipment, performs histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction and size cropping on each frame of video image in sequence, and obtains four intermediate processing results.

[0058] The spatial affine transformation method includes: first determining the linear relationship between the image plane coordinate system and the millimeter unit, and then using the projection transformation principle under the pinhole model to determine the relationship between the camera coordinate system and the image plane coordinate system.

[0059] It's worth noting that histogram statistics can be quickly implemented on a graphics processing unit using software algorithms. By analyzing the number of pixels at each gray level in an image, understanding the brightness distribution and revealing the image's tonal gradations is crucial for subsequent gray-scale consistency correction, ensuring good coordination in brightness and color between images from different video sources. Lens distortion is often a major obstacle in image stitching. By pre-calibrating distortion parameters and then using inverse distortion algorithms, barrel and pincushion distortions can be eliminated, restoring the image to its natural shape. Gray-scale consistency correction can be adjusted by comparing the average brightness of the images, ensuring consistency in gray levels between images from different video sources and eliminating the visual fragmentation caused by brightness differences. Furthermore, to ensure accurate alignment during stitching, images can be cropped as needed based on the final stitching size. Through a series of preprocessing steps, the quality of the stitched image can be effectively improved, reducing stitching defects caused by differences in brightness, color, and size between different video sources, achieving refined image processing, and providing high-quality material for subsequent stitching.

[0060] This application proposes an optimized method for spatial affine transformation. First, the pixel coordinates of an image captured by a camera are typically two-dimensional coordinates in pixels, while the actual physical dimensions involve units of length in the physical world. By calibrating the camera's intrinsic parameters, such as focal length and image sensor size, the actual physical size of each pixel can be calculated, thus establishing a linear relationship between the image plane coordinate system and millimeters. Then, a pinhole model is used for projection transformation, in which the lens is assumed to be a pinhole through which light travels in a straight line to the image plane. The relationship between the camera coordinate system and the image plane coordinate system can be determined using the camera's intrinsic and extrinsic parameters.

[0061] The spatial affine transformation method proposed in this application ensures seamless stitching of four video images, avoiding significant visual misalignment and greatly improving the accuracy of the stitched image. After spatial affine transformation, the spatial position of the images is more accurate, avoiding image distortion and loss due to positional deviations, and significantly enhancing the visual effect of the stitched image. Furthermore, since spatial affine transformation relies on the parameters of optical precision measurement equipment, the mathematical model-based processing method can be widely adapted to different optical precision measurement devices, ensuring accurate image stitching. Therefore, it not only solves the spatial alignment problem but also ensures high-quality output of the stitched image through the precise application of the mathematical model.

[0062] S103, concatenate the four intermediate processing results according to the target format to obtain the concatenated result.

[0063] It should be noted that the target format takes into account the resolution and color depth of the final stitched image. In practical applications, the spatial relationship of each video image and the overlapping areas between them can be considered, and a complete image can be generated by fusing different image segments.

[0064] This application ensures the synchronization of video streams and optical parameters by simultaneously acquiring each frame of video image from four video inputs, along with the corresponding optical precision measurement equipment parameters, thereby improving the accuracy and stability of the stitched images. It achieves seamless real-time stitching of four or more video sources, significantly expanding the field of view while maintaining the detail and clarity of high-resolution imaging. This overcomes the contradiction between large field of view coverage and high-resolution imaging, and provides a more reliable data source for optical precision measurement through precise matching of measurement parameters. Combining various image preprocessing techniques effectively eliminates the effects of inconsistencies in brightness, color, and distortion, significantly improving the visual quality and measurement accuracy of the images. It possesses the capability to process ultra-high bandwidth video streams, adapting to the big data processing needs of modern optoelectronic precision measurement equipment and demonstrating excellent scalability.

[0065] like Figure 2 The diagram illustrates an implementation method for histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction, and size trimming according to this application. In some embodiments of this application, the following methods may be specifically employed:

[0066] (1) Perform histogram statistics, spatial affine transformation and distortion correction on each frame of video image in sequence to obtain the correction results of the video images corresponding to the four video streams, which are recorded as the first correction result to the fourth correction result.

[0067] (2) Calculate the gray values ​​of the overlapping area between the first correction result and the second correction result to obtain the first statistical result, and then perform size cutting.

[0068] (3) Combine the first statistical results and perform grayscale consistency correction on the second correction results to obtain the first grayscale correction results; count the grayscale of the overlapping area between the first grayscale correction results and the fourth correction results to obtain the second statistical results, and then perform size cutting.

[0069] (4) Combine the first statistical results and perform grayscale consistency correction on the third correction results to obtain the second grayscale correction results; count the grayscale of the overlapping area between the second grayscale correction results and the fourth correction results to obtain the third statistical results, and then perform size trimming.

[0070] (5) Combining the second and third statistical results, perform grayscale consistency correction on the fourth correction result to obtain the third grayscale correction result, and then perform size trimming on the third grayscale correction result.

[0071] In this embodiment, a phased, multi-channel collaborative grayscale correction mechanism is constructed to achieve color consistency across four video streams. The grayscale correction of the four video streams is decomposed into three iterative stages. Each stage uses the corrected video stream as a benchmark and dynamically adjusts the gain and bias parameters of subsequent video streams by statistically analyzing the grayscale distribution characteristics of overlapping areas. By gradually eliminating grayscale differences between the video streams, end-to-end color consistency is ultimately achieved. Simultaneously, a size trimming operation is performed after each grayscale correction, removing invalid edge areas and establishing a geometric foundation for subsequent stitching through forced spatial alignment. Furthermore, performing size trimming immediately after each grayscale correction reduces data volume and, while ensuring the integrity of effective information, lowers the computational load of subsequent stitching processing. Compared to direct stitching schemes, this embodiment reduces the brightness jump at the stitching seams.

[0072] In some embodiments of this application, after obtaining the stitching result, enhancement processing can be performed. The intermediate processing results are sent to the main processing module, where image stitching and enhancement, as well as measurement information matching, local magnification, and OSD (On-Screen Display) are completed, and finally the results are input. In specific execution, relevant implementation methods in the prior art can be directly adopted. Among them, measurement information matching can match the image with the information of the device that captured the image frame, such as absolute time, lens focal length, aperture, platform orientation, target position information, etc., to obtain complete data for precise measurement.

[0073] When performing spatial affine transformations, the following methods can be used:

[0074] The goal of spatial affine transformation is to transform the actual position in 3D space within the stitched field of view between the camera's imaging point and the actual position in the 3D space. The transformation relationship model can be established based on optical imaging principles. For example... Figure 3 The diagram shown illustrates the three-coordinate system of the camera. The coordinates are centered on the optical axis. (The optical center) is the origin of the coordinate system. Z The axes follow the right-hand rule. Image origin. The plane in question is the image plane coordinate system, while the actual object's coordinate system is the world coordinate system. Among these, The value in the world coordinate system is , yes At the projection point in the image plane coordinate system, its value in the camera coordinate system is... . It is the angle between the camera coordinate system's Z-axis and the image plane. Generally, the Z-axis is perpendicular to the image plane. The value is 90°. And the camera coordinate system... With the image plane parallel, This is the focal length of the camera. Wherein, The horizontal direction of the image plane The direction perpendicular to the image plane.

[0075] For the transformation from the camera coordinate system to the image plane coordinate system, the image plane coordinate system is represented in pixels, while the camera coordinate system is represented in millimeters. Therefore, to complete this transformation, it is necessary to first obtain the linear relationship between the pixel units and millimeter units of the image plane. Figure 3 In the middle, the center of the camera's optical axis Intersection point of the direction with the image plane This is called the projection center, with coordinates of , is a unit of measurement for pixels, and each pixel is in and The physical dimensions are: The unit is pixels per millimeter. Therefore, the linear relationship between pixels and millimeters in the image plane is:

[0076]

[0077] in, Represents the x-coordinate of a pixel in the image plane coordinate system. Represents the ordinate of a pixel in the image plane coordinate system. Indicates each pixel in Physical dimensions Indicates each pixel in Physical dimensions Indicates the x-coordinate of the projection center. Indicates the ordinate of the projection center. The abscissa represents the physical coordinates in a planar coordinate system. The ordinate represents the physical coordinates in a planar coordinate system. Indicates the horizontal direction of the detector. This indicates the vertical direction of the detector.

[0078] Based on the projection transformation principle of the pinhole model, the physical coordinates of the image plane coordinate system... The corresponding camera coordinates satisfy:

[0079]

[0080] The corresponding matrix form is:

[0081]

[0082] Combining the above formulas, we obtain the matrices of the camera coordinate system and the image plane coordinate system:

[0083]

[0084] in, These are the camera's six intrinsic parameters, and the matrix formed by them is the intrinsic parameter matrix.

[0085] The calibration of camera intrinsic parameters can be solved using known feature point image coordinates and camera coordinates.

[0086] This application can simultaneously acquire each frame of video images from four video inputs under synchronous signal drive. It inputs the parameters of the optical precision measurement equipment corresponding to each frame via the network port. First, it performs histogram statistics, spatial affine transformation, distortion correction, size cropping, and grayscale consistency processing on the input video in a predetermined order. Then, it stitches the four received image frames from the same moment according to the final format and outputs the enhanced version. It can simultaneously receive and stitch four video images of the same format, with a maximum video processing bandwidth of no less than 8.9 GB / s, and supports simultaneous output of multiple processed video streams, with a maximum output resolution of up to 8 kHz.

[0087] like Figure 4 The diagram shown is a schematic of a four-channel video image seamless stitching system according to this application, which may include:

[0088] The data module is used to synchronously acquire each frame of video image from the four video inputs, as well as the optical precision measurement equipment parameters corresponding to each frame of video image;

[0089] The processing module is used to combine the parameters of the corresponding optical precision measurement equipment to perform histogram statistics, spatial affine transformation, distortion correction, grayscale consistency correction and size cropping on each frame of video image in sequence, and obtain four intermediate processing results. The spatial affine transformation method includes: first determining the linear relationship between the image plane coordinate system and the millimeter unit, and then determining the relationship between the camera coordinate system and the image plane coordinate system by using the projection transformation principle under the pinhole model.

[0090] The splicing module is used to splice the four intermediate processing results according to the target format to obtain the spliced ​​result.

[0091] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of each module is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0092] Furthermore, the modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0093] This application also provides an electronic device, which may include one or more processors, memory and communication interfaces.

[0094] The memory, communication interface, and processor are coupled together. For example, the memory, communication interface, and processor can be coupled together via a bus.

[0095] The communication interface is used for data transmission with other devices. The memory stores computer program code. This computer program code includes computer instructions, which, when executed by the processor, cause the electronic device to perform the steps of the aforementioned seamless four-channel video image stitching method.

[0096] The processor can be a processor or controller, such as a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The processor can be used to support an electronic device in performing the method steps provided in the above embodiments.

[0097] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. These buses can be categorized as address buses, data buses, control buses, etc.

[0098] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the above-described seamless stitching method for four-channel video images.

[0099] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage media known in the art.

[0100] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A seamless tiling method for four-way video images, characterized by, The method comprises the following steps: Synchronously acquiring each frame of video images of four video inputs and corresponding optical precision measurement device parameters; Respectively performing histogram statistics, spatial affine transformation, distortion correction, gray consistency correction and size cutting on each frame of video images in sequence to obtain four intermediate processing results in combination with the corresponding optical precision measurement device parameters; wherein the method of spatial affine transformation comprises the following steps: The method of determining the linear relationship between the image plane coordinate system and the millimeter unit comprises the following steps: wherein denotes the horizontal coordinate of a pixel in the image plane coordinate system, denotes the vertical coordinate of a pixel in the image plane coordinate system, denotes the physical dimension of each pixel in the horizontal direction, denotes the physical dimension of each pixel in the vertical direction, denotes the physical dimension of each pixel in the horizontal direction, denotes the physical dimension of each pixel in the vertical direction, denotes the horizontal coordinate of the projection center, denotes the vertical coordinate of the projection center, denotes the horizontal coordinate of the physical coordinate of the image plane coordinate system, denotes the vertical coordinate of the physical coordinate of the image plane coordinate system, denotes the horizontal direction of the detector, denotes the vertical direction of the detector; The method of determining the relationship between the camera coordinate system and the image plane coordinate system by means of the projection transformation principle under the pinhole model comprises the following steps: Determining the coordinates of the camera coordinate system corresponding to the physical coordinates of the image plane coordinate system: wherein, represents the focal length of the camera, represents the X-axis coordinate of the projection point of the image plane coordinate system, represents the Y-axis coordinate of the projection point of the image plane coordinate system, represents the Z-axis coordinate of the projection point of the image plane coordinate system, represents the angle between the Z-axis of the camera coordinate system and the image plane; According to the coordinates of the camera coordinate system corresponding to the physical coordinates of the image plane coordinate system, a corresponding matrix form is obtained: According to the linear relationship between the image plane coordinate system and the millimeter unit and the corresponding matrix form, the relationship between the camera coordinate system and the image plane coordinate system is obtained: Splicing the four intermediate processing results according to a target format to obtain a splicing result.

2. The method of claim 1, wherein, The method of respectively performing histogram statistics, spatial affine transformation, distortion correction, gray consistency correction and size cutting on each frame of video images in sequence comprises the following steps: Respectively performing histogram statistics, spatial affine transformation and distortion correction on each frame of video images in sequence to obtain correction results of video images corresponding to the four videos, which are recorded as a first correction result to a fourth correction result; Statistically obtaining a first statistical result by overlapping the gray scales of the first correction result and the second correction result, and then performing size cutting; Performing gray consistency correction on the second correction result in combination with the first statistical result to obtain a first gray correction result; statistically obtaining a second statistical result by overlapping the gray scales of the first gray correction result and the fourth correction result, and then performing size cutting; Performing gray consistency correction on the third correction result in combination with the first statistical result to obtain a second gray correction result; statistically obtaining a third statistical result by overlapping the gray scales of the second gray correction result and the fourth correction result, and then performing size cutting; Performing gray consistency correction on the fourth correction result in combination with the second statistical result and the third statistical result to obtain a third gray correction result, and then performing size cutting on the third gray correction result.

3. The method of claim 1, wherein, After the splicing of the four intermediate processing results according to the target format, image enhancement is further included.

4. The method of claim 3, wherein, After the image enhancement, the following steps are further included: Matching the splicing result and information of the optical precision measurement device to complete measurement information matching.

5. The method of claim 4, wherein, The information of the optical precision measurement device comprises absolute time, lens focal length, lens aperture, platform pointing and target position information.

6. A seamless four-way video image tiling system, characterized by, The method comprises the following steps: A data module is configured to synchronously acquire each frame of video images of four video inputs and corresponding optical precision measurement device parameters; The processing module is used for sequentially performing histogram statistics, spatial affine transformation, distortion correction, gray consistency correction and size cutting on each frame of video image respectively in combination with corresponding optical precision measurement equipment parameters, and four intermediate processing results are obtained correspondingly; wherein, the method of the spatial affine transformation comprises the following steps: firstly determining the linear relationship between the image plane coordinate system and the millimeter unit, and then determining the relationship between the camera coordinate system and the image plane coordinate system by means of the projection transformation principle under the pinhole model; The method for determining the linear relationship between the image plane coordinate system and the millimeter unit comprises the following steps: wherein denotes the horizontal coordinate of a pixel in the image plane coordinate system, denotes the vertical coordinate of a pixel in the image plane coordinate system, denotes the physical dimension of each pixel in the horizontal direction, denotes the physical dimension of each pixel in the vertical direction, denotes the physical dimension of each pixel in the horizontal direction, denotes the physical dimension of each pixel in the vertical direction, denotes the horizontal coordinate of the projection center, denotes the vertical coordinate of the projection center, denotes the horizontal coordinate of the physical coordinates of the image plane coordinate system, denotes the vertical coordinate of the physical coordinates of the image plane coordinate system, denotes the horizontal direction of the detector, denotes the vertical direction of the detector; The method for determining the relationship between the camera coordinate system and the image plane coordinate system by means of the projection transformation principle under the pinhole model comprises the following steps: determining the coordinates of the camera coordinate system corresponding to the physical coordinates of the image plane coordinate system: wherein, represents the focal length of the camera, represents the X-axis coordinate of the projection point of the image plane coordinate system, represents the Y-axis coordinate of the projection point of the image plane coordinate system, represents the Z-axis coordinate of the projection point of the image plane coordinate system, represents the angle between the Z-axis of the camera coordinate system and the image plane; obtaining the corresponding matrix form according to the coordinates of the camera coordinate system corresponding to the physical coordinates of the image plane coordinate system: obtaining the relationship between the camera coordinate system and the image plane coordinate system according to the linear relationship between the image plane coordinate system and the millimeter unit and the corresponding matrix form: The splicing module is used for splicing the four intermediate processing results according to a target format to obtain a splicing result.

7. An electronic device, comprising: comprise: a memory and one or more processors; the memory is coupled with the processor; wherein the memory has computer program code stored therein, the computer program code comprises computer instructions, when the computer instructions are executed by the processor, the electronic device executes the steps of the four-way video image seamless splicing method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium has a computer program stored therein, and the computer program is executed by the processor to realize the steps of the four-way video image seamless splicing method according to any one of claims 1-5.

Images