Image splicing system and method, electronic equipment and medium

Abstract

The invention provides an image stitching system and method, electronic equipment and a medium, and the system comprises an image sensor which is configured to operate in a periodic mode and outputs interlaced image data. The data processing device comprises a receiving and analyzing module, a calculation module and a storage module, the receiving and analyzing module is used for receiving each row of interlaced image data output by the image sensor and analyzing and acquiring a row of interlaced images and data information corresponding to each row of interlaced image data; the calculation module is used for calculating a storage address of the current row interlaced image; and the storage module is used for writing each row of interlaced images into a corresponding storage address to complete splicing of a plurality of frames of positive sequence images. A receiving module, an analysis module and a calculation module are arranged in the data processing device, image data are received, analyzed and spliced in real time, a CPU does not need to be additionally occupied, and stability and real-time performance are high; meanwhile, all the collected image data are processed in real time and are not transmitted or transferred, so that the problem of frame loss is avoided.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an image splicing system, method, electronic equipment and medium. Background Art

[0002] In image acquisition, in order to meet the acquisition requirements, an image with interlaced effective information is obtained in similar scenes with multiple angles or multiple light sources, or in application scenarios where the exposure frequency, time and other conditions are changed. The arrangement rule of the effective information in the image is the same as the setting rule of the specific application scenario, such as the angle arrangement rule, the light source arrangement rule or the delay time arrangement rule. For an image with interlaced effective information, it is necessary to decompose it into multiple non-interlaced continuous images. If the interlaced image is interlaced in sequence according to the rules of the first angle (light source) and the second angle (light source), it is necessary to extract the odd and even rows of the image respectively, and reorganize them into the sub-image of the first angle (light source) and the sub-image of the second angle (light source).

[0003] In application scenarios similar to the above, the prior art uses a host computer to implement image processing of interlaced images.

[0004] For example, patent CN 116482942 A describes an image exposure method and device. In this patent, the control module of the laser direct writing device uses the time interval formed by each laser exposing the image strip back and forth to alternately read a number of data packets corresponding to all odd-numbered image strips in the first memory and a number of data packets corresponding to all even-numbered image strips in the second memory, and controls N lasers to simultaneously expose the corresponding K lines of image strips row by row in order from top to bottom, so as to improve the image exposure efficiency of the laser.

[0005] Chinese patent CN 117372230 A proposes an in-situ image reconstruction method, system, storage medium and device. The patent allocates storage space for a buffer of the size of a single row of images, which consumes less memory. At the same time, by utilizing the arrangement rules of interlaced images and establishing a buffer, the CPU cache is maximized to reconstruct the interlaced images, thereby improving image processing efficiency.

[0006] However, using a host computer to process interlaced images of similar scenes as mentioned above requires, on the one hand, a high CPU usage rate and low image processing efficiency; on the other hand, if the operating system is bumpy, resulting in untimely CPU processing, image stitching errors or frame loss will occur.

[0007] In view of the above technical problems, the present invention provides an image stitching system, method, electronic device and medium, which can solve at least one of the above technical problems. Summary of the invention

[0008] The present invention provides an image stitching system, method, electronic device and medium, which can solve at least one of the above technical problems.

[0009] To achieve the above object, the present invention provides an image stitching system, comprising: an image sensor configured to operate in a periodic mode to output interleaved image data; The data processing device comprises: a receiving and analyzing module, a computing module, and a storage module; A receiving and analyzing module receives each line of interlaced image data output by the image sensor, and analyzes and obtains the line interlaced image and data information corresponding to each line of interlaced image data; A calculation module, calculating the storage address of the current row of interlaced images; The storage module writes each row of interlaced images to the corresponding storage address to complete the splicing of several frames of normal sequence images.

[0010] Further, a frame of interlaced image data includes image data generated by each group of pixels in a plurality of groups of pixels; wherein each group of pixels is configured to receive one of a plurality of different exposures running in a periodic sequence and generate image data corresponding to one exposure.

[0011] Furthermore, the data information includes: a row number of any row of the interlaced image and a resolution of any row of the interlaced image.

[0012] Furthermore, it also includes: a host computer, which receives a plurality of frames of positive sequence images output by the data processing device to realize image display or subsequent image processing operations.

[0013] Furthermore, the data processing device transmits several frames of normal sequence images to a corresponding memory buffer of the host computer through DMA.

[0014] Furthermore, the data processing device is an acquisition card.

[0015] Based on the same inventive concept, the present invention also proposes an image stitching method, comprising: Obtain interlaced image data and period information; Analyze and obtain the row interlaced image and data information corresponding to each row of interlaced image data; Based on the period information and the data information, the storage address of the current row interlaced image is calculated; Based on the calculation, write the current row interlaced image into the corresponding storage address; Based on the data information, repeated calculation and writing are performed to complete the splicing of several frames of positive sequence images.

[0016] Furthermore, the method also includes: outputting a plurality of spliced ​​frames of positive sequence images for image display or subsequent image processing operations.

[0017] On the other hand, the present invention further proposes an electronic device, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method as described above.

[0018] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image stitching method as described above is implemented.

[0019] The beneficial effects of the present invention are: The present invention has built-in receiving, analyzing modules and computing modules in the data processing device, which can receive, analyze and splice image data in real time without occupying additional CPU, and has high stability and real-time performance. At the same time, all collected image data can be processed in real time without any transmission or transfer, and frame loss will not occur.

[0020] The data processing device in the present invention is directly transmitted to the memory buffer corresponding to the host computer through DMA. The image obtained by the host computer is the stitched positive sequence image, which speeds up the processing efficiency of image stitching and maximizes the integrity and correctness of the stitched positive sequence image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural diagram of the image stitching system; Figure 2 is a flow chart of the image stitching method; Figure 3 is a schematic diagram of an implementation of the system of the present invention; Figure 4 It is a method flow diagram of an implementation method of the system in the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0023] like Figure 1 As shown, the present invention proposes an image stitching system, comprising: An image sensor is configured to operate in a periodic mode and output interlaced image data. A frame of interlaced image data includes image data generated by each group of pixels in a plurality of groups of pixels; wherein each group of pixels is configured to receive one of a plurality of different exposures operating in a periodic sequence and generate image data corresponding to the one exposure.

[0024] The data processing device comprises: a receiving and analyzing module, a computing module, and a storage module; The receiving and analyzing module receives each line of interlaced image data output by the image sensor, and analyzes and obtains the line interlaced image and data information corresponding to each line of interlaced image data, wherein the data information includes: the line number of any line of interlaced image and the resolution of any line of interlaced image.

[0025] The calculation module calculates the storage address of the current row of interlaced images.

[0026] The storage module writes each row of interlaced images to the corresponding storage address to complete the splicing of several frames of normal sequence images.

[0027] The system also includes: a host computer, which receives a plurality of frames of positive sequence images output by the data processing device and realizes image display or subsequent image processing operations.

[0028] The data processing device transmits several frames of positive sequence images to the corresponding memory buffer of the host computer through DMA.

[0029] Furthermore, the data processing device is an acquisition card.

[0030] like Figure 2 As shown, based on the same inventive concept, the present invention also proposes an image stitching method to adapt to the above-mentioned image stitching system, and the specific method is as follows: Obtain interlaced image data and period information; Analyze and obtain the row interlaced image and data information corresponding to each row of interlaced image data; Based on the period information and the data information, the storage address of the current row interlaced image is calculated; Based on the calculation, write the current row interlaced image into the corresponding storage address; Based on the data information, repeated calculation and writing are performed to complete the splicing of several frames of positive sequence images.

[0031] The method also includes: outputting a plurality of spliced ​​frames of positive sequence images for image display or subsequent image processing operations.

[0032] like Figure 3 As shown, in this embodiment, the operation method of the image stitching system is: The image sensor is configured to operate in periodic mode and outputs interleaved image data.

[0033] A frame of interlaced image data includes image data generated by each group of pixels in multiple groups of pixels (e.g., two or more groups of pixels); wherein each group of pixels is configured to receive one of multiple different exposures running in a periodic sequence and generate image data corresponding to one exposure.

[0034] In the periodic mode, each of the plurality of groups of pixels is configured to receive one of a plurality of different exposures; and image data generated by each of the plurality of groups of pixels is output as a frame of interlaced image data.

[0035] In image acquisition, in order to meet the acquisition requirements, an image with interlaced effective information is obtained in similar scenes with multiple angles or multiple light sources, or in application scenarios with changing exposure frequency, time, etc. The arrangement rule of effective information in the image is the same as the setting rule of the specific application scenario, such as the angle arrangement rule, light source arrangement rule, or delay time arrangement rule.

[0036] The arrangement rule of the above-mentioned effective information is based on the periodic mode in which the image sensor is configured. Taking any arrangement rule of effective information as an example, the periodic mode in which the image sensor is configured is explained.

[0037] When the image sensor collects a row of images, it switches the angle of the light source, that is, it collects the interlaced image data generated by the exposure of the first angle light source, the second angle light source, the third angle light source, ..., and the Nth angle light source. After completing a cycle, it collects the next row of images again in the same way to obtain a frame of interlaced image data. According to the acquisition rule of one frame of interlaced image data, the next frame of interlaced image data is obtained.

[0038] Furthermore, in addition to the switching of the light source angle, the type of light source can also be changed, such as light sources of different colors or light of different wavelengths, etc. Or the brightness of the light source can be changed so that the image sensor can sequentially collect each line of image of the first brightness, the second brightness, ..., the Nth brightness, and output a plurality of interlaced frames of image of the light source brightness.

[0039] Furthermore, in addition to changing the angle and type of the light source, the light source exposure can be set at equal time intervals, so that the image sensor can sequentially capture each row of images at the first time, the second time, ..., the Nth time, and output several frames of interlaced images at equal time intervals.

[0040] In this embodiment, the arrangement period of the multi-angle light source is taken as an example, that is, the image sensor is configured with light sources of N angles to expose the scene, and the exposure of the N angle light sources is periodic.

[0041] like Figure 3 As shown, in this embodiment, the image sensor includes N groups of pixels, and a group of pixels corresponds to the exposure response of an angle light source. The image sensor operates in a periodic mode and outputs interlaced image data; a frame of interlaced image data includes image data generated by each group of N groups of pixels, and N groups of pixels are read in the frame of interlaced image data.

[0042] The data processing device includes: a receiving and analyzing module, a computing module and a storage module. In this embodiment, the data processing device is an acquisition card.

[0043] The receiving and analyzing module receives each line of interlaced image data output by the image sensor, and analyzes and obtains the line interlaced image and data information corresponding to each line of interlaced image data, wherein the data information includes: the line number of any line of interlaced image and the resolution of any line of interlaced image.

[0044] According to the order in which the image sensor outputs interlaced image data, each row of interlaced image data is received, and the row interlaced image corresponding to each row of interlaced image data is parsed and obtained; at the same time, data information is obtained, including the row number of the current row of interlaced image and the width and height of any row of interlaced image.

[0045] The first line of interlaced image data is received, and the first line of interlaced image is parsed. Meanwhile, the first line of interlaced image data is written starting from the cache start address A0 set by the storage module.

[0046] According to the order in which the image sensor outputs interlaced image data, the Mth row of interlaced image data (M=2,3,4...) is received, and the Mth row of interlaced image corresponding to the Mth row of interlaced image data is parsed and obtained; at the same time, data information is obtained, specifically including the row number M and the width and height of the Mth row of interlaced image.

[0047] The calculation module calculates the storage address of the current row of interlaced images. The storage address of the current row of interlaced images is calculated according to the data information obtained by the receiving and parsing modules, as follows: Receive the Mth row of interlaced image data (M=2,3,4...), parse and obtain the Mth row of interlaced image data, and simultaneously obtain the resolution of the Mth row of interlaced image data, that is, obtain the width and height of the Mth row of interlaced image data.

[0048] Based on the period N of the image sensor configuration and the number of rows M of the interlaced image, the k+1th row of the current row of interlaced image belonging to the jth exposure is calculated, where j is the modulo of M over N, and k is the quotient of M over N.

[0049] Based on the width and height of the interlaced image of the Mth row, the starting address of the interlaced image of the Mth row stored in the storage module is calculated as A L ×H×(j-1)+A L ×k+A0. Among them, A L is the address width of the storage module occupied by each row of pixel data, H is the number of rows required to be collected for each exposure, and A0 is the cache start address set by the storage module.

[0050] In general, the first line of interlaced image data is the first line of the first exposure by default, and the first line of interlaced image data corresponding to the first line of interlaced image data is directly cached to the cache start address A0 set by the storage module. Therefore, the storage address of the first line of interlaced image is not specifically calculated, but the above calculation formula is also applicable to the calculation of the storage address of the first line of interlaced image.

[0051] The storage module writes the interlaced image of the current row to the corresponding storage address to complete the splicing of several frames of normal sequence images.

[0052] According to the above calculation results, the Mth row of interlaced images is written to the corresponding storage address A L ×H×(j-1)+A L ×k+A0. Several rows of interlaced images are stored in sequence to complete the acquisition and cache of one frame of interlaced image data. At this time, in the storage module, several rows of interlaced images stored are spliced ​​according to the exposure of light sources at different angles to form a frame of N groups of positive sequence images.

[0053] In this embodiment, the storage module is a RAM.

[0054] Furthermore, the acquisition and transmission process of one frame of the positive sequence image can be completed by transferring the positive sequence image stored in the RAM to the corresponding memory buffer of the host computer through DMA.

[0055] Repeat the above operations to complete the acquisition and transmission of the next frame of positive sequence image in sequence until the image acquisition is completed, so that the host computer can receive several frames of positive sequence images output by the data processing device to realize image display or subsequent image processing operations.

[0056] On the other hand, the present invention further proposes an electronic device, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method as described above.

[0057] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image stitching method as described above is implemented.

[0058] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0059] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image stitching system, characterized in that: include: an image sensor configured to operate in a periodic mode to output interleaved image data; The data processing device comprises: a receiving and analyzing module, a computing module, and a storage module; A receiving and analyzing module receives each line of interlaced image data output by the image sensor, and analyzes and obtains the line interlaced image and data information corresponding to each line of interlaced image data; A calculation module, calculating the storage address of the current row of interlaced images; The storage module writes each row of interlaced images to the corresponding storage address to complete the splicing of several frames of normal sequence images.

2. The image stitching system according to claim 1, characterized in that: A frame of interlaced image data includes image data generated by each group of pixels in a plurality of groups of pixels; wherein each group of pixels is configured to receive one of a plurality of different exposures running in a periodic sequence and generate image data corresponding to one exposure.

3. The image stitching system according to claim 1, characterized in that: The data information includes: the row number of any row of the interlaced image and the resolution of any row of the interlaced image.

4. The image stitching system according to claim 1, characterized in that: Also includes: The host computer receives several frames of positive sequence images output by the data processing device to realize image display or subsequent image processing operations.

5. The image stitching system according to claim 4, characterized in that: The data processing device transmits several frames of positive sequence images to the corresponding memory buffer of the host computer through DMA.

6. The image stitching system according to claim 1, characterized in that: The data processing device is an acquisition card.

7. An image stitching method, characterized in that: include: Obtain interlaced image data and period information; Analyze and obtain the row interlaced image and data information corresponding to each row of interlaced image data; Based on the period information and the data information, the storage address of the current row interlaced image is calculated; Based on the calculation, write the current row interlaced image into the corresponding storage address; Based on the data information, repeated calculation and writing are performed to complete the splicing of several frames of positive sequence images.

8. The image stitching method according to claim 7, characterized in that: Also includes: Output several frames of stitched positive sequence images for image display or subsequent image processing operations.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to claim 7 or 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image stitching method according to claim 7 or 8 is implemented.

Images