Method for splicing and displaying panoramic scanning image of infrared warning equipment based on SOC (system on chip)

Abstract

The invention discloses an SOC-based splicing and displaying method for panoramic scanning images of infrared warning equipment. The method comprises the following steps: step 1, designing an image scaling processing module; 2, performing image acquisition on the zoomed image through row and column extraction, and storing the acquired image in an FIFO (First In First Out) module to form a new image; 3, carrying out the optimization design of an FDMAIP module, carrying out the input universalization of the IP module during the strip splicing, and employing an FDMA read-write address control mode; and 4, carrying out panoramic scanning image splicing display design. According to the invention, firstly, the original image is shrunk, so that the subsequent splicing operation is facilitated, meanwhile, the data processing is more convenient when the image is shrunk, then the shrunk image is stored in the DDR memory according to a rule, and finally, the image is read out from the DDR memory according to a 1080p line-field time sequence for display. According to the image splicing display method, image display processing is more flexible and rapid.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for splicing and displaying panoramic scanning images of infrared warning equipment based on SOC. Background Art

[0002] With the continuous development of image acquisition technology, people's demand for large-size, high-resolution images is increasing. Especially for infrared warning equipment, since the warning device is in a state of panorama scanning, the image formed is a panoramic 360° image, and the single-frame staring image is in a state of constant refreshing, and the image content cannot be seen clearly. Based on the above actual demand scenarios, it is necessary to splice and display the panorama 360° image, and realize the function of zooming in on a single-frame small image.

[0003] When processing complex scenes and large amounts of image data, traditional image stitching methods require a large memory to read images and then stitch them, which will lead to reduced image stitching efficiency. Therefore, this scheme proposes a stitching and display method for panoramic scanning images of infrared warning equipment based on SOC to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide a method for stitching and displaying panoramic scanning images of an infrared warning device based on SOC, so as to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for stitching and displaying panoramic scanning images of an infrared warning device based on SOC, the method comprising the following steps:

[0006] The first step is to design an image scaling processing module, and use the image scaling processing module to perform image scaling processing before the image enters the storage module;

[0007] In the second step, the scaled image is collected by row and column extraction, and the collected image is stored in the FIFO module to form a new image;

[0008] The third step is to optimize the design of the FDMA IP module. When splicing strips, the IP module input is universalized and the FDMA read and write address control method is used;

[0009] The fourth step is to design the panoramic scanning image stitching display, including the read address control of the strip image, the read address control of the enlarged area image, the data reading line, the input control of the field signal and the image stitching display control.

[0010] Preferably, the image includes a strip area image and an enlarged area image, and image processing of two resolutions is required when the image is scaled.

[0011] Preferably, during the image acquisition process, it is continuously determined which row and which column the corresponding data is in, and the points meeting the requirements are stored in the FIFO module.

[0012] Preferably, after the number of pixels stored in the FIFO module exceeds the number of scaling target pixels, the number in the FIFO is read out one row according to the number of scaling target pixels, and the row synchronization signal and the field synchronization signal are redesigned simultaneously during the data reading process to obtain a new image under the scaling target pixels.

[0013] Preferably, the design of the IP module improves two signals, including a w_addr_lf[31:0] signal and an r_addr_lf[31:0] signal, and both the w_addr_lf[31:0] signal and the r_addr_lf[31:0] signal are used as input signals.

[0014] Preferably, if the resolution of the stored strip image is 64*80120 frames, the base address of the first frame image written by the input logic of the w_addr_lf[31:0] signal is 0x0500000, then the base address of the nth frame image written in the first row is expressed as:

[0015] 0x0500000+64*(n-1) (1);

[0016] And the write base address of the image of the nth frame in the mth row is expressed as:

[0017] 0x0500000+1920*80*(m-1)+64*(n-1) (2);

[0018] Among them, 1920*80 represents the storage space occupied by storing a row of images.

[0019] Preferably, when the stripe partial image reading is controlled, the read base address r_addr_lf[31:0] is selected as the base address of the first frame image.

[0020] Preferably, when the image reading of the enlarged area is controlled, the display is performed based on the selected column as the starting point, and the base address r_addr_lf[31:0] for reading the image of the enlarged area is added with a column signal on the basis of the written base address.

[0021] Preferably, the base address of the partial image of the enlarged area is expressed as:

[0022] 0x0XXXXXXX~0x0XXXXXXX+(m*n-1) (3);

[0023] Wherein, m*n represents the product of the width and height of the image to be read.

[0024] Preferably, when the images are spliced ​​and displayed, the strip image and the magnified area image are divided according to the number of data source rows, and the remaining data sources after the division are displayed in a superimposed manner.

[0025] Technical effects and advantages of the present invention:

[0026] The present invention first realizes the reduction processing of the original image to facilitate the subsequent splicing operation. At the same time, the data processing is more convenient when the image is reduced. Then the reduced image is stored in the DDR memory according to the rules. Finally, the image is read out from the DDR memory according to the 1080p line field timing for display. Such an image splicing display method makes the image display processing more flexible and fast. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is an operation flow chart of the splicing display method of the present invention.

[0028] Figure 2 This is a schematic diagram of the image reduction IP core of the present invention.

[0029] Figure 3 This is a schematic diagram of basic parameter settings of the video_in_buf IP core of the present invention.

[0030] Figure 4 Schematic diagram of the FDMA IP core of the present invention.

[0031] Figure 5 This is a schematic diagram of the FDMA IP core parameter settings of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The present invention provides Figure 1 The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC shown includes the following steps:

[0034] The first step is to design an image scaling processing module, and use the image scaling processing module to perform image scaling processing before the image enters the storage module;

[0035] It should be noted that, since the image output by the alarm device needs to be spliced ​​in multiple frames, the image is first scaled before entering the storage module.

[0036] Specifically, the image includes a strip area image and an enlarged area image, and image processing of two resolutions needs to be performed when the image is scaled.

[0037] In the second step, the scaled image is collected by row and column extraction, and the collected image is stored in the FIFO module to form a new image;

[0038] Specifically, during the image acquisition process, it is continuously determined which row and column the corresponding data is in, and the points that meet the requirements are stored in the FIFO module.

[0039] Example 1: In a certain type of infrared warning equipment, the input original image size is 512*640, and the image output resolution is 1920*1080. In the upper half of the display area, i.e., the 1920*540 area, three strips of 120 frames of images are spliced, and the enlarged area image is displayed in the lower half of the display area. Regarding the above two different images, two resolution image processing is performed when the image is scaled. First, the image is reduced from 512*640 to 64*80 (strip area image), and secondly, the image is reduced from 512*640 to 480*600 (enlarged area image). When scaling the image, the image is mainly processed by row and column extraction;

[0040] When reducing a 512*640 image to 64*80, one point is extracted for every eight columns in the column direction, and one row is extracted for every eight rows in the row direction. During the image acquisition process, it is constantly determined which row and column the corresponding data is in. The points that meet the requirements are stored in the FIFO module. When the stored data meets the usage, the data in the FIFO is read out as 64 pixels per row. During the data reading process, the row synchronization signal and the field synchronization signal are redesigned at the same time. After completing the above operations, a new image with a resolution of 64*80 can be obtained.

[0041] When reducing a 512*640 image to 480*600, remove one dot for every sixteen points in the column direction and remove one row for every sixteen rows in the row direction. During the image acquisition process, it is constantly determined which row and column the corresponding data is in. The points that meet the requirements are stored in the FIFO module. When the stored number meets the usage, the number in the FIFO is read out as 480 rows per row. During the data reading process, the row synchronization signal and the field synchronization signal are redesigned at the same time. After completing the above operations, a new image with a resolution of 480*600 can be obtained.

[0042] In the process of realizing the above functions, an image reduction IP core module video_in_buf is designed, as shown in the attached figure. Figure 2 As shown, the input and output signals of the module are shown in Table 1 below;

[0043] Table 1 video_in_buf input and output signal table

[0044]

[0045] During the calling process, the video_in_buf module only needs to be double-clicked to set the new line and field signals. The setting block diagram is shown in the attached figure. Figure 3 As shown in the figure, Img_Width_suo represents the number of columns of the image after reduction, and Img_High_suo represents the number of rows of the image after reduction.

[0046] Furthermore, when the number of pixels stored in the FIFO module exceeds the number of scaling target pixels, the number in the FIFO is read out one row according to the number of scaling target pixels, and the row synchronization signal and the field synchronization signal are redesigned during the data reading process to obtain a new image under the scaling target pixels.

[0047] The third step is to optimize the design of the FDMA IP module. When splicing strips, the IP module input is universalized and the FDMA read and write address control method is used;

[0048] It should be noted that, in the process of panoramic stitching, multiple frames of images need to be stored and then displayed, so the FDMA IP module is optimized and designed. The storage space of the SOC on-chip RAM is limited, which is not enough for storing a large number of images. The off-chip DDR addressing space can reach 512Mb, and the data bit width is 16bit. The total storage space of a DDR can reach 8Gb. The storage space required for a 512*640 8-bit infrared image is 2.5Mb. For 120 frames of images in a weekly scan, the total storage area required is 300Mb. Therefore, DDR fully meets the storage space required for storing a circle of images scanned by the warning infrared device.

[0049] The existing DDR read and write operations use the datamover method, but the control method of datamover is complicated and requires writing a set of control logic for each signal of the AXI bus. This makes development difficult in actual use. Secondly, data transmission based on datamover is less efficient than data transmission based on FDMA. During the test, it was found that for the same 1080p resolution image, it takes 20ms to write to the DDR space based on the datamover method, but it takes 5ms to write to the DDR based on the FDMA method.

[0050] Based on the above reasons, the FDMA method is adopted when designing the strip splicing image reading and writing of infrared warning equipment.

[0051] Embodiment 2: In the SOC application field, there is an FDMA control module, but the module cannot be universalized for strip splicing, so the module is improved and designed. The designed IP module diagram is shown in the accompanying figure. Figure 4 As shown, the input and output signals of the module are shown in Table 2 below;

[0052] Table 2 IP module input and output signal table

[0053]

[0054]

[0055] When calling the module, double-click it to set the corresponding parameters. The setting diagram is shown in the attached figure. Figure 5 As shown, the parameter settings are consistent with the mature FDMA parameter design in the existing field.

[0056] Specifically, the design of the IP module improves two signals, including a w_addr_lf[31:0] signal and an r_addr_lf[31:0] signal, and both the w_addr_lf[31:0] signal and the r_addr_lf[31:0] signal are used as input signals.

[0057] It should be noted that, since these two parameters in the mature FDMA modules in the existing field are fixed as a parameter during use and cannot be modified in the middle, but for the weekly scan image, the storage of 120 frames of images needs to be written in a certain pattern in order to be displayed as strips, so these two parameters need to be continuously transformed in the middle. In the improved design, both w_addr_lf[31:0] and r_addr_lf[31:0] signals are designed as input signals instead of the original parameters, and are used together with the read address controller and the write address controller to flexibly realize the splicing and storage of multiple frames of images.

[0058] In the third embodiment, if the resolution of the stored strip image is 64*80120 frames, the base address of the first frame image written by the input logic of the w_addr_lf[31:0] signal is 0x0500000, then the base address of the nth frame image written in the first row is expressed as:

[0059] 0x0500000+64*(n-1) (1);

[0060] The write base address of the nth frame image in the second row is expressed as:

[0061] 0x0500000+1920*80+64*(n-1) (2);

[0062] And the write base address of the image of the nth frame in the mth row is expressed as:

[0063] 0x0500000+1920*80*(m-1)+64*(n-1) (3);

[0064] Among them, 1920*80 represents the storage space occupied by storing a row of images.

[0065] For the processing of the 480*600 resolution image in the enlarged area, considering that the image needs to be able to be displayed by random column selection, the input logic of w_addr_lf[31:0] is different from strip splicing. Since the image in the enlarged area does not need to be wrapped, if the base address written in the first frame is 0x0600000, the base address written in the nth frame is expressed as:

[0066] 0x0500000+480*(n-1) (4);

[0067] When storing the image of the enlarged area, the total storage space required for storing 120 frames of images is taken into consideration. The parameter setting of the WDsizebits signal is calculated and then input. The default value of the FDMA module is 22 bits, and the corresponding storage space is 4Mb. The storage space required for 120 frames of 480*600 8-bit infrared images is 264M. Therefore, the WDsizebits parameter should be entered as 29 to meet the storage requirements, and the corresponding storage space is 512Mb.

[0068] The fourth step is to design the panoramic scanning image stitching display, including the read address control of the strip image, the read address control of the enlarged area image, the data reading line, the input control of the field signal and the image stitching display control.

[0069] Specifically, when the stripe partial image reading is controlled, the read base address r_addr_lf[31:0] is selected as the base address of the first frame image.

[0070] When the reading of the enlarged area part of the image is controlled, the display is performed based on the selected column as the starting point, and the base address r_addr_lf[31:0] for reading the enlarged area image adds a column signal based on the written base address.

[0071] The base address of the image in the enlarged area is expressed as:

[0072] 0x0XXXXXXX~0x0XXXXXXX+(m*n-1) (5);

[0073] Wherein, m*n represents the product of the width and height of the image to be read.

[0074] When the images are stitched and displayed, the strip image and the enlarged area image are divided according to the number of data source rows, and the remaining data sources after division are displayed in a superimposed manner.

[0075] Embodiment 4: In a certain type of security scanning device, the stripe portion image is written in the order of 1920*320 regions, so when reading the image, the base address r_addr_lf[31:0] to be read can be directly selected as the base address of the first frame image.

[0076] The reading of the image in the enlarged area needs to be displayed based on the column selected by the mouse as the starting point, so the base address r_addr_lf[31:0] for reading the image in the enlarged area needs to add a column signal on the basis of the written base address. 480*120 has a total of 57600 columns, so the base address of the image in the enlarged area is between 0x0XXXXXXX and 0x0XXXXXXX+57599.

[0077] Secondly, the data is read by the input of line and field signals. Since the image is finally read out according to the standard 30hz SDI of 1920*1080, the input line and field signals are the line and field signals of the standard 30hz SDI of 1920*1080.

[0078] The last step is image stitching display. When outputting the display, the logic needs to be designed so that the data source for the first 320 rows is the compressed strip video, rows 321 to 921 are the magnified area images, and the remaining rows are used to overlay other information.

[0079] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC, characterized in that: The splicing and display method comprises the following steps: The first step is to design an image scaling processing module, and use the image scaling processing module to perform image scaling processing before the image enters the storage module; In the second step, the scaled image is collected by row and column extraction, and the collected image is stored in the FIFO module to form a new image; The third step is to optimize the design of the FDMA IP module. When splicing strips, the IP module input is universalized and the FDMA read and write address control method is used; The fourth step is to design the panoramic scanning image stitching display, including the read address control of the strip image, the read address control of the enlarged area image, the data reading line, the input control of the field signal and the image stitching display control.

2. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 1 is characterized in that: The image includes a strip area image and an enlarged area image, and image processing with two resolutions needs to be performed when the image is scaled.

3. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 1 is characterized in that: During the image acquisition process, it is continuously determined which row and which column the corresponding data is in, and the points that meet the requirements are stored in the FIFO module.

4. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 3 is characterized in that: When the number of pixels stored in the FIFO module exceeds the number of scaling target pixels, the number in the FIFO is read out one row according to the number of scaling target pixels, and the row synchronization signal and the field synchronization signal are redesigned during the data reading process to obtain a new image under the scaling target pixels.

5. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 1 is characterized in that: The design of the IP module improves two signals, including a w_addr_lf[31:0] signal and an r_addr_lf[31:0] signal, and both the w_addr_lf[31:0] signal and the r_addr_lf[31:0] signal are used as input signals.

6. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 5 is characterized in that: If the resolution of the stored strip image is 64*80120 frames, the base address of the first frame image written by the input logic of the w_addr_lf[31:0] signal is 0x0500000, then the base address of the nth frame image written in the first row is expressed as: 0x0500000+64*(n-1) (1); And the write base address of the image of the nth frame in the mth row is expressed as: 0x0500000+1920*80*(m-1)+64*(n-1) (2); Among them, 1920*80 represents the storage space occupied by storing a row of images.

7. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 1, characterized in that: When the stripe partial image reading is controlled, the read base address r_addr_lf[31:0] is selected as the base address of the first frame image.

8. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 1, characterized in that: When the enlarged area partial image is read and controlled, the display is performed based on the selected column as the starting point, and the base address r_addr_lf[31:0] for reading the enlarged area image is added with a column signal on the basis of the written base address.

9. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 8, characterized in that: The base address of the enlarged area partial image is expressed as: 0x0XXXXXXX~0x0XXXXXXX+(m*n-1) (3); Wherein, m*n represents the product of the width and height of the image to be read.

10. The method for stitching and displaying panoramic scanning images of infrared warning equipment based on SOC according to claim 1, characterized in that: When the images are displayed in mosaic, the strip image and the magnified area image are divided according to the number of data source rows, and the remaining data sources after the division are displayed in a superimposed manner.

Images