Method and system for superposing data in digital image with any rectangular size based on FPGA (Field Programmable Gate Array)

Abstract

The invention provides a method and a system for superposing data in a digital image with any rectangular size based on an FPGA, and the method comprises the following steps: receiving and caching imager image information and aircraft state information, reading pixels at each caching moment after all imager image information is received and cached, and storing the pixels at the caching moment; judging whether the pixel is in the replacement area or not; if the pixels are located in the replacement area, outputting aircraft state information corresponding to the pixels to a data recorder; if the pixel is not in the replacement area, outputting imager image information at least comprising the pixel to a data recorder; all pixels of the imager image are traversed until the count of the pixels conforms to the total number of pixels corresponding to the rectangular size. According to the invention, the image information of the imager and the state information of the aircraft are fused in one image for recording, so that the unified storage and management of data are realized, the comprehensive analysis and diagnosis of the operation condition of the aircraft are facilitated, and the increasing monitoring requirements on the aircraft are met.

Description

Technical Field

[0001] The present application relates to the field of infrared imaging information processing technology, and in particular to a method and system for superimposing data in a digital image of arbitrary rectangular size based on FPGA. Background Art

[0002] During the operation of the aircraft, the image data collected by the infrared imager, the aircraft working status data and other information need to be output to the data recorder for storage. After the aircraft crashes to the ground or the flight test is completed, the data stored in the data recorder can be read out in a specific way and analyzed, which is of great significance to the evaluation of the flight test effect and fault analysis.

[0003] Traditional aircraft data recording methods may only record the aircraft's status information and ignore the importance of image data; or image data and status information are recorded separately, which is not convenient for comprehensive analysis, and it is impossible to fully understand the aircraft's operating conditions and meet the increasingly high requirements for aircraft monitoring. Summary of the invention

[0004] In view of the above defects or deficiencies in the prior art, the present application aims to provide a method and system for overlaying data in a digital image of arbitrary rectangular size based on FPGA, including:

[0005] In a first aspect, the present application proposes a method for overlaying data in a digital image of arbitrary rectangular size based on FPGA, comprising the following steps:

[0006] Initializing the rectangular size of an imager image and replacement area parameters of a replacement area in the imager image; the imager image is composed of a plurality of pixel arrangements;

[0007] Receive and cache image information of the imager and aircraft status information; wherein the image information of the imager includes at least the pixels at each cache moment, and each cache moment corresponds to the aircraft status information at the cache moment;

[0008] When all the imager image information of the imager image is completely received and buffered, outputting the information to a data recorder;

[0009] The output of information to the data recorder specifically comprises the following steps:

[0010] Reading the pixel at each cached moment, and determining whether the pixel is within the replacement area;

[0011] If the pixel is within the replacement area, outputting the aircraft state information corresponding to the pixel at the cache time to the data recorder;

[0012] If the pixel is not within the replacement region, outputting the image information of the imager including at least the pixel to the data recorder;

[0013] All pixels of the imager image are traversed until the count of the pixels meets the total number of pixels corresponding to the size of the rectangle.

[0014] According to the technical solution provided in the embodiment of the present application, the rectangular size of the imager image is (M, N), the rectangular size represents the image coordinate system, the upper left corner of the imager image is the origin of the image coordinate system, each row of the imager image has M pixels, and there are N rows of pixels in total;

[0015] The counting of pixels until it matches the total number of pixels corresponding to the size of the rectangle specifically includes the following steps:

[0016] Until the total number of all the pixels is the product of M and N.

[0017] According to the technical solution provided in the embodiment of the present application, the replacement area parameters include the starting position of the replacement area, the starting position at least includes the starting coordinates of the starting pixel in the image coordinate system, and the size information of the replacement area, and the size information includes the area length and the area width.

[0018] According to the technical solution provided by the embodiment of the present application, after the initialization of the rectangular size of the imager image and the replacement area parameters of the replacement area in the imager image, the following steps are also included:

[0019] Obtaining a feature coordinate set according to the rectangle size and the replacement area parameter, wherein the feature count set includes the feature coordinates of all the pixels within the replacement area;

[0020] The determining whether the pixel is within the replacement area specifically comprises the following steps:

[0021] The coordinates of the pixel are obtained, and whether the pixel is within the replacement area is determined based on whether the coordinates of the pixel have a matching item in the feature coordinate set.

[0022] According to the technical solution provided in the embodiment of the present application, if the pixel is within the replacement area, outputting the aircraft state information corresponding to the pixel at the cache time to the data recorder specifically includes the following steps:

[0023] If the coordinates of the pixel have matching feature coordinates in the feature coordinate set, the aircraft state information corresponding to the pixel at the cache time is output to the data recorder.

[0024] According to the technical solution provided in the embodiment of the present application, if the pixel is not in the replacement area, outputting the image information of the imager including at least the pixel to the data recorder specifically includes the following steps:

[0025] If the coordinates of the pixel do not have any matching feature coordinates in the feature count set, the imager image information including at least the pixel is output to the data recorder.

[0026] According to the technical solution provided in the embodiment of the present application, obtaining a feature coordinate set according to the rectangle size and the replacement area parameter specifically includes the following steps:

[0027] The coordinates whose horizontal coordinates are greater than or equal to the horizontal coordinates of the starting coordinates and less than the sum of the horizontal coordinates of the starting coordinates and the length of the area, and whose vertical coordinates are greater than or equal to the vertical coordinates of the starting coordinates and less than the sum of the vertical coordinates of the starting coordinates and the width of the area are calibrated as the characteristic coordinates, thereby obtaining the characteristic coordinate set.

[0028] According to the technical solution provided by the embodiment of the present application, the method further includes a method for determining replacement area parameters of the replacement area in the image of the imager, and the determination method includes the following steps:

[0029] A non-important area of ​​the imager image is selected as a replacement area, and the parameters of the non-important area are the replacement area parameters. The non-important area at least includes four end corners and four edges of a rectangular size.

[0030] According to the technical solution provided by the embodiment of the present application, the method further includes a method for determining replacement area parameters of the replacement area in the image of the imager, and the determination method includes the following steps:

[0031] Analyze all the pixels in the image of the imager by using image recognition technology to obtain pixel features of each pixel, wherein the pixel features include at least brightness value and color value;

[0032] Taking a region composed of all adjacent pixels whose similarity of pixel features is higher than a first preset threshold as a candidate region;

[0033] If the size of the candidate area meets the preset size, the candidate area is used as the replacement area, and the parameters of the candidate area are the replacement area parameters.

[0034] In the second aspect, the present application proposes to provide a system for superimposing data in digital images of arbitrary rectangular sizes based on FPGA, which is used to implement the method for superimposing data in digital images of arbitrary rectangular sizes based on FPGA as described above, including a management and control processing board, the management and control processing board includes an FPGA, the FPGA realizes data interaction with the DSP through the EMIF bus, and the FPGA is respectively connected to the data recorder and the imager through the LVDS interface.

[0035] Compared with the prior art, the beneficial effect of the present application is that the present application integrates the image information of the imager and the aircraft status information into one image for recording, thereby realizing unified storage and management of data. The user can view the image data and the aircraft status information simultaneously in one record file, thereby facilitating comprehensive analysis and diagnosis of the aircraft's operation status, avoiding the inconvenience caused by recording image data and status information separately, improving data utilization efficiency and analysis accuracy, and meeting the increasingly high requirements for aircraft monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of the steps of a method for superimposing data in a digital image of any rectangular size based on FPGA provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of the structure of a system for superimposing data in a digital image of any rectangular size based on FPGA provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of pixel distribution of an imager image of a rectangular size provided in an embodiment of the present application;

[0039] Figure 4 A flowchart of the steps of an implementation method of a method for overlaying data within a digital image of arbitrary rectangular size based on FPGA provided in an embodiment of the present application.

[0040] The text annotations in the figure represent:

[0041] 1. Imager; 2. Data recorder; 3. DSP; 4. FPGA; 5. Imager image; 6. Replacement area. DETAILED DESCRIPTION

[0042] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0043] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] Example 1

[0045] As mentioned in the background technology, in view of the problems in the prior art, the present application proposes a method for superimposing data in a digital image of arbitrary rectangular size based on FPGA4, comprising the following steps:

[0046] S1, initializing the rectangular size of the imager image 5 and the replacement area parameters of the replacement area 6 in the imager image 5; the imager image 5 is composed of a plurality of pixel arrangements;

[0047] S2, receiving and buffering image information of the imager and aircraft status information; wherein the image information of the imager includes at least the pixels at each buffering moment, and each buffering moment corresponds to the aircraft status information at the buffering moment;

[0048] S3, when all the imager image information of the imager image 5 is received and cached, output the information to the data recorder 2;

[0049] The output of information to the data recorder 2 specifically includes the following steps:

[0050] S3-1, reading the pixel at each cache moment, and determining whether the pixel is within the replacement area 6;

[0051] S3-2, if the pixel is in the replacement area 6, outputting the aircraft state information corresponding to the pixel at the cache time to the data recorder 2;

[0052] S3-3, if the pixel is not in the replacement area 6, outputting the image information of the imager including at least the pixel to the data recorder 2;

[0053] S3-4, traverse all pixels of the imager image 5 until the count of the pixels meets the total number of pixels corresponding to the rectangle size.

[0054] Further, the rectangular size of the imager image 5 is (M, N), the rectangular size represents the image coordinate system, the upper left corner of the imager image 5 is the origin of the image coordinate system, each row of the imager image 5 has M pixels, and there are N rows of pixels in total;

[0055] The counting of pixels until it matches the total number of pixels corresponding to the size of the rectangle specifically includes the following steps:

[0056] Until the total number of all the pixels is the product of M and N.

[0057] For further information, please refer to Figure 3 As shown, the replacement area parameters include the starting position of the replacement area 6, the starting position at least includes the starting coordinates of the starting pixel in the image coordinate system, and the size information of the replacement area 6, and the size information includes the area length and the area width.

[0058] Optionally, the starting pixel is A, and the starting coordinate is (M A ,N A ), the region length is L, the region width is W, and the end pixel is B, such as Figure 3 .

[0059] Furthermore, after the initialization of the rectangular size of the imager image 5 and the replacement area parameters of the replacement area 6 in the imager image 5, the following steps are also included:

[0060] According to the rectangle size and the replacement area parameter, a feature coordinate set is obtained, wherein the feature count set includes the feature coordinates of all the pixels in the replacement area 6;

[0061] The step of determining whether the pixel is within the replacement area 6 specifically includes the following steps:

[0062] The coordinates of the pixel are obtained, and it is determined whether the pixel is within the replacement area 6 according to whether the coordinates of the pixel have a matching item in the feature coordinate set.

[0063] Furthermore, if the pixel is in the replacement area 6, outputting the aircraft state information corresponding to the pixel at the cache time to the data recorder 2 specifically includes the following steps:

[0064] If the coordinates of the pixel have matching feature coordinates in the feature coordinate set, the aircraft state information corresponding to the pixel at the cache time is output to the data recorder 2 .

[0065] If the pixel is not in the replacement area 6, outputting the image information of the imager including at least the pixel to the data recorder 2 specifically comprises the following steps:

[0066] If the coordinates of the pixel do not have matching feature coordinates in the feature count set, the imager image information including at least the pixel is output to the data recorder 2 .

[0067] Furthermore, obtaining a feature coordinate set according to the rectangle size and the replacement area parameter specifically includes the following steps:

[0068] The coordinates whose horizontal coordinates are greater than or equal to the horizontal coordinates of the starting coordinates and less than the sum of the horizontal coordinates of the starting coordinates and the length of the area, and whose vertical coordinates are greater than or equal to the vertical coordinates of the starting coordinates and less than the sum of the vertical coordinates of the starting coordinates and the width of the area are calibrated as the characteristic coordinates, thereby obtaining the characteristic coordinate set.

[0069] Optionally, if the horizontal coordinate of the pixel is greater than or equal to M A and less than M A +L, and the ordinate is greater than or equal to N A and less than N A +W, then the coordinates of the pixel are the feature coordinates and are within the feature coordinate set.

[0070] In a preferred embodiment, the method further comprises a method for determining replacement area parameters of the replacement area 6 in the imager image 5, the determination method comprising the following steps:

[0071] A non-important area of ​​the imager image 5 is selected as the replacement area 6, and the parameters of the non-important area are the replacement area parameters. The non-important area at least includes four corners and four edges of a rectangular size.

[0072] Specifically, the non-key area can be understood as an area that will not affect the overall presentation of the imager image 5 and the user's perception, and can be four corners, an upper row, a lower row, a left column, and a right column.

[0073] Optionally, the image data has a fixed format, generally 640 pixels per line, for a total of 513 lines, ie, M is 640 and N is 513.

[0074] Furthermore, the above provides a method for determining replacement area parameters, which can meet most usage scenarios, but has poor flexibility. Now another method for determining replacement area parameters is proposed. Specifically:

[0075] Analyze all the pixels in the imager image 5 using image recognition technology to obtain pixel features of each pixel, wherein the pixel features include at least brightness value and color value;

[0076] Taking a region composed of all adjacent pixels whose similarity of pixel features is higher than a first preset threshold as a candidate region;

[0077] Specifically, if the pixel features of pixels in a region are very similar, it means that the color and brightness are similar, which means that this region may be a large open area that does not contain important information, such as a large grassland or a large sky. In this case, this region can be used as a candidate region.

[0078] If the size of the candidate area meets the preset size, the candidate area is used as the replacement area 6, and the parameters of the candidate area are the replacement area parameters.

[0079] If the area of ​​the candidate region is sufficient, these empty spaces can be used as replacement regions 6 , which will not affect the user's visual perception of the image 5 of the imager.

[0080] By initializing the rectangular size and replacement area parameters of the imager image 5, the parameters of the replacement area 6 in the image for displaying the aircraft status information can be flexibly set according to actual needs. A relatively empty or less important area can be selected as the replacement area 6 to reduce interference with the main content of the image. At the same time, different replacement areas 6 can be set for different application scenarios to meet diverse needs.

[0081] Example 2

[0082] On the basis of Example 1, this example proposes a system for superimposing data in a digital image of any rectangular size based on FPGA4, which is used to implement the method for superimposing data in a digital image of any rectangular size based on FPGA4 as described above, including a control and processing board, wherein the control and processing board includes FPGA4, FPGA4 realizes data interaction with DSP3 through EMIF bus, and FPGA4 is respectively connected to data recorder 2 and imager 1 through LVDS interface.

[0083] Specifically, FPGA4 (Chinese name: Field Programmable Gate Array) has high-speed parallel processing capability and flexibility, which can meet the requirements of fast processing and real-time output of image data. At the same time, the continuous advancement of digital image technology enables the imager 1 to output high-resolution image data, providing a basis for the fusion of image data and status information.

[0084] DSP3 (Chinese name: Digital Signal Processor) can perform complex digital signal processing algorithms and can accurately collect and process aircraft status information. The combination of FPGA4 and DSP3 can achieve efficient data processing and interaction, providing technical support for the implementation of this solution.

[0085] LVDS (Chinese name: Low Voltage Differential Signal) is a high-speed digital signal transmission technology that uses low-voltage differential signal transmission. It has the advantages of low power consumption, low noise, and high speed. It is often used in high-speed data transmission, such as in image transmission, communication and other fields.

[0086] The high-speed parallel processing capability of FPGA4 and the fast data transmission characteristics of the LVDS interface enable the solution to quickly receive, cache and output image data. At the same time, the participation of DSP3 can realize the real-time acquisition and processing of aircraft status information, improving the response speed and real-time performance of the system. It traverses all pixels of the image for judgment and output to ensure the accuracy and integrity of the data. This solution provides more comprehensive information for aircraft monitoring. By superimposing the aircraft status information in the image, you can intuitively understand the status of the aircraft at a specific moment, such as position, speed, attitude, etc. This helps to detect abnormal conditions of the aircraft in a timely manner and improve flight safety. It is also of great significance for the research and development and debugging of aircraft. By analyzing the recorded images and status information, the performance and control algorithm of the aircraft can be optimized.

[0087] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and its core ideas of this application. The above is only the preferred implementation method of this application. It should be pointed out that due to the limitations of textual expression and the objective existence of infinite specific structures, ordinary technicians in this technical field can make several improvements, modifications or changes without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the scope of protection of this application.

Claims

1. A method for superimposing data in a digital image of arbitrary rectangular size based on FPGA, characterized in that: The following steps are involved: Initializing the rectangular size of an imager image (5) and replacement area parameters of a replacement area (6) within the imager image (5); the imager image (5) is composed of a plurality of pixel arrangements; Receive and cache image information of the imager and aircraft status information; wherein the image information of the imager includes at least the pixels at each cache moment, and each cache moment corresponds to the aircraft status information at the cache moment; When all the imager image information of the imager image (5) has been received and cached, outputting the information to the data recorder (2); The step of outputting information to the data recorder (2) specifically comprises the following steps: Reading the pixel at each cache moment, and determining whether the pixel is within the replacement area (6); If the pixel is within the replacement area (6), outputting the aircraft state information corresponding to the pixel at the cache time to the data recorder (2); If the pixel is not within the replacement area (6), outputting the image information of the imager including at least the pixel to the data recorder (2); All pixels of the imager image (5) are traversed until the count of the pixels matches the total number of pixels corresponding to the size of the rectangle.

2. The method for superimposing data in a digital image of arbitrary rectangular size based on FPGA according to claim 1, characterized in that: The rectangular size of the imager image (5) is (M, N), the rectangular size represents the image coordinate system, the upper left corner of the imager image (5) is the origin of the image coordinate system, each row of the imager image (5) has M pixels, and there are N rows of pixels in total; The counting of pixels until it matches the total number of pixels corresponding to the size of the rectangle specifically includes the following steps: Until the total number of all the pixels is the product of M and N.

3. The method for superimposing data in a digital image of arbitrary rectangular size based on FPGA according to claim 2, characterized in that: The replacement area parameters include the starting position of the replacement area (6), the starting position at least including the starting coordinates of the starting pixel in the image coordinate system, and the size information of the replacement area (6), the size information including the area length and the area width.

4. The method for superimposing data in a digital image of arbitrary rectangular size based on FPGA according to claim 2, characterized in that: After the initialization of the rectangular size of the imager image (5) and the replacement area parameters of the replacement area (6) in the imager image (5), the following steps are also included: According to the rectangular size and the replacement area parameter, a feature coordinate set is obtained, wherein the feature count set includes the feature coordinates of all the pixels in the replacement area (6); The step of judging whether the pixel is within the replacement area (6) specifically comprises the following steps: The coordinates of the pixel are obtained, and whether the pixel is within the replacement area (6) is determined based on whether the coordinates of the pixel have a matching item in the feature coordinate set.

5. The method for superimposing data in a digital image of arbitrary rectangular size based on FPGA according to claim 3, characterized in that: If the pixel is within the replacement area (6), outputting the aircraft state information corresponding to the pixel at the cache time to the data recorder (2) specifically comprises the following steps: If the coordinates of the pixel have matching characteristic coordinates in the characteristic coordinate set, the aircraft state information corresponding to the pixel at the cache time is output to the data recorder (2).

6. The method for superimposing data in a digital image of arbitrary rectangular size based on FPGA according to claim 3, characterized in that: If the pixel is not in the replacement area (6), outputting the image information of the imager including at least the pixel to the data recorder (2) specifically comprises the following steps: If the coordinates of the pixel do not have matching feature coordinates in the feature count set, the imager image information including at least the pixel is output to the data recorder (2).

7. The method for superimposing data in a digital image of arbitrary rectangular size based on FPGA according to claim 3, characterized in that: The step of obtaining a feature coordinate set according to the rectangle size and the replacement area parameter specifically includes the following steps: The coordinates whose horizontal coordinates are greater than or equal to the horizontal coordinates of the starting coordinates and less than the sum of the horizontal coordinates of the starting coordinates and the length of the area, and whose vertical coordinates are greater than or equal to the vertical coordinates of the starting coordinates and less than the sum of the vertical coordinates of the starting coordinates and the width of the area are calibrated as the characteristic coordinates, thereby obtaining the characteristic coordinate set.

8. The method for superimposing data in a digital image of arbitrary rectangular size based on FPGA according to claim 1, characterized in that: The method also comprises a method for determining replacement area parameters of a replacement area (6) within the imager image (5), the determination method comprising the following steps: A non-important area of ​​the imager image (5) is selected as a replacement area (6), the parameters of the non-important area being the replacement area parameters, and the non-important area at least includes four end corners and four edges of a rectangular size.

9. The method for superimposing data in a digital image of arbitrary rectangular size based on FPGA according to claim 1, characterized in that: The method also comprises a method for determining replacement area parameters of a replacement area (6) within the imager image (5), the determination method comprising the following steps: Analyzing all the pixels in the imager image (5) using image recognition technology to obtain pixel features of each pixel, wherein the pixel features at least include brightness value and color value; Taking a region composed of all adjacent pixels whose similarity of pixel features is higher than a first preset threshold as a candidate region; If the size of the candidate area meets the preset size, the candidate area is used as the replacement area (6), and the parameters of the candidate area are the replacement area parameters.

10. A system for overlaying data in a digital image of arbitrary rectangular size based on FPGA, used to implement the method for overlaying data in a digital image of arbitrary rectangular size based on FPGA as claimed in any one of claims 1 to 9, characterized in that: The control and processing board comprises an FPGA (4), the FPGA (4) realizes data interaction with a DSP (3) via an EMIF bus, and the FPGA (4) is respectively connected to a data recorder (2) and an imager (1) via an LVDS interface.

Images