A Video Character Superposition Method Based on FPGA Time Series

Abstract

The present invention belongs to the application field of image processing technology in FPGA, and particularly relates to a method for realizing character superposition in a time series manner on an FPGA platform. The method includes the following steps: Step 1: The host computer module configures the superposition mode through RS232; Step 2: Establish logic generation modules for each common character based on the time series; Step 3: Call each character module to be displayed in the video stream. Based on the FPGA timing sequence logic, the method designs modules for common characters in the way of offsetting the row and column address positions, and calls the timing modules of each character according to the needs of the display content to realize the complete superposition of the character string on the video image.

Description

Technical Field

[0001] The present invention belongs to the application field of image processing technology in FPGA, and relates to a video character overlay method based on FPGA time series. Background Art

[0002] With the development of integrated circuit technology, the processing power of high-performance processors is getting stronger and stronger, and various complex image and video processing functions can be realized, such as image acquisition, image processing, video compression, video storage, character overlay, etc. The character overlay function is widely used in the fields of image annotation and video surveillance. By overlaying necessary characters in video data, the annotation of key positions in images or key frames of videos can be realized, which is of great significance for obtaining key information and analyzing images. Therefore, character overlay has become an essential function in fields such as video surveillance. At present, there are many and relatively mature ways to implement character overlay. For example, chips such as ARM, DSP, and FPGA can all realize the real-time processing of this function. For character overlay without real-time requirements, it is more simple and fast to implement it in a pure software manner.

[0003] The character overlay method based on DSP has been maturely applied at home and abroad. For example, in foreign countries, there are solutions such as TMS320DM8148 and TMS320DM8168 of Texas Instruments in the United States. This chip has a built-in character overlay OSD (On-Screen Display) module. By simply calling and configuring the module, characters or pictures can be overlaid on high-definition videos. In China, there are high-performance ARM processors such as Hi3516 and Hi3559 of Huawei HiSilicon. Their applications in the security field are relatively mature. They have built-in character overlay OSD modules based on the Linux operating system, and can overlay the required characters or pictures on each video while collecting and displaying multiple videos. The function interface is simple and the configuration is flexible.

[0004] The character overlay method based on FPGA is mainly applied in solutions with FPGA as the main processing module. While realizing high-speed image or video acquisition, FPGA realizes character overlay through logic. In traditional solutions, the font library is generally stored in external FLASH or internal BRAM. The size and font of characters in the font library are fixed. When display is required, the font library is accessed to call the corresponding characters for display. The pixel value at the position where characters need to be displayed in the font library is 255, and the position where no display is required is 0. Using this method, only a single FPGA chip is needed to implement, and the solution is simple and has strong real-time performance. However, its programming implementation logic is complex, the internal RAM resources are consumed greatly, and the function is single. Implementing advanced functions brings great challenges to programming and debugging.

[0005] Chinese Patent CN104023182A discloses a character overlay device and an overlay method based on FPGA. The implementation of this patent requires reading characters from a font library. Patent CN202772993U discloses a device for quickly loading characters in character overlay, which is connected to an FPGA chip through the parallel interface of a font library chip, effectively improving the character loading speed. Patent CN207530948U relates to a video character overlay system, in which characters are stored in a block RAM memory. A method for character overlay based on FPGA disclosed in Patent CN111311479A stores characters in a read-only memory. Summary of the Invention

[0006] (1) Objectives of the Invention

[0007] The objective of the present invention is to provide a video character overlay method based on the time sequence of FPGA, which does not require an external storage module or the BRAM resource inside the FPGA to establish a font library, but makes full use of its LUT unit to establish the dynamic display logic of characters in the form of time sequence, and embeds characters in a pipelined manner during the video display process to achieve high-speed, efficient, simple, fast and resource-saving character overlay.

[0008] (2) Technical Solutions

[0009] To solve the above technical problems, the present invention provides a method for character overlay based on the time sequence of FPGA. The method is based on the timing sequence logic of FPGA, designs modules for common characters in the way of offset of row and column addresses, and calls the timing modules of each character according to the needs of the display content to achieve the complete overlay of the string on the video image.

[0010] The method includes the following steps:

[0011] Step 1: The host computer module configures the overlay mode through RS232.

[0012] Step 2: Establish logic generation modules for each common character based on the time sequence.

[0013] Step 3: Call each character module to be displayed in the video stream.

[0014] Among them, in Step 1, the host computer module communicates with the FPGA through the RS232 serial port, and the RS232 serial port uses a DB9 connector; a logic module for serial communication is designed in the FPGA.

[0015] The character overlay mode can be configured through the upper computer serial port communication: sending a character switch instruction can turn on or off the character overlay function; sending a starting address command can configure the starting position of the character in the image pixels; sending a font color instruction can configure the display color of the font in the image; sending a character length instruction configures how many characters need to be displayed; the character content instruction configures the character content to be displayed.

[0016] Among them, in the said step 2, it includes the following sub-steps:

[0017] Step 21: Generate the dot matrix arrangement of the required characters by the font tool module. The dot matrix arrangements are different for different character sizes.

[0018] Step 22: Generate the pixel clock of the video by the FPGA phase-locked loop PLL. Taking this clock as the reference and the line signal as the auxiliary, perform row and column counting respectively according to the video image resolution size, and the starting relative addresses of the row and column counting are both 0.

[0019] Step 23: When the row and column counters count to the corresponding position of the dot matrix of the character, set the pixel value at this position to 255, and keep the pixel values of other positions unchanged as the original image pixel values. Advance in the above pipeline manner until all the dot matrix positions of the entire character are covered.

[0020] Step 24: Repeat the above steps to write corresponding character generation modules for common characters such as 0-9, a-z, A-Z, and punctuation marks.

[0021] Among them, in the said step 21, different dot matrix arrangements are generated according to different character sizes. The display part is 1, and the display pixel value is 255 in actual application; the non-display part is 0, and the display pixel value is 0 in actual application. Common character size formats are 8×8, 16×16, etc.

[0022] Among them, in the said step 22, the position formula of the character dot matrix is:

[0023] Pixel point position = (row start + row count) × width of the image + (column start + column count);

[0024] Both the row start and column start positions are specified by the external interface.

[0025] Among them, in the said step 3, it includes the following sub-steps:

[0026] Step 31: Obtain the length of the character, the size of a single character, and the starting row and column addresses of the first character according to the instruction input by the external interface.

[0027] Step 32: Perform logical judgment according to the character content to be displayed and select the character module to be called.

[0028] Step 33: According to the total character length, when the character length exceeds the width of the image, automatically switch to the next character line, and the starting address becomes the line start address plus the height of a single character;

[0029] Step 34: For the characters in the same line, the starting position of the line remains the same. According to the width of a single character, the column position of the second character is the column address of the first character plus the width of a single character; and so on for subsequent characters;

[0030] Step 35: When outputting the video image, make a judgment according to the row and column addresses. When the position is a character position, output the character pixel data; when the position is not a character, output the pixel data of the original image.

[0031] (III) Beneficial effects

[0032] The video character overlay method based on the FPGA time series provided by the above technical solution has the following beneficial effects:

[0033] (1) Compared with the traditional method of storing the font library using BRAM, generating characters by the method of time series counting only requires a small amount of LUT resources for the entire module, with a simple implementation logic, strong replicability, and does not occupy a large amount of BRAM resources or external storage modules.

[0034] (2) This method adopts a pipeline method, and the delay time is at the pixel clock level, with strong real-time performance.

[0035] (3) This implementation method has no limitation on the image resolution size. It only needs to specify the starting position of the character to be implemented, with a simple configuration interface and does not require complex system configuration of the DSP platform. Description of the drawings

[0036] Figure 1 is a flowchart for configuring the overlay mode through the RS232 serial port.

[0037] Figure 2 is an example of the character dot matrix arrangement generated by the font tool module.

[0038] Figure 3 is a program flowchart for generating a single character using row and column counting.

[0039] Figure 4 is a flowchart for overlaying all characters as required. Detailed implementation manners

[0040] To make the objectives, contents, and advantages of the present invention clearer, the following further describes in detail the specific implementation manners of the present invention with reference to the drawings and embodiments.

[0041] This embodiment is based on a time - series FPGA character overlay method. Based on the FPGA timing sequence logic, the method designs modules for common characters in the way of offsetting the row - column address positions. According to the needs of the display content, it calls the timing modules of each character to achieve the complete overlay of the string on the video image.

[0042] The video character overlay method based on the FPGA time series in this embodiment includes the following steps:

[0043] Step 1: The host computer module configures the overlay mode through RS232.

[0044] Step 2: Establish logic generation modules for each common character based on the time series.

[0045] Step 3: Call each character module to be displayed in the video stream.

[0046] Combined with Figure 1 As shown, the host computer module communicates with the FPGA through the RS232 serial port, and the RS232 serial port uses a DB9 connector; a logic module for serial communication is designed in the FPGA.

[0047] After the system is powered on, the host computer sends the configuration parameters required for character overlay to the serial communication module of the FPGA. Sending a character switch instruction can turn on or off the character overlay function; sending a starting address command configures the starting position of the character in the image pixels; sending a font color instruction can configure the display color of the font in the image; sending a character length instruction configures how many characters need to be displayed; sending a character content instruction configures the character content to be displayed.

[0048] Combined with Figure 2 As shown, for the generation of the character module, first, the dot - matrix arrangement of the required character is generated by the dot - matrix tool module. Different character sizes have different dot - matrix arrangements; according to different character sizes, different dot - matrix arrangements are generated, with the displayed part being 1 and the non - displayed part being 0; common character size formats such as CharWidth×CharHeight are 8×8, 16×16, etc.

[0049] Combined with Figure 3 As shown, the pixel clock of the video image is generated by the FPGA phase - locked loop PLL. Based on this clock and with the row signal as an auxiliary, row - column counting is performed according to the size of the video image resolution, which are RowCnt and ColCnt respectively, and the starting relative addresses of both row - column counting are 0.

[0050] According to Figure 2For the dot matrix arrangement of the character generated by the character mold tool module, when the row and column counter counts to the corresponding position of the dot matrix, the pixel value at this position is set to 255, and the pixel values at other positions remain unchanged as the pixel values of the original image. Through the above pipeline method, it progresses until all 8×8 or 16×16 dot matrix positions of the entire character are covered. Thus, the design of the single-character generation module is completed.

[0051] Among them, the calculation formula for the position CharPos of the character dot matrix is:

[0052] CharPos = (RowStart + RowCnt) × ImageWidth + (ColStart + ColCnt);

[0053] Both the row start address RowStart and the column start address ColStart are specified by the external interface.

[0054] Repeat the above steps to write the corresponding character generation modules for common characters such as 0 - 9, a - z, A - Z, and punctuation marks; through this method, an FPGA generation module for common characters can be established. To display a certain character, only this module needs to be called.

[0055] Combined with Figure 4 As shown, when receiving the open character overlay instruction sent by the host computer, according to the instruction input by the external interface, obtain the length of the character, the size of a single character, and the starting row and column addresses of the first character; then make a logical judgment based on the character content to be displayed and select the character module to be called.

[0056] According to the length of the entire string to be displayed, when this length exceeds the width of the image, it automatically switches to the next character row, and the starting address becomes the row start address plus the height of a single character CharHeight. The calculation formula becomes:

[0057] CharPos = (RowStart + RowCnt + CharHeight) × ImageWidth + (ColStart + ColCnt);

[0058] For the characters in the same row, the row starting position remains the same. According to the width of a single character, the column position of the second character is the column address of the first character plus the width of a single character; and so on for subsequent characters.

[0059] When the video image is output in a pipeline, call the character overlay module designed by the present invention. Make a judgment based on the row and column addresses. When this position is a character position, output the character pixel data, and when this position is not a character, output the pixel data of the original image.

[0060] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for video character overlay based on FPGA time series, characterized in that, The method is based on the FPGA time series logic. It designs modules for common characters in the way of offsetting the row and column addresses. According to the needs of the display content, it calls the timing modules of each character to achieve the complete superposition of the string in the video image; The method includes the following steps: Step 1: The host computer module configures the superposition mode through RS232; Step 2: Establish logic generation modules for each common character based on the time series; Step 3: Call each character module to be displayed in the video stream; In Step 1, the host computer module communicates with the FPGA through the RS232 serial port. The RS232 serial port uses a DB9 connector; a logic module for serial communication is designed in the FPGA; In Step 1, the superposition mode of characters is configured through the serial communication of the host computer: send a character switch instruction to turn on or off the character superposition function; send a start address command to configure the starting position of the character in the image pixels; send a font color instruction to configure the display color of the font in the image; send a character length instruction to configure how many characters need to be displayed; send a character content instruction to configure the character content to be displayed; Step 2 includes the following sub-steps: Step 21: Generate the dot matrix arrangement of the required characters by the font tool module. The dot matrix arrangements are different for different character sizes; Step 22: Generate the pixel clock of the video by the FPGA phase-locked loop PLL. Based on this clock and with the row signal as the auxiliary, perform row and column counting respectively according to the size of the video image resolution, and the starting relative addresses of the row and column counting are both 0; Step 23: When the row and column counters count to the corresponding position of the dot matrix of the character, set the pixel value at this position to 255, and keep the pixel values of other positions unchanged as the pixel values of the original image; Advance in the above pipeline manner until the entire dot matrix position of the character is completely covered; Step 24: Repeat the above steps to write corresponding character generation modules for common characters; In Step 21, according to different character sizes, generate different dot matrix arrangements. The display part is 1, and the display pixel value is 255 in actual application; the non-display part is 0, and the display pixel value is 0 in actual application; In Step 21, common characters include 0-9, a-z, A-Z, and punctuation marks; In Step 21, the common character size formats are 8×8 and 16×16; In Step 22, the position formula of the character dot matrix is: Pixel position = (row start + row count) × width of the image + (column start + column count); Both the row start and column start positions are specified by the external interface; Step 3 includes the following sub-steps: Step 31: According to the instructions input by the external interface, obtain the length of the character, the size of a single character, and the starting row and column addresses of the first character; Step 32: Make a logical judgment according to the character content to be displayed and select the character module to be called; Step 33: According to the total character length, when the character length exceeds the width of the image, automatically switch to the next character line, and the starting address becomes the row start address plus the height of a single character; Step 34: For the characters in the same row, the starting position of the row remains consistent. According to the width of a single character, the column position of the second character is the column address of the first character plus the width of a single character; and so on for subsequent characters. Step 35: When outputting the video image, make a judgment based on the row and column addresses. When the position is a character position, output the character pixel data; when the position is not a character, output the pixel data of the original image.

Images