A multi-division video image processing display device and display terminal

Abstract

The application provides a multi-division video image processing display device, comprising an input matrix tracking decoding unit, a multi-channel video image frame loss processing unit, a multi-channel video image scaling processing unit and a write frame storage controller unit. The input matrix tracking decoding unit is used for outputting channel data of multiple channels after ITU input data is processed by input matrix and decoded by channel tracking; the multi-channel video image frame loss processing unit is used for receiving the multiple channel data and performing frame loss processing on each channel data according to needs; the multi-channel video image scaling processing unit is used for performing scaling processing on the input video image to make the image size meet the needs of multi-division display; and the write frame storage controller unit is used for issuing a request of write frame cache to the write frame storage controller, and the write frame storage controller writes the image data processed by the channel into the frame cache through a bus in sequence. The device realizes flexible and efficient division display under the condition of single display chip and on the premise of guaranteeing chip pin packaging and PCB wiring.

Description

Technical Field

[0001] This invention relates to the field of digital image processing technology, and in particular to a multi-segment video image processing and display device and display terminal. Background Technology

[0002] In fields such as in-vehicle multimedia-assisted driving and video surveillance, to better adapt to market development, it is necessary to display the images input from multiple cameras on the same terminal, achieving simultaneous display of two-split, four-split, or six-split images. Current technical solutions typically use ITU656 / 601 signals generated by the camera front-end as input to multi-split processing systems. These multi-channel ITU656 / 601 signals are then processed and synchronously displayed on a single terminal. Ordinary multi-channel ITU signal transmission requires multiple sets of ports to transmit multiple channels of ITU signals; that is, there is a one-to-one correspondence between the number of ports and the number of channels. This brings significant inconvenience and costs to chip pin packaging and PCB routing, and may even require multiple chips to achieve the fused split display function.

[0003] How to achieve flexible and efficient split display under the condition of a single display chip while ensuring chip pin packaging and PCB routing is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the technical problem to be solved by the present invention is to provide a multi-segment video image processing and display device and display terminal to overcome the shortcomings of the prior art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] This invention provides a multi-segment video image processing and display device, comprising: an input matrix tracking and decoding unit, a multi-channel video image frame dropping processing unit, a multi-channel video image scaling processing unit, and a write frame storage controller unit;

[0007] The input matrix tracking and decoding unit is used to process ITU input data through an input matrix and then output channel data for multiple channels after channel tracking and decoding.

[0008] The multi-channel video image frame dropping processing unit is used to receive data from multiple channels and perform frame dropping processing on each channel data as needed.

[0009] The multi-channel video image scaling processing unit is used to scale the input video image so that the image size meets the requirements of multi-segment display;

[0010] The write frame storage controller unit is used to send a write frame buffer request to the write frame storage controller, which then writes the processed image data of the channel into the frame buffer via the bus.

[0011] Furthermore, the input matrix tracking decoding unit includes: an input matrix subunit, multiple port channel tracking decoding subunits, and a channel selection matrix subunit;

[0012] The input matrix subunit is used to select the input signal and output it to the corresponding port channel tracking decoding subunit;

[0013] The port channel tracking and decoding subunit is used to track and decode the channel according to the channel information embedded in the ITU signal time base synchronization code in order to track the receiving end and the transmitting end channel in real time.

[0014] The channel selection matrix subunit is used to receive the channel signal output by the port channel tracking decoding subunit and output the corresponding number of valid channel data according to the number of segments.

[0015] Furthermore, the port channel tracking decoding subunit samples the input data using the port input clock and presets a channel counter at intervals based on the number of port multiplexed channels. When the channel counter equals 0, the input data is temporarily stored. The continuously stored channel data is then judged. After the synchronization header of the first channel signal is detected, the low-order bits of the channel information embedded after the synchronization code are compared. If the low-order bits are equal to 0, it proves that the port channel tracking decoding is correct. Otherwise, the channel counter is reset according to the value of the low-order bits, and the channel information is tracked again until the detected channel information is equal to 0, at which point the channel tracking decoding is considered to be correct.

[0016] Furthermore, the multi-channel video image frame loss processing unit includes: a multi-channel video frame synchronization control subunit, multiple channel control subunits, and multiple channel image output subunits;

[0017] The multi-channel video frame synchronization control subunit is used to generate a corresponding number of system reset signals, frame synchronization reset signals, and frame dropping control signals according to the number of segments. The system reset signal is determined by the system power-on time and soft reset. The frame synchronization reset signal is a synchronization pulse signal issued before frame dropping processing to synchronize the frame dropping interval of each channel. The frame dropping control signal is used to control the frame dropping frequency of each channel.

[0018] The channel control subunit is used to receive the signal output by the multi-channel video frame synchronization control subunit and the timing signal in the corresponding channel signal, and generate the channel output enable signal.

[0019] The channel image output subunit is used to receive the ITU video signal CH0 input to the corresponding channel, including the input horizontal and vertical synchronization signals, valid timing signals, and image data. Based on the enable signal output by the corresponding channel control subunit, it controls and selects the video signal to be processed by the corresponding channel for output.

[0020] Further, the channel control subunit sets the frame counter clear signal fcnt_clr as follows: (1) When the system reset signal output by the multi-channel video frame synchronization control subunit is received, the frame counter clear signal fcnt_clr is set to 0; (2) When the frame synchronization reset signal is received, the frame counter clear signal fcnt_clr is set to 1; (3) When fcnt_clr is equal to 1 and the timing signal field synchronization vsyn in the ITU video signal CH0 is equal to 1, fcnt_clr is set to 0; The channel control subunit has a built-in frame counter, which is set as follows: (1) When the system reset signal is received, the frame counter is set to 0; (2) When the timing signal field synchronization vsyn is equal to 1 and fcnt_clr is equal to 1, the frame counter is set to 0; (3) When the timing signal field synchronization vsyn is equal to 1 and fcnt_clr is not equal to 1, the frame counter is incremented by 1.

[0021] Furthermore, the multi-channel video image frame loss processing unit is configured with a 2 5 A frame display register of bits; each bit represents a frame display enable signal; when the frame display enable signal is 1, the output image data of the channel image output subunit is equal to the input image data; when the frame display enable signal is 0, it indicates that a frame should be dropped, and the output image data is equal to 0.

[0022] Furthermore, the multi-channel video image scaling processing unit includes: a scaling data acquisition subunit, a first scaling buffer, a second scaling buffer, a horizontal scaling control subunit, a third scaling buffer, a vertical scaling control subunit, a scaling processing control subunit, and a scaling logic operation unit; the scaling data acquisition subunit is used to convert the image signal in the ITU input pixel clock domain into image data in the system bus clock domain and send a scaling indication signal to the scaling processing control subunit; the scaling processing control subunit is used to coordinate and control the scaling process of the horizontal scaling subunit and the vertical scaling control subunit; the first scaling buffer and the second scaling buffer are used to cache the data output by the scaling data acquisition subunit for use by the horizontal scaling control subunit; the third scaling buffer is used to cache the data output by the horizontal scaling control subunit for use by the vertical scaling control subunit; the scaling logic operation unit is used to provide calculation operations for image-related pixel values ​​and weight values.

[0023] Furthermore, the output system bus clock frequency is an integer multiple of the input pixel clock frequency.

[0024] Furthermore, both the horizontal scaling control subunit and the vertical scaling control subunit employ a multi-phase interpolation algorithm or a bilinear interpolation algorithm for horizontal scaling.

[0025] This invention also provides a display device, including the multi-segment video image processing display device described above.

[0026] Compared with existing technologies, the present invention has the following advantages: The multi-segment video image processing and display device provided by the present invention changes the existing one-to-one correspondence between the number of ports and the number of channels, realizing a one-port multi-channel transmission mode. It can achieve efficient segmentation display under a single display chip condition, while ensuring chip pin packaging and PCB routing. Simultaneously, the image of each channel needs to be scaled to ensure that the image and video of each channel can be displayed perfectly on the multi-segment system according to the required ratio. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a structural diagram of a multi-segment video image processing and display device provided in an embodiment of the present invention;

[0029] Figure 2 This is a structural diagram of the input matrix tracking decoding unit;

[0030] Figure 3 This is a structural diagram of a multi-channel video image frame dropping processing unit;

[0031] Figure 4 This is a structural diagram of a multi-channel video image scaling processing unit. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] The input matrix tracking and decoding unit is used to process ITU input data through an input matrix and then output channel data for multiple channels after channel tracking and decoding.

[0034] The multi-channel video image frame dropping processing unit is used to receive data from multiple channels and perform frame dropping processing on each channel data as needed.

[0035] The multi-channel video image scaling processing unit is used to scale the input video image so that the image size meets the requirements of multi-segment display;

[0036] The write frame storage controller unit is used to send a write frame buffer request to the write frame storage controller, which then writes the processed image data of the channel into the frame buffer via the bus.

[0037] like Figure 1 The diagram shown is an overall structural diagram of a multi-segment video image processing and display device according to an embodiment of the present invention. It mainly includes: an input matrix tracking and decoding unit, a multi-channel video image frame dropping processing unit, a multi-channel video image scaling processing unit, and a write frame storage controller unit.

[0038] The input matrix tracking decoding unit receives multi-port ITU656 / 601 input data ITUAIN, ITUBIN, and ITUCIN. After input matrix processing and channel tracking decoding, the output port actually includes channel data CH0, CH1, CH2, CH3, CH4, and CH5 from multiple channels. This embodiment assumes 3 input ports and supports 6 output channels; however, in practical applications, the number of input ports and channels is not limited to these and can be adapted. A more detailed structure of the input matrix tracking decoding unit is shown below. Figure 2 As shown, it includes: an input matrix subunit, three port channel tracking and decoding subunits, and a channel selection matrix subunit. The three port channel tracking and decoding subunits are, in order: port A channel tracking and decoding subunit, port B channel tracking and decoding subunit, and port C channel tracking and decoding subunit.

[0039] The input matrix sub-unit is mainly for the convenience of PCB board-level routing. The input signal selection of the port configurable matrix is ​​as follows: ITUAIN1 can be arbitrarily selected as one of the ports of ITUAIN, ITUBIN, and ITUCIN. ITUBIN1 can be arbitrarily selected as one of the two ports after ITUAIN1 is selected. ITUCIN1 is the remaining port after ITUAIN1 and ITUBIN1 are selected. The matrix parameters can be configured by the system.

[0040] The Port A channel tracking and decoding subunit receives the port signal ITUAIN1. This invention assumes that Port ITUAIN1 can transmit up to four channels of ITU656 data. The Port A channel tracking and decoding subunit is responsible for hardware-based channel tracking and decoding of the four channels in ITUAIN1. In single-port multi-channel transmission mode, there is an asynchronous relationship between the ITU receiver and transmitter systems, including inconsistent power-on times. Direct downsampling by the ITU receiver based on the channel multiplexing number may cause channel misalignment. For example, the downsampled first channel signal A_CH0 may not correspond to the first channel signal of the transmitter and may be misjudged as data from the second, third, or fourth channel. This can lead to indeterminate and disordered image positions during multi-segment display, requiring significant software resource consumption for subsequent software channel adjustments. This module tracks and decodes the channels based on the channel information embedded in the ITU signal time base synchronization code during channel multiplexing, enabling real-time tracking of the receiver and transmitter channels. Specifically, ITUAIN1 is sampled using the clock input at the port. A channel counter CNT is preset at intervals equal to the number of multiplexed channels. When CNT equals 0, ITUAIN1 is temporarily stored. The continuously stored channel data (presumably the first channel) is then judged. When the synchronization header of the first channel signal is detected (for example, 0xFF0000XY), the lower bit Y of the channel information XY embedded after the synchronization code is compared. If Y equals 0, it proves that the port channel tracking decoding is correct; otherwise, the channel counter is reset according to the value of Y, and the channel information is tracked again until the detected channel information equals 0, at which point the channel tracking decoding is considered correct. In this embodiment, 4 channels are transmitted through port A as an example. When CNT equals 0, A_CH0 is output; when CNT equals 1, A_CH1 is output; when CNT equals 2, A_CH2 is output; and when CNT equals 3, A_CH3 is output. The port A channel tracking decoder can also work in a mode that transmits only two channels, one-to-one with port B and port C. The principle is the same as the four-channel mode, but the channel counter only needs to count 0 and 1. In this case, outputs A_CH1 and A_CH2 are valid, while A_CH2 and A_CH3 are invalid.

[0041] The working principle of the port B channel tracking decoding subunit is the same as that of the port A channel tracking decoding subunit. The difference is that the output port supports two channels, B_CH0 and B_CH1. During tracking decoding, the channel counter only needs to count 0 and 1.

[0042] The working principle of the port C channel tracking decoding subunit is the same as that of the port A channel tracking decoding subunit. The difference is that the output port supports two channels, C_CH0 and C_CH1. During tracking decoding, the channel counter only needs to count 0 and 1.

[0043] The port B channel tracking and decoding subunit and the port C channel tracking and decoding subunit will not be described in detail. After the port channel tracking and decoding is performed using the hardware method of this invention, the problems of image position instability and disorder that may occur in multi-segment displays due to conventional methods can be solved, reducing the huge software resource consumption required for software adjustment of display channels. Since each line synchronization header is detected, even if the system is interfered with, this module can detect the correct channel information in the shortest possible time.

[0044] The channel selection matrix subunit receives the channel signals A_CH0, A_CH1, A_CH2, A_CH3, B_CH0, B_CH1, C_CH0, and C_CH1 output by the tracking decoding subunit at channels A / B / C. It can output up to six valid channels (CH0, CH1, CH2, CH3, CH4, and CH5) depending on actual needs. For example, optional six-segment configurations include: CH0 = A_CH0, CH1 = A_CH1, CH2 = A_CH2, CH3 = A_CH3, CH4 = B_CH0, CH5 = B_CH1; or CH0 = A_CH0, CH1 = A_CH1, CH2 = B_CH0, CH3 = B_CH, CH4 = C_CH0, CH5 = C_CH1. Of course, the matrix selection method is not limited to this; it can be flexibly selected according to multi-port multi-channel or single-port multi-channel transmission methods.

[0045] After the ITU data is processed by the input matrix tracking decoding unit, the chip that supports multi-segment video image processing can not only support multi-port multi-channel, but also single-port multi-channel transmission. The hardware performs real-time channel decoding and tracking of the port input ITU data, which improves the efficiency of channel decoding and tracking for multiple camera input split display, making the real-time multi-segment image display more accurate and stable.

[0046] This multi-channel video image frame dropping processing unit receives multiple channels of CH0, CH1, CH2, CH3, CH4, and CH5 signals. It can perform frame dropping processing on the data from each channel as needed, and then output multi-channel signals CH0_SI, CH1_SI, CH2_SI, CH3_SI, CH4_SI, and CH5_SI. In multi-segment video image processing, the frame rates of the input and output video images may differ. Some applications also require recording and saving multiple channels of video. To ensure that the video from each channel is preserved relatively completely, when the system's data processing capacity reaches a bottleneck, if the frame rate is too high, it may be impossible to process the data. Frame dropping processing is a relatively effective choice without affecting video smoothness. The existing processing method is that after the ITU signal is decoded, scaled, and written back to the frame storage, the software responds to the interrupt and determines whether to drop the image frame written back to the storage. Although this saves some resources by eliminating the need for subsequent processing units to process the dropped frame, the processing and writing back process itself consumes a significant amount of bandwidth and power. This application, combined with multi-channel, multi-segment display, performs frame dropping processing on multiple video channels before the video images are written back, such as... Figure 3 The diagram shows the specific structure of the multi-channel video image frame loss processing unit. It mainly includes: a multi-channel video frame synchronization control subunit, six channel control subunits, and six channel image output subunits. The six channel control subunits are, in order, channel 0 control subunit, channel 1 control subunit, channel 2 control subunit, channel 3 control subunit, channel 4 control subunit, and channel 5 control subunit; the six channel image output subunits are, in order, channel 0 image output subunit, channel 1 image output subunit, channel 2 image output subunit, channel 3 image output subunit, channel 4 image output subunit, and channel 5 image output subunit.

[0047] The multi-channel video frame synchronization control subunit generates system reset signals, frame synchronization reset signals, and frame drop control signals for six channels (channels 0 to 5) according to system application requirements. The system reset signal is determined by the system power-on time and soft reset. The frame synchronization reset signal is a synchronization pulse signal issued by software to synchronize the frame drop intervals of each channel before frame drop processing is required. The frame drop control signal is also configured by software and can control the frame drop frequency of each channel.

[0048] The Channel 0 control subunit receives the output signal from the multi-channel video frame synchronization control subunit and the timing signal from Channel 0 signal CH0, and generates the Channel 0 output enable signal. Specifically, the frame counter clear signal fcnt_clr is generated as follows: ① When the system reset signal output by the synchronization control module is received, the frame counter clear signal fcnt_clr is set to 0; ② When the frame synchronization reset signal is received, the frame counter clear signal fcnt_clr is set to 1; ③ When fcnt_clr is equal to 1 and the field synchronization vsyn in CH0 is equal to 1, fcnt_clr is set to 0.

[0049] The channel 0 control subunit has a built-in frame counter, which operates as follows: 1) Upon receiving the system reset signal from the multi-channel video frame synchronization control subunit, the frame counter is set to 0; 2) When the timing signal field synchronization vsyn equals 1 and fcnt_clr equals 1, the frame counter is set to 0; 3) Otherwise, when vsyn equals 1, the frame counter is incremented by 1. The frame counter width is set to 5 bits, and overflow is ignored. Since the camera video frame rate is generally 25 or 30 frames per second, this application sets a 2-bit frame counter to correspond to the frame counter. 5 The frame display register frame_en_num0 has a default value of 0xffffffff on power-on reset, indicating that all frames are displayed within a 32-frame interval without dropping any frames. If half of the frames are to be dropped within a 32-frame interval, the frame display register can be configured to 0x55555555. This can be understood as bit 0 in the frame display register representing the first processed frame and bit 31 representing the 32nd processed frame within 32 frames. If sequentially arranged video frames are to be dropped, this bit is set to 0. When the frame counter value is equal to the corresponding bit value in the frame display register, it indicates that frame dropping is required and a frame display enable signal frame_ratio_en is generated as 0; otherwise, frame_ratio_en is generated as 1 and no frames are dropped.

[0050] Since the frame counters change when the timing signal field synchronization vsyn equals 1, frame_ratio_en is perfectly aligned with the start of each video frame. It is entirely feasible to use this signal to control whether the channel data of the current channel is dropped.

[0051] The Channel 0 image output subunit receives the ITU video signal CH0 input from Channel 0, including the input horizontal and vertical sync, valid timing signal, and image data. Based on the frame display enable signal frame_ratio_en output by the Channel 0 control subunit, it controls and selects the video signal to be processed by Channel 0 for output. Specifically, when frame_ratio_en is 1, the output horizontal and vertical sync and valid timing signal are equal to the input horizontal and vertical sync and valid timing signal, and the output image data is equal to the input image data; when frame_ratio_en is 0, it indicates that a frame should be dropped, the output horizontal and vertical sync and valid timing signal are equal to 0, and the output image data is equal to 0.

[0052] Since `frame_ratio_en` is implemented by a software-configurable frame display register, and its changes are perfectly aligned with the start and end of image frames, after selection processing, video images can achieve frame dropping at any configurable interval, and valid and invalid frames can be seamlessly switched in hardware. When frame dropping is selected, the output data is all 0, and the subsequent video processing module can enter a low-power mode. Video data does not need to be written to the frame memory, thus reducing power consumption and alleviating the system's bandwidth requirements.

[0053] The frame dropping processing for channels 1 to 5 follows the same principle as channel 0, and the specific module operation process will not be elaborated further. It is worth noting that the multi-channel video frame synchronization control subunit of this application considers the situation of multi-channel collaborative operation. When the system is reset, it simultaneously resets the processing paths of all six channels. When the application needs to be reset, it also simultaneously generates frame synchronization reset signals for all six channels, ensuring that the processing of each channel can be completely synchronized. Furthermore, the advantage of this invention lies in the fact that when multi-channel video image processing needs to be activated simultaneously, and the system needs to perform multi-channel recording and multi-segment display simultaneously, high input resolution may cause a bottleneck in system processing capacity. This invention can collaboratively control the frame dropping interval of each channel, performing appropriate frame dropping processing on each channel without affecting video smoothness, thereby reducing the system's data processing load and improving the receiving and processing capability of multi-channel input video. For ease of understanding, the following detailed explanation uses four valid input channels as an example. Assuming all four channels are 1920x1080@120Hz input sources, the system struggles to process them and may even fail to receive them correctly. This invention can perform frame dropping processing on the 1920x1080@120Hz video, reducing the data volume of each channel to 1920x1080@30Hz, as detailed below:

[0054] 1) Set the frame display register frame_en_num0 of channel 0 to 0x11111111, which means that channel 0 will discard 3 frames every 4 frames, and the display frames of channel 0 will be the 0th, 4th, 8th, 12th, ... 28th...

[0055] 2) Set the frame display register frame_en_num1 of channel 1 to 0x22222222, which means that channel 0 will discard 3 frames every 4 frames, and channel 1 will display frames as 0+1, 4+1, 8+1, 12+1, ... 28+1...

[0056] 3) Set the frame display register frame_en_num2 of channel 2 to 0x44444444, which means that channel 0 will discard 3 frames every 4 frames, and channel 2 will display frames as 0+2, 4+2, 8+2, 12+2, ... 28+2...

[0057] 4) Set the frame display register frame_en_num3 of channel 3 to 0x88888888, which means that channel 0 will discard 3 frames every 4 frames, and channel 3 will display frames as 0+3, 4+3, 8+3, 12+3, ... 28+3...

[0058] As can be seen from the above processing method, although four channels receive data simultaneously, under the control of frame display, each channel drops 3 frames every 4 frames, and the display frame times are staggered. Only one channel's image needs to be processed within the time of a single video frame. This reduces the frame rate of each channel from 120Hz to 30Hz. Furthermore, due to the uniform frame dropping, the amount of video data processed by the system is reasonably reduced, and the video image does not experience stuttering in any channel while meeting the frame rate requirements. The frame dropping method of this invention reduces system storage bandwidth and power consumption while satisfying display requirements, resulting in better multi-segment video image processing capabilities.

[0059] The multi-channel video image scaling processing unit scales the input video image to ensure the image size meets the requirements of multi-segment display. The scaling method is the same for each channel; the following details the process using channel 0 as an example. Figure 4 As shown, the multi-channel video image scaling processing unit specifically includes: a scaling data acquisition subunit, a first scaling buffer Y scaling buffer 10, a second scaling buffer Y scaling buffer 11, a horizontal scaling control subunit, a third scaling buffer Y scaling buffer 20, a vertical scaling control subunit, a scaling processing control subunit, and a scaling logic operation unit.

[0060] The scaling data acquisition subunit receives the video image data CHO_SI of the current channel. CHO_SI is the signal in the ITU input pixel clock domain. Based on the input horizontal and vertical valid signals, it converts the image data into image data in the system bus clock domain. Assuming the input pixel clock is CLK_PIX and the output clock is the system bus clock CLK_AXI, since CLK_AXI is generally faster than CLK_PIX (typically several times faster in multi-split display applications), after conversion, the proportion of valid data occupying one line of time is greatly reduced, which can be understood as a proportional increase in the data processing capacity per line of time. The image scaling processing of this invention utilizes the speed relationship between CLK_AXI and CLK_PIX, ensuring that both horizontal and vertical scaling processes occur within the CLK_AXI clock domain. This increases the data processing speed within the same time frame. The scaling data acquisition unit writes the acquired image data to the first scaling buffer 10 and the second scaling buffer 11 for use in the horizontal scaling process. The scaling data acquisition subunit converts the line and field valid signals and outputs the line valid signal HACT and the field valid signal VACT to the scaling processing control subunit, respectively. Simultaneously, after writing a line of data to the first scaling buffer 10 and the second scaling buffer 11, the scaling data acquisition subunit sends a line scaling indication signal DO_SCALER to the scaling processing control subunit.

[0061] The second scaling buffer 11 and the first scaling buffer 10, this buffer unit first buffers the luminance component Y of the image data converted to the CLK_AXI domain, and the processing of the chrominance component C is similar to that of the luminance component Y. The scaling part will only be described in detail using Y as an example. The scaling data acquisition subunit writes to the first scaling buffer 10 and the second scaling buffer 11 simultaneously to meet the data requirements of the horizontal scaling step.

[0062] The scaling control subunit coordinates and controls the entire horizontal and vertical scaling process. The image scaling steps are described in detail below, focusing on horizontal scaling, vertical scaling, and scaling buffers:

[0063] 1) Upon receiving the rising edge of the field valid signal VACT, the scaling control enters the initial standby state.

[0064] 2) In the initial standby state, when the DO_SCALER signal is 1, it indicates that a line of image data is ready. The command is issued to control the horizontal scaling unit to read the contents Hdst_data0 and Hdst_data1 in Y buffer 10 and Y buffer 11 for horizontal scaling. Buffer 10 reads the X address and buffer 11 reads the X+1 address to meet the algorithm requirements of horizontal scaling. X is a natural number.

[0065] 3) After horizontal scaling, write the result Hdst_data to the first line address in the Y scaling cache 20 until all the data of the horizontally scaled line has been written.

[0066] 4) Horizontal scaling simultaneously reads the contents Hdst_data0 and Hdst_data1 from Y cache 10 and Y cache 11 for horizontal scaling. Cache 10 reads address X and cache 11 reads address X+1 to meet the algorithm requirements for horizontal scaling. X is a natural number.

[0067] 5) After horizontal scaling, write the result Hdst_data to the second address in the Y scaling cache 20 until the data of the next horizontally scaled row is written.

[0068] 6) The second line of input data is written to cache 10 and cache 11.

[0069] 7) When the DO_SCALER signal is 1, the horizontal scaling unit reads the image data Hdst_data0 and Hdst_data1 from buffers 10 and 11 at the same time for horizontal scaling. Buffer 10 reads address X and buffer 11 reads address X+1 to meet the algorithm requirements for horizontal scaling. X is a natural number.

[0070] 8) After horizontal scaling, write the result Hdst_data to the third line address of Y cache 20 until all the data of the horizontally scaled line has been written.

[0071] 9) The third line of input data is written to cache 10 and cache 11.

[0072] 10) Horizontal scaling control: Simultaneously read image data Hdst_data0 and Hdst_data1 from caches 10 and 11 for horizontal scaling. Cache 10 reads address X and cache 11 reads address X+1 to meet the algorithm requirements for horizontal scaling. X is a natural number.

[0073] 11) After horizontal scaling, write the result Hdst_data to the fourth line address of Y cache 20 until all the data of the horizontally scaled line is written.

[0074] 12) The vertical scaling unit sequentially and simultaneously reads the four rows of horizontally scaled Y data Vsrc_data from Y buffer 20, performs vertical scaling, and writes the vertical scaling result Vdst_data to Y buffer 10 to obtain CH0_SO. Finally, it requests the frame storage controller to write the data to the frame buffer via a bus such as AXI. After inputting the fourth row, the process repeats from step 9 to step 12 until all rows of image data within a frame have been scaled.

[0075] As can be seen from the above steps, the output of the vertical scaling control subunit of this invention is still written to the first scaling buffer 10, sharing the buffer space with the input data, thus saving additional storage resources. This is mainly due to the fact that this invention utilizes the relationship between the input clock and the system clock during multi-segment video image processing. Since the system needs to process image data from multiple channels, the system bus clock CLK_AXI must be greater than the pixel clock CLK_PIX of a single channel input to maintain the balance of data processing throughput. Specifically, for a single channel, the input image data has already been converted to the faster clock domain CLK_AXI for processing. Therefore, under the processing of the CLK_AXI clock, the effective processing time for a line of image data is much more generous than that of CLK_PIX. This ensures that when a new line of image data arrives, the previous line of image data has already been vertically scaled and written back to the Y buffer 10, and the output has already been written back to the frame buffer. In other words, it ensures that consecutive lines of image data will not overwrite each other in terms of time. This processing satisfies the requirements of multi-channel image scaling processing while saving storage resources. In particular, the more channels there are, the greater the resource consumption. This invention can reduce the cost of multi-segment video image processing display system chips while meeting the requirements.

[0076] The scaling logic operation unit is generally a multiplication and addition unit for pixel values ​​and weight values. Since horizontal scaling and vertical scaling are coordinated in a time-sharing manner under the control of the scaling processing control subunit, the operation units of the horizontal scaling and vertical scaling units can also be shared, which can also reduce the resources of the logic circuit.

[0077] Horizontal and vertical scaling can be implemented using multi-phase interpolation or bilinear interpolation algorithms; the specific implementation steps will not be detailed here. The processing of the chromaticity component C is similar in principle to that of the luminance component Y, and will not be elaborated upon here either.

[0078] The write frame storage controller unit is used to send a write frame buffer request to the write frame storage controller after the video images of each channel have been scaled. The write frame storage controller sequentially writes the processed image data CH0_SO, CH1_SO, CH2_SO, CH3_SO, CH4_SO, and CH5_SO of each channel to the frame buffer through the bus. After subsequent processing by the system, the processing and display of multi-segment video images can be realized.

[0079] The embodiments of the present invention are described using a six-segment display as an example, but it can also be extended to a multi-segment display system that supports any channel.

[0080] This invention provides a multi-segment video image processing and display device, enabling chips that support multi-port multi-channel and single-port multi-channel transmission modes. It uses hardware to perform real-time channel decoding and tracking of the input ITU signals, improving the efficiency of decoding and tracking multiple camera input segmented display channels, resulting in more accurate and stable real-time multi-segment image display. Multi-channel video image processing requires scaling of each channel image. Based on optimal chip performance and cost considerations, the multi-segment video processing system provided by this invention fully utilizes the frequency relationship between the input clock and the system clock, as well as the collaborative processing time relationship of each channel. While satisfying real-time video processing, it performs multi-time-segment and multi-step resource sharing in horizontal and vertical scaling, greatly improving the efficiency and resource utilization of multi-segment video image scaling processing, resulting in better chip performance and lower cost. Furthermore, this invention provides a hardware-based frame dropping mechanism for selecting input image frames. Compared to software processing, it eliminates the need for the CPU to respond to frame interrupts before dropping frames, thus reducing frequent CPU resource consumption. Additionally, the video image processing system can ignore hardware-selected frames to be dropped and avoid writing them back to frame storage, thereby reducing system power consumption and bandwidth.

[0081] This invention also provides a display device, including the multi-segment video image processing display device described above.

[0082] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods.

[0083] The above-described preferred embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention. The scope of rights claimed by the present invention should be determined by the scope of the invention application, and not limited to the above-described embodiments.

Claims

1. A multi-segment video image processing and display device, characterized in that, include: Input matrix tracking decoding unit, multi-channel video image frame dropping processing unit, multi-channel video image scaling processing unit, and write frame storage controller unit; The input matrix tracking and decoding unit is used to process the ITU656 / 601 data stream through the input matrix and output channel data for multiple channels after channel tracking and decoding. The input matrix tracking and decoding unit includes: an input matrix subunit, multiple port channel tracking and decoding subunits, and a channel selection matrix subunit; The input matrix subunit is used to select the input signal and output it to the corresponding port channel tracking decoding subunit; The port channel tracking and decoding subunit is used to track and decode the channel according to the channel information embedded in the ITU656 / 601 data stream time base synchronization code in order to track the receiving end and the transmitting end channel in real time. The channel selection matrix subunit is used to receive the channel signal output by the port channel tracking decoding subunit and output the corresponding number of valid channel data according to the number of segments; The multi-channel video image frame dropping processing unit is used to receive data from multiple channels and perform frame dropping processing on each channel data as needed. The multi-channel video image scaling processing unit is used to scale the input video image so that the image size meets the requirements of multi-segment display; The write frame storage controller unit is used to send a write frame buffer request to the write frame storage controller, which then writes the processed image data of the channel into the frame buffer sequentially via the bus. The multi-channel video image frame loss processing unit includes: a multi-channel video frame synchronization control subunit, multiple channel control subunits, and multiple channel image output subunits; The multi-channel video frame synchronization control subunit is used to generate a corresponding number of system reset signals, frame synchronization reset signals, and frame dropping control signals according to the number of segments. The system reset signal is determined by the system power-on time and soft reset. The frame synchronization reset signal is a synchronization pulse signal issued before frame dropping to synchronize the frame dropping interval of each channel. The frame dropping control signal is used to control the frame dropping frequency of each channel. The channel control subunit is used to receive the signal output by the multi-channel video frame synchronization control subunit and the timing signal in the corresponding channel signal, and generate the channel output enable signal. The channel image output subunit is used to receive the ITU656 / 601 data stream input to the corresponding channel, including the input horizontal and vertical synchronization signals, valid timing signals, and image data. Based on the enable signal output by the corresponding channel control subunit, it controls and selects the video signal to be processed by the corresponding channel for output.

2. The multi-segment video image processing and display device according to claim 1, characterized in that, The port channel tracking decoding subunit samples the input data using the port input clock. It presets a channel counter at intervals based on the number of port multiplexed channels. When the channel counter equals 0, the input data is temporarily stored. The continuously stored channel data is then judged. After the synchronization header of the first channel signal is detected, the low-order bits of the channel information embedded after the synchronization code are compared. If the low-order bits are equal to 0, it proves that the port channel tracking decoding is correct. Otherwise, the channel counter is reset according to the value of the low-order bits, and the channel information is tracked again until the detected channel information is equal to 0, at which point the channel tracking decoding is considered to be correct.

3. The multi-segment video image processing and display device according to claim 1, characterized in that, The channel control subunit sets the frame counter clear signal fcnt_clr as follows: (1) When the system reset signal output by the multi-channel video frame synchronization control subunit is received, the frame counter clear signal fcnt_clr is set to 0; (2) When the frame synchronization reset signal is received, the frame counter clear signal fcnt_clr is set to 1; (3) When fcnt_clr is equal to 1 and the timing signal field synchronization vsyn in the ITU656 / 601 data stream is equal to 1, fcnt_clr is set to 0; The channel control subunit has a built-in frame counter, which is set as follows: (1) When the system reset signal is received, the frame counter is set to 0; (2) When the timing signal field synchronization vsyn is equal to 1 and fcnt_clr is equal to 1, the frame counter is set to 0; (3) When the timing signal field synchronization vsyn is equal to 1 and fcnt_clr is not equal to 1, the frame counter is incremented by 1.

4. The multi-segment video image processing and display device according to claim 1, characterized in that, The multi-channel video image frame loss processing unit is configured with 2 5 A bit-based frame display register; each bit represents a frame display enable signal; when the frame display enable signal is 1, the channel image output subunit outputs image data equal to the input image data; when the frame display enable signal is 0, it indicates that a frame should be dropped, and the output image data is equal to 0.

5. The multi-segment video image processing and display device according to claim 1, characterized in that, The multi-channel video image scaling processing unit includes: a scaling data acquisition subunit, a first scaling buffer, a second scaling buffer, a horizontal scaling control subunit, a third scaling buffer, a vertical scaling control subunit, a scaling processing control subunit, and a scaling logic operation unit. The scaling data acquisition subunit converts the image signal from the ITU656 / 601 input pixel clock domain into image data from the system bus clock domain and sends a scaling indication signal to the scaling processing control subunit. The scaling processing control subunit coordinates and controls the scaling processes of the horizontal scaling subunit and the vertical scaling control subunit. The first and second scaling buffers cache the data output from the scaling data acquisition subunit for use by the horizontal scaling control subunit. The third scaling buffer caches the data output from the horizontal scaling control subunit for use by the vertical scaling control subunit. The scaling logic operation unit provides calculation operations for image-related pixel values ​​and weight values.

6. The multi-segment video image processing and display device according to claim 5, characterized in that, The system bus clock frequency is an integer multiple of the input pixel clock frequency.

7. The multi-segment video image processing and display device according to claim 5, characterized in that, The horizontal scaling control subunit performs horizontal scaling, and the vertical scaling control subunit performs vertical scaling, both using a multi-phase interpolation algorithm or a bilinear interpolation algorithm.

8. A display terminal, characterized in that, include: The multi-segment video image processing and display device according to any one of claims 1-7.

Images