Low-delay image processing device and method

Abstract

The invention provides a low-delay image processing device and method. A synchronizing signal extraction unit extracts a synchronizing word from an input video stream and obtains an input frame synchronizing signal InVsync; the image caching unit receives an input video stream, performs single-frame caching or ping-pong caching, and outputs ROIstream; the digital zooming unit is used for zooming the ROIstream to an image format required by the video output unit; the video output unit outputs a video stream Outstream and outputs a frame synchronization signal OutVsync to the read-write control unit at the same time; the read-write control unit receives InVsync and OutVsync, generates a write pointer WritePointer and a read pointer ReadPointer, and dynamically adjusts the frequency of an output pixel clock OutPclk of the video output unit according to the position relation between the write pointer WritePointer and a top row ROItop of a region of interest ROI when an effective output frame synchronizing signal OutVsync is received; and the ePTZ control unit sets control parameters of the read-write control unit. According to the invention, the clock frequency precision of the output pixel of the output video stream can be controlled in an allowable range, and image delay in 2-4 line periods can be realized at the same time.

Description

Technical Field

[0001] The present invention relates to the technical fields of video communication, video surveillance, computer vision, medical imaging and vehicle-mounted imaging, and in particular to a low-latency image processing device and method. Background Art

[0002] PTZ cameras are widely used in video communications, video surveillance, computer vision, medical imaging, and automotive imaging. P stands for Pan, meaning horizontal panning; T stands for Tilt, meaning vertical panning; and Z stands for Zoom, meaning push-in / pull-out shooting. Electronic PTZ is abbreviated as ePTZ or EPTZ. Manual and motorized PTZ achieve PTZ functionality by changing the lens's azimuth and focal length. EPTZ does not change the lens's azimuth and focal length, but instead achieves PTZ by changing the ROI (Region of Interest) and using digital zoom. ROI stands for Region of Interest (ROI).

[0003] To ensure stable frame synchronization of the video stream output by an EPTZ camera, existing technology only starts reading the top row of the ROI data after the DDR has fully written a frame of data. This causes an image delay of less than one frame. When the ROI_top (RT) and Sensor_top (ST) are on the same row, the delay is one frame period. Therefore, the disadvantage of existing technology is that it supports the EPTZ function but adds an excessively long image delay. Summary of the Invention

[0004] In response to the above technical problems, the purpose of the present invention is to provide a low-latency image processing device and method, which can ensure that the output pixel clock frequency accuracy of the output video stream is controlled within an allowable range, while achieving image delay within 2-4 line cycles, thereby shortening the image cache delay.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] The present invention provides a low-latency image processing device, comprising:

[0007] A synchronization signal extraction unit is used to extract a synchronization word from an input video stream In_stream and obtain an input frame synchronization signal In_Vsync;

[0008] An image buffer unit is used to receive an input video stream In_stream, perform single-frame buffering or ping-pong buffering, and output an ROI video image ROI_stream;

[0009] a digital scaling unit, configured to scale the ROI video image ROI_stream from the image buffer unit to an image format required by a video output unit, and transmit the image to the video output unit;

[0010] The video output unit is used to output a frame synchronization signal Out_Vsync to the read / write control unit while outputting the video stream Out_stream. The output video stream Out_stream has Out_Vtotal*Out_Htotal Out_Pclk clock cycles between two adjacent Out_Vsyncs, where Out_Vtotal, Out_Htotal, and Out_Pclk represent the total number of output lines (including blanking lines and valid lines), the total number of pixels per line (including blanking pixels and valid pixels), and the output pixel clock, respectively.

[0011] A read-write control unit is configured to receive an input frame synchronization signal In_Vsync from a synchronization signal extraction unit and an output frame synchronization signal Out_Vsync from a video output unit, generate a write pointer Write_Pointer for writing video data from an input video stream In_stream into a suitable location in an image cache unit, and generate a read pointer Read_Pointer for reading an ROI video image ROI_stream at a suitable location in the image cache unit; the read-write control unit dynamically adjusts the frequency of an output pixel clock Out_Pclk of the video output unit based on a positional relationship between the write pointer Write_Pointer and the top row ROI_top of the region of interest ROI when receiving a valid output frame synchronization signal Out_Vsync;

[0012] The ePTZ control unit is used to set the control parameters of the read / write control unit.

[0013] Furthermore, the single-frame buffer caches one frame of image and refreshes it frame by frame, and the ping-pong buffer caches two frames of image, frame A and frame B, and performs ping-pong refresh.

[0014] Furthermore, the ping-pong refresh is that when the write pointer Write_Pointer points to a certain row of frame A, frame A is the current frame and frame B is the previous frame; when the write pointer Write_Pointer points to a certain row of frame B, frame B is the current frame and frame A is the previous frame.

[0015] Furthermore, when the image cache unit performs single-frame caching, the specific method in which the read / write control unit dynamically adjusts the frequency of the output pixel clock Out_Pclk of the video output unit is as follows:

[0016] When receiving a valid output frame synchronization signal Out_Vsync, if the write pointer Write_Pointer≤ROI_top_Target, set ROI_top=Write_Pointer-1 line, otherwise set ROI_top=ROI_top_Target; wherein ROI_top_Target represents the top line target position of the region of interest ROI;

[0017] If ROI_top<Write_Pointer≤ROI_top+TH1, decrease the Out_Pclk frequency within the accuracy range required by the standard;

[0018] If ROI_top+TH1<Write_Pointer≤ROI_top+TH2, the Out_Pclk frequency remains unchanged;

[0019] If Write_Pointer>ROI_top+TH2, increase the Out_Pclk frequency within the accuracy range required by the standard; the read pointer Read_Pointer moves to ROI_top.

[0020] Further, when the image buffer unit is performing ping-pong buffering, the read-write control unit dynamically adjusts the output pixel clock Out_Pclk frequency of the video output unit in the following specific manner:

[0021] When receiving a valid output frame synchronization signal Out_Vsync, if the write pointer Write_Pointer≤ROI_top, the read pointer Read_Pointer moves to the ROI_top of the previous frame, and the Out_Pclk frequency is increased to the maximum allowed value Out_Pclk_Max;

[0022] If ROI_top<Write_Pointer≤ROI_top+TH3, the Read_Pointer moves to the ROI_top of the current frame, and the Out_Pclk frequency is decreased;

[0023] If ROI_top+TH3<Write_Pointer≤ROI_top+TH4, the Read_Pointer moves to the ROI_top of the current frame, and the Out_Pclk frequency remains unchanged;

[0024] If Write_Pointer>ROI_top+TH4, the Read_Pointer moves to the ROI_top of the current frame, and the Out_Pclk frequency is increased.

[0025] Further, the TH1 and TH2 are settable parameters, TH1=2 lines, and TH2=4 lines.

[0026] Further, the TH3 and TH4 are settable parameters, TH3=2 lines, and TH4=4 lines.

[0027] Further, the control parameters of the read-write control unit include a horizontal offset of a region of interest (ROI_H_Offset), a vertical offset of a region of interest (ROI_V_Offset), a horizontal size of a region of interest (ROI_H_Size), a vertical size of a region of interest (ROI_V_Size), a video output image format, and a digital image scaling ratio, wherein the ROI_V_Offset is also referred to as ROI_top_Target.

[0028] Further, the Out_Vtotal, Out_Htotal, and Out_Pclk frequencies comply with the CTA-861 standard.

[0029] The application further provides a low-delay image processing method applied to the low-delay image processing device.

[0030] A synchronization word is extracted from the input video stream In_stream to obtain an input frame synchronization signal In_Vsync, and the input video stream In_stream is subjected to single-frame buffering or ping-pong buffering to output a ROI video image ROI_stream;

[0031] The ROI video image ROI_stream is scaled to a required image format of a video output unit, and is output through the video output unit;

[0032] The read-write control unit is output with a frame synchronization signal Out_Vsync while the output video stream Out_stream is output, wherein the output video stream Out_stream has Out_Vtotal*Out_Htotal Out_Pclk clock periods between two adjacent Out_Vsyncs.

[0033] The control parameters of the read-write control unit are set by the ePTZ control unit. The read-write control unit generates a write pointer Write_Pointer based on the input frame synchronization signal In_Vsync and the output frame synchronization signal Out_Vsync, which is used to write the video data from the input video stream In_stream to the appropriate position of the image cache unit, and generates a read pointer Read_Pointer for reading the ROI video image ROI_stream at the appropriate position in the image cache unit; the read-write control unit dynamically adjusts the output pixel clock Out_Pclk frequency of the video output unit based on the position relationship between the write pointer Write_Pointer and ROI_top when receiving a valid output frame synchronization signal Out_Vsync.

[0034] Compared with the prior art, the present invention has the following advantages: The present invention provides a low-latency image processing device and method. The image cache unit implements two caching schemes for the input video stream In_stream: single-frame caching or ping-pong caching. For different caching schemes, the read / write control unit dynamically adjusts the output pixel clock Out_Pclk frequency of the video output unit based on the positional relationship between the write pointer Write_Pointer and the ROI_top when receiving a valid output frame synchronization signal Out_Vsync. This ensures that the output pixel clock frequency accuracy of the output video stream is within an allowable range while achieving an image processing delay within 2-4 line cycles. The image cache unit (single-frame or ping-pong caching) combined with the precise read / write pointers (Write_Pointer, Read_Pointer) generated by the read / write control unit supports efficient extraction and processing of the region of interest (ROI). The digital zoom unit further adapts the ROI image to the target output format, ensuring the final low-latency, high-quality output. Furthermore, the ePTZ control unit allows the configuration of key parameters of the read / write control unit (such as ROI position and size), making the entire image processing device (especially low-latency ROI extraction and scaling) highly flexible and configurable.

[0035] This invention innovatively reduces image processing latency to just 2-4 line cycles. Simultaneously, through a dynamic output pixel clock adjustment mechanism, it strictly guarantees the high precision of the output pixel clock of the output video stream, thereby achieving a perfect synergistic optimization of latency and output clock accuracy in the field of ultra-low-latency image processing. This invention has extremely important application value in applications that require low latency and extremely high requirements for output timing stability, such as video communications, video surveillance, computer vision, medical imaging, and automotive imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0037] Figure 1 It is a schematic diagram of the working principle of the EPTZ camera equipment in the prior art;

[0038] Figure 2 Schematic diagram of the working principle of the EPTZ camera device when the (image sensor) master synchronization mode is adopted in the prior art;

[0039] Figure 3 Schematic diagram of the working principle of the EPTZ camera device when the (image sensor) slave synchronization mode is adopted in the second prior art;

[0040] Figure 4 A schematic diagram of the working principle of a low-latency image processing device when the image cache unit provided in the first embodiment of the present invention operates in a single-frame cache working mode;

[0041] Figure 5 A timing diagram of a frame period of dynamically adjusting the frequency of the output pixel clock (Out_Pclk) when the image buffer unit provided in the first embodiment of the present invention operates in a single-frame buffer operation mode;

[0042] Figure 6 A flow chart of a method for dynamically adjusting the frequency of an output pixel clock (Out_Pclk) provided in the first embodiment of the present invention;

[0043] Figure 7 A schematic diagram of the working principle of a low-latency image processing device when the image cache unit provided in the second embodiment of the present invention operates in a ping-pong cache working mode;

[0044] Figure 8 This is a flow chart of a method for dynamically adjusting the frequency of the output pixel clock (Out_Pclk) provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0045] To make the technical solutions and advantages achieved by the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0046] References in the prior art Figure 1As shown, EPTZ common technology: sensor image size (H_active, V_active), through the standard MIPI interface to the DDR memory, where the ROI region size (H_size, V_size) of the top left corner coordinates (H_offset, V_offset) is scaled to the display image size (H_display, V_display) by the scaler. In the EPTZ camera process, the horizontal offset H_offset is increased or decreased frame by frame to realize the function of horizontal panning (Pan), the vertical offset V_offset is increased or decreased frame by frame to realize the function of vertical panning (Tilt), the horizontal size H_size and the vertical size V_size are increased or decreased frame by frame to realize the function of push-pull shooting (Zoom) while keeping the ratio of horizontal size H_size:vertical size V_size unchanged.

[0047] Prior art one, using (image sensor) master Sensor master synchronization mode, delay one frame.

[0048] Reference Figure 2 As shown, prior art one uses Sensor master synchronization mode, which means that the image sensor frame synchronization signal Sensor_Sync (SS) is generated by the image sensor Sensor and is not controlled externally. Sensor_Stream is the video data stream output by Sensor, which also includes the frame synchronization code. In order to realize the frame synchronization stability of the display image data stream Display_Stream in the EPTZ process, the ROI_Sync (RS) generated by the synchronization controller must cooperate with the readout controller to ensure that:

[0049] The frame periods of Sensor_Stream, ROI_Stream and Display_Stream are consistent, that is, Figure 2 The ROI frame period R_total in the above formula is equal to the (image sensor) frame period S_total;

[0050] Figure 2 The phase difference between the ROI synchronization signal ROI_Sync (RS) and the (image sensor) synchronization signal Sensor_Sync (SS) is stable.

[0051] In order to achieve the above two goals, prior art one starts to read out ROI data only when the DDR is full of one frame of image. From Figure 2 It can be seen from the above formula that:

[0052] The delay of the image data in the ROI_top (RT) row from being written into the image buffer DDR to being read out of DDR is within one frame;

[0053] When ROI_top (RT) and Sensor_top (ST) are in the same row, the delay is 1 frame period.

[0054] The second existing technology adopts the (image sensor) slave synchronization mode, which delays one frame.

[0055] refer to Figure 3 As shown, the second prior art adopts the Sensor slave synchronization mode, which means that the frame synchronization signal Sensor_trigger is generated by the synchronization controller. In order to achieve stable frame synchronization of Display_Stream during the EPTZ process, the synchronization controller generates ROI_Sync (RS) and cooperates with the readout controller to ensure that the frame periods of Sensor_Stream, ROI_Stream, and Display_Stream are consistent, that is, Figure 3 R_total=S_total in Figure 3 The phase difference between ROI_Sync (RS) and Sensor_Sync (SS) is stable.

[0056] In order to achieve the above two goals, the second existing technology starts to read out the ROI data only when the DDR is fully written with a frame of image. Figure 3 It can be seen that:

[0057] The delay from writing the image data of the top row of ROI (ROI_top(RT)) into DDR to reading it out of DDR is within one frame.

[0058] When ROI_top(RT) and Sensor_top(ST) are in the same row, the delay is 1 frame period.

[0059] To address the issue of excessive image delay in the aforementioned prior art, the present invention discloses a low-latency image processing device and method that ensures the output pixel clock frequency accuracy of the output video stream is within an acceptable range while achieving an image delay within 2-4 line periods. Taking the 1080p image format as an example, each frame has 1125 lines, including the blanking period. By dynamically adjusting the output pixel clock (Out_Pclk) frequency, the ePTZ video delay can be reduced to a range of 2 to 4 lines, less than 0.36% of the frame period, achieving a highly significant effect.

[0060] Example 1

[0061] like Figure 4As shown, the first embodiment of the present invention provides a low-latency image processing device, including: a synchronization signal extraction unit, an image buffer unit, a digital zoom unit, an image output unit, an ePTZ control unit, and a read-write control unit;

[0062] A synchronization signal extraction unit is used to extract a synchronization word from an input video stream In_stream and obtain an input frame synchronization signal In_Vsync;

[0063] The image buffer unit is used to receive the input video stream In_stream, perform single-frame buffering, that is, buffer one frame of image and refresh it frame by frame, and output the ROI video image ROI_stream;

[0064] a digital scaling unit, configured to scale the ROI video image ROI_stream from the image buffer unit to an image format required by a video output unit, and transmit the image to the video output unit;

[0065] The video output unit is configured to output a frame synchronization signal Out_Vsync to the read / write control unit while outputting the video stream Out_stream. The output video stream Out_stream has Out_Vtotal*Out_Htotal Out_Pclk clock cycles between two adjacent Out_Vsyncs, where Out_Vtotal, Out_Htotal, and Out_Pclk represent the total number of output lines (including blanking lines and valid lines), the total number of pixels per output line (including blanking pixels and valid pixels), and the output pixel clock, respectively. The frequencies of Out_Vtotal, Out_Htotal, and Out_Pclk comply with the CTA-861 standard.

[0066] A read-write control unit is configured to receive an input frame synchronization signal In_Vsync from a synchronization signal extraction unit and an output frame synchronization signal Out_Vsync from a video output unit, generate a write pointer Write_Pointer for writing video data from an input video stream In_stream into a suitable location in an image cache unit, and generate a read pointer Read_Pointer for reading an ROI video image ROI_stream at a suitable location in the image cache unit; the read-write control unit dynamically adjusts the frequency of an output pixel clock Out_Pclk of the video output unit based on a positional relationship between the write pointer Write_Pointer and the top row ROI_top of the region of interest ROI when receiving a valid output frame synchronization signal Out_Vsync;

[0067] The ePTZ control unit is used to set the control parameters of the read / write control unit, including the horizontal offset (ROI_H_Offset), vertical offset (ROI_V_Offset), horizontal size (ROI_H_Size), vertical size (ROI_V_Size), video output image format, and digital image scaling. ROI_V_Offset is also called ROI_top_Target. ROI_H_Offset indicates the horizontal offset of the top-left corner of the ROI rectangle relative to the top-left origin of the image. ROI_V_Offset indicates the vertical offset of the top-left corner of the ROI rectangle relative to the top-left origin of the image. ROI_H_Size indicates the horizontal width of the ROI rectangle. ROI_V_Size indicates the vertical height of the ROI rectangle.

[0068] When the image cache unit performs single-frame caching, the specific method of the read-write control unit dynamically adjusting the frequency of the output pixel clock Out_Pclk of the video output unit is as follows:

[0069] When receiving a valid output frame synchronization signal Out_Vsync, if the write pointer Write_Pointer ≤ ROI_top_Target, then set ROI_top = Write_Pointer - 1 row, otherwise set ROI_top = ROI_top_Target; wherein ROI_top_Target represents the top row target position of the region of interest ROI;

[0070] If ROI_top<Write_Pointer≤ROI_top+TH1, reduce the Out_Pclk frequency within the accuracy required by the standard;

[0071] If ROI_top+TH1<Write_Pointer≤ROI_top+TH2, the Out_Pclk frequency remains unchanged;

[0072] If Write_Pointer>ROI_top+TH2, then the Out_Pclk frequency is increased within the accuracy range required by the standard; and the read pointer Read_Pointer is moved to ROI_top.

[0073] like Figure 6 As shown, specifically, the steps of the read / write control unit dynamically adjusting the frequency of the output pixel clock Out_Pclk of the video output unit are as follows:

[0074] S1, OUT_Vsync is valid, enter S2;

[0075] S2. If Write_Pointer>ROI_top_Target, go to S3; otherwise go to S4;

[0076] S3, set ROI_top = ROI_top_Target, and go to S5;

[0077] S4. Set ROI_top = Write_Pointer - 1 and go to S5.

[0078] S5. Set a temporary variable s = Write_Pointer - ROI_top, where s must be greater than 0; if 0 < s ≤ TH1, proceed to S6; if TH1 < s ≤ TH2, proceed to S7; if s > TH2, proceed to S8;

[0079] S6. Reduce the Out_Pclk frequency within the frequency accuracy range required by the standard and enter S9;

[0080] S7, Out_Pclk frequency remains unchanged, enter S9;

[0081] S8. Increase or decrease the Out_Pclk frequency within the frequency accuracy range required by the standard and enter S9;

[0082] S9, Read_Pointer moves to ROI_top and enters S10;

[0083] S10, wait for OUT_Vsync, until OUT_Vsync is valid, enter S1;

[0084] For example, TH1 and TH2 are configurable parameters, TH1 = 2 rows, TH2 = 4 rows.

[0085] According to the first embodiment of the present invention, the video delay can be converged to a time range of 2 to 4 lines, and the frequency accuracy of OUT_Pclk can be controlled within the allowable range, such as Figure 5As shown in the figure. IT is the abbreviation of Input_top, which represents the first row of image data in the input frame; IB is the abbreviation of Input_bottom, which represents the last row of image data in the input frame; RT is the abbreviation of ROI_top, which represents the first row of image data in the ROI, that is, the image data of the ROI_V_Offsetth row in the input video image; RB is the abbreviation of ROI_bottom, which represents the last row of image data in the ROI, that is, the image data of the ROI_V_Offset+ROI_V_Sizeth row in the input video image; VS_delay in the figure is the delay from the "input frame synchronization signal (In_Vsync)" to the "output frame synchronization signal (Out_Vsync)", not the video delay; the video delay is the delay from "writing RT image data into the image buffer" to "reading RT image data from the image buffer."

[0086] Taking the 1080p image format as an example, each frame has 1125 lines, including the blanking period. By dynamically adjusting the output pixel clock (Out_Pclk) frequency, the video delay can be guaranteed to converge to a time range of 2 to 4 lines, which is less than 0.36% of the frame period.

[0087] Example 2

[0088] like Figure 7 As shown, the second embodiment of the present invention provides a low-latency image processing device, including: a synchronization signal extraction unit, an image buffer unit, a digital zoom unit, an image output unit, an ePTZ control unit, and a read-write control unit;

[0089] A synchronization signal extraction unit is used to extract a synchronization word from an input video stream In_stream and obtain an input frame synchronization signal In_Vsync;

[0090] The image cache unit is used to receive the input video stream In_stream, perform ping-pong caching, that is, cache two frames of images, frame A and frame B, and perform ping-pong refresh, and output the ROI video image ROI_stream; wherein, the ping-pong refresh is when the write pointer Write_Pointer points to a certain row of frame A, frame A is the current frame and frame B is the previous frame; when the write pointer Write_Pointer points to a certain row of frame B, frame B is the current frame and frame A is the previous frame;

[0091] a digital scaling unit, configured to scale the ROI video image ROI_stream from the image buffer unit to an image format required by a video output unit, and transmit the image to the video output unit;

[0092] The video output unit is configured to output a frame synchronization signal Out_Vsync to the read / write control unit while outputting the video stream Out_stream. The output video stream Out_stream has Out_Vtotal*Out_Htotal Out_Pclk clock cycles between two adjacent Out_Vsyncs. The Out_Vtotal, Out_Htotal, and Out_Pclk frequencies represent the total number of output lines (including blanking lines and valid lines), the total number of pixels per line (including blanking pixels and valid pixels), and the output pixel clock, respectively. The Out_Vtotal, Out_Htotal, and Out_Pclk frequencies comply with the CTA-861 standard.

[0093] A read-write control unit is configured to receive an input frame synchronization signal In_Vsync from a synchronization signal extraction unit and an output frame synchronization signal Out_Vsync from a video output unit, generate a write pointer Write_Pointer for writing video data from an input video stream In_stream into a suitable location in an image cache unit, and generate a read pointer Read_Pointer for reading an ROI video image ROI_stream at a suitable location in the image cache unit; the read-write control unit dynamically adjusts the frequency of an output pixel clock Out_Pclk of the video output unit based on a positional relationship between the write pointer Write_Pointer and the top row ROI_top of the region of interest ROI when receiving a valid output frame synchronization signal Out_Vsync;

[0094] The ePTZ control unit is used to set the control parameters of the read / write control unit, including ROI_H_Offset, ROI_V_Offset, ROI_H_Size, ROI_V_Size, video output image format, and digital image scaling ratio. ROI_V_Offset is also called ROI_top_Target.

[0095] When the image cache unit performs ping-pong caching, the specific method of the read-write control unit dynamically adjusting the frequency of the output pixel clock Out_Pclk of the video output unit is as follows:

[0096] When receiving a valid output frame synchronization signal Out_Vsync, if the write pointer Write_Pointer ≤ ROI_top, the read pointer Read_Pointer moves to the ROI_top of the previous frame, and at the same time increases the Out_Pclk frequency to the maximum allowable value Out_Pclk_Max;

[0097] If ROI_top<Write_Pointer≤ROI_top+TH3, then Read_Pointer moves to ROI_top of the current frame and reduces the Out_Pclk frequency;

[0098] If ROI_top+TH3<Write_Pointer≤ROI_top+TH4, then Read_Pointer moves to ROI_top of the current frame and the Out_Pclk frequency remains unchanged;

[0099] If Write_Pointer>ROI_top+TH4, the Read_Pointer moves to the ROI_top of the current frame, and the Out_Pclk frequency is increased within the frequency accuracy range required by the standard.

[0100] Specifically, the method of dynamically adjusting the output pixel clock (Out_Pclk) frequency is more flexible. Figure 8 According to a specific embodiment of the present invention, the steps for dynamically adjusting the frequency of the output pixel clock (Out_Pclk) are as follows:

[0101] S1, OUT_Vsync is valid, enter S2;

[0102] S2. Set a temporary variable s = Write_Pointer-ROI_top; if s≤0, go to S3; if 0<s≤TH3, go to S4; if TH3<s≤TH4, go to S5; if s>TH4, go to S6;

[0103] S3. Within the frequency accuracy range required by the standard, the Out_Pclk frequency increases to the maximum allowable value Out_Pclk_Max, and then enters S7.

[0104] S4. Reduce the Out_Pclk frequency within the frequency accuracy range required by the standard and enter S8;

[0105] S5, Out_Pclk frequency remains unchanged, enter S8;

[0106] S6. Increase or decrease the Out_Pclk frequency within the frequency accuracy range required by the standard and enter S8.

[0107] S7, Read_Pointer moves to ROI_top of the previous frame and enters S9;

[0108] S8, Read_Pointer moves to ROI_top of the current frame and enters S9;

[0109] S9, wait for OUT_Vsync. Until OUT_Vsync is valid, enter S1.

[0110] According to a specific embodiment of the present invention, HDMI allows a Pclk frequency accuracy of ±0.5%, and SDI allows a Pclk frequency accuracy of ±0.1%, and Out_Pclk_Max and Out_Pclk_Min can be determined accordingly.

[0111] The TH3 and TH4 are configurable parameters, TH3 = 2 rows, TH4 = 4 rows.

[0112] According to the second embodiment of the present invention, during the ePTZ transition period, video delay may exceed one frame, but this transition period is brief. During the ePTZ stabilization period, video delay converges to within two to four lines. HDMI allows for a Pclk frequency accuracy of ±0.5%, while SDI allows for a Pclk frequency accuracy of ±0.1%. This allows Out_Pclk_Max and Out_Pclk_Min to be determined accordingly.

[0113] Example 3

[0114] The third embodiment of the present invention further provides a low-latency image processing method, which is applied to the low-latency image processing device of the first or second embodiment, and includes the following steps:

[0115] Extract the synchronization word from the input video stream In_stream and obtain the input frame synchronization signal In_Vsync; and perform single-frame buffering or ping-pong buffering on the input video stream In_stream and output the ROI video image ROI_stream;

[0116] Scale the ROI video image ROI_stream to the image format required by the video output unit and output it through the video output unit;

[0117] While outputting the video stream Out_stream, the frame synchronization signal Out_Vsync is output to the read / write control unit, wherein the output video stream Out_stream has Out_Vtotal*Out_Htotal Out_Pclk clock cycles between two adjacent Out_Vsyncs;

[0118] The control parameters of the read-write control unit are set by the ePTZ control unit. The read-write control unit generates a write pointer Write_Pointer based on the input frame synchronization signal In_Vsync and the output frame synchronization signal Out_Vsync, which is used to write the video data from the input video stream In_stream to the appropriate position of the image cache unit, and generates a read pointer Read_Pointer for reading the ROI video image ROI_stream at the appropriate position in the image cache unit; the read-write control unit dynamically adjusts the output pixel clock Out_Pclk frequency of the video output unit based on the position relationship between the write pointer Write_Pointer and ROI_top when receiving a valid output frame synchronization signal Out_Vsync.

[0119] In summary, in the technical solution for ePTZ image processing, to ensure a stable frame period for the displayed image, the existing technology requires an image cache delay of less than 1 frame. The present invention discloses a low-latency ePTZ image processing device and method. This method dynamically adjusts the output pixel clock (Out_Pclk) of the video output unit based on the "position relationship between the write pointer Write_Pointer and ROI_top" when the output frame synchronization signal (Out_Vsync) arrives. This ensures that the output pixel clock frequency accuracy of the output video stream is within the allowable range, while achieving an image delay of 2-4 line periods.

[0120] Matters not covered by the present invention are known technologies.

[0121] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A low-latency image processing device, characterized in that: include: A synchronization signal extraction unit is used to extract a synchronization word from an input video stream In_stream and obtain an input frame synchronization signal In_Vsync; An image buffer unit is used to receive an input video stream In_stream, perform single-frame buffering or ping-pong buffering, and output an ROI video image ROI_stream; a digital scaling unit, configured to scale the ROI video image ROI_stream from the image buffer unit to an image format required by a video output unit, and transmit the image to the video output unit; The video output unit is used to output a frame synchronization signal Out_Vsync to the read / write control unit while outputting the video stream Out_stream. The output video stream Out_stream has Out_Vtotal*Out_Htotal Out_Pclk clock cycles between two adjacent Out_Vsyncs, where Out_Vtotal, Out_Htotal, and Out_Pclk represent the total number of output rows, the total number of pixels per row, and the output pixel clock, respectively. A read-write control unit is configured to receive an input frame synchronization signal In_Vsync from a synchronization signal extraction unit and an output frame synchronization signal Out_Vsync from a video output unit, generate a write pointer Write_Pointer for writing video data from an input video stream In_stream into a suitable location in an image cache unit, and generate a read pointer Read_Pointer for reading an ROI video image ROI_stream at a suitable location in the image cache unit; the read-write control unit dynamically adjusts the frequency of an output pixel clock Out_Pclk of the video output unit based on a positional relationship between the write pointer Write_Pointer and the top row ROI_top of the region of interest ROI when receiving a valid output frame synchronization signal Out_Vsync; The ePTZ control unit is used to set the control parameters of the read / write control unit.

2. The low-latency image processing device according to claim 1, wherein: The single-frame buffer caches one frame of image and refreshes it frame by frame, and the ping-pong buffer caches two frames of image, frame A and frame B, and performs ping-pong refresh.

3. The low-latency image processing device according to claim 2, characterized in that: The ping-pong refresh is that when the write pointer Write_Pointer points to a certain row of frame A, frame A is the current frame and frame B is the previous frame; when the write pointer Write_Pointer points to a certain row of frame B, frame B is the current frame and frame A is the previous frame.

4. The low-latency image processing device according to claim 1, wherein: When the image cache unit performs single-frame caching, the specific method of the read-write control unit dynamically adjusting the frequency of the output pixel clock Out_Pclk of the video output unit is as follows: When receiving a valid output frame synchronization signal Out_Vsync, if the write pointer Write_Pointer ≤ ROI_top_Target, then set ROI_top = Write_Pointer - 1 row, otherwise set ROI_top = ROI_top_Target; wherein ROI_top_Target represents the top row target position of the region of interest ROI; If ROI_top<Write_Pointer≤ROI_top+TH1, reduce the Out_Pclk frequency within the accuracy required by the standard; If ROI_top+TH1<Write_Pointer≤ROI_top+TH2, the Out_Pclk frequency remains unchanged; If Write_Pointer>ROI_top+TH2, then the Out_Pclk frequency is increased within the accuracy range required by the standard; and the read pointer Read_Pointer is moved to ROI_top.

5. The low-latency image processing device according to claim 1, wherein: When the image cache unit performs ping-pong caching, the specific method of the read-write control unit dynamically adjusting the frequency of the output pixel clock Out_Pclk of the video output unit is as follows: When receiving a valid output frame synchronization signal Out_Vsync, if the write pointer Write_Pointer ≤ ROI_top, the read pointer Read_Pointer moves to the ROI_top of the previous frame, and at the same time increases the Out_Pclk frequency to the maximum allowable value Out_Pclk_Max; If ROI_top<Write_Pointer≤ROI_top+TH3, then Read_Pointer moves to ROI_top of the current frame and reduces the Out_Pclk frequency; If ROI_top+TH3<Write_Pointer≤ROI_top+TH4, then Read_Pointer moves to ROI_top of the current frame and the Out_Pclk frequency remains unchanged; If Write_Pointer>ROI_top+TH4, the Read_Pointer moves to the ROI_top of the current frame and the Out_Pclk frequency increases.

6. The low-latency image processing device according to claim 4, characterized in that: The TH1 and TH2 are configurable parameters, TH1 = 2 rows, TH2 = 4 rows.

7. The low-latency image processing device according to claim 5, characterized in that: The TH3 and TH4 are configurable parameters, TH3 = 2 rows, TH4 = 4 rows.

8. The low-latency image processing device according to claim 1, wherein: The control parameters of the read-write control unit include the horizontal offset ROI_H_Offset of the region of interest, the vertical offset ROI_V_Offset of the region of interest, the horizontal size ROI_H_Size of the region of interest, the vertical size ROI_V_Size of the region of interest, the video output image format and the digital image scaling ratio t.

9. The low-latency image processing device according to claim 1, wherein: The Out_Vtotal, Out_Htotal, and Out_Pclk frequencies comply with the requirements of the CTA-861 standard.

10. A low-latency image processing method, applied to the low-latency image processing device according to any one of claims 1 to 9, characterized in that: include: Extract the synchronization word from the input video stream In_stream and obtain the input frame synchronization signal In_Vsync; and perform single-frame buffering or ping-pong buffering on the input video stream In_stream and output the ROI video image ROI_stream; Scale the ROI video image ROI_stream to the image format required by the video output unit and output it through the video output unit; While outputting the video stream Out_stream, the frame synchronization signal Out_Vsync is output to the read / write control unit, wherein the output video stream Out_stream has Out_Vtotal*Out_Htotal Out_Pclk clock cycles between two adjacent Out_Vsyncs; The control parameters of the read-write control unit are set by the ePTZ control unit. The read-write control unit generates a write pointer Write_Pointer based on the input frame synchronization signal In_Vsync and the output frame synchronization signal Out_Vsync, which is used to write the video data from the input video stream In_stream to the appropriate position of the image cache unit, and generates a read pointer Read_Pointer for reading the ROI video image ROI_stream at the appropriate position in the image cache unit; the read-write control unit dynamically adjusts the output pixel clock Out_Pclk frequency of the video output unit based on the position relationship between the write pointer Write_Pointer and ROI_top when receiving a valid output frame synchronization signal Out_Vsync.

Images