Image signal processing method and device, storage medium and electronic device

Abstract

This invention provides an image signal processing method, apparatus, storage medium, and electronic device. The method includes: determining the current data rate of the target serializer and the clock frequency of the current operating clock of the target encoder based on an acquired target data rate and rate constraint parameters of the target serializer; when the target data rate is less than the minimum data rate, performing first signal processing on the original image signal to obtain a first encoded input signal; encoding the first encoded input signal to obtain a first encoded output signal; then performing target signal processing on the first encoded output signal to obtain a second encoded output signal matching the current data rate; and serializing the second encoded output signal using a dual-mode target serializer to obtain a first serialized output signal. This invention solves the problem in related technologies where serializers operating in dual mode cannot be used in low-resolution scenarios.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to an image signal processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] HDMI stands for High Definition Multimedia Interface. It's a fully digital video and audio transmission interface that can send uncompressed audio and video signals. HDMI can be used in set-top boxes, DVD players, personal computers, televisions, game consoles, integrated amplifiers, digital audio systems, and televisions. HDMI can transmit audio and video signals simultaneously, greatly simplifying system wiring by using the same cable for both. HDMI versions have evolved to include HDMI 1.0, HDMI 1.1, HDMI 1.2, HDMI 1.3, HDMI 1.4, HDMI 2.0, and HDMI 2.1. Different versions differ in supported data rates, transmission bandwidth, color format support, color depth support, spatial color support, audio specifications, and resolution refresh rate limitations. FPGA is a semi-custom circuit within the category of application-specific integrated circuits (ASICs). It is a programmable logic array with abundant wiring resources, reprogrammability, high integration, and relatively low investment, making it widely used in digital circuit design. Control systems implemented using FPGAs can improve system response speed, enhance product scalability, and reduce development costs.

[0003] The HDMI display interface is the most commonly used display interface in the field of video image processing. It eliminates the need for digital-to-analog or analog-to-digital conversion before signal transmission, ensuring the highest quality audio and video signal transmission. Some technologies use FPGA chips to support HDMI display interfaces, such as using HDMI encoding IP provided by FPGA chip manufacturers. However, this requires payment, and porting the product to different platforms necessitates purchasing different HDMI encoding IPs for each platform, resulting in high costs.

[0004] In related technologies, HDMI encoding combined with a serializer is commonly used for encoded display. The serializer operates in both single and double modes. When using HDMI encoding with a serializer, the data rate (data rate, which can be understood as the rate at which the serializer outputs data) supported by the serializer is limited, whether it operates in single or double mode. For example, the SerDes serializer in Intel's Stratix IVGX110 device only supports a maximum data rate of 3.75Gbps in single mode. While it can support higher data rates in double mode, there is a minimum data rate limitation, such as a requirement greater than 1Gbps. However, some commonly used resolutions have lower data rates than this minimum. Therefore, if a lower resolution is selected as the display parameter for the serializer's output signal, the serializer can only operate in single mode. If a higher resolution (with a corresponding data rate greater than the maximum data rate) is selected, the serializer can only operate in double mode.

[0005] There is currently no effective solution to the problem that serializers operating in dual mode cannot be used in low-resolution scenarios in related technologies. Summary of the Invention

[0006] This invention provides an image signal processing method, apparatus, storage medium, and electronic device to at least solve the problem in related technologies that serializers operating in dual mode cannot be used in low-resolution scenarios.

[0007] According to an embodiment of the present invention, an image signal processing method is provided, comprising: determining the current data rate of the output data of the target serializer based on an acquired target data rate and a rate constraint parameter of the target serializer, and determining the clock frequency of the current operating clock of the target encoder based on the current data rate, wherein the target data rate is data corresponding to a target resolution and a target frequency, the target resolution and the target frequency are display parameters for displaying the original image signal, the rate constraint parameter is the minimum data rate of the output data allowed when the target serializer is in dual-mode operation, the target serializer operation modes include single-mode and dual-mode, and the first serialization ratio of the target serializer in single-mode is less than the second serialization ratio in dual-mode; when the target data rate is less than the minimum data rate... In this case, the original image signal undergoes first signal processing to obtain a first encoded input signal, wherein the data in the first encoded input signal is discrete; the first encoded input signal is encoded by the target encoder whose working clock is the current working clock to obtain a first encoded output signal, wherein the data in the first encoded output signal is discrete; the first encoded output signal undergoes target signal processing to obtain a second encoded output signal that matches the current data rate, wherein the data in the second encoded output signal is continuous; the second encoded output signal is serialized by the target serializer whose working mode is the dual-mode according to the second serialization ratio to obtain a first serialized output signal, wherein the first serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters.

[0008] In one exemplary embodiment, determining the current data rate of the output data of the target serializer based on the acquired target data rate and the rate constraint parameters of the target serializer includes: determining the current data rate to be equal to the target data rate when the target data rate is greater than or equal to the minimum data rate; and determining the current data rate to be equal to N times the target data rate when the target data rate is less than the minimum data rate, where N = 2. n , where n is an integer greater than or equal to 1.

[0009] In one exemplary embodiment, determining the clock frequency of the current operating clock of the target encoder based on the current data rate includes: when the target data rate is greater than or equal to the minimum data rate, determining the clock frequency of the current operating clock to be equal to half of the pixel clock frequency corresponding to the original image signal; when the target data rate is less than the minimum data rate, determining the clock frequency of the current operating clock to be equal to the pixel clock frequency corresponding to the original image signal * N / 2.

[0010] In an exemplary embodiment, the first signal processing of the original image signal to obtain a first coded input signal includes: when the target data rate is less than the minimum data rate, adjusting the time length of every two consecutive pixel data in the original image signal from N clock cycles in the current working clock to one clock cycle, and setting an invalid signal with a time length of one clock cycle * (N-1) between every two adjusted pixel data to obtain the first coded input signal, wherein every two consecutive pixel data in the original image signal are consecutive, and the time length of every two consecutive pixel data in the original image signal is N clock cycles in the current working clock.

[0011] In an exemplary embodiment, target signal processing is performed on the first encoded output signal to obtain a second encoded output signal that matches the current data rate. This includes: when the target data rate is less than the minimum data rate, performing a discrete-to-continuous operation on the first encoded output signal to obtain a third encoded output signal, wherein the encoded data in the third encoded output signal is continuous; and performing oversampling processing on the third encoded output signal to obtain the second encoded output signal, wherein the data in the second encoded output signal corresponding to one clock cycle of the current operating clock includes one encoded data, wherein the one encoded data is obtained by copying each bit of the encoded data in the third encoded output signal corresponding to one clock cycle (N-1) times, the number of bits of each encoded data in the second encoded output signal is equal to the number of bits of one encoded data in the third encoded output signal * N, and the encoded data in the second encoded output signal is continuous.

[0012] In an exemplary embodiment, the step of performing a discrete-to-continuous operation on the first encoded output signal to obtain a third encoded output signal includes: adjusting the duration of each group of continuous encoded data in the first encoded output signal from one clock cycle of the current working clock to N clock cycles to obtain the third encoded output signal. Each group of continuous encoded data includes two continuous encoded data sets. The duration of each group of continuous encoded data in the first encoded output signal is one clock cycle of the current working clock. Between each two groups of continuous encoded data in the first encoded output signal is an invalid signal with a duration of one clock cycle * (N-1). The encoded data in the third encoded output signal is continuous, and the duration of each encoded data set in the third encoded output signal is N / 2 clock cycles.

[0013] In one exemplary embodiment, each group of consecutive coded data in the first coded output signal is split into N coded data in the third coded output signal. The N coded data are arranged in clock cycle order from front to back. The first N / 2 coded data in the N coded data are arranged in clock cycle order from back to front to form one coded data in each group of consecutive coded data. The last N / 2 coded data in the N coded data are arranged in clock cycle order from back to front to form another coded data in each group of consecutive coded data. The number of bits of each coded data in the N coded data is 2 / N times the number of bits of each coded data in each group of consecutive coded data.

[0014] In an exemplary embodiment, the oversampling process of the third encoded output signal to obtain the second encoded output signal includes: copying each bit of each encoded data in the third encoded output signal (N-1) times to obtain the second encoded output signal, wherein each encoded data in the second encoded output signal includes M groups of bits arranged in sequence, each group of bits includes N identical bits, and M is the number of bits of one encoded data in the third encoded output signal.

[0015] In one exemplary embodiment, the method further includes: when the target data rate is greater than or equal to the minimum data rate, performing a second signal processing on the original image signal to obtain a second coded input signal, wherein the data in the second coded input signal is continuous; encoding the second coded input signal using the target encoder whose operating clock is the current operating clock to obtain a fourth coded output signal, wherein the data in the fourth coded output signal is continuous; and serializing the fourth coded output signal according to the second serialization ratio using the target serializer whose operating mode is the dual-mode to obtain a second serialized output signal, wherein the second serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters.

[0016] In an exemplary embodiment, the second signal processing of the original image signal to obtain a second coded input signal includes: merging the signals of every two consecutive pixel data in the original image signal into a signal transmitted within one clock cycle of the current operating clock to obtain the second coded input signal, wherein each pixel data in the original image signal is continuous, and the time length of each pixel data in the original image signal is half a clock cycle of the current operating clock.

[0017] According to another embodiment of the present invention, an image signal processing apparatus is also provided, comprising: a first determining module, configured to determine the current data rate of the output data of the target serializer based on an acquired target data rate and a rate constraint parameter of the target serializer, and to determine the clock frequency of the current operating clock of the target encoder based on the current data rate, wherein the target data rate is data corresponding to a target resolution and a target frequency, the target resolution and the target frequency are display parameters for displaying the original image signal, the rate constraint parameter is the minimum data rate of the output data allowed when the target serializer is in a dual-mode operating mode, the operating mode of the target serializer includes a single mode and the dual-mode, and the first serialization ratio of the target serializer in the single-mode is less than the second serialization ratio in the dual-mode; and a first processing module, configured to, when the target data rate is less than the minimum data rate... The original image signal is subjected to a first signal processing to obtain a first encoded input signal, wherein the data in the first encoded input signal is discrete; an encoding module is used to encode the first encoded input signal using the target encoder whose working clock is the current working clock to obtain a first encoded output signal, wherein the data in the first encoded output signal is discrete; a second processing module is used to perform target signal processing on the first encoded output signal to obtain a second encoded output signal that matches the current data rate, wherein the data in the second encoded output signal is continuous; a third processing module is used to perform serialization processing on the second encoded output signal according to the second serialization ratio using the target serializer whose working mode is the dual mode to obtain a first serialized output signal, wherein the first serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters.

[0018] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0019] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0020] According to this invention, the current data rate of the target serializer output data is determined based on the target data rate corresponding to the display parameters of the original image signal and the minimum data rate of the output data allowed by the target serializer in dual mode. The clock frequency of the target encoder's current operating clock is then determined based on the current data rate. When the target data rate is less than the minimum data rate, the original image signal undergoes first signal processing to obtain a first encoded input signal, wherein the data in the first encoded input signal is discrete. The first encoded input signal is then encoded by the target encoder to obtain a first encoded output signal, wherein the data in the first encoded output signal is discrete. The first encoded output signal undergoes target signal processing to obtain a second encoded output signal matching the current data rate, wherein the data in the second encoded output signal is continuous. Finally, the second encoded output signal is serialized using the dual-mode target serializer to obtain a first serialized output signal for display. This invention achieves the goal of processing raw image signals with a target data rate lower than the minimum data rate in dual-mode, avoiding the problem in related technologies that can only process raw image signals with a target data rate lower than the minimum data rate in single-mode, thus failing to achieve compatibility between high and low resolution encoding display schemes. Therefore, it solves the problem in related technologies that the serializer working in dual-mode cannot be used in low-resolution scenarios. Attached Figure Description

[0021] Figure 1 This is a block diagram of the mobile terminal hardware structure of the image signal processing method according to an embodiment of the present invention.

[0022] Figure 2 This is an example diagram of an HDMI display system environment in related technologies;

[0023] Figure 3 This is an example diagram of image signal timing in related technologies;

[0024] Figure 4 This is a schematic diagram illustrating the implementation of the single mode in related technologies;

[0025] Figure 5 This is a schematic diagram of the double mode implementation in related technologies;

[0026] Figure 6 This is an example diagram illustrating the rate limiting of serializers under different operating modes in related technologies;

[0027] Figure 7 This is a flowchart of an image signal processing method according to an embodiment of the present invention;

[0028] Figure 8 This is a block diagram of an HDMI encoding and display scheme according to an embodiment of the present invention;

[0029] Figure 9 This is a high-resolution double-mode encoded input timing diagram according to an embodiment of the present invention;

[0030] Figure 10 This is a low-resolution double-mode encoded input timing diagram according to an embodiment of the present invention;

[0031] Figure 11 This is a timing diagram of the high-resolution double-mode encoded output according to an embodiment of the present invention;

[0032] Figure 12 This is a timing diagram of the low-resolution double-mode encoded output according to an embodiment of the present invention;

[0033] Figure 13 This is a low-resolution discrete-to-continuous timing diagram according to an embodiment of the present invention;

[0034] Figure 14 This is a low-resolution oversampling timing diagram according to an embodiment of the present invention;

[0035] Figure 15 This is a structural block diagram of an image signal processing apparatus according to an embodiment of the present invention. Detailed Implementation

[0036] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0038] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a block diagram of the mobile terminal hardware structure of the image signal processing method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0039] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the image signal processing method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0041] Figure 2 This is an example diagram of an HDMI display system environment in related technologies. In an HDMI display solution implemented based on FPGA, the data source for the image display part may come from the graphics card or other device outputs. After image acquisition and processing, the image is cached in buffer units such as DDR and RAM. Before display, there may also be other information overlay or image processing, such as OSD and GUI overlay, GAMMA brightness adjustment, color conversion, etc.

[0042] Before entering the HDMI encoding module, the image signal needs to be time-converted according to video standard formats (such as CEA-861-D, CEA-861-F, VESA) to generate parallel RGB signals and image control signals (such as the DVI-D interface signals in the aforementioned patent). These image control signals include the data enable signal DE, the horizontal sync signal HS (HSYNC), and the vertical sync signal VS (VSYNC). Figure 3 As shown, Figure 3Taking 1920x1080P@60Hz as an example, the HDMI encoding module encodes the input control signal and 24-bit RGB data (3 channels) according to the HDMI protocol. This module also supports the insertion of audio and auxiliary data, but these are not the focus of this application and will not be discussed further. The encoding module's operating clock, user_clk, depends on the pixel rate of the resolution. For example, the pixel clock for 1920x1080P@60Hz is 148.5MHz, calculated as: 2200 x 1125 x 60 = 148.5 (MHz), where 2200 is the total clock count for one row and 1125 is the total number of rows.

[0043] In related technologies, when implementing HDMI encoded display, there are low-resolution single mode and high-resolution double mode. Figure 4 This is a schematic diagram of the single-mode implementation in related technologies. The serializer serializes the encoded 10-bit data from each channel and transmits it in differential pairs, with a line rate of 10 * 148.5 Mbps = 1.485 Gbps. The serializer varies depending on the FPGA platform and device; common types include LVDS and SerDes. Regardless of the type, the supported data rate is limited in Single-mode (10:1 serialization). For example, Intel's Stratix IV GX110 device only supports LVDS up to 1.6 Gbps, and the Arria10 GX 027 device's SerDes single-mode only supports data rates of 600 Mbps to 3.75 Gbps. Figure 5 This is a schematic diagram of the double-mode implementation in related technologies. If the serializer supports double mode (20:1 serialization), the supported date rate can be as high as 12G or even higher. In this case, the encoding module and other preceding image processing modules operate at a user_clk clock of Data_rate / 20. Each user_clk processes 2 pixels, therefore user_clk = 1 / 2 * pixel_clk. Taking 4K@30Hz resolution as an example, pixel_clk is 297M, date_rate is 2.97G, and operating in double mode, user_clk is 148.5M.

[0044] When implementing HDMI encoding and serialization using an FPGA, the serializer varies depending on the FPGA platform and device. Common types include LVDS and SerDes. Regardless of the type, the supported Date_rate in Single mode (10:1 serialization) is limited, depending on the FPGA platform and device. Generally, the maximum supported resolution is around 4K@30. For example, the SerDes serializer on Intel's Stratix IV GX110 device is limited to supporting only 3.75G. Figure 6 This is an example diagram illustrating the rate limits of serializers under different operating modes in related technologies. While Double mode can support high resolutions, it has a minimum rate limit, such as... Figure 6 As shown in Table 1, a speed greater than 1Gbps is required. However, some commonly used low-resolution resolutions have speeds lower than this limit.

[0045] Table 1

[0046] resolution frequency 800*600@60Hz 40 1024*768@60Hz 65 1366*768@60Hz 72 1280*720@60Hz 74.25 1280*720@50Hz 74.25 1920*1080I@50Hz 74.25 1920*1080I@60Hz 74.25 1024*768@75Hz 78.75 1280*800@60Hz 83.5

[0047] In related technologies, the HDMI encoder + serializer method can only be fixed in double mode and single mode, and switching between single mode and double mode is not supported.

[0048] This embodiment provides a method for processing image signals. Figure 7 This is a flowchart of an image signal processing method according to an embodiment of the present invention, such as... Figure 7 As shown, the process includes the following steps:

[0049] Step S702: Based on the obtained target data rate and the rate constraint parameters of the target serializer, determine the current data rate of the output data of the target serializer, and determine the clock frequency of the current working clock of the target encoder based on the current data rate. Here, the target data rate is the data corresponding to the target resolution and the target frequency, the target resolution and the target frequency are display parameters used to display the original image signal, and the rate constraint parameters are the minimum data rate of the output data allowed when the target serializer is in dual mode. The working mode of the target serializer includes single mode and dual mode. The first serialization ratio of the target serializer in single mode is less than the second serialization ratio in dual mode.

[0050] Step S704: When the target data rate is less than the minimum data rate, perform a first signal processing on the original image signal to obtain a first coded input signal, wherein the data in the first coded input signal is discrete;

[0051] Step S706: The first encoded input signal is encoded by the target encoder whose working clock is the current working clock to obtain a first encoded output signal, wherein the data in the first encoded output signal is discrete;

[0052] Step S708: Perform target signal processing on the first encoded output signal to obtain a second encoded output signal that matches the current data rate, wherein the data in the second encoded output signal is continuous;

[0053] Step S710: The target serializer, operating in the dual-mode mode, performs serialization processing on the second encoded output signal according to the second serialization ratio to obtain a first serialized output signal, wherein the first serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters.

[0054] Through the above steps, based on the target data rate corresponding to the display parameters of the original image signal and the minimum data rate of the target serializer's output data allowed in dual mode, the current data rate of the target serializer's output data is determined, and the clock frequency of the target encoder's current operating clock is determined based on the current data rate. When the target data rate is less than the minimum data rate, the original image signal undergoes first signal processing to obtain a first encoded input signal, wherein the data in the first encoded input signal is discrete. The first encoded input signal is then encoded by the target encoder to obtain a first encoded output signal, wherein the data in the first encoded output signal is discrete. The first encoded output signal undergoes target signal processing to obtain a second encoded output signal that matches the current data rate, wherein the data in the second encoded output signal is continuous. Finally, the second encoded output signal is serialized using the dual-mode target serializer to obtain a first serialized output signal for display. This invention achieves the goal of processing raw image signals with a target data rate lower than the minimum data rate in dual-mode, avoiding the problem in related technologies that can only process raw image signals with a target data rate lower than the minimum data rate in single-mode, thus failing to achieve compatibility between high and low resolution encoding display schemes. Therefore, it solves the problem in related technologies that the serializer operating in dual-mode cannot be used in low-resolution scenarios.

[0055] The entity performing the above steps can be a processor, an application program, or a terminal, but is not limited to these.

[0056] In the above embodiment, taking a target resolution of 1280x720@60Hz as an example, the display parameters for displaying the original image signal include a target resolution of 1280x720 and a target frequency (or frame rate) of 60Hz. If the serializer uses an Arria10 GX 027 SerDes device, the single mode (corresponding to the single mode mentioned above) can only support a data rate of 600Mbps–3.75Gbps, while the minimum rate (corresponding to the minimum data rate) limit of the double mode (corresponding to the dual mode mentioned above) is 1Gbps. The pixel clock (or original resolution pixel clock, or original resolution pixel clock frequency) corresponding to the above display parameters is pixel_clk = 74.25M, so the target data rate (or original data rate) is 0.7425Gbps (less than the minimum rate limit). If the original image signal is to be encoded and displayed in double mode, the target data rate does not match the minimum rate limit. Figure 6As shown, a speed greater than 1Gbps is required. In this case, the current data rate of the target serializer output data is determined. For example, the current data rate is determined to be twice the target data rate (or original data rate), such as 1.485Gbps. The clock frequency of the target encoder's current operating clock is also determined. For example, the clock frequency of the current operating clock = the aforementioned pixel clock = 74.25MHz. When the target data rate is less than the minimum data rate, for example, in applications where low resolution is selected as the display parameter for the output signal, the original image signal undergoes first signal processing to obtain a first encoded input signal. The data in the first encoded input signal is discrete, meaning the original image signal is processed discretely. For example, each clock cycle of the current operating clock in the first encoded input signal includes two pixel data points, with one clock cycle of invalid data between each two pixel data points. The first encoded input signal is encoded using the target encoder, whose current operating clock is used, to obtain a first encoded output signal. For example, a discrete data valid signal DE(Data) is constructed for the target encoder. Enable (or data enable signal) timing, for example, the clock period of the DE timing is twice the clock period of the current working clock, and the data in the first encoded output signal is discrete; then the first encoded output signal is processed by target signal processing to obtain a second encoded output signal that matches the current data rate. For example, the data in the first encoded output signal is converted from discrete to continuous and then oversampled to obtain the second encoded output signal. In this way, the data in the second encoded output signal is continuous; then, the second encoded output signal is serialized according to the second serialization ratio (e.g., 20:1) through a target serializer with dual working mode to obtain a first serialized output signal. This first serialized output signal is used to be displayed with the target resolution and target frequency as display parameters. This invention achieves the goal of processing raw image signals with a target data rate lower than the minimum data rate in dual-mode, avoiding the problem in related technologies that can only process raw image signals with a target data rate lower than the minimum data rate in single-mode, thus failing to achieve compatibility between high and low resolution encoding display schemes. Therefore, it solves the problem in related technologies that the serializer operating in dual-mode cannot be used in low-resolution scenarios.

[0057] In an optional embodiment, determining the current data rate of the target serializer output data based on the acquired target data rate and the rate constraint parameters of the target serializer includes: if the target data rate is greater than or equal to the minimum data rate, determining the current data rate to be equal to the target data rate; if the target data rate is less than the minimum data rate, determining the current data rate to be equal to N times the target data rate, where N = 2.n In this embodiment, when the target resolution (or original resolution) is 4k@60Hz, the corresponding pixel clock (pixel_clk) is 594MHz, and the target data rate (or original data rate) is 5.94Gbps (greater than 1Gbps), the current data rate of the target serializer is determined to be equal to the target data rate, i.e., 5.94Gbps. Conversely, when the target resolution (or original resolution) is 1280x720@60Hz, the corresponding pixel clock (pixel_clk) is 74.25MHz, and the target data rate (or original data rate) is 0.7425Gbps (less than 1Gbps), the current data rate of the target serializer is determined to be N times the target data rate, for example, twice (i.e., 1.485Gbps), four times, or eight times, etc. This embodiment achieves the goal of setting a reasonable current data rate for the target serializer according to the requirements of different display parameters.

[0058] In an optional embodiment, determining the clock frequency of the target encoder's current operating clock based on the current data rate includes: when the target data rate is greater than or equal to the minimum data rate, determining the clock frequency of the current operating clock to be equal to half of the pixel clock frequency corresponding to the original image signal; when the target data rate is less than the minimum data rate, determining the clock frequency of the current operating clock to be equal to the pixel clock frequency corresponding to the original image signal * N / 2. Where N = 2 n n is an integer greater than or equal to 1, such as N = 2, 4, 8 or other values. In this embodiment, if the target resolution (or original resolution) is 4k@60Hz, the corresponding pixel clock (or pixel clock frequency) pixel_clk = 594M, and the target data rate (or original data rate) is 5.94Gbps (greater than 1Gbps), the clock frequency of the current working clock of the target encoder is determined to be equal to 1 / 2 of the pixel clock frequency, i.e., 297M; while if the target resolution (or original resolution) is 1280x720@60Hz, the corresponding pixel clock pixel_clk = 74.25M, and the target data rate (or original data rate) is 0.7425Gbps (less than 1Gbps), the clock frequency of the current working clock of the target encoder is determined to be equal to the pixel clock frequency * N / 2. For example, when N = 2, user_clk / pixel_clk = 1, i.e., 74.25M; when N = 4, user_clk / pixel_clk = 2. The user_clk mentioned above refers to the current clock frequency of the target encoder. This embodiment achieves the goal of setting a reasonable clock frequency for the target encoder according to the requirements of different display parameters.

[0059] In this embodiment of the application, unless otherwise specified, user_clk and pixel_clk both refer to clock frequency.

[0060] In an optional embodiment, the first signal processing of the original image signal to obtain a first coded input signal includes: when the target data rate is less than the minimum data rate, adjusting the time length of every two consecutive pixel data in the original image signal from N clock cycles in the current working clock to one clock cycle, and setting an invalid signal with a time length of one clock cycle * (N-1) between every two adjusted pixel data to obtain the first coded input signal, wherein every two consecutive pixel data in the original image signal are continuous, and the time length of every two consecutive pixel data in the original image signal is N clock cycles in the current working clock. In this embodiment, the time length of every two pixel data in the original image signal is adjusted from N clock cycles of the current working clock to one clock cycle, that is, one clock cycle includes two pixel data. Then, an invalid signal of (N-1) clock cycles (referring to the clock cycle of the current working clock) is set between every two pixel data. That is, only one clock cycle of the current working clock is valid data, and the other (N-1) clock cycles correspond to invalid data. For example, if N=2, then valid data appears once every two cycles, and the interval between valid data is one cycle. When N=4, there is one valid data point out of four cycles, with a three-cycle interval between valid data points. These cycles correspond to the period of user_clk (i.e., the clock cycle of the current operating clock). N user_clk cycles correspond to two cycles of the original image pixel_clk, i.e., user_period / pixel_period = 2 / N. user_period refers to the clock cycle of the current operating clock, and pixel_period refers to the clock cycle of the original image signal. Thus, the data in the resulting first encoded input signal is discrete. This embodiment achieves the goal of discrete processing of the original image signal.

[0061] In an optional embodiment, target signal processing is performed on the first encoded output signal to obtain a second encoded output signal that matches the current data rate. This includes: when the target data rate is less than the minimum data rate, performing a discrete-to-continuous operation on the first encoded output signal to obtain a third encoded output signal, wherein the encoded data in the third encoded output signal is continuous; and performing oversampling processing on the third encoded output signal to obtain the second encoded output signal, wherein the data in the second encoded output signal corresponding to one clock cycle of the current operating clock includes one encoded data, wherein the one encoded data is obtained by copying each bit of the encoded data in the third encoded output signal corresponding to one clock cycle (N-1) times, the number of bits of each encoded data in the second encoded output signal is equal to the number of bits of one encoded data in the third encoded output signal * N, and the encoded data in the second encoded output signal is continuous.In this embodiment, the data in the first coded output signal is discrete, requiring a discrete-to-continuous conversion operation. For example, the effective data portion of the first coded output signal consists of two coded data units (e.g., 20 bits), where each coded data unit corresponds to a pixel in the original image signal. After performing the discrete-to-continuous conversion operation on the first coded output signal, it is transformed into continuous coded data (e.g., two 10-bit units or four 5-bit units), resulting in a continuous data unit in the third coded output signal. Then, the third coded output signal undergoes oversampling processing, for example, oversampling the data in the third coded output signal... The continuous encoded data (e.g., 10 bits) is copied to obtain continuous 20 bits of data. Taking N=2 as an example, the encoded data corresponding to each clock cycle in the third encoded output signal is 10 bits, such as bit[9:0]. Then each bit in the encoded data is copied (N-1) times, such as the encoded data obtained after copying: {bit[9], bit[9], bit[8], bit[8], ..., bit[0], bit[0]}. If N=4, then the encoded data corresponding to each clock cycle in the third encoded output signal is 5 bits, such as the first The first clock cycle corresponds to bit[4:0], and the second clock cycle corresponds to bit[9:5]. Therefore, each bit of the encoded data in the first clock cycle is copied (N-1) times. For example, the encoded data obtained after copying in the first clock cycle is: {bit[4], bit[4], bit[4], bit[4], ..., bit[0], bit[0], bit[0], bit[0]}, and so on. The encoded data in each of the other clock cycles is copied to obtain the second encoded output signal. Each bit in the second encoded output signal obtained after sampling processing... The number of bits in the code data is equal to the number of bits in one coded data segment of the third coded output signal multiplied by N. For example, when N = 2, one coded data segment in the third coded output signal is 10 bits, and the number of bits in each coded data segment of the second coded output signal is 10 bits * 2 = 20 bits. When N = 4, one coded data segment in the third coded output signal is 5 bits, and the number of bits in each coded data segment of the second coded output signal is 5 bits * 4 = 20 bits. That is, each coded data segment in the second coded output signal is a continuous 20 bits, in order to achieve matching with the current data rate of the target serializer. Through this embodiment, the purpose of processing the first coded output signal to obtain a second coded output signal that matches the current data rate of the target serializer is achieved.

[0062] In an optional embodiment, the step of performing a discrete-to-continuous operation on the first encoded output signal to obtain a third encoded output signal includes: adjusting the time length of each group of continuous encoded data in the first encoded output signal from one clock cycle of the current working clock to N clock cycles to obtain the third encoded output signal. Each group of continuous encoded data includes two continuous encoded data sets. The time length of each group of continuous encoded data in the first encoded output signal is one clock cycle of the current working clock. Between each two groups of continuous encoded data in the first encoded output signal is an invalid signal with a time length of one clock cycle * (N-1). The encoded data in the third encoded output signal is continuous, and the time length of each encoded data set in the third encoded output signal is N / 2 clock cycles. In this embodiment, the discrete-to-continuous operation may include adjusting the duration of each group of continuous encoded data in the first encoded output signal from one clock cycle of the current working clock to N clock cycles. For example, a group of continuous encoded data in the first encoded output signal is 20 bits, and the duration of each group of continuous encoded data in the first encoded output signal is one clock cycle of the current working clock. A group of continuous encoded data includes two continuous encoded data, such as each continuous encoded data being 10 bits. That is, the duration of a group of continuous encoded data (e.g., 20 bits) is adjusted from one clock cycle to N clock cycles to obtain the third encoded output signal. That is, the duration of each encoded data in the adjusted third encoded output signal is N / 2 clock cycles, and the pixel data in the third encoded output signal is continuous. The duration of each group of continuous coded data in the first coded output signal is one clock cycle of the current working clock, while there is an invalid signal with a duration of (N-1) clock cycles between every two groups of continuous coded data. For example, when N=2, there is 1 clock cycle of invalid data between every two groups of continuous coded data, and when N=4, there is 3 clock cycles of invalid data between every two groups of continuous coded data. Through this embodiment, the discrete data in the first coded output signal is converted into continuous data.

[0063] In an optional embodiment, each group of consecutive coded data in the first coded output signal is split into N coded data in the third coded output signal. The N coded data are arranged in clock cycle order from front to back. The first N / 2 coded data in the N coded data are arranged in clock cycle order from back to front to form one coded data in each group of consecutive coded data. The last N / 2 coded data in the N coded data are arranged in clock cycle order from back to front to form the other coded data in each group of consecutive coded data. The number of bits in each coded data in the N coded data is 2 / N times the number of bits in each coded data in each group of consecutive coded data. For example, each group of consecutive encoded data in the first encoded output signal is 20 bits, such as bit[19:0]. This 20-bit data is split into N encoded data in the third encoded output signal. When N=2, the third encoded output signal has 2 encoded data. The first encoded data (corresponding to the first N / 2 encoded data) corresponds to one encoded data of a group of consecutive encoded data in the first encoded output signal (e.g., bit[9:0]), and the second encoded data (corresponding to the last N / 2 encoded data) corresponds to another encoded data of a group of consecutive encoded data in the first encoded output signal (e.g., bit[19:10]). When N=4, the third encoded output signal has 4 encoded data. The first two encoded data form the third encoded data in the clock cycle from back to front. A set of continuously encoded data in an encoded output signal is one encoded data. For example, in the third encoded output signal, the first encoded data is bit[4:0], the second encoded data is bit[9:5], and the two are arranged in reverse order to form bit[9:0] (i.e., one encoded data of a set of continuously encoded data in the first encoded output signal). The last two encoded data are arranged in reverse order according to the clock cycle to form another encoded data of a set of continuously encoded data in the first encoded output signal. For example, in the third encoded output signal, the third encoded data is bit[14:10], the fourth encoded data is bit[19:15], and the two are arranged in reverse order to form bit[19:10] (i.e., another encoded data of a set of continuously encoded data in the first encoded output signal). Through this embodiment, the purpose of converting discrete data in the first encoded output signal into continuous data is achieved.

[0064] In an optional embodiment, the oversampling process of the third encoded output signal to obtain the second encoded output signal includes: copying each bit of each encoded data in the third encoded output signal (N-1) times to obtain the second encoded output signal, wherein each encoded data in the second encoded output signal includes M groups of bits arranged in sequence, each group of bits includes N identical bits, and M is the number of bits of one encoded data in the third encoded output signal.Taking N=2 as an example, the duration of each group of continuous encoded data (e.g., 20 bits) in the first encoded output signal is one clock cycle. The encoded data in the third encoded output signal is continuous, and the duration of each encoded data (e.g., 20 / N=10 bits) in the third encoded output signal is N / 2 clock cycles (i.e., one clock cycle). That is, the encoded data corresponding to the first clock cycle in the third encoded output signal is bit[9:0], and the encoded data corresponding to the second clock cycle in the third encoded output signal is bit[19:10]. At this time, the corresponding M=10 bits are obtained. Therefore, each bit of each encoded data is copied once to obtain... The encoded data of the first clock cycle in the second encoded output signal is {bit[9], bit[9], bit[8], bit[8], ..., bit[0], bit[0]}, and the encoded data of the second clock cycle in the second encoded output signal is {bit

[19] , bit

[19] , bit

[18] , bit

[18] , ..., bit

[10] , bit

[10] }. When N=4, the time length of each group of continuous encoded data (e.g., 20 bits) in the first encoded output signal is one clock cycle, that is, the time length of each group of continuous encoded data in the first encoded output signal is adjusted from one clock cycle. The encoding data in the third encoded output signal is continuous, and the time length of each encoded data (e.g., 20 / N = 5 bits) in the third encoded output signal is one clock cycle. That is, the encoded data corresponding to the first clock cycle of the third encoded output signal is bit[4:0], the encoded data corresponding to the second clock cycle of the third encoded output signal is bit[9:5], the encoded data corresponding to the third clock cycle of the third encoded output signal is bit[14:10], and the encoded data corresponding to the fourth clock cycle of the third encoded output signal is bit[19:15]. At this time, the corresponding M = 5 bits, then... Each bit of each encoded data is copied three times. The encoded data for the first clock cycle of the resulting second encoded output signal is {bit[4], bit[4], bit[4], bit[4], ..., bit[0], bit[0], bit[0], bit[0]}. The encoded data for the second clock cycle of the resulting second encoded output signal is {bit[9], bit[9], bit[9], bit[9], ..., bit[5], bit[5], bit[5]}, and so on. The encoded data for the third and fourth clock cycles of the second encoded output signal can be obtained in this way. Through this embodiment, the purpose of oversampling the third encoded output signal is achieved to obtain a second encoded output signal that matches the current data rate of the target serializer.

[0065] In an optional embodiment, the method further includes: when the target data rate is greater than or equal to the minimum data rate, performing a second signal processing on the original image signal to obtain a second coded input signal, wherein the data in the second coded input signal is continuous; encoding the second coded input signal using the target encoder whose operating clock is the current operating clock to obtain a fourth coded output signal, wherein the data in the fourth coded output signal is continuous; and serializing the fourth coded output signal according to the second serialization ratio using the target serializer whose operating mode is the dual-mode to obtain a second serialized output signal, wherein the second serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters. In this embodiment, when the target data rate is greater than or equal to the minimum data rate, for example, in an application scenario where high resolution is selected as the display parameter for the output signal, the original image signal undergoes a second signal processing to obtain a second encoded input signal. The data in the second encoded input signal is continuous. For example, when the target data rate is greater than or equal to the minimum data rate, the clock frequency of the current operating clock of the target encoder can be determined to be equal to half the pixel clock frequency. Each clock cycle of the current operating clock in the second encoded input signal includes two pixel data points, which can be obtained by merging every two consecutive pixel data points in the original image signal. The second encoded input signal is then encoded by the target encoder to obtain a fourth encoded output signal. For example, a continuous valid data signal DE(Data) is constructed for the target encoder. The enabled signal (or data enable signal) timing results in continuous data in the fourth encoded output signal; that is, 20 bits of continuous data are output in each clock cycle of the current operating clock. Then, the fourth encoded output signal is serialized by the dual-mode target serializer according to a second serialization ratio (e.g., 20:1) to obtain a second serialized output signal. This second serialized output signal is used to display the image with the target resolution and target frequency as display parameters. This embodiment achieves the goal of processing raw image signals with a target data rate greater than or equal to the minimum data rate in dual-mode, thus achieving simultaneous compatibility with processing schemes for high and low resolution image signals and reducing development costs.

[0066] In an optional embodiment, the second signal processing of the original image signal to obtain a second encoded input signal includes: merging the signals of every two consecutive pixel data in the original image signal into a signal transmitted within one clock cycle of the current operating clock to obtain the second encoded input signal. Each pixel data in the original image signal is consecutive, and the time length of each pixel data in the original image signal is half a clock cycle of the current operating clock. In this embodiment, by merging every two consecutive pixel data in the original image signal, a second encoded input signal transmitting two pixel data within one clock cycle of the current operating clock is obtained. In practical applications, for example, merging two consecutive pixel data D1 and D2 in the original image signal to obtain D2D1, that is, using the earlier-timed single pixel data D1 as the low-order bit of the merged data, and using the later-timed single pixel data D2 as the high-order bit of the merged data. Through this embodiment, the purpose of encoding high-resolution image signals is achieved.

[0067] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. The present invention will be specifically described below with reference to the embodiments.

[0068] Figure 8 This is a block diagram of an HDMI encoding and display scheme according to an embodiment of the present invention. The implementation process of the scheme is as follows:

[0069] 1. Serializer rate (corresponding to the aforementioned current data rate) setting: If the original data rate of the resolution (corresponding to the aforementioned target data rate) is greater than or equal to the resolution limit of the minimum rate of the serializer double mode (corresponding to the aforementioned dual mode) (corresponding to the aforementioned minimum data rate), then set the Date rate (i.e., the aforementioned current data rate) to be the same as the original rate; otherwise, set the serializer rate to twice the original rate.

[0070] 2. The user_clk (corresponding to the clock frequency of the current working clock) obtained from the above data rate settings are as follows: the former corresponds to user_clk = original resolution pixel clock pixel_clk / 2, and the latter corresponds to user_clk = pixel_clk (corresponding to the aforementioned pixel clock frequency);

[0071] 3. If the target resolution is high (the original data rate of the resolution is greater than or equal to the minimum rate of the serializer in double mode), the user_clk for image processing and encoding is pixel / 2. The timing sequence is constructed according to continuous DE, such as... Figure 9 As shown, the encoded data is also continuous and valid, and its timing is consistent with the timing of entering the encoding module.

[0072] 4. If the target resolution is low (the original data rate is less than the minimum rate of the serializer in double mode), and the user_clk for image processing and encoding is pixel, construct the timing sequence according to discrete DE, such as... Figure 10 As shown, the encoded data is also discrete and valid, and its timing is consistent with the timing of entering the encoding module.

[0073] 5. The HDMI encoding module operates in double mode. Based on the input Double_Data_Enable signal, it determines whether the current data belongs to the video data (active data) to be encoded; based on the input Double_blank_Enable signal, it determines whether the current data belongs to the blanking area data (including Vsync / Hsync signals, audio signals and other auxiliary signal encoding) to be encoded; and it determines the boundary between the two areas through the Double_Valid input signal.

[0074] 6. If the target resolution is high, the encoding output timing is as follows: Figure 11 As shown, the encoded output data is directly selected as the input data of the serializer, and then serialized and output.

[0075] 7. If the target resolution is low, the encoding output timing is as follows: Figure 12 As shown, the encoded output data still needs to be converted from discrete to continuous and oversampled before it can be used as the input data for the serializer.

[0076] 8. The discrete-to-continuous module converts the input discrete double-mode data (20 bits) into continuous single-mode data (10 bits). The timing diagram is as follows: Figure 13 As shown.

[0077] 9. The oversampling module copies the input continuous single-mode data pixel by pixel and oversamples it into continuous double-mode data (20-bit) Single_to_Double. The timing diagram is as follows. Figure 14 As shown, Figure 14The relationship between enc_b1b1 and enc_b1 is enc_b1b1={enc_b1[9],enc_b1[9],enc_b1[8],enc_b1[8],enc_b1[7],enc_b1[7],enc_b1[6],enc_b1[6],enc_b1[5],enc_b1[5],enc_b1[4],enc_b1[4],enc_b1[3],enc_b1[3],enc_b1[2],enc_b1[2],enc_b1[1],enc_b1[1],enc_b1[0],enc_b1[0]}, that is, each bit of the 10-bit data is copied once to become 20 bits.

[0078] The present invention will now be described in conjunction with specific embodiments. Specific Implementation Example 1

[0080] 1-1. In this implementation case, it is assumed that the target resolution is 4k@60Hz, the pixel clock frequency is pixel_clk = 594M, and the data rate is 5.94Gbps, according to the VESA standard (e.g. Figure 3 Each row has a total of 4400 pixels and 3840 effective pixels. The serializer uses an Arria10 GX 027 Serdes device. In single mode, it can only support a data rate of 600Mbps–3.75Gbps, and in double mode, the minimum rate limit is 1Gbps.

[0081] 1-2. Serializer rate setting: Since 5.94G > 3.75G and 5.94G > 1G, use double mode and set the Serdes' Date rate to be the same as the original rate, i.e., 5.94G.

[0082] 1-3. The user_clk obtained from the above data tate settings is: original resolution pixel clock pixel_clk / 2 = 594M / 2 = 297M;

[0083] 1-4. According to clock 297M, according to Figure 9 A continuous DE timing sequence is constructed and sent to the encoding module. The total number of clocks per line is 4400 / 2 = 2200, and the number of clocks for effective pixels is 3840 / 2 = 1920.

[0084] 1-5. The HDMI encoding module operates in double mode. Based on the input Double_Data_Enable signal, it determines whether the current data belongs to the video data (active data) to be encoded; based on the input Double_blank_Enable signal, it determines whether the current data belongs to the blank area data (including Vsync / Hsync signals, audio signals and other auxiliary signal encoding) to be encoded; and it determines the boundary between the two areas through the Double_Valid input signal.

[0085] 1-6, Encoding output timing is as follows Figure 11 As shown, the encoded output data is directly selected as the input data of the serializer, and then serialized and output. Specific Implementation Example 2

[0087] 2-1. In this implementation case, it is assumed that the target resolution is 1280x720@60Hz, the pixel clock frequency is pixel_clk = 74.25M, and the data rate is 0.7425Gbps, according to the VESA standard (e.g. Figure 3 Each row has 1650 total pixels and 1280 effective pixels. The serializer uses an Arria10 GX 027 SerDes device. In single mode, it can only support a data rate of 600Mbps–3.75Gbps, and in double mode, the minimum rate limit is 1Gbps.

[0088] 2-2. Serializer rate setting: Since 0.7425G < 1G, set the Date rate to twice the original rate, i.e., 1.485G;

[0089] 2-3. The user_clk obtained from the above data tate settings is the original resolution pixel clock (pixel_clk = 74.25M).

[0090] 2-4. According to the clock 74.25M, according to Figure 10 Discrete DE timing sequences are constructed and sent to the encoding module. The total number of clock cycles per line is 1650, and the number of clock cycles for effective pixels is 1280 / 2 = 640.

[0091] 2-5. The HDMI encoding module operates in double mode. Based on the input Double_Data_Enable signal, it determines whether the current data belongs to the video data (active data) to be encoded; based on the input Double_blank_Enable signal, it determines whether the current data belongs to the blank area data (including Vsync / Hsync signals, audio signals and other auxiliary signal encoding) to be encoded; and it determines the boundary between the two areas through the Double_Valid input signal.

[0092] 2-6. Encoding output timing is as follows Figure 12 In each row of 1650 cycles, only 1650 / 2 = 825 data points are actually encoded data; the other half of the cycles are invalid data. That is, the valid data is 20 bits / clk * (1650 / 2)clk. Then it enters the discrete-to-continuous conversion module.

[0093] 2-7. The discrete-to-continuous module converts the input discrete double-mode data (20 bits) into continuous single-mode data (10 bits), with the effective data being 10 bits / clk * 1650clk. The timing diagram is as follows. Figure 13 As shown;

[0094] 2-8. The oversampling module copies the input continuous single-mode data pixel by pixel and oversamples it into continuous double-mode data (20-bit) Single_to_Double. See the timing diagram. Figure 14 .

[0095] In the above embodiments, in double mode, the low resolution is first time-discretized, then encoded and converted from discrete to continuous, and then oversampling is used to achieve compatibility with high resolution; thus achieving the goal of using FPGA logic and serializer to replace IP and peripheral HDMI conversion chips.

[0096] The above embodiments overcome the maximum date rate limitation of single mode and the minimum rate limitation of double mode. High resolution and low resolution can be compatible in a single solution. Compared with related technologies, it has the following advantages: 1) No additional HDMI conversion chip is required, and there is no need to purchase HDMI decoding IP, saving product costs; 2) It supports various versions of HDMI protocols, is easily expandable, and can be simplified and customized according to product needs, improving the product's cost-effectiveness and reusability, making it highly practical. The encoding module is implemented purely with logic, and it can be ported and promoted to different platforms simply by replacing the serializer.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0098] This embodiment also provides an image signal processing device. Figure 15 This is a structural block diagram of an image signal processing apparatus according to an embodiment of the present invention, such as... Figure 15 As shown, the device includes:

[0099] The first determining module 1502 is used to determine the current data rate of the output data of the target serializer based on the acquired target data rate and the rate constraint parameters of the target serializer, and to determine the clock frequency of the current operating clock of the target encoder based on the current data rate. The target data rate is the data corresponding to the target resolution and the target frequency. The target resolution and the target frequency are display parameters used to display the original image signal. The rate constraint parameters are the minimum data rate of the output data allowed when the target serializer is in dual mode. The operating mode of the target serializer includes single mode and dual mode. The first serialization ratio of the target serializer in single mode is less than the second serialization ratio in dual mode.

[0100] The first processing module 1504 is configured to perform first signal processing on the original image signal to obtain a first coded input signal when the target data rate is less than the minimum data rate, wherein the data in the first coded input signal is discrete.

[0101] The encoding module 1506 is used to encode the first encoding input signal using the target encoder whose current working clock is the working clock, to obtain a first encoding output signal, wherein the data in the first encoding output signal is discrete;

[0102] The second processing module 1508 is used to perform target signal processing on the first encoded output signal to obtain a second encoded output signal that matches the current data rate, wherein the data in the second encoded output signal is continuous;

[0103] The third processing module 1510 is used to perform serialization processing on the second encoded output signal according to the second serialization ratio through the target serializer in the dual-mode working mode to obtain a first serialized output signal, wherein the first serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters.

[0104] In an optional embodiment, the first determining module 1502 includes: a first determining unit, configured to determine the current data rate as equal to the target data rate when the target data rate is greater than or equal to the minimum data rate; and a second determining unit, configured to determine the current data rate as equal to N times the target data rate when the target data rate is less than the minimum data rate, wherein N = 2. n , where n is an integer greater than or equal to 1.

[0105] In an optional embodiment, the first determining module 1502 includes: a third determining unit, configured to determine the clock frequency of the current working clock as equal to half of the pixel clock frequency corresponding to the original image signal when the target data rate is greater than or equal to the minimum data rate; and a fourth determining unit, configured to determine the clock frequency of the current working clock as equal to the pixel clock frequency * N / 2 corresponding to the original image signal when the target data rate is less than the minimum data rate.

[0106] In an optional embodiment, the first processing module 1504 includes: a first processing unit, configured to, when the target data rate is less than the minimum data rate, adjust the time length of every two consecutive pixel data in the original image signal from N clock cycles in the current working clock to one clock cycle, and set an invalid signal with a time length of one clock cycle * (N-1) between every two adjusted pixel data to obtain the first encoded input signal, wherein every two consecutive pixel data in the original image signal are consecutive, and the time length of every two consecutive pixel data in the original image signal is N clock cycles in the current working clock.

[0107] In an optional embodiment, the second processing module 1508 includes: a conversion unit, configured to perform a discrete-to-continuous operation on the first encoded output signal to obtain a third encoded output signal when the target data rate is less than the minimum data rate, wherein the encoded data in the third encoded output signal is continuous; and a sampling unit, configured to perform oversampling processing on the third encoded output signal to obtain a second encoded output signal, wherein the data in the second encoded output signal corresponding to one clock cycle of the current working clock includes one encoded data, the encoded data being data obtained by copying each bit of the encoded data in the third encoded output signal corresponding to one clock cycle (N-1) times, the number of bits of each encoded data in the second encoded output signal being equal to the number of bits of one encoded data in the third encoded output signal * N, and the encoded data in the second encoded output signal being continuous.

[0108] In an optional embodiment, the conversion unit includes a processing subunit, configured to adjust the duration of each group of continuous encoded data in the first encoded output signal from one clock cycle of the current operating clock to N clock cycles to obtain the third encoded output signal, wherein each group of continuous encoded data includes two continuous encoded data, the duration of each group of continuous encoded data in the first encoded output signal is one clock cycle of the current operating clock, and there is an invalid signal with a duration of one clock cycle * (N-1) between every two groups of continuous encoded data in the first encoded output signal, and the encoded data in the third encoded output signal is continuous, and the duration of each encoded data in the third encoded output signal is N / 2 clock cycles.

[0109] In an optional embodiment, each group of consecutive coded data in the first coded output signal is split into N coded data in the third coded output signal. The N coded data are arranged in clock cycle order from front to back. The first N / 2 coded data in the N coded data are arranged in clock cycle order from back to front to form one coded data in each group of consecutive coded data. The last N / 2 coded data in the N coded data are arranged in clock cycle order from back to front to form the other coded data in each group of consecutive coded data. The number of bits in each coded data in the N coded data is 2 / N times the number of bits in each coded data in each group of consecutive coded data.

[0110] In an optional embodiment, the sampling unit includes a copying subunit for copying each bit of each coded data in the third coded output signal (N-1) times to obtain the second coded output signal, wherein each coded data in the second coded output signal includes M groups of bits arranged in sequence, each group of bits includes N identical bits, and M is the number of bits of one coded data in the third coded output signal.

[0111] In an optional embodiment, the above apparatus further includes: a fourth processing module, configured to perform second signal processing on the original image signal to obtain a second encoded input signal when the target data rate is greater than or equal to the minimum data rate, wherein the data in the second encoded input signal is continuous; a fifth processing module, configured to encode the second encoded input signal using the target encoder whose working clock is the current working clock to obtain a fourth encoded output signal, wherein the data in the fourth encoded output signal is continuous; and a sixth processing module, configured to perform serialization processing on the fourth encoded output signal according to the second serialization ratio using the target serializer whose working mode is the dual-mode, to obtain a second serialized output signal, wherein the second serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters.

[0112] In an optional embodiment, the fourth processing module includes: a second processing unit, configured to merge the signals of every two consecutive pixel data in the original image signal into a signal transmitted within one clock cycle of the current working clock to obtain the second coded input signal, wherein each pixel data in the original image signal is continuous, and the time length of each pixel data in the original image signal is half a clock cycle of the current working clock.

[0113] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0114] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0115] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0116] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0117] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0118] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0119] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0120] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing image signals, characterized in that, include: Based on the acquired target data rate and the rate constraint parameters of the target serializer, the current data rate of the output data of the target serializer is determined, and the clock frequency of the current operating clock of the target encoder is determined based on the current data rate. Here, the target data rate is the data corresponding to the target resolution and the target frequency, the target resolution and the target frequency are display parameters used to display the original image signal, and the rate constraint parameters are the minimum data rate of the output data allowed when the target serializer is in dual mode. The operating modes of the target serializer include single mode and dual mode, and the first serialization ratio of the target serializer in single mode is less than the second serialization ratio in dual mode. When the target data rate is less than the minimum data rate, the original image signal is subjected to a first signal processing to obtain a first coded input signal, wherein the data in the first coded input signal is discrete; The first encoded input signal is encoded by the target encoder whose working clock is the current working clock to obtain a first encoded output signal, wherein the data in the first encoded output signal is discrete; The first encoded output signal is subjected to target signal processing to obtain a second encoded output signal that matches the current data rate, wherein the data in the second encoded output signal is continuous; The target serializer, operating in dual-mode, serializes the second encoded output signal according to the second serialization ratio to obtain a first serialized output signal. This first serialized output signal is used to display the target resolution and target frequency as display parameters. The second serialization ratio is N times the first serialization ratio, where N=2. n , where n is an integer greater than or equal to 1.

2. The method according to claim 1, characterized in that, The step of determining the current data rate of the target serializer output data based on the acquired target data rate and the rate constraint parameters of the target serializer includes: If the target data rate is greater than or equal to the minimum data rate, the current data rate is determined to be equal to the target data rate. If the target data rate is less than the minimum data rate, the current data rate is determined to be N times the target data rate, where N=2. n , where n is an integer greater than or equal to 1.

3. The method according to claim 2, characterized in that, Determining the clock frequency of the target encoder's current operating clock based on the current data rate includes: When the target data rate is greater than or equal to the minimum data rate, the clock frequency of the current working clock is determined to be equal to half of the pixel clock frequency corresponding to the original image signal; If the target data rate is less than the minimum data rate, the clock frequency of the current operating clock is determined to be equal to the pixel clock frequency corresponding to the original image signal * N / 2.

4. The method according to claim 2, characterized in that, The first signal processing of the original image signal to obtain the first coded input signal includes: When the target data rate is less than the minimum data rate, the time length of every two consecutive pixel data in the original image signal is adjusted from N clock cycles in the current working clock to one clock cycle, and an invalid signal with a time length of one clock cycle * (N-1) is set between every two adjusted pixel data to obtain the first coded input signal, wherein every two consecutive pixel data in the original image signal are continuous, and the time length of every two consecutive pixel data in the original image signal is N clock cycles in the current working clock.

5. The method according to claim 2, characterized in that, Performing target signal processing on the first encoded output signal to obtain a second encoded output signal that matches the current data rate includes: When the target data rate is less than the minimum data rate, the first encoded output signal is subjected to a discrete-to-continuous operation to obtain a third encoded output signal, wherein the encoded data in the third encoded output signal is continuous; The third encoded output signal is oversampled to obtain the second encoded output signal. The data in the second encoded output signal corresponding to one clock cycle of the current working clock includes one encoded data. The encoded data is obtained by copying each bit of the encoded data in the third encoded output signal corresponding to one clock cycle (N-1) times. The number of bits of each encoded data in the second encoded output signal is equal to the number of bits of one encoded data in the third encoded output signal * N. The encoded data in the second encoded output signal is continuous.

6. The method according to claim 5, characterized in that, The step of performing a discrete-to-continuous conversion operation on the first encoded output signal to obtain the third encoded output signal includes: The duration of each group of continuous encoded data in the first encoded output signal is adjusted from one clock cycle of the current working clock to N clock cycles to obtain the third encoded output signal. Each group of continuous encoded data includes two continuous encoded data. The duration of each group of continuous encoded data in the first encoded output signal is one clock cycle of the current working clock. Between each two groups of continuous encoded data in the first encoded output signal is an invalid signal with a duration of one clock cycle * (N-1). The encoded data in the third encoded output signal is continuous, and the duration of each encoded data in the third encoded output signal is N / 2 clock cycles.

7. The method according to claim 6, characterized in that, Each group of consecutive coded data in the first coded output signal is split into N coded data in the third coded output signal. The N coded data are arranged in order from front to back according to the clock cycle. The first N / 2 coded data in the N coded data are arranged in order from back to front according to the clock cycle to form one coded data in each group of consecutive coded data. The last N / 2 coded data in the N coded data are arranged in order from back to front according to the clock cycle to form the other coded data in each group of consecutive coded data. The number of bits of each coded data in the N coded data is 2 / N times the number of bits of each coded data in each group of consecutive coded data.

8. The method according to claim 5, characterized in that, The step of oversampling the third encoded output signal to obtain the second encoded output signal includes: Each bit in each encoded data in the third encoded output signal is copied (N-1) times to obtain the second encoded output signal, wherein each encoded data in the second encoded output signal includes M groups of bits arranged in sequence, each group of bits includes N identical bits, and M is the number of bits in one encoded data in the third encoded output signal.

9. The method according to claim 1, characterized in that, The method further includes: When the target data rate is greater than or equal to the minimum data rate, the original image signal is subjected to a second signal processing to obtain a second coded input signal, wherein the data in the second coded input signal is continuous; The second encoded input signal is encoded by the target encoder whose current working clock is used to obtain a fourth encoded output signal, wherein the data in the fourth encoded output signal is continuous; The target serializer, operating in the dual-mode mode, performs serialization processing on the fourth encoded output signal according to the second serialization ratio to obtain a second serialized output signal, wherein the second serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters.

10. The method according to claim 9, characterized in that, The second signal processing of the original image signal to obtain the second coded input signal includes: The signals of every two consecutive pixels in the original image signal are combined into a signal transmitted within one clock cycle of the current working clock to obtain the second coded input signal, wherein each pixel data in the original image signal is continuous, and the time length of each pixel data in the original image signal is half a clock cycle of the current working clock.

11. An image signal processing apparatus, characterized in that, include: The first determining module is used to determine the current data rate of the output data of the target serializer based on the acquired target data rate and the rate constraint parameters of the target serializer, and to determine the clock frequency of the current operating clock of the target encoder based on the current data rate. The target data rate is the data corresponding to the target resolution and the target frequency. The target resolution and the target frequency are display parameters used to display the original image signal. The rate constraint parameters are the minimum data rate of the output data allowed when the target serializer is in dual mode. The operating mode of the target serializer includes single mode and dual mode. The first serialization ratio of the target serializer in single mode is less than the second serialization ratio in dual mode. A first processing module is configured to perform a first signal processing on the original image signal to obtain a first coded input signal when the target data rate is less than the minimum data rate, wherein the data in the first coded input signal is discrete; An encoding module is used to encode the first encoding input signal using the target encoder whose current operating clock is the operating clock, to obtain a first encoding output signal, wherein the data in the first encoding output signal is discrete; The second processing module is used to perform target signal processing on the first encoded output signal to obtain a second encoded output signal that matches the current data rate, wherein the data in the second encoded output signal is continuous; The third processing module is used to serialize the second encoded output signal according to the second serialization ratio using the target serializer operating in the dual-mode mode, to obtain a first serialized output signal. The first serialized output signal is used to be displayed with the target resolution and the target frequency as display parameters. The second serialization ratio is N times the first serialization ratio, where N=2. n , where n is an integer greater than or equal to 1.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 10.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 10.

Images