Image processing method and image processing apparatus

Abstract

Embodiments of the present application disclose an image processing method and device, the method comprising: obtaining scaling configuration parameters of a to-be-scaled image to select one of a first direction scaling module and a second direction scaling module as a first scaling module and the other as a second scaling module; inputting a plurality of pixel data of the to-be-scaled image into a first storage module in sequence based on a system clock signal to store the pixel data; performing first scaling processing on first pixel data in the stored pixel data by the first scaling module in a plurality of first clock cycles of the system clock signal respectively to obtain a plurality of processed pixel data and storing the processed pixel data into a second storage module; and performing second scaling processing on second pixel data in the stored processed pixel data by the second scaling module in a plurality of second clock cycles of the system clock signal respectively to output a target image. The present application can shorten the delay time.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to an image processing method and an image processing device. Background Technology

[0002] With the rapid development of computing and storage capabilities, the video processing industry faces a wide variety of image sizes. When displaying images, the input image size needs to be scaled up to the display size point-to-point. The image and display sizes can be adapted to each other arbitrarily, but this introduces a problem: excessively high instantaneous bandwidth leads to a mismatch between system processing capabilities and instantaneous bandwidth, resulting in a delay between the captured and displayed images. For example, existing image scaling methods have a one-frame delay between the image and the display, which seriously affects the user experience.

[0003] Therefore, providing an image scaling method to shorten the latency time is a problem that this invention urgently needs to solve. Summary of the Invention

[0004] The embodiments of the present invention disclose an image processing method and an image processing device, which can avoid the problem of a one-frame delay between the image frame and the display frame in existing image scaling methods, shorten the delay time, and improve the user experience.

[0005] An image processing method disclosed in this invention includes: obtaining scaling configuration parameters of an image to be scaled; selecting one of a first-direction scaling module and a second-direction scaling module as a first scaling module and the other as a second scaling module based on the scaling configuration parameters; sequentially storing multiple pixel data of the image to be scaled into a first storage module based on a system clock signal; having the first scaling module read first pixel data from the stored pixel data in the first storage module during multiple first clock cycles of the system clock signal and perform a first scaling process to obtain multiple processed pixel data stored in a second storage module; and having the second scaling module read second pixel data from the stored processed pixel data in the second storage module during multiple second clock cycles of the system clock signal and perform a second scaling process to output a target image.

[0006] Existing image scaling methods suffer from a mismatch between system processing capacity and instantaneous bandwidth due to excessively high instantaneous bandwidth, resulting in a one-frame delay between the captured and displayed images, severely impacting user experience. The image processing method disclosed in this invention obtains scaling configuration parameters of the image to be scaled, selecting one of a first-direction scaling module and a second-direction scaling module as the first scaling module and the other as the second scaling module. Based on the system clock signal, multiple pixel data of the image to be scaled are sequentially stored in a first storage module. The first scaling module reads the first pixel data stored in the first storage module during multiple first clock cycles, performs a first scaling process to obtain multiple processed pixel data. The second scaling module reads the second pixel data from the processed pixel data during multiple second clock cycles, performs a second scaling process to output the target image. This avoids the problem of a mismatch between system processing capacity and instantaneous bandwidth caused by excessively high instantaneous bandwidth, and avoids the one-frame delay between the captured and displayed images. It reduces the delay time from the image frame delay to the inter-pixel row delay, greatly shortening the delay time and improving user experience.

[0007] In one embodiment of the present invention, the step of having the first scaling module read first pixel data from the stored pixel data in the first storage module during multiple first clock cycles of the system clock signal and perform a first scaling process to obtain multiple processed pixel data stored in the second storage module includes: the first scaling module determining at least one first pixel data to be processed in each first clock cycle based on a first preset scaling algorithm and a first scaling coefficient, thereby reading the corresponding at least one first pixel data in each first clock cycle and performing the first scaling process to obtain one processed pixel data stored in the second storage module.

[0008] By setting the first scaling module to perform a first scaling process on at least one first pixel data in the first clock cycle to obtain a processed pixel data, the problem of excessive instantaneous bandwidth can be avoided, and the delay time can be effectively shortened.

[0009] In one embodiment of the present invention, the step of having the second scaling module read second pixel data from the stored processed pixel data in the second storage module during multiple second clock cycles of the system clock signal and perform second scaling processing to output a target image includes: the second scaling module determining at least one second pixel data to be processed in each second clock cycle based on a second preset scaling algorithm and a second scaling coefficient, thereby performing the second scaling processing on the corresponding at least one second pixel data in each second clock cycle to obtain a target pixel data; wherein, the target image is composed of multiple target pixel data.

[0010] By setting the second scaling module to perform a second scaling process on at least one second pixel data in the second clock cycle to obtain a target pixel data, the problem of excessive instantaneous bandwidth can be avoided, and the delay time can be effectively scaled.

[0011] In one embodiment of the present invention, the scaling configuration parameter is the scaling method of the image to be scaled; wherein, the scaling method is shrinking in a first direction and magnifying in a second direction, the first direction scaling module is the first scaling module, and the second direction scaling module is the second scaling module; or the scaling method is magnifying in a first direction and shrinking in a second direction, the second direction scaling module is the first scaling module, and the first direction scaling module is the second scaling module; or the scaling method is shrinking in a first direction and shrinking in a second direction, either the first direction scaling module or the second direction scaling module is the first scaling module, and the other is the second scaling module; or the scaling method is magnifying in a first direction and magnifying in a second direction, either the first direction scaling module or the second direction scaling module is the first scaling module, and the other is the second scaling module; wherein, when the scaling method is any one of shrinking in a first direction and magnifying in a second direction, magnifying in a first direction and shrinking in a second direction, and shrinking in a first direction and shrinking in a second direction, the plurality of first clock cycles are discontinuous, and a portion of the plurality of second clock cycles is located between two discontinuous first clock cycles.

[0012] By selecting the first scaling module and the second scaling module from the first scaling module and the second scaling module in the scaling mode, the horizontal scaling and vertical scaling can be processed separately, which can solve the problem of excessive instantaneous bandwidth. When the scaling mode includes shrinking, the second clock cycle is set to be located between two discontinuous first clock cycles, which can further reduce the peak value of instantaneous image bandwidth and provide a slow data flow for the next scaling process. In this way, more pixel data can be processed within a certain period of time, thereby improving image scaling efficiency.

[0013] In one embodiment of the present invention, the step of sequentially storing multiple pixel data of the image to be scaled into a first storage module based on a system clock signal includes: determining the number of pixel data to be stored per unit time based on the image frequency and image resolution of the image to be scaled; and determining the clock cycle of the system clock signal corresponding to each pixel data based on the number of clock cycles of the system clock signal and the number of pixel data per unit time, so as to input each pixel data into the first storage module for storage within the corresponding clock cycle.

[0014] The number of pixel data points per unit time is determined by the image frequency and image resolution, thereby determining the clock cycle corresponding to each pixel data point. Pixel data is stored within the clock cycle corresponding to the pixel data point, which avoids the over-storage problem of existing image scaling methods that require storing the entire image before scaling. It also avoids excessive instantaneous bandwidth and eliminates the need to add a frame delay, effectively shortening the latency time.

[0015] In one embodiment of the present invention, obtaining the scaling configuration parameters of the image to be scaled includes: calculating an image scaling factor based on the image resolution of the image to be scaled and the screen resolution of the display screen; and determining a scaling method as the scaling configuration parameter based on the image scaling factor.

[0016] In one embodiment of the present invention, the first direction scaling module is a horizontal scaling module, and the second direction scaling module is a vertical scaling module; when the scaling method is magnification in the first direction and magnification in the second direction, the second direction scaling module is the first processing module, and the first direction scaling module is the second processing module.

[0017] By using the vertical scaling module as the first processing module and the horizontal scaling module as the second processing module when zooming is required in both directions, additional resource consumption can be avoided.

[0018] In one embodiment of the present invention, the scaling method is either shrinking in the first direction and keeping the second direction unchanged, or enlarging in the first direction and keeping the second direction unchanged, wherein the first direction scaling module is the first scaling module and the second direction scaling module is the second scaling module; or the scaling method is either keeping the first direction unchanged and shrinking in the second direction, or keeping the first direction unchanged and enlarging in the second direction, wherein the second direction scaling module is the first scaling module and the first direction scaling module is the second scaling module.

[0019] The aforementioned image processing method is also applicable to unidirectional scaling, thus expanding its scope of application.

[0020] Furthermore, embodiments of the present invention also provide an image processing device, including: an image processor and a controller connected to the image processor, wherein the image processor and the controller cooperate to execute any of the aforementioned image processing methods.

[0021] In one embodiment of the present invention, the image processor includes: a plurality of image scaling channels and an overlay module connecting the plurality of image scaling channels, each image scaling channel including: a first storage module, a first direction scaling module, a second storage module and a second direction scaling module; wherein, each image scaling channel cooperates with the controller to execute any of the aforementioned image processing methods, and the overlay module overlays the target image output by each image scaling channel and then outputs and displays it.

[0022] By setting multiple image scaling channels in the image processor, each image scaling channel works with the controller to execute image processing methods, and the overlay module overlays the target images output by each image scaling channel before displaying them, the flexibility of application scenarios can be further improved.

[0023] The above technical solution can have one or more of the following advantages: In this embodiment of the invention, by obtaining the scaling configuration parameters of the image to be scaled, one of the first direction scaling module and the other of the second direction scaling module is selected as the first scaling module and the other as the second scaling module. Based on the system clock signal, multiple pixel data of the image to be scaled are sequentially stored in the first storage module. The first scaling module reads the first pixel data stored in the first storage module in multiple first clock cycles and performs a first scaling process to obtain multiple processed pixel data. The second scaling module reads the second pixel data in the processed pixel data in multiple second clock cycles and performs a second scaling process to output the target image. This can avoid the problem of mismatch between system processing capacity and instantaneous bandwidth caused by excessive instantaneous bandwidth, and avoid the problem of a one-frame delay between the captured image and the displayed image. The delay time can be shortened from the delay of the image frame to the delay between pixel rows, which greatly shortens the delay time and improves the user experience. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the structure of an image processing device disclosed in one embodiment of the present invention.

[0026] Figure 2 This is a flowchart of the steps of an image processing method disclosed in an embodiment of the present invention.

[0027] Figure 3This is a schematic diagram illustrating an example structure of an image processor in an image processing device disclosed in an embodiment of the present invention.

[0028] Figures 4a-4e This is a schematic diagram illustrating the pixel data and clock cycle involved in an example of an image processing device performing an image processing method according to an embodiment of the present invention.

[0029] Figure 5 This is a schematic diagram of another structure of an image processing device disclosed in one embodiment of the present invention.

[0030] Figure 6 This is a schematic diagram illustrating an example structure of an image processing device disclosed in an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] like Figure 1 As shown, one embodiment of the present invention discloses an image processing device 100, which includes, for example, an image processor 10 and a controller 20 connected to the image processor 10.

[0033] The image processor 10 is, for example, a programmable logic device, such as an FPGA (Field-Programmable Gate Array) or other similar logic device. The controller 20 is, for example, a MCU (Microcontroller Unit), also known as a single-chip microcomputer or microcontroller; or other microprocessors with certain data processing and computing capabilities, such as ARM processors and DSP processors.

[0034] Specifically, the image processor 10 includes, for example, a data input module 12, a first-direction scaling module 111, a second-direction scaling module 112, a storage module 113, and a storage module 114. The data input module 12 is connected to the controller 20, the storage module 113, and the storage module 114; the storage module 113 is connected to the first-direction scaling module 111 and the second-direction scaling module 112; and the storage module 114 is connected to both the first-direction scaling module 111 and the second-direction scaling module 112.

[0035] In this embodiment, the first scaling module 111 is, for example, a vertical scaling module, i.e., an H Scaler, and the second scaling module 112 is, for example, a horizontal scaling module, i.e., a V Scaler; or the first scaling module 111 is, for example, a horizontal scaling module, i.e., a V Scaler, and the second scaling module 112 is, for example, a vertical scaling module, i.e., an H Scaler. Storage modules 113 and 114 are, for example, FIFOs (First-In, First-Out registers). However, this embodiment is not limited to this; storage modules 113 and 114 can also be other modules with storage capabilities, such as RAM modules.

[0036] In one embodiment of the present invention, the image processor 10 and the controller 20 in the image processing device 100 cooperate to execute an image processing method 200. One embodiment of the present invention discloses an image processing method 200, such as... Figure 2 As shown, the image processing method 200 includes, for example:

[0037] Step S21: Obtain the scaling configuration parameters of the image to be scaled;

[0038] Step S22: Based on the scaling configuration parameters, select one of the first direction scaling module and the second direction scaling module as the first scaling module and the other as the second scaling module;

[0039] Step S23: Based on the system clock signal, store multiple pixel data of the image to be scaled sequentially into the first storage module;

[0040] Step S24: The first scaling module reads first pixel data from the stored pixel data from the first storage module during multiple first clock cycles of the system clock signal, performs first scaling processing, and stores the processed pixel data into the second storage module; and

[0041] Step S25: The second scaling module reads the second pixel data from the stored processed pixel data from the second storage module during multiple second clock cycles of the system clock signal and performs the second scaling process to output the target image.

[0042] Step S21 is executed, for example, in the image processor 10. In step S21, the scaling configuration parameter is, for example, the scaling method of the image to be scaled. This scaling method is calculated and issued by the controller 20, or it can be calculated by the image processor 10 itself. The scaling configuration parameter can also be, for example, an instruction indicating the output path of the image to be scaled, which is issued by the controller 20 to the image processor. It is worth noting that the instruction indicating the output path of the image to be scaled is an instruction generated by the controller 20 based on the scaling method.

[0043] Step S22 is executed, for example, in an image processor. In step S22, when the scaling configuration parameter is a scaling mode, the first scaling module and the second scaling module are selected based on the principle of performing reduction processing first and then enlargement processing. When the scaling configuration parameter is an instruction indicating the output path of the image to be scaled, the first scaling module and the second scaling module can be selected directly based on the received instruction. The first scaling module can be understood as the module that performs scaling processing first, and the second scaling module can be understood as the module that performs scaling processing later.

[0044] Step S23 is executed, for example, in the image processor 10. In step S23, the system clock signal mentioned is, for example, the clock signal inside the image processing device 100. This can be understood as the system clock signal representing the throughput and data computation clock of the image processing device 100. It is worth noting that the system clock signal involved in this embodiment is different from the image clock signal. The image clock signal is generally related to the image resolution; as the image resolution increases, the image clock signal speeds up (i.e., the period becomes shorter), and as the image resolution decreases, the image clock signal slows down (i.e., the period becomes longer). The period of the system clock signal is generally greater than the period of the image clock signal. The pixel data mentioned is, for example, grayscale data. The storage of one pixel data corresponds to one clock cycle. The first storage module mentioned is the storage module connected to the first scaling module. That is, when the first scaling module is the first direction scaling module 111, the first storage module is storage module 113; when the first scaling module is the second direction scaling module 112, the first storage module is storage module 114.

[0045] Step S24 is executed, for example, in the image processor 10. In step S24, the first clock cycle is a certain clock cycle of the system clock signal. Multiple first clock cycles can be consecutive clock cycles or discontinuous clock cycles. Reading the first pixel data refers to reading a single first pixel or reading multiple first pixel data. The number of first pixel data read depends on the scaling algorithm executed by the first scaling module. The second storage module is a storage module connected to the second scaling module. That is, when the second scaling module is the first direction scaling module 111, the second storage module is storage module 113; when the second scaling module is the second direction scaling module 112, the second storage module is storage module 114. The first scaling process refers to, for example, magnification, reduction, or invariance. For instance, the scaling factor corresponding to magnification is greater than 1, the scaling factor corresponding to reduction is less than 1, and the scaling factor corresponding to invariance is equal to 1.

[0046] Step S25 is executed, for example, in the image processor 10. In step S25, the second clock cycle is a specific clock cycle of the system clock signal, and it is a different clock cycle from the first clock cycle. Multiple second clock cycles can be consecutive or discontinuous. Reading the second pixel data refers to reading one or multiple second pixel data points. The number of second pixel data points read depends on the scaling algorithm executed by the second scaling module. The second scaling process refers to, for example, magnification, invariance, or reduction. For instance, magnification corresponds to a scaling factor greater than 1, reduction corresponds to a scaling factor less than 1, and invariance corresponds to a scaling factor equal to 1.

[0047] The above method obtains the scaling configuration parameters of the image to be scaled, selects one of the first and second direction scaling modules as the first scaling module and the other as the second scaling module, and then stores multiple pixel data of the image to be scaled sequentially into the first storage module based on the system clock signal. The first scaling module reads the first pixel data stored in the first storage module in multiple first clock cycles, performs a first scaling process to obtain multiple processed pixel data, and the second scaling module reads the second pixel data from the processed pixel data in multiple second clock cycles to perform a second scaling process to output the target image. This method avoids the problem of mismatch between system processing capacity and instantaneous bandwidth caused by excessive instantaneous bandwidth, and avoids the problem of a one-frame delay between the captured image and the displayed image. It can shorten the delay time from the delay of image frames to the delay between pixel rows, greatly reducing the delay time and improving the user experience.

[0048] In other embodiments of the present invention, the aforementioned step S21 includes, for example:

[0049] S211: Calculate the image scaling factor based on the image resolution of the image to be scaled and the screen resolution of the display screen; and

[0050] S212: Determine the scaling method based on the image scaling factor as the scaling configuration parameter.

[0051] For example, step S211 is executed in controller 20, where controller 20 calculates the image scaling factor and sends it to image processor 10. Image processor 10 then executes step S212, which determines the scaling method based on the image scaling factor. However, this embodiment is not limited to this. Steps S211 and S212 can both be executed in image processor 10 or in controller 20. When executed in controller 20, controller 20 sends scaling configuration parameters to image processor 10. The display screen is, for example, an LED display screen. For instance, the screen resolution is 1080P (1920*1080), and the image resolution is 720P (1280*720). Therefore, the image scaling factor can be calculated as follows: horizontal scaling factor 1920 / 1280 = 1.5, and vertical scaling factor 1080 / 720 = 1.5. Thus, horizontal scaling is determined based on the horizontal scaling factor of 1.5, and vertical scaling is determined based on the vertical scaling factor of 1.5, meaning the scaling method is both horizontal and vertical scaling.

[0052] The image scaling factor is calculated using the image resolution and screen resolution, and the scaling method is obtained as the scaling configuration parameter, which can make the image to be scaled input to the image processing device 100 arbitrarily adapt to the display screen.

[0053] In other embodiments of the present invention, step S22 may include, for example:

[0054] Step S221: Determine the number of pixel data points that need to be stored per unit time based on the image frequency and image resolution of the image to be scaled; and

[0055] Step S222: Determine the clock cycle of the system clock signal corresponding to each pixel data based on the number of clock cycles of the system clock signal within the unit time and the number of pixel data, so as to input each pixel data into the first storage module for storage within the corresponding clock cycle.

[0056] For example, steps S221 and S222 are executed in the image processor 10, where the unit time mentioned is, for example, 1µs. The image processor 10 can roughly determine the number of pixel data that need to be stored in the first storage module in 1µs based on the image frequency and image resolution, and thus determine the clock cycle corresponding to each pixel data based on the number of clock cycles and the number of pixel data in 1µs. For example, the image processor 10 determines that 5 pixel data points need to be stored in 1µs based on the image frequency and image resolution. The system clock signal period is 0.1µs. It's worth noting that the system clock signal period is preset by the image processing device 100, meaning there are 10 clock cycles within 1µs. Distributing the 5 pixel data points evenly within 1µs ensures that the clock cycles between adjacent pixel data points are spaced one clock cycle apart. For instance, the first pixel data point corresponds to the first clock cycle out of 10, the second to the third, the third to the fifth, the fourth to the seventh, and the fifth to the ninth. Of course, when the image resolution of the image to be scaled is relatively large, the clock cycles corresponding to the pixel data points can also be consecutive clock cycles.

[0057] The above method determines the number of pixel data per unit time by using image frequency and image resolution, thereby determining the clock cycle corresponding to each pixel data. Pixel data is stored in the clock cycle corresponding to the pixel data. The number of pixel data stored in the first storage module per unit time is relatively small, which avoids the over-storage problem of existing image scaling methods that require storing the entire image before scaling. It also avoids excessive instantaneous bandwidth and eliminates the need to add a frame delay, effectively shortening the latency time.

[0058] In other embodiments of the present invention, step S23 includes, for example, the first scaling module determining at least one first pixel data to be processed in each first clock cycle based on a first preset scaling algorithm and a first scaling factor, thereby reading the corresponding at least one first pixel data in each first clock cycle, performing the first scaling process to obtain a processed pixel data and storing it in the second storage module.

[0059] The first preset scaling algorithm mentioned herein is an existing scaling algorithm, such as the sampling method, averaging method, or bilinear method. This embodiment does not limit the specific type of the first preset scaling algorithm; any algorithm that can achieve image scaling is applicable to this embodiment. The first scaling factor mentioned herein is either a horizontal scaling factor or a vertical scaling factor. When the first scaling module corresponds to HScaler, the first scaling factor is a vertical scaling factor; when the first scaling module corresponds to VScaler, the first scaling factor is a horizontal scaling factor. The first scaling factor is, for example, issued by the controller 20 to the image processor 10. The phrase "reading at least one first pixel data in the first clock cycle" can be understood as requiring the processing of at least one first pixel data in one first clock cycle. The number of first pixel data processed depends on the first preset scaling algorithm. Different scaling algorithms require different numbers of first pixel data. For example, when the first preset scaling algorithm is the sampling method, the number of first pixel data processed in one first clock cycle is one; when the first preset scaling algorithm is the averaging method, the number of first pixel data processed in one first clock cycle is, for example, two.

[0060] By setting the first scaling module to perform a first scaling process on at least one first pixel data in the first clock cycle to obtain a processed pixel data, the problem of excessive instantaneous bandwidth can be avoided, and the delay time can be effectively shortened.

[0061] In other embodiments of the present invention, step S24 includes, for example, the second scaling module determining at least one second pixel data to be processed in each second clock cycle based on a second preset scaling algorithm and a second scaling factor, thereby performing the second scaling process on the corresponding at least one second pixel data in each second clock cycle to obtain a target pixel data; wherein, the target image is composed of a plurality of target pixel data.

[0062] The second preset scaling algorithm mentioned herein is an existing scaling algorithm, such as the sampling method, averaging method, or bilinear method. This embodiment does not limit the specific type of the second preset scaling algorithm; any algorithm that can achieve image scaling is applicable to this embodiment. The second preset scaling algorithm may be the same as or different from the first preset scaling algorithm. The second scaling factor mentioned herein is a horizontal scaling factor or a vertical scaling factor. When the second scaling module corresponds to HScaler, the second scaling factor is a vertical scaling factor; when the second scaling module corresponds to VScaler, the second scaling factor is a horizontal scaling factor. The second scaling factor is, for example, issued by the controller 20 to the image processor 10. The mention of reading at least one second pixel data in the second clock cycle can be understood as requiring the processing of at least one second pixel data in one second clock cycle. The number of second pixel data processed depends on the second preset scaling algorithm. Different scaling algorithms require different numbers of second pixel data. For example, when the second preset scaling algorithm is the sampling method, the number of second pixel data processed in one second clock cycle is one; when the second preset scaling algorithm is the averaging method, the number of second pixel data processed in one second clock cycle is, for example, two.

[0063] By setting the second scaling module to perform a second scaling process on at least one second pixel data in the second clock cycle to obtain a target pixel data, the problem of excessive instantaneous bandwidth can be avoided, and the delay time can be effectively shortened.

[0064] In other embodiments of the present invention, the scaling configuration parameter mentioned in step S22 is the scaling method of the image to be scaled. Wherein, the scaling method is shrinking in a first direction and enlarging in a second direction, the first direction scaling module is the first scaling module, and the second direction scaling module is the second scaling module; or

[0065] The scaling method is to enlarge in a first direction and shrink in a second direction, wherein the second-direction scaling module is the first scaling module, and the first-direction scaling module is the second scaling module; or

[0066] The scaling method is to reduce in a first direction and reduce in a second direction, wherein either the first-direction scaling module or the second-direction scaling module is the first scaling module, and the other is the second scaling module; or

[0067] The scaling method is to zoom in in a first direction and zoom in in a second direction. Either the first-direction scaling module or the second-direction scaling module is the first scaling module, and the other is the second scaling module.

[0068] Wherein, when the scaling method is any one of the following: shrinking in the first direction and expanding in the second direction, expanding in the first direction and shrinking in the second direction, and shrinking in the first direction and shrinking in the second direction, the plurality of first clock cycles are discontinuous, and some of the plurality of second clock cycles are located between two discontinuous first clock cycles.

[0069] Wherein, when the scaling method is a first direction of magnification and a second direction of magnification, the plurality of first clock cycles are, for example, consecutive clock cycles, the plurality of second clock cycles are, for example, consecutive clock cycles, and the plurality of consecutive second clock cycles are located after the plurality of consecutive first clock cycles, wherein the plurality of consecutive second clock cycles and the plurality of consecutive first clock cycles are consecutive clock cycles in the system clock signal.

[0070] For example, the first direction mentioned is, for instance, a horizontal direction, and the second direction mentioned is, for instance, a vertical direction; or, the first direction mentioned is a vertical direction, and the second direction mentioned is a horizontal direction.

[0071] By selecting the first scaling module and the second scaling module from the first scaling module and the second scaling module in the scaling mode, the horizontal scaling and vertical scaling can be processed separately, which can solve the problem of excessive instantaneous bandwidth. When the scaling mode includes shrinking, the second clock cycle is set to be located between two discontinuous first clock cycles, which can further reduce the peak value of instantaneous image bandwidth and provide a slow data flow for the next scaling process. In this way, more pixel data can be processed within a certain period of time, thereby improving image scaling efficiency.

[0072] Furthermore, in other embodiments of the present invention, the scaling method is either shrinking in the first direction and keeping the second direction unchanged, or enlarging in the first direction and keeping the second direction unchanged, wherein the first direction scaling module is the first scaling module and the second direction scaling module is the second scaling module; or the scaling method is either keeping the first direction unchanged and shrinking in the second direction, or keeping the first direction unchanged and enlarging in the second direction, wherein the second direction scaling module is the first scaling module and the first direction scaling module is the second scaling module.

[0073] Wherein, when the scaling method is to shrink in the first direction and keep the second direction unchanged, or to keep the first direction unchanged and shrink in the second direction, the plurality of first clock cycles are, for example, discontinuous, the plurality of second clock cycles are, for example, discontinuous, and some of the plurality of second clock cycles are, for example, located between two discontinuous first clock cycles.

[0074] When the scaling method is to enlarge in the first direction and keep the second direction unchanged, or to keep the first direction unchanged and enlarge in the second direction, the plurality of first clock cycles are, for example, consecutive, the plurality of second clock cycles are, for example, consecutive, and the plurality of consecutive second clock cycles are located after the plurality of consecutive first clock cycles, wherein the plurality of consecutive second clock cycles and the plurality of consecutive first clock cycles are consecutive clock cycles in the system clock signal.

[0075] That is, the image processing method 200 performed by the image processing device 100 disclosed in this embodiment is also applicable to the case of unidirectional scaling, thus expanding the scope of application.

[0076] In other embodiments of the present invention, the first direction scaling module is, for example, a horizontal scaling module, and the second direction scaling module is, for example, a vertical scaling module; when the scaling method is magnification in the first direction and magnification in the second direction, the second direction scaling module is the first processing module, and the first direction scaling module is the second processing module.

[0077] Since horizontal scaling requires caching column data and vertical scaling requires caching row data, if horizontal scaling is performed first, more row data will be cached, resulting in additional resource consumption. By performing vertical scaling first and then horizontal scaling, this additional resource consumption can be avoided.

[0078] like Figure 3 The diagram shown is an example structural schematic of an image processor in an image processing device disclosed in an embodiment of the present invention. The image processing method executed by the image processing device will be illustrated below using the first direction scaling module 111 as VSCALER, the second direction scaling module 112 as HSCALER, the storage module 113 as FIFO0, and the storage module 114 as FIFO1 as examples.

[0079] Image processor 10 includes, for example, FIFO0, V SCALER, FIFO1, and H SCALER, wherein one input terminal of FIFO0 is connected to a data input module ( Figure 3 (Not shown), another input terminal is connected to H SCALER, the output terminal of FIFO0 is connected to VSCALER, one input terminal of FIFO1 is connected to the data input module, the other input terminal is connected to V SCALER, and the output terminal of FIFO1 is connected to H SCALER. V SCALER is used to perform horizontal scaling, and HSCALER is used to perform vertical scaling.

[0080] The data input module is used to obtain the scaling method of the module to be scaled, and to select one of VSCALER and HSCALER as the first scaling module and the other as the second scaling module based on the scaling configuration parameters. The first scaling module is the module that performs scaling processing first, and the second scaling module is the module that performs scaling processing later. Specifically, when the scaling method is horizontal shrinking and vertical zooming, V SCALER is the first scaling module and H SCALER is the second scaling module; when the scaling method is horizontal zooming and vertical shrinking, H SCALER is the first scaling module and V SCALER is the second scaling module; when the scaling method is horizontal shrinking and vertical shrinking, either V SCALER or H SCALER is the first scaling module and the other is the second scaling module; when the scaling method is horizontal zooming and vertical zooming, or horizontal zooming and vertical zooming, V SCALER is the first scaling module and H SCALER is the second scaling module; when the scaling method is horizontal zooming and vertical zooming, or horizontal zooming and vertical zooming, H SCALER is the first scaling module and V SCALER is the second scaling module.

[0081] To illustrate image processing methods more clearly, the following will combine... Figures 4a-4e The following example illustrates the scaling process, where the image resolution of the image to be scaled is 4*4, the screen resolution of the display is 2*8, and the 16 pixel data in the image to be scaled correspond to 16 consecutive clock cycles.

[0082] like Figure 4a As shown, the data input module acquires the image to be scaled and its corresponding scaling method. The controller, based on the image resolution of 4x4 and the screen resolution of 2x8, calculates a horizontal scaling factor of 0.5 and a vertical scaling factor of 2, thus determining the scaling method as horizontal reduction and vertical enlargement. This scaling method is then communicated to the data input module. Based on this scaling method, the data input module determines VSCALER as the first scaling module and HSCALER as the second scaling module. In other words, the data input module inputs the image to be scaled into FIFO0 for storage. Specifically, as... Figure 4a As shown, the image resolution of the image to be scaled is, for example, 4*4, which includes 16 pixels. P11 represents the pixel data located in the first row and first column, P12 represents the pixel data located in the first row and second column, and so on. Figure 4bAs shown, one pixel data is transmitted in each clock cycle. The data input module starts from the initial clock cycle of the system clock signal, for example t1, and stores 16 pixel data into FIFO0 in sequence. That is, it takes 16 clock cycles to store 16 pixel data into FIFO0, that is, from t1 to t16.

[0083] The V SCALER employs a preset scaling algorithm, such as the sampling method. Based on the sampling method and a horizontal scaling factor of 0.5, the V SCALER reads the first pixel data from the stored pixel data in FIFO0 during multiple first clock cycles of the system clock signal, performs horizontal scaling, and stores the resulting processed pixel data in FIFO1. Specifically, as shown... Figure 4c As shown, multiple first clock cycles are t1-t4 and t9-t12. The first pixel data corresponding to the first clock cycle t1 is P11, the first pixel data corresponding to the first clock cycle t2 is P12, the first pixel data corresponding to the first clock cycle t3 is P13, the first pixel data corresponding to the first clock cycle t4 is P14, the first pixel data corresponding to the first clock cycle t9 is P31, the first pixel data corresponding to the first clock cycle t10 is P32, the first pixel data corresponding to the first clock cycle t11 is P33, and the first pixel data corresponding to the first clock cycle t12 is P34. During the first clock cycle t1, VSCALER reads the first pixel data P11 from FIFO0. Since the sampling method is used, the read first pixel data P11 is directly stored in FIFO1 as the processed pixel data. Similarly, after the first clock cycles t2, t3, t4, t9, t10, t11 and t12, the first pixel data P12, P13, P14, P31, P32, P33 and P34 are stored in FIFO1 as the processed pixel data. Among them, the first clock cycles t1-t4 and t9-t12 are not completely continuous clock cycles, and there is a four-clock-cycle interval between the first clock cycle t4 and the first clock cycle t9.

[0084] The H SCALER employs a preset scaling algorithm, such as the averaging method. Based on the averaging method and a vertical scaling factor 2, the H SCALER reads the second pixel data from the stored processed pixel data in FIFO1 during multiple second clock cycles of the system clock signal, performs vertical scaling, and outputs the target image. Specifically, as... Figure 4dAs shown, multiple second clock cycles are t5-t8 and t13-t24. For example, the second pixel data corresponding to second clock cycle t5 is P11, the second pixel data corresponding to second clock cycle t6 is P11 and P12, the second pixel data corresponding to second clock cycle t7 is P12, the second pixel data corresponding to second clock cycle t8 is P12 and P13, the second pixel data corresponding to second clock cycle t13 is P13, the first pixel data corresponding to second clock cycle t14 is P13 and P14, the second pixel data corresponding to second clock cycle t15 is P14, and the second pixel data corresponding to second clock cycle t16 is P14. The corresponding second pixel data are P14 and P15. The second pixel data corresponding to the second clock cycle t17 is P31. The second pixel data corresponding to the second clock cycle t18 is P31 and P32. The second pixel data corresponding to the second clock cycle t19 is P32. The second pixel data corresponding to the second clock cycle t20 is P32 and P33. The second pixel data corresponding to the second clock cycle t21 is P33. The second pixel data corresponding to the second clock cycle t22 is P33 and P34. The second pixel data corresponding to the second clock cycle t23 is P34. The second pixel data corresponding to the second clock cycle t24 is P34. During the second clock cycle t5, H SCALER reads the second pixel data P11 from FIFO1 and outputs it as the target pixel data P11. During the second clock cycle t6, H SCALER reads the second pixel data P11 and the second pixel data P12 from FIFO1, performs scaling processing, and outputs the target pixel data P11.5. Similarly, after the second clock cycles t7-t8 and t13-t24, the target pixel data P12, P12.5, P13, P13.5, P14, P14.5, P31, P31.5, P32, P32.5, P33, P33.5, P34, and P34.5 are obtained respectively, thus obtaining the target image. See [link to relevant documentation]. Figure 4e Among these, multiple second clock cycles are not completely consecutive clock cycles; specifically, second clock cycles t5-t8 lie between the first clock cycles t4 and t9. From... Figure 4b and Figure 4d As can be seen, image scaling only delays the process by 8 clock cycles. Since a single clock cycle is typically on the order of nanoseconds, this delay is negligible.

[0085] It should be noted that the foregoing description is only for better understanding of this embodiment, but the present invention does not limit the scaling algorithms executed by the first direction scaling module and the second direction scaling module, and the first pixel data corresponding to the first clock cycle and the second pixel data corresponding to the second clock cycle are not limited to... Figures 4b-4d For details regarding scaling methods, please refer to the foregoing description and we will not repeat them here.

[0086] The foregoing embodiment is illustrated by taking an image processor 100 including an image scaling channel as an example, which can be understood as the obtained target image being directly output to an external display screen for on-screen display.

[0087] In other embodiments of the invention, such as Figure 5 As shown, the image processor 100 includes, for example, a plurality of image scaling channels 11 and an overlay module 13 connecting the plurality of image scaling channels 11. Each image scaling channel 11 includes, for example, a storage module 113, a first-direction scaling module 111, a storage module 114, and a second-direction scaling module 112. Figure 5 The data input module 12 shown is connected to each image scaling channel 11. However, this embodiment is not limited to this, and the data input module 12 can also be set in each image scaling channel 11. Each image scaling channel 11 cooperates with the controller 20 to execute the aforementioned image processing method, and the overlay module 13 overlays the target image output from each image scaling channel and then outputs and displays it.

[0088] For example, such as Figure 6 As shown, the image processor includes, for example, four image scaling channels 11. The image processor receives a stitched image, including: upper left, lower right, lower left, and lower right. Then, it sends the four parts of the stitched image to the four image scaling channels as the images to be processed for each image scaling channel. Each image scaling channel 11 performs the image processing method disclosed in the aforementioned embodiment on the received images to be processed and outputs the target image to the overlay module. The overlay module overlays the target image and outputs it. The overlay image can be understood as the image after scaling the stitched image.

[0089] By setting multiple image scaling channels in the image processor, each image scaling channel works with the controller to execute image processing methods, and the overlay module overlays the target images output by each image scaling channel before displaying them, the flexibility of application scenarios can be further improved.

[0090] It is worth mentioning that the image processing device 100 disclosed in the foregoing embodiments of the present invention is, for example, a transmitting card in an LED display control system. It is connected to a receiving card in the LED display to send images to the LED display for display. In addition to the receiving card, the LED display also includes an LED cabinet connected to the receiving card. The receiving card is used to carry the LED cabinet for image display.

[0091] In summary, the aforementioned embodiments of the present invention can avoid the problem of mismatch between system processing capacity and instantaneous bandwidth caused by excessive instantaneous bandwidth, and avoid the problem of a one-frame delay between the captured image and the displayed image. The delay time can be shortened from the delay of image frames to the delay between pixel rows, which greatly reduces the delay time and improves the user experience.

[0092] In the several embodiments provided by this invention, it should be understood that the disclosed systems, devices, and / or methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units / modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0093] The units / modules described as separate components may or may not be physically separate. The components shown as units / modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units / modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] Furthermore, in the various embodiments of the present invention, the functional units / modules can be integrated into one processing unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated into one unit / module. The integrated unit / module described above can be implemented in hardware or in the form of hardware plus software functional units / modules.

[0095] The integrated units / modules implemented as software functional units / modules described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause one or more processors of a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that, include: Get the scaling configuration parameters of the image to be scaled; Based on the scaling configuration parameters, one of the first-direction scaling module and the second-direction scaling module is selected as the first scaling module, and the other is selected as the second scaling module. Based on the system clock signal, multiple pixel data of the image to be scaled are sequentially stored into the first storage module; The first scaling module reads the first pixel data from the stored pixel data from the first storage module during multiple first clock cycles of the system clock signal, performs a first scaling process, and stores the processed pixel data into the second storage module. as well as The second scaling module reads the second pixel data from the stored processed pixel data from the second storage module during multiple second clock cycles of the system clock signal, performs the second scaling process, and outputs the target image. The step of sequentially storing multiple pixel data of the image to be scaled into the first storage module based on the system clock signal includes: determining the number of pixel data to be stored per unit time based on the image frequency and image resolution of the image to be scaled; and determining the clock cycle of the system clock signal corresponding to each pixel data based on the number of clock cycles of the system clock signal per unit time and the number of pixel data, so as to input each pixel data into the first storage module for storage within the corresponding clock cycle.

2. The image processing method according to claim 1, characterized in that, The first scaling module reads first pixel data from the stored pixel data in the first storage module during multiple first clock cycles of the system clock signal, performs first scaling processing, and stores multiple processed pixel data in the second storage module, including: The first scaling module determines at least one first pixel data to be processed in each first clock cycle based on a first preset scaling algorithm and a first scaling factor, and then reads the corresponding at least one first pixel data in each first clock cycle to perform the first scaling process to obtain a processed pixel data which is then stored in the second storage module.

3. The image processing method according to claim 1, characterized in that, The step of the second scaling module reading second pixel data from the stored processed pixel data in the second storage module during multiple second clock cycles of the system clock signal, performing second scaling processing to output the target image includes: The second scaling module determines at least one second pixel data to be processed in each second clock cycle based on a second preset scaling algorithm and a second scaling factor, and then performs the second scaling process on the corresponding at least one second pixel data in each second clock cycle to obtain a target pixel data; wherein, the target image is composed of multiple target pixel data.

4. The image processing method according to claim 1, characterized in that, The scaling configuration parameter is the scaling method of the image to be scaled; wherein, The scaling method is to shrink in a first direction and enlarge in a second direction, wherein the first direction scaling module is the first scaling module, and the second direction scaling module is the second scaling module; or The scaling method is to enlarge in a first direction and shrink in a second direction, wherein the second-direction scaling module is the first scaling module, and the first-direction scaling module is the second scaling module; or The scaling method is to reduce in a first direction and reduce in a second direction, wherein either the first-direction scaling module or the second-direction scaling module is the first scaling module, and the other is the second scaling module; or The scaling method is to zoom in in a first direction and zoom in in a second direction. Either the first-direction scaling module or the second-direction scaling module is the first scaling module, and the other is the second scaling module. Wherein, when the scaling method is any one of the following: shrinking in the first direction and expanding in the second direction, expanding in the first direction and shrinking in the second direction, and shrinking in the first direction and shrinking in the second direction, the plurality of first clock cycles are discontinuous, and some of the plurality of second clock cycles are located between two discontinuous first clock cycles.

5. The image processing method according to claim 1, characterized in that, The process of obtaining the scaling configuration parameters of the image to be scaled includes: The image scaling factor is calculated based on the image resolution of the image to be scaled and the screen resolution of the display screen; and The scaling method is determined based on the image scaling factor and used as the scaling configuration parameter.

6. The image processing method according to claim 4, characterized in that, The first direction scaling module is a horizontal scaling module, and the second direction scaling module is a vertical scaling module; When the scaling method is both zooming in the first direction and zooming in the second direction, the second direction scaling module is the first processing module, and the first direction scaling module is the second processing module.

7. The image processing method according to claim 4, characterized in that, The scaling method is either shrinking in the first direction and keeping the second direction unchanged, or enlarging in the first direction and keeping the second direction unchanged, wherein the scaling module in the first direction is the first scaling module, and the scaling module in the second direction is the second scaling module; or The scaling method is either keeping the first direction unchanged and shrinking in the second direction, or keeping the first direction unchanged and enlarging in the second direction, wherein the second direction scaling module is the first scaling module, and the first direction scaling module is the second scaling module.

8. An image processing device, characterized in that, include: An image processor and a controller connected to the image processor, wherein the image processor and the controller cooperate to perform the image processing method according to any one of claims 1-7.

9. The image processing apparatus according to claim 8, characterized in that, The image processor includes: a plurality of image scaling channels and an overlay module connecting the plurality of image scaling channels, each of the image scaling channels including: a first storage module, a first direction scaling module, a second storage module and a second direction scaling module; Each of the image scaling channels works in conjunction with the controller to execute the image processing method according to any one of claims 1-7, and the overlay module overlays the target image output by each of the image scaling channels and then outputs and displays it.

Images