Image processing method and apparatus

Abstract

The embodiment of the present application discloses an image processing method, which is executed in an image processor, the image processor comprises a plurality of layer processing channels, each of the layer processing channels is provided with a scaling module, the image processing method comprises the following steps: receiving an inputted image to be processed; acquiring a scaling mode corresponding to each of the image to be processed; determining a processing sequence corresponding to a target image to be processed according to the corresponding scaling mode; performing at least cache processing and scaling processing on the target image to be processed according to the processing sequence, so as to obtain a scaled image; wherein the scaling processing is performed by the same scaling module. The present application also discloses an image processing device. The present application can reduce resource consumption in the image processing process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image processing method and an image processing device. BACKGROUND

[0002] In the process of video processing, it is often necessary to perform scaling processing on video images. A Scaler (scaling module) is a scaling engine widely used in image processing devices. In an image processor based on FPGA (field programmable gate array), the Scaler performs scaling processing, which often consumes a large amount of resources, such as Lut (lookup table) resources and REG (register) resources. The more the number of Scalers performing scaling processing, the more resources they consume.

[0003] Therefore, how to reduce the resource consumption in the process of image scaling processing has become a problem to be solved by the present application. SUMMARY

[0004] Therefore, in order to overcome the prior art, the present application proposes an image processing method and an image processing device to reduce the resource consumption in the process of image scaling processing.

[0005] Specifically, in a first aspect, an image processing method is disclosed, which is executed in an image processor, the image processor comprising a plurality of layer processing channels, each of the layer processing channels being provided with a scaling module, and the image processing method comprising:

[0006] receiving an input image to be processed;

[0007] obtaining a scaling mode corresponding to each of the image to be processed;

[0008] determining a processing order corresponding to a target image to be processed according to the corresponding scaling mode;

[0009] performing at least cache processing and scaling processing on the target image to be processed according to the processing order, to obtain a scaled image; wherein the scaling processing is performed by the same scaling module.

[0010] In the existing single-layer scaling processing, a scaling module is fixedly arranged before and after a memory, the scaling module before the memory only performs a reducing processing, the scaling module after the memory only performs an enlarging processing, and through a fixed processing sequence, scaling of an image at any scale can be realized, a large number of scaling modules are used, resource consumption is large, if only one scaling module before or after the memory is reserved in each layer, only reducing or enlarging can be realized through the fixed processing sequence, although the number of scaling modules is reduced, but the implementable scaling scale is limited. The image processing method disclosed in the embodiment of the application can realize scaling of an image at any scale through arranging only one scaling module in each layer processing channel and performing scaling on a target image to be processed through the same scaling module, while reducing the number of scaling modules, the scaling function at any scale is also ensured, and resource consumption can be reduced.

[0011] In an embodiment of the application, each scaling module comprises a first direction scaling unit and a second direction scaling unit.

[0012] The scaling through the same scaling module comprises:

[0013] performing first direction scaling processing through the first direction scaling unit to output an image after first direction scaling processing, and / or

[0014] performing second direction scaling processing through the corresponding second direction scaling unit to output an image after second direction scaling processing.

[0015] The caching processing comprises caching one of the target image to be processed, the image after first direction scaling processing and the image after second direction scaling processing to a memory, and obtaining an image after caching processing.

[0016] By separating the first direction scaling unit and the second direction scaling unit in each scaling module, at least one direction scaling module can be used to perform scaling in at least one direction on the target image to be processed, and the image processing method can be flexibly applied to scaling of an image at any scale, and has the characteristics of flexibility and comprehensiveness.

[0017] In an embodiment of the application, each layer processing channel further comprises an input module, a first selector, a second selector and a third selector, wherein the input module is used to receive the target image to be processed.

[0018] The determining of the processing sequence of the target image to be processed according to the corresponding scaling mode comprises:

[0019] The first selector receives a first control signal corresponding to the scaling mode, and transmits one of the target image to be processed, the image processed in the second direction, or the image processed in the cache to the first direction scaling unit based on the first control signal.

[0020] The second selector receives a second control signal corresponding to the scaling mode, and transmits one of the target image to be processed, the image processed in the first direction, or the image processed in the cache to the second direction scaling unit based on the second control signal.

[0021] The third selector receives a third control signal corresponding to the scaling mode, and transmits one of the target image to be processed, the image processed in the first direction, or the image processed in the second direction to the memory based on the third control signal, so as to determine the processing order.

[0022] In the image processing method disclosed in the embodiment, the first selector, the second selector and the third selector are arranged in each layer processing channel, and the first selector, the second selector and the third selector are used to receive corresponding first control signals, second control signals and third control signals respectively according to different scaling modes, and each of the first selector, the second selector and the third selector selects one processed image to transmit based on the corresponding control signal, and the processing order corresponding to each of the first selector, the second selector and the third selector is determined by the input and output data of each of the first selector, the second selector and the third selector. The image processing method disclosed in the embodiment adopts fewer modules, and can conveniently and flexibly realize scaling processing of images in any proportion, and has the characteristics of simplicity and efficiency.

[0023] In an embodiment of the present application, the processing order corresponding to the target image to be processed is determined according to the corresponding scaling mode, and the processing order corresponding to the target image to be processed includes:

[0024] The scaling mode is first direction enlargement and second direction reduction, first direction enlargement and second direction enlargement, first direction enlargement and second direction invariable, or first direction invariable and second direction reduction, the first selector selects to transmit the image processed in the cache to the first direction scaling unit based on the first control signal; or

[0025] The scaling mode is first direction reduction and second direction reduction, first direction reduction and second direction enlargement, or first direction reduction and second direction invariable, the first selector selects to transmit the target image to be processed to the first direction scaling unit based on the first control signal; or

[0026] The scaling mode is that the first direction is unchanged and the second direction is enlarged, and the first selector selects one of the buffered processed image or the target image to be processed to transmit to the first direction scaling unit based on the first control signal.

[0027] No matter what the scaling mode is, the first selector only selects one of the buffered processed image or the target image to be processed to transmit to the first direction scaling unit, without receiving the second direction scaling processed image, so that the scheduling is more convenient, the output transmission link is simpler, and the resource occupation is less.

[0028] In an embodiment of the present application, the determining the processing sequence of the target image to be processed according to the corresponding scaling mode further comprises:

[0029] The first control signal, the second control signal and the third control signal are generated according to the corresponding scaling mode and are output to the first selector, the second selector and the third selector respectively.

[0030] In an embodiment of the present application, the determining the processing sequence of the target image to be processed according to the corresponding scaling mode comprises:

[0031] The scaling mode is that the first direction is enlarged and the second direction is reduced, the first direction is enlarged and the second direction is enlarged, the first direction is enlarged and the second direction is unchanged, or the first direction is unchanged and the second direction is reduced, and the first direction scaling processing is after the buffering processing; or

[0032] The scaling mode is that the first direction is reduced and the second direction is reduced, the first direction is reduced and the second direction is enlarged, or the first direction is reduced and the second direction is unchanged, and the first direction scaling processing is before the buffering processing; or

[0033] The scaling mode is that the first direction is unchanged and the second direction is enlarged, and the first direction scaling processing is before the buffering processing or the buffering processing is before the first direction scaling processing.

[0034] Different processing sequences are set through different scaling modes, which are suitable for various scaling modes, and the first direction scaling processing is either after the buffering processing or before the buffering processing and before the second direction scaling processing, so that the scheduling is more convenient, the output transmission link is simpler, and the resource occupation is less.

[0035] In an embodiment of the present application, the determining the processing sequence of the target image to be processed according to the corresponding scaling mode comprises:

[0036] The scaling manner is that the first direction is enlarged and the second direction is reduced, the first direction is enlarged and the second direction is unchanged, or the first direction is unchanged and the second direction is reduced, and the processing order is that the second direction scaling processing, the cache processing and the first direction scaling processing are sequentially performed; or

[0037] The scaling manner is that the first direction is enlarged and the second direction is enlarged, and the processing order is that the cache processing, the first direction scaling processing and the second direction scaling processing are sequentially performed; or

[0038] The scaling manner is that the first direction is reduced and the second direction is reduced, and the processing order is that the first direction scaling processing, the second direction scaling processing and the cache processing are sequentially performed.

[0039] The scaling manner is that the first direction is reduced and the second direction is enlarged, and the processing order is that the first direction scaling processing, the cache processing and the second direction scaling processing are sequentially performed; or

[0040] The scaling manner is that the first direction is unchanged and the second direction is enlarged, and the processing order is that the first direction scaling processing, the cache processing and the second direction scaling processing are sequentially performed or the cache processing, the first direction processing and the second direction scaling processing are sequentially performed; or

[0041] The scaling manner is that the first direction is reduced and the second direction is unchanged, and the processing order is that the first direction scaling processing, the cache processing and the second direction scaling processing are sequentially performed or the first direction scaling processing, the second direction processing and the cache processing are sequentially performed.

[0042] Different processing orders are set through different scaling manners, which is suitable for multiple scaling manners, and the images processed by the cache processing are always not enlarged images, which is beneficial to reducing the bandwidth consumption of the cache processing.

[0043] In one embodiment of the present application, each of the layer processing channels further comprises an input module, a first target selector and a second target selector, wherein the input module is configured to receive the target image to be processed;

[0044] The processing order corresponding to the target image to be processed is determined according to the corresponding scaling manner, and the processing order comprises:

[0045] The first target selector receives a first selection signal corresponding to the scaling manner, and transmits one selected from the target image to be processed and the image processed by the cache processing to one of the first direction scaling unit and the second direction scaling unit based on the first selection signal; and

[0046] The second target selector receives a second selection signal corresponding to the scaling mode, and transmits one of the first direction scaling processed image and the second direction scaling processed image and the target image to be processed to the memory based on the second selection signal, so as to determine the processing order.

[0047] The scaling mode is that the first direction is reduced and the second direction is unchanged, the first direction is enlarged and the second direction is unchanged, the first direction is unchanged and the second direction is reduced, or the first direction is unchanged and the second direction is enlarged.

[0048] In the image processing method disclosed in the embodiment, when the scaling mode of the image is that one direction is scaled and the other direction is unchanged, the processing of the scaling unit corresponding to the unchanged direction is skipped by controlling the different selections of the first target selector and the second target selector, so that the processing time is saved, and the effect of reducing resource consumption is achieved.

[0049] In an embodiment of the present application, the image processing method further comprises:

[0050] The pre-monitoring picture is obtained by superimposing the plurality of scaling processed images output by the plurality of layer processing channels, and the pre-monitoring picture is output to an external display for display.

[0051] In a second aspect, the embodiment of the present application discloses an image processing device, comprising: an image processor and a memory connected to the image processor; wherein the image processor is used to execute the image processing method of any one of the preceding embodiments.

[0052] In a third aspect, the embodiment of the present application discloses another image processing device, comprising: an image processor, a memory connected to the image processor, and a controller connected to the image processor; wherein the image processor is used to execute the image processing method of the preceding embodiments, the controller is used to generate the first control signal, the second control signal and the third control signal according to the scaling mode, and output the first control signal to the first selector, output the second control signal to the second selector, and output the third control signal to the third selector.

[0053] As can be seen from the above, the embodiment of the present application can achieve one or more of the following beneficial effects: by setting only one scaling module in a single layer processing channel, for a target image to be processed, the same scaling module can realize scaling processing of any ratio, reducing the number of scaling modules while ensuring the scaling function of any ratio of the image, thereby reducing the occupancy rate of the scaling module on the resources of the image processing device, improving the image processing efficiency and reducing the resource consumption.

[0054] Other aspects and features of the present application will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating the principles of the application. It should be understood, however, that the drawings solely are designed for purposes of illustration only and not as a definition of the limits of the application. It should be further understood that the drawings are not necessarily drawn to scale and that, unless otherwise specified, they are merely intended to conceptually illustrate the structures and procedures described herein. BRIEF DESCRIPTION OF DRAWINGS

[0055] The specific embodiments of the present application will be described below with reference to the accompanying drawings.

[0056] Figure 1 A step flow chart of the image processing method disclosed for an embodiment of the present application.

[0057] Figure 2 A structure schematic diagram of the image processing device disclosed for an embodiment of the present application.

[0058] Figure 3 A connection relationship diagram of the controller and the first selector, the second selector and the third selector in another image processing device disclosed for an embodiment of the present application.

[0059] REFERENCE NUMERALS

[0060] 10: image processor; 11: layer processing channel; 111: scaling module, 1111: first direction scaling unit; 1112: second direction scaling unit; 112: input module; 1131: first selector; 1132: second selector; 1133: third selector; 20: memory; 30: controller; 200: image processing device; 300: image processing device; M100: image processing method.

[0061] S1-S4: steps of the image processing method. DETAILED DESCRIPTION

[0062] In order to make the above objectives, features and advantages of the present application more apparent, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0063] In order to make the above objectives, features and advantages of the present application more apparent, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0064] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described drawings are used to distinguish similar objects and are not necessarily used to describe a particular sequential or chronological order. It should be understood that the terms as used can be interchanged, where appropriate, to refer to embodiments of the application described herein in other than the order described herein. Furthermore, the terms "comprising" and "including" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that includes a list of steps or units not necessarily limited to those specifically listed, but can include other steps or units not expressly listed or inherent to such process, method, product or apparatus.

[0065] It should also be noted that the division of the various embodiments of the present application is only for the convenience of description, and should not constitute a special limitation. The features in the various embodiments can be combined with each other without contradiction, and can be mutually referred to.

[0066] In the existing image processor, when performing scaling processing of a single layer, a scaling module is generally fixedly arranged before and after the memory, each scaling module includes a vertical scaling unit V Scaler and a horizontal scaling unit H Scaler for vertical and horizontal scaling processing respectively, the scaling module before the memory, for example, Scaler 0, is only used for scaling down processing, and the scaling module after the memory, for example, Scaler 1, is only used for scaling up processing, so as to ensure that the bandwidth consumption of the memory is minimized, and smooth transition when dynamically adjusting the scaling ratio of the video is ensured. In the scaling processing, no matter how the scaling mode is, the processing path of the image is always: input image→Scaler 0→memory→Scaler 1→output image, so that the image entering the memory does not exceed the size of the original input image, and arbitrary scaling of the input image can be realized.

[0067] However, the scaling module itself occupies a large amount of consumed resources as the core module of scaling processing, and the more the number of scaling modules, the more resources are consumed. If the number of scaling modules is directly reduced, for example, only Scaler 0 is used for scaling processing of the image, only vertical scaling down and horizontal scaling down processing can be performed, and if only Scaler 1 is used, only vertical scaling up and horizontal scaling up processing can be performed, resulting in limited scaling ratio of the image.

[0068] Therefore, an embodiment of the present application provides an image processing device 200, as shown in Figure 2As shown, the image processing device 200 comprises an image processor 10 and a memory 20. The image processor 10 is, for example, a programmable logic device, specifically, for example, an FPGA (Field-Programmable Gate Array) device or other similar logic device. The memory 20 is used to cache images, and the memory 20 is, for example, a DDR. When the resource capacity permits, the memory 20 can also be a memory inside the FPGA, and the embodiment is not limited thereto. When the to-be-processed image is large, multiple memories 20 can also be set according to actual needs to meet the processing needs.

[0069] With reference to Figure 2 , the image processor 10 comprises at least one layer processing channel 11 (only one is shown in the figure, but the embodiment is not limited thereto, and the layer processing channel 11 can also be multiple, for example). Figure 2 Each layer processing channel 11 is provided with a scaling module 111.

[0070] The image processor 10 is used to execute an image processing method, as shown in the figure. Figure 1 An embodiment of the present application discloses an image processing method M100, which comprises, for example:

[0071] S1: receiving an input to-be-processed image;

[0072] S2: obtaining a scaling mode corresponding to each to-be-processed image;

[0073] S3: determining a processing order of a target to-be-processed image according to the corresponding scaling mode;

[0074] S4: performing at least cache processing and scaling processing on the target to-be-processed image according to the processing order to obtain a scaling-processed image; wherein the scaling processing is performed by the same scaling module 111.

[0075] The image to be processed received in step S1 can be multiple, each of which corresponds to a layer processing channel 11, and the target image to be processed mentioned in step S3 can be any one of the multiple images to be processed. Specifically, the scaling mode mentioned in step S2 can be obtained by directly inputting control information (for example, directly inputting the scaling ratio of the image), or by obtaining the current resolution and the target resolution of the image to be processed and then comparing the current resolution with the target resolution. For example, the initial resolution is 128*512 pixels, and the target resolution is 1024*256 pixels. Since 128<1024, the scaling mode in the horizontal direction is enlargement, and since 512>256, the scaling in the vertical direction is reduction. The scaling mode of the image is vertical reduction and horizontal enlargement. In this way, the scaling mode can be divided into the following cases: reduction in both directions (vertical reduction and horizontal reduction), enlargement in both directions (vertical enlargement and horizontal enlargement), reduction in one direction and enlargement in the other direction (vertical reduction and horizontal enlargement / vertical enlargement and horizontal reduction), scaling in only one direction and no change in the other direction (vertical reduction and no change in the horizontal direction / vertical enlargement and no change in the horizontal direction / no change in the vertical direction and horizontal reduction / no change in the vertical direction and horizontal enlargement).

[0076] In the embodiment, each layer processing channel 11 of the image processing device 200 is provided with only one scaling module 111. For a target image to be processed, the scaling of the target image to be processed in any proportion can be realized by only one scaling module 111 in the layer processing channel 11 corresponding to the target image to be processed according to the different scaling modes described above. Compared with the structure of the prior art in which each layer corresponds to two scaling modules, the embodiment reduces the number of scaling modules and reduces resource consumption.

[0077] In one embodiment of the present application, with reference to Figure 2 Each scaling module 111 includes a first direction scaling unit 1111 and a second direction scaling unit 1112. The scaling in step S4 mentioned above, for example, includes first direction scaling processing by the first direction scaling unit 1111 to output a first direction scaling processed image, and / or second direction scaling processing by the corresponding second direction scaling unit 1112 to output a second direction scaling processed image. The buffering processing, for example, includes buffering one of the target image to be processed, the first direction processed image, and the second direction processed image to the memory 20 to obtain a buffered image. The specific buffering processing content is different according to the different scaling modes.

[0078] In the above embodiments, the target image to be processed is scaled in at least one direction by at least one of the first direction scaling unit 1111 and the second direction scaling unit 1112 in the scaling module 111. Specifically, the first direction scaling unit 1111 is configured to perform scaling in a first direction on the input image, and the second direction scaling unit 1112 is configured to perform scaling in a second direction on the input image. The first direction is, for example, the vertical direction, and the second direction is, for example, the horizontal direction. The first direction scaling unit 1111 is a vertical scaling unit, and the second direction scaling unit 1112 is a horizontal scaling unit. Of course, the first direction can be the horizontal direction, and the second direction can be the vertical direction. In this case, the first direction scaling unit 1111 is a horizontal scaling unit, and the second direction scaling unit 1112 is a vertical scaling unit. For the convenience of understanding, the embodiments of the present application are described by taking the first direction as the vertical direction and the second direction as the horizontal direction.

[0079] As in the prior art described above, the scaling module and the memory have a fixed processing order, and the positions of the two direction scaling units in each scaling module relative to the memory are also fixed. When only one scaling module is provided, the image can only be scaled in both directions to be enlarged or both directions to be reduced, which limits the scaling ratio of the image. The image processing device 200 disclosed in the embodiments can perform all scaling modes through the first direction scaling unit 1111 and the second direction scaling unit 1112 in the same scaling module 111, which can be flexibly applied to the scaling of the image in any ratio and has the characteristics of flexibility and comprehensiveness.

[0080] In an embodiment of the present application, with reference to Figure 2 Each layer processing channel 11 further includes, for example, an input module 112, a first selector 1131, a second selector 1132, and a third selector 1133.

[0081] The step S1 is performed in the input module 112, for example. After the step S1 is performed, the input module 112 further performs, for example, cropping and other processing on the image to facilitate the scaling in the subsequent steps.

[0082] For example, the step S3 includes, for example:

[0083] S311: The first selector 1131 receives the first control signal corresponding to the scaling mode, and transmits one of the target image to be processed, the image after the second direction scaling processing and the image after the buffering processing to the first direction scaling unit 1111 based on the first control signal corresponding to the scaling mode. The second selector 1132 receives the second control signal corresponding to the scaling mode, and transmits one of the target image to be processed, the image after the first direction scaling processing and the image after the buffering processing to the second direction scaling unit 1112 based on the second control signal corresponding to the scaling mode. The third selector 1133 receives the third control signal corresponding to the scaling mode, and transmits one of the target image to be processed, the image after the second direction scaling processing and the image after the first direction scaling processing to the memory 20 based on the third control signal corresponding to the scaling mode.

[0084] It should be noted that the first control signal corresponding to different scaling modes in step S311 is not the same, the second control signal corresponding to different scaling modes is not the same, and the third control signal corresponding to different scaling modes is not the same. Specifically, for example, the scaling mode is first direction reduction and second direction enlargement, the first selector 1131 receives the first control signal, selects to transmit the target image to be processed to the first direction scaling unit 1111 based on the first control signal, the second selector 1132 receives the second control signal, selects to transmit the image after the buffering processing to the second direction scaling unit 1112 based on the second control signal, and the third selector 1133 receives the third control signal, selects to transmit the image after the first direction scaling processing to the memory 20 based on the third control signal, so as to determine the processing order as: the first direction scaling processing, the buffering processing and the second direction scaling processing are sequentially performed. For another example, the scaling mode is first direction enlargement and second direction reduction, the first selector 1131 receives the first control signal, selects to transmit the image after the buffering processing to the first direction scaling unit 1111 based on the first control signal, the second selector 1132 receives the second control signal, selects to output the target image to be processed to the second direction scaling unit 1112 based on the second control signal, and the third selector 1133 receives the third control signal, selects to transmit the image after the second direction scaling processing to the memory 20 based on the third control signal, so as to determine the processing order as: the second direction scaling processing, the buffering processing and the first direction scaling processing are sequentially performed. Obviously, the two first control signals in the above two scaling modes are different, the two second control signals are different, and the two third control signals are different. Of course, in some scaling modes, they can also be the same, and the embodiment is not limited. Of course, the process of determining the processing order by the first selector 1131, the second selector 1132 and the third selector 1133 based on the received selection control signal corresponding to the two scaling modes in the above examples is only used for illustration, and cannot be regarded as a limitation on the process of determining the processing order in the embodiment.

[0085] For example, each layer processing channel 11 also includes a fourth selector ( Figure 2 (Not shown in the image), step S3, as described above, may further include S312: the fourth selector receives a fourth control signal corresponding to the scaling mode, and selects one of the images scaled in the first direction, scaled in the second direction, and buffered images as the output scaled image based on the fourth control signal. It should be noted that the fourth control signals corresponding to different scaling modes are not entirely the same. For example, in the aforementioned example scaling mode of shrinking in the first direction and enlarging in the second direction, the fourth selector receives the fourth control signal and selects the image scaled in the second direction as the output scaled image based on the fourth control signal. In the scaling mode of enlarging in the first direction and shrinking in the second direction, the fourth selector receives the fourth control signal and selects the image scaled in the first direction as the output scaled image based on the fourth control signal. Clearly, these two fourth control signals are also different.

[0086] When the scaling mode is first direction enlargement and second direction reduction, first direction enlargement and second direction enlargement, first direction enlargement and second direction invariable, or first direction invariable and second direction reduction, the first selector selects to transmit the buffered processed image to the first direction scaling unit based on the first control signal. When the scaling mode is first direction reduction and second direction reduction, first direction reduction and second direction enlargement, or first direction reduction and second direction invariable, the first selector selects to transmit the target image to be processed to the first direction scaling unit based on the first control signal. When the scaling mode is first direction invariable and second direction enlargement, the first selector selects to transmit one of the buffered processed image or the target image to be processed to the first direction scaling unit based on the first control signal. As mentioned above, the first direction is the vertical direction and the second direction is the horizontal direction. In the image scaling process, for example, when the scaling mode is horizontal enlargement and vertical enlargement, vertical enlargement is performed first and then horizontal enlargement is performed, which can make the buffering of the row data less and the resource consumption less. At this time, the processing order is that the buffering processing, the vertical scaling processing, and the horizontal scaling processing are sequentially performed, and the first selector 1131 corresponding to the vertical scaling unit transmits the buffered processed data to the vertical scaling unit. If the scaling mode is vertical reduction and horizontal enlargement, in order to minimize the bandwidth consumption of the memory 20, the processing order should be that the vertical reduction processing, the buffering processing, and the horizontal enlargement processing are sequentially performed. At this time, the first selector 1131 corresponding to the vertical scaling unit transmits the target input image to the vertical scaling unit. Therefore, the vertical scaling unit always selects one of the buffered processed image or the target image to be processed for processing, without directly processing the image after the horizontal scaling processing. Therefore, in order to make the scheduling of the scaling module 111 more convenient and the data transmission path more simple, no matter what the scaling mode is, the first selector 1131 is uniformly set to select one of the target image to be processed and the buffered processed image to transmit to the first direction scaling unit 1111, without receiving the image after the second direction scaling processing. Of course, the first selector 1131 in other embodiments of the present application can not be limited in this way.

[0087] For example, the processing order determined according to the step S3 is specifically, for example, as follows:

[0088] When the scaling mode is first direction reduction and second direction reduction, the processing order is that the first direction scaling processing, the second direction scaling processing, and the buffering processing are sequentially performed, or the second direction scaling processing, the first direction scaling processing, and the buffering processing are sequentially performed.

[0089] When the scaling mode is first direction zoom-in and second direction zoom-in, the processing sequence is that the buffer processing, the first direction scaling processing and the second direction scaling processing are sequentially performed; or the buffer processing, the second direction scaling processing and the first direction scaling processing are sequentially performed. Preferably, the vertical direction zoom-in is performed first, and then the horizontal direction zoom-in is performed. For example, the first direction is the vertical direction, and the second direction is the horizontal direction. When both directions are zoomed in, the processing sequence is that the buffer processing, the first direction scaling processing and the second direction scaling processing are sequentially performed. The vertical zoom-in processing is performed first, and thus the row data buffer is less, and the resource consumption is less.

[0090] When the scaling mode is first direction zoom-out and second direction zoom-in, the processing sequence is that the first direction scaling processing, the buffer processing and the second direction scaling processing are sequentially performed.

[0091] When the scaling mode is first direction zoom-in and second direction zoom-out, the processing sequence is that the second direction scaling processing, the buffer processing and the first direction scaling processing are sequentially performed.

[0092] When the scaling mode is first direction zoom-out and second direction unchanged, the processing sequence is that the first direction scaling processing and the buffer processing are sequentially performed. The second direction scaling processing can be located before, after or between the first direction scaling processing and the buffer processing. Preferably, the second direction scaling processing is located after the first direction scaling processing, so that the data processed by the second direction scaling unit 1112 is the data after being zoomed out once, and the resource consumption is less.

[0093] When the scaling mode is first direction zoom-in and second direction unchanged, the processing sequence is that the buffer processing and the first direction scaling processing are sequentially performed. The second direction scaling processing can be located before, after or between the buffer processing and the first direction scaling processing. Preferably, the second direction scaling processing is located after the first direction scaling processing.

[0094] When the scaling mode is first direction unchanged and second direction zoom-out, the processing sequence is that the second direction scaling processing and the buffer processing are sequentially performed. The first direction scaling processing can be located before, after or between the second direction scaling processing and the buffer processing. Preferably, the first direction scaling processing is located after the buffer processing.

[0095] When the scaling mode is first direction unchanged and second direction zoom-in, the processing sequence is that the buffer processing and the second direction scaling processing are sequentially performed. The first direction scaling processing can be located before, after or between the buffer processing and the second direction scaling processing. Preferably, the first direction scaling processing is located before the second direction scaling processing.

[0096] The above sets different processing sequences by different scaling modes, is applicable to various scaling modes, and makes image scaling more flexible and reduces the instantaneous bandwidth of the memory to the maximum extent. The preferred selector 1131 in the foregoing embodiment only transmits one of the target image to be processed and the buffered processed image to the first direction scaling unit 1111, and the corresponding processing sequence is preferably that the first direction scaling processing is performed after the buffering processing or before the buffering processing and before the second direction scaling processing. The preferred processing sequence corresponding to each scaling mode can be referred to the foregoing content, and specific examples of each scaling mode will not be described here.

[0097] In another embodiment of the application, when the scaling mode is one-direction scaling and the other direction is unchanged, another specific implementation of the foregoing step S3 includes, for example:

[0098] S321: The first target selector receives a first selection signal corresponding to the scaling mode, and transmits one of the target image to be processed and the buffered processed image to one of the first direction scaling unit 1111 and the second direction scaling unit 1112 based on the first selection signal; and the second target selector receives a second selection signal corresponding to the scaling mode, and transmits one of the target image to be processed, one of the first direction scaling processed image and the second direction scaling processed image, and the target image to be processed to the memory 20 based on the second selection signal, thereby determining the processing sequence.

[0099] The second target selector is the third selector 1133 mentioned above, and the second selection signal is the third control signal mentioned above. The first target selector is selected from one of the first selector 1131 and the second selector 1132 according to the scaling mode. Specifically, when the scaling mode is the first direction scaling and the second direction is unchanged, the first target selector is the first selector 1131, and the corresponding first selection signal is the first control signal mentioned above. When the scaling mode is the first direction unchanged and the second direction scaling, the first target selector is the second selector 1132, and the corresponding first selection signal is the second control signal mentioned above. The data transmission relationship between the first target selector and the second target selector and the target image to be processed, the first direction scaling processed image, the second direction scaling processed image, and the cache processed image is described above with reference to the first selector 1131, the second selector 1132, and the third selector 1133. As described above, each layer processing channel 11 further includes a fourth selector. Step S3 further includes S322: when the scaling mode is the first direction scaling and the second direction is unchanged, the fourth selector receives the fourth control signal, and selects one of the first direction scaling processed image and the cache processed image as the scaling processed image based on the fourth control signal; when the scaling mode is the first direction unchanged and the second direction scaling, the fourth selector receives the fourth control signal, and selects one of the second direction scaling processed image and the cache processed image as the scaling processed image based on the fourth control signal.

[0100] For example, when the scaling mode is the first direction scaling and the second direction is unchanged, the first target selector is the first selector 1131, and the second target selector is the third selector 1133. The first selector 1131 receives the first control signal, and transmits the target image to be processed to the first direction scaling unit 1111 based on the first control signal from the target image to be processed and the cache processed image. The third selector receives the third control signal, and transmits the first direction processed image to the memory 20 based on the third control signal from the target image to be processed and the first direction scaling processed image, so that the processing order is the first direction scaling processing and the cache processing in sequence, and the second direction scaling processing is skipped. For example, the fourth selector receives the fourth control signal, and selects the cache processed image as the scaling processed image based on the fourth control signal. The remaining scaling modes can be derived by referring to this example, and will not be described one by one.

[0101] For example, the processing order determined according to step S321 includes:

[0102] The scaling manner is first direction reduction and second direction invariable, the processing order is first direction scaling processing and buffer processing sequentially, and second direction scaling processing is skipped.

[0103] The scaling manner is first direction enlargement and second direction invariable, the processing order is buffer processing and first direction scaling processing sequentially, and second direction scaling processing is skipped.

[0104] The scaling manner is first direction invariable and second direction reduction, the processing order is second direction scaling processing and buffer processing sequentially, and first direction scaling processing is skipped.

[0105] The scaling manner is first direction invariable and second direction enlargement, the processing order is buffer processing and second direction scaling processing sequentially, and first direction scaling processing is skipped.

[0106] In the two specific implementation manners of the foregoing step S3, step S311 is suitable for any scaling manner, wherein the first selector 1131, the second selector 1132, and the third selector 1133 all receive corresponding control signals, and perform corresponding selection based on the received selection control signals, so as to determine the processing order corresponding to the scaling manner, and the control is simpler. Step S321 is suitable for the case that the scaling manner is that only one direction is scaled and the other direction is invariable, wherein the selector connected with the scaling unit corresponding to the invariable direction does not receive the corresponding control signal, that is, the output interface of the selector does not output data, so that the scaling unit corresponding to the invariable direction is skipped, and the processing time length and the resources occupied in the processing process of the scaling unit corresponding to the invariable direction can be saved.

[0107] For example, if the scaling mode is vertical invariant and horizontal zooming (i.e. the first direction is invariant and the second direction is zooming), step S311 is selected, and the specific processing content of step S4 is that the first direction scaling unit 1111 (vertical scaling unit) performs the vertical invariant processing (first direction scaling processing) on the target image to be processed and outputs the first direction scaling processed image to the memory 20 for caching, the cached image is read out and transmitted to the second direction scaling unit 1112 (horizontal scaling unit) for horizontal zooming processing (second direction scaling processing), and finally the second direction scaling processed image is output as the scaling processed image. Alternatively, the target image to be processed is first input to the memory 20 for caching, and then the cached image is transmitted to the first direction scaling unit 1111 (vertical scaling unit) for vertical invariant processing (first direction scaling processing), and finally the first direction scaling unit 1111 (vertical scaling unit) outputs the first direction scaling processed image to the second direction scaling unit 1112 (horizontal scaling unit) for horizontal zooming processing (second direction scaling processing), and finally the second direction scaling processed image is output as the scaling processed image.

[0108] If step S321 is selected, the specific scaling processing content is that the vertical direction processing (first direction scaling processing) is skipped, the target image to be processed is directly input to the memory 20 for caching, the cached image is transmitted to the second direction scaling unit 1112 (horizontal scaling unit) for horizontal zooming processing (second direction scaling processing), and finally the second direction scaling processed image is output as the scaling processed image. It should be noted that the vertical invariant processing can be understood as the first direction scaling unit 1111 performing scaling processing with a scaling ratio of 1:1 on the input image, although the output image is invariant to the input image, but it occupies resources and processing time, and therefore skipping the vertical direction processing reduces the resource occupation and saves the processing time.

[0109] In summary, in the above embodiments, the first selector 1131, the second selector 1132 and the third selector 1133 are arranged in each layer processing channel 11, and the data transmission sequence between the first direction scaling unit 1111, the second direction scaling unit 1112 and the memory 20 is determined by a specific working mode, which is only a specific example of step S3 implemented in the image processor 10 of the image processing device provided by the embodiments of the present application, and by adding a few modules in each layer processing channel 11, the scaling processing of the image with any ratio can be conveniently and flexibly implemented, which has the characteristics of simplicity and efficiency. However, the above specific embodiments cannot be regarded as a limitation of the image processor 10, and any structure that can realize the processing sequence of step S3 according to the scaling mode is within the protection scope of the present application.

[0110] Further, the image processor 10 is also configured to perform step S5 (performed between step S2 and step S3): generating the first control signal, the second control signal and the third control signal according to the zoom mode, and outputting the first control signal to the first selector 1131, outputting the second control signal to the second selector 1132, and outputting the third control signal to the third selector 1133. Wherein, as each layer processing channel 11 in the foregoing examples also includes a fourth selector, step S5 also includes generating a fourth control signal according to the zoom mode, and outputting the fourth control signal to the fourth selector.

[0111] According to the different embodiments described above, the image processor 10 can only generate the first control signal and the third control signal, or only generate the second control signal and the third control signal, or generate the first control signal, the second control signal and the third control signal according to the zoom mode.

[0112] In another embodiment of the present application, another image processing device 300 for performing the foregoing image processing method is also provided, which is based on the image processing device 200, and further includes a controller 30 connected with the image processor 10. Specifically, referring to Figure 3 , the controller 30 is connected with the first selector 1131, the second selector 1132 and the third selector 1133 in the image processor 10 respectively, and the controller 30 is configured to generate the first control signal, the second control signal and the third control signal, and output the first control signal to the first selector 1131, output the second control signal to the second selector 1132, and output the third control signal to the third selector 1133. Wherein, as each layer processing channel 11 in the foregoing examples also includes a fourth selector, the controller 30 is also connected with the fourth selector (not shown in Figure 3 ), and the controller 30 generates a fourth control signal according to the zoom mode, and outputs the fourth control signal to the fourth selector.

[0113] According to the different embodiments described above, the controller 30 can only generate the first control signal and the third control signal, or only generate the second control signal and the third control signal, or generate the first control signal, the second control signal and the third control signal according to the zoom mode. The controller 30 is specifically, for example, an MCU or an ARM processor or a controller with similar control functions, and the present embodiment is not limited thereto.

[0114] In an embodiment of the present application, the image processor 10 in the foregoing examples also includes a superimposition module, which is configured to perform step S6: superimposing the plurality of zoom-processed images output by the plurality of layer processing channels 11 to obtain a pre-monitoring picture, and outputting the pre-monitoring picture to an external display for display.

[0115] The pre-monitoring picture can also be displayed on the screen after step S6. The plurality of images to be processed input into the plurality of layer processing channels 11 can be a plurality of sub-images obtained by splitting a source image, and the plurality of sub-images are scaled and superimposed to obtain a scaled picture of the source image. Of course, the plurality of images to be processed can also be a plurality of different images, and the scaling modes can be the same or different. The image processing device disclosed in the embodiment can be applied to scaling pre-monitoring of a video processor. In actual applications, the plurality of scaled images are output and then buffered or processed in other manners. The foregoing embodiments of the present application emphasize that the image processing device can be applied to scaling pre-monitoring, but are not limited thereto. The image processing device can also be applied to other systems that have scaling processing requirements.

[0116] In the foregoing embodiments, the number of scaling modules of a single layer in the image processing device is reduced to one, and the processing order is determined according to the scaling mode of the image to be processed, so that the image to be processed is scaled based on the processing order, thereby ensuring that the scaling of the image at any scale is achieved, and the total number of scaling modules in the entire image processing device is reduced by half, so that the resources occupied by the scaling modules in the scaling of the image of the same number of layers are less, and the resource consumption is reduced. Alternatively, using the same total number of scaling modules, the image processing device can process more layers of scaling than the prior art, and more functions can be achieved under the same resource consumption. The image processing method M100 is executed in the image processing device, and has the same beneficial effects as the image processing device.

[0117] The above description is only the preferred embodiments of the present application, and does not limit the present application in any form. Although the present application has been disclosed as the above preferred embodiments, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical contents to obtain equivalent embodiments, without departing from the technical solution of the present application. Any simple modification, equivalent change and modification of the above embodiments according to the technical essence of the present application are still within the scope of the technical solution of the present application.

Claims

1. An image processing method, characterized by, The image processing method is executed in an image processor, the image processor comprises a plurality of layer processing channels, each of the layer processing channels is provided with a scaling module, and the image processing method comprises the following steps: receiving an input image to be processed; obtaining a scaling mode corresponding to each of the images to be processed; determining a processing order of a target image to be processed according to the corresponding scaling mode; performing at least cache processing and scaling processing on the target image to be processed according to the processing order to obtain a scaled image; wherein the scaling processing is performed by the same scaling module; wherein each of the scaling modules comprises a first direction scaling unit and a second direction scaling unit; the scaling processing by the same scaling module comprises: performing first direction scaling processing by the first direction scaling unit to output a first direction scaled image, and / or performing second direction scaling processing by the corresponding second direction scaling unit to output a second direction scaled image; the cache processing comprises caching one of the target image to be processed, the first direction scaled image and the second direction scaled image to a memory to obtain a cached image; each of the layer processing channels further comprises an input module, a first selector, a second selector and a third selector, wherein the input module is used to receive the target image to be processed; the determination of the processing order of the target image to be processed according to the corresponding scaling mode comprises: the first selector receives a first control signal corresponding to the scaling mode and transmits one of the target image to be processed, the second direction scaled image or the cached image to the first direction scaling unit based on the first control signal; and the second selector receives a second control signal corresponding to the scaling mode and transmits one of the target image to be processed, the first direction scaled image or the cached image to the second direction scaling unit based on the second control signal; and the third selector receives a third control signal corresponding to the scaling mode and transmits one of the target image to be processed, the first direction scaled image or the second direction scaled image to the memory based on the third control signal, thereby determining the processing order.

2. The image processing method of claim 1, wherein, the determination of the processing order of the target image to be processed according to the corresponding scaling mode comprises: when the scaling mode is first direction enlargement and second direction reduction, first direction enlargement and second direction enlargement, first direction enlargement and second direction unchanged, or first direction unchanged and second direction reduction, the first selector transmits the cached image to the first direction scaling unit based on the first control signal; or when the scaling mode is first direction reduction and second direction enlargement, first direction reduction and second direction unchanged, or first direction unchanged and second direction enlargement, the second selector transmits the cached image to the second direction scaling unit based on the second control signal; or when the scaling mode is first direction unchanged and second direction unchanged, the third selector transmits the cached image to the memory based on the third control signal. The scaling manner is first direction reduction and second direction reduction, first direction reduction and second direction enlargement, or first direction reduction and second direction invariable, the first selector selects to transmit the target image to be processed to the first direction scaling unit based on the first control signal; or The scaling manner is first direction invariable and second direction enlargement, the first selector selects to transmit one of the buffered processed image or the target image to be processed to the first direction scaling unit based on the first control signal.

3. The image processing method of claim 1, wherein, The determining of the processing sequence of the target image to be processed according to the corresponding scaling manner further comprises: The first control signal, the second control signal and the third control signal are generated according to the corresponding scaling manner and output to the first selector, the second selector and the third selector respectively.

4. The image processing method of claim 1, wherein, The determining of the processing sequence of the target image to be processed according to the corresponding scaling manner comprises: The scaling manner is first direction enlargement and second direction reduction, first direction enlargement and second direction enlargement, first direction enlargement and second direction invariable, or first direction invariable and second direction reduction, the first direction scaling processing is after the buffering processing; or The scaling manner is first direction reduction and second direction reduction, first direction reduction and second direction enlargement, or first direction reduction and second direction invariable, the first direction scaling processing is before the buffering processing; or The scaling manner is first direction invariable and second direction enlargement, the first direction scaling processing is before the buffering processing or the buffering processing is before the first direction scaling processing.

5. The image processing method of claim 4, wherein, The determining of the processing sequence of the target image to be processed according to the corresponding scaling manner comprises: The scaling manner is first direction enlargement and second direction reduction, first direction enlargement and second direction invariable, or first direction invariable and second direction reduction, the processing sequence is that the second direction scaling processing, the buffering processing and the first direction scaling processing are sequentially performed; or The scaling manner is first direction enlargement and second direction enlargement, the processing sequence is that the buffering processing, the first direction scaling processing and the second direction scaling processing are sequentially performed; or The scaling manner is first direction reduction and second direction reduction, the processing sequence is that the first direction scaling processing, the second direction scaling processing and the buffering processing are sequentially performed; The scaling manner is first direction reduction and second direction enlargement, the processing sequence is that the first direction scaling processing, the buffering processing and the second direction scaling processing are sequentially performed; or The scaling manner is first direction invariable and second direction enlargement, the processing sequence is that the first direction scaling processing, the buffering processing and the second direction scaling processing are sequentially performed or the buffering processing, the first direction processing and the second direction scaling processing are sequentially performed; or The scaling manner is first direction reduction and second direction invariable, and the processing sequence is that the first direction scaling processing, the buffer processing and the second direction scaling processing are sequentially performed or the first direction scaling processing, the second direction processing and the buffer processing are sequentially performed.

6. The image processing method of claim 1, wherein, Each of the layer processing channels further comprises: An input module, a first target selector and a second target selector, wherein the input module is configured to receive the target image to be processed; The processing sequence is determined according to the corresponding scaling manner, comprising: The first target selector receives a first selection signal corresponding to the scaling manner, and transmits one selected from the target image to be processed and the buffered image to one of the first direction scaling unit and the second direction scaling unit based on the first selection signal; and The second target selector receives a second selection signal corresponding to the scaling manner, and transmits one selected from the first direction scaling processed image and the second direction scaling processed image, and The target image to be processed to the memory, so as to determine the processing sequence; The scaling manner is: first direction reduction and second direction invariable, first direction enlargement and second direction invariable, first direction invariable and second direction reduction, or first direction invariable and second direction enlargement.

7. The method of claim 1 6. The image processing method according to any one of claims 1 to 5, wherein The image processing method further comprises: Superimposing the plurality of scaling processed images output by the plurality of layer processing channels to obtain a pre-monitoring picture, and outputting the pre-monitoring picture to an external display for display.

8. An image processing apparatus characterized by comprising: Comprises: An image processor and a memory connected to the image processor, wherein the image processor is configured to perform the method of claim 1 6. The image processing method of any one of claims 1 to 5.

Images