Image Processing Apparatus, Image Processing System, and Image Processing Method

Abstract

The present invention provides an image processing apparatus, an image processing system, and an image processing method for determining image adjustment parameters in real time to facilitate modification of the image adjustment parameters. The image processing apparatus includes: an image input device that acquires an original image signal and outputs a first image signal; a parameter adjustment device that modifies the image adjustment parameters; an image processing circuit that receives the first image signal and the image adjustment parameters and adjusts the first image signal according to the image adjustment parameters to output a second image signal; an information marking circuit that is coupled to the image processing circuit and receives the second image signal of the image processing circuit and internal information and marks at least a part of the second image signal according to the internal information to generate a marked output image; and an image output device that is coupled to the information marking circuit and outputs the marked output image having at least one mark indicating the internal information.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing system, and an image processing method. Background Art

[0002] The image quality of an object obtained from an image system may depend on the environmental conditions of the object. For example, the image quality of the object may become lower as the environmental brightness becomes lower. A typical image processing method for improving the image quality in the above conditions is to use an image processing unit (such as a spatio-temporal filter) to filter the noise of the image. However, it may not be possible to determine the appropriate image adjustment parameters of the image processing unit for effectively filtering the noise of the image. The side effect of using the wrong filter is that there are unexpected images in the video or the noise cannot be significantly eliminated. As the image processing algorithm becomes more complex, it becomes more difficult to adjust the parameters only according to the final effect.

[0003] Nothing in this document should be construed as an admission of knowledge in the prior art of any part of the present invention. Summary of the Invention

[0004] The present invention introduces an image processing apparatus, an image processing system, and an image processing method for determining image adjustment parameters in real time.

[0005] In an embodiment of the present invention, the image processing apparatus includes: an image processing circuit configured to receive a first image signal and an image adjustment parameter and process the first image signal according to the image adjustment parameter to output a second image signal; and an information marking circuit coupled to the image processing circuit and configured to receive the second image signal of the image processing circuit and internal information and mark at least a part of the second image signal according to the internal information to generate a marked output image having at least one mark indicating the internal information, so as to display in real time the adjustment result generated by the image adjustment parameter for facilitating the modification of the image adjustment parameter.

[0006] In an embodiment of the present invention, the image processing system includes: an image input device configured to acquire an original image signal and output a first image signal; a parameter adjustment device configured to modify an image adjustment parameter; an image processing device configured to receive the first image signal and the image adjustment parameter and process the first image signal according to the image adjustment parameter to generate a marked output image for displaying in real time the adjustment result generated by the image adjustment parameter for facilitating the modification of the image adjustment parameter; and an image output device coupled to the information marking circuit and configured to output the marked output image having at least one mark indicating the internal information of the image processing device.

[0007] In an embodiment of the present invention, an image processing method includes: receiving a first image signal and an image adjustment parameter, and processing the first image signal according to the image adjustment parameter by an image processing circuit to output a second image signal; and receiving the second image signal of the image processing circuit and internal information, and marking at least a part of the second image signal according to the internal information to generate a marked output image having at least one mark indicating the internal information, so as to display in real time the adjustment result generated by the image adjustment parameter for facilitating the modification of the image adjustment parameter.

[0008] In summary, the image processing apparatus, the image processing system, and the image processing method provided by the present invention facilitate the modification of the image adjustment parameter to enhance the image quality.

[0009] To make the foregoing content easier to understand, several embodiments with accompanying drawings are described in detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is an image processing system according to an embodiment of the present invention.

[0011] Figure 2 is an image processing apparatus according to an embodiment of the present invention.

[0012] Figures 3A to 3C are three marked output images according to an embodiment of the present invention.

[0013] Figure 4 is the signal flow of an image processing apparatus according to an embodiment of the present invention.

[0014] Figure 5 is the signal flow of different modes according to an embodiment of the present invention.

[0015] Figure 6 is the signal flow of an information marking circuit according to an embodiment of the present invention.

[0016] Figure 7 is a flowchart of an image processing method according to an embodiment of the present invention.

[0017] DESCRIPTION OF THE REFERENCE NUMERALS IN THE DRAWINGS

[0018] 1, 2: movement conditions;

[0019] 100: image processing system;

[0020] 101: image input device;

[0021] 102: image processing device;

[0022] 103: image output device;

[0023] 104: parameter adjustment device;

[0024] 201: Image processing circuit;

[0025] 202: Information marking circuit;

[0026] 401: Color gain;

[0027] 402: CFA;

[0028] 403: Edge enhancement unit;

[0029] 404: Spatial - temporal filter;

[0030] 405: Image sensor;

[0031] 501: Multiplexer circuit;

[0032] 502: Calibration mode;

[0033] 503: Normal mode;

[0034] 601: Receiving unit;

[0035] 602: Mark generation unit;

[0036] 603: Overlap unit;

[0037] A: First vehicle;

[0038] B: Second vehicle;

[0039] C: Third vehicle;

[0040] S701, S702, S703, S704, S705: Steps. Detailed implementation manner

[0041] Embodiments of the present invention will be described below with reference to the drawings.

[0042] Figure 1 It is an image processing system 100 according to an embodiment of the present invention. The image processing system 100 includes an image input device 101, an image processing device 102, an image output device 103, and a parameter adjustment device 104. The image input device 101 may include an image sensor or an image reading device that obtains an original image signal from an image sensor or a storage device, and outputs a first image signal to the image processing device 102. The image processing device 102 receives the first image signal, adjusts the first image signal according to the image adjustment parameters from the parameter adjustment device 104, and outputs a marked output image. The image output device 103 receives and outputs the marked output image.

[0043] After modifying the image adjustment parameters, the internal operation state or condition or settings of the image processing device 102 are modified, which may further generate a second image signal having different marks indicating different internal information.

[0044] The image output device 103 can present the marked output image to the user of the image processing system 100 in real time, so as to display the adjustment result generated by the image adjustment parameters to facilitate the user to modify the image adjustment parameters. In other words, different values of the image adjustment parameters can make the marks on the marked output image different. The user can view the marked output images with different marks to observe the different adjustment effects generated by different values of the image adjustment parameters. Due to different marks (with different observable indications) of different categories (for example, different moving states, different edge intensities), the user can more easily judge the reason for the unsatisfactory image. For example, the user can more easily judge whether the moving area of the burr moving area is a bad judgment of the covered area of the moving area or an over-strong filtering associated with the moving area.

[0045] Therefore, a user with more knowledge about the output image can make a more appropriate modification to the image adjustment parameters. In other words, the user can then modify the value of the image adjustment parameters according to the user's observation, so that the marked output image generated according to the modified value of the image adjustment parameters can better meet the user's requirements. The final value of the image adjustment parameters determined by the user in the calibration mode can be stored in the storage device, so that in the normal mode, the image processing device 102 can operate according to the final value of the image adjustment parameters.

[0046] Figure 2 is the signal flow of the image processing device 102 according to an embodiment of the present invention. The image processing device 102 includes an image processing circuit 201 and an information marking circuit 202. The image processing circuit 201 receives the first image signal and the image adjustment parameters and adjusts the first image signal according to the image adjustment parameters to output a second image signal. The information marking circuit 202 receives the second image signal of the image processing circuit 201 and the internal information and marks at least a part of the second image signal according to the internal information to generate a marked output image for presenting the adjustment result generated by the image adjustment parameters in real time to facilitate the modification of the image adjustment parameters. The parameter adjustment device 104 is configured to modify the image adjustment parameters, and the image processing circuit 201 processes the first image signal according to the image adjustment parameters. After modifying the image adjustment parameters, the internal operation state or condition or settings of the image processing circuit 201 are modified, which may further generate a second image signal having different marks indicating different internal information.

[0047] The final values of the image adjustment parameters determined by the user in the calibration mode can be stored in the storage device, so that in the normal mode, the image processing apparatus 102 can operate according to the final values of the image adjustment parameters.

[0048] The image processing circuit 201 can classify at least a part of the second image signal into different categories and the internal information indicates the result of the classification, and the marking generation unit 602 of the information marking circuit 202 is configured to generate different marks for different categories according to the internal information. The different marks for different categories can enable the user or operator to have more observations on the adjustment result of the image adjustment parameters. For example, the different marks can have different colors or any observable indications of different categories.

[0049] At least a part of the second image signal includes a first part belonging to the first category among different categories and a second part belonging to the second category among different categories, and the overlapping unit 603 of the information marking circuit 202 can be configured to overlap the first part with a first mark indicating the first category and overlap the second part with a second mark indicating the second category.

[0050] As an example, the image processing circuit 201 includes a spatio-temporal filter 404 configured to filter the first image signal according to the image adjustment parameters. The different categories can correspond to different movement conditions, where the different movement conditions can include at least one of a stationary state, a moving state, and a transition state. Therefore, the marks can have a first color (or any first indication), a second color (or any second indication), and a third color (or any third indication) respectively for the stationary state, the moving state, and the transition state.

[0051] As an example, the image processing circuit 201 includes an edge enhancement unit 403 or a sharpening unit configured to enhance or sharpen the edges of the first image signal according to the image adjustment parameters. The different categories correspond to different edge intensities. Therefore, the marks can have a first color (or any first indication) and a second color (or any second indication) respectively for the first edge intensity and the second edge intensity, and so on.

[0052] Figure 3A and Figure 3B are two marked output images according to an embodiment of the present invention. Figure 3A and Figure 3B are street view images. In Figure 3A , the street view image is at the first time, the first car (A) that is moving and about to pass through the zebra crossing is in the moving state and the first car (A) is marked with a first color (or line width, line type). The second car (B) and the third car (C) are behind the first car (A), and the second car (B) that starts to move is in the transition state and is marked with a second color (or line width, line type). Figure 3APerson (D) is walking (i.e., in a moving state) and marked with a first color. In Figure 3B at a second time later than the first time, three cars (A, B, C) and person (D) are all in a moving state and marked with a first color.

[0053] Figure 3C The different objects in can be classified into different categories according to their edge strengths and marked with different edge strengths corresponding to different categories. For example, an object with an edge strength less than a first threshold (which is a set of parameters in the image processing device 102) can be marked with a first edge strength, and an object with an edge strength equal to or greater than the first threshold can be marked with a second edge strength.

[0054] The operator or user can judge the adjustment result of the image adjustment parameter in real time by observing the marked output image with different marks corresponding to different values of the image adjustment parameter so as to modify the image adjustment parameter.

[0055] Reference Figure 2 and Figures 3A to 3C The internal information of the image processing circuit 201 indicates Figure 3A and Figure 3C the results of the classification mentioned in, for example, when the image processing circuit 201 includes a spatio-temporal filter 404 configured to filter the first image signal according to the image adjustment parameter, different categories correspond to different movement conditions, where different movement conditions include at least one of a stationary state, a moving state, and a transitional state. When the image processing circuit 201 includes an edge enhancement unit 403 or a sharpening unit configured to enhance or sharpen the edges of the first image signal according to the image adjustment parameter, different categories correspond to different edge strengths.

[0056] Figure 4 is an image processing circuit 201 according to an embodiment of the present invention. Figure 4 The CFA 402 in represents a color filter array. The first image signal received by the image processing circuit 201 is processed by at least one of a series of functional blocks (i.e., color gain 401, CFA 402, edge enhancement unit 403, and spatio-temporal filter 404) to obtain an improved image signal compared to the original image signal obtained from the image sensor 405 or the storage device. In other words, the image adjustment parameter provided by the parameter adjustment device 104 can be used by any one or more of the functional blocks to process the image input device 101 (e.g., the image sensor 405).

[0057] Figure 5is a signal flow of different modes according to an embodiment of the present invention. The information marking circuit 202 further includes a multiplexer circuit 501 configured to receive a marked output image and a second image signal and output the marked output image or the second image signal according to a mode selection signal. In a calibration mode 502, the multiplexer circuit 501 can output a marked output image signal. In a normal mode 503, the multiplexer circuit 501 can directly output the second image signal without a mark.

[0058] Figure 6 is a signal flow of the information marking circuit 202 according to an embodiment of the present invention. The information marking circuit 202 includes: a receiving unit 601 configured to receive a second image signal and internal information of the image processing circuit 201; a marking generation unit 602 configured to generate at least one mark according to the second image signal and the internal information of the image processing circuit 201; and an overlapping unit 603 configured to overlap at least one part of the second image signal with at least one mark to generate a marked output image.

[0059] In some embodiments, the marking generation unit 602 can generate different marks for overlapping the second image signal by modifying the U value and the V value of the second image signal into different values according to different values of the internal information while maintaining the Y value of the second image signal.

[0060] In some alternative embodiments, the overlapping unit 603 generates different marks for overlapping the second image signal by modifying the Y value of the second image signal into different values according to different values of the internal information while modifying each of the U value and the V value of the second image signal into a predetermined value (e.g., 128 of an 8-bit signal).

[0061] As an example, the marking generation unit 602 generates different colors as at least one mark according to different movement conditions (i.e., different categories) of at least one part of the second image signal. This can occur when the image processing circuit 201 includes a spatio-temporal filter 404. The overlapping unit 603 can overlap a first part of at least one part of the second image signal having a movement condition 1 (i.e., a movement state) with a first color by respectively modifying the U value and the V value of the second image signal into a first predetermined value while maintaining the Y value of the second image signal. Additionally or alternatively, the overlapping unit 603 can overlap a second part of at least one part of the second image signal having a movement condition 2 (i.e., a transition state) with a second color by respectively modifying the U value and the V value of the second image signal into a second predetermined value while maintaining the Y value of the second image signal.

[0062] As another example, the marker generation unit 602 generates different colors as at least one marker according to different edge intensities (i.e., different categories) of at least one part of the second image signal. This may occur when the image processing circuit 201 includes an edge enhancement unit 403 or a sharpening unit. The overlapping unit 603 may overlap a first part of at least one part of the second image signal having a first edge intensity with a first color by respectively modifying the U value and the V value of the second image signal to a first predetermined value while maintaining the Y value of the second image signal. Additionally or alternatively, the overlapping unit 603 may overlap a second part of at least one part of the second image signal having a second edge intensity with a second color by respectively modifying the U value and the V value of the second image signal to a second predetermined value while maintaining the Y value of the second image signal.

[0063] The overlapping unit 603 may also overlap at least one part of the second image signal with a different color by respectively modifying the R value, the G value, and the B value of the second image signal to a predetermined value. For example, for an 8-bit image signal, the R value, the G value, and the B value may be modified to 255. The marker generation unit 602 may also generate different line widths or line types as at least one marker.

[0064] Figure 7 is a flowchart of an image processing method according to an embodiment of the present invention. The image processing method may be executed by the image processing apparatus in the above embodiments, but is not limited thereto. The image processing method may be executed in a calibration mode of the image processing apparatus. The image processing method includes: receiving a first image signal and image adjustment parameters (S701); processing the first image signal through an image processing circuit according to the image adjustment parameters to output a second image signal (S702); receiving the second image signal and internal information of the image processing circuit (S703); marking at least one part of the second image signal according to the internal information (S704); generating a marked output image having at least one marker indicating the internal information for real-time presentation of the adjustment result generated by the image adjustment parameters (S705) so as to facilitate a user or an operator of the image processing apparatus to determine final image adjustment parameters. By viewing the marker having observable features / characteristics that can indicate the internal information (e.g., different blocks in the image processing circuit), the user or the operator can better judge the cause of the non-desired image and make better adjustments to improve the image quality. The final image adjustment parameters determined or required by the user or the operator may then be stored in a storage device and accessed by the image processing apparatus to process image data and / or display an image based on the final image adjustment parameters in a normal mode. The details of each step can be analogized from the descriptions in other embodiments and are therefore omitted here for the sake of brevity.

[0065] In summary, the image processing apparatus, image processing system, and image processing method provided by the present invention facilitate modifying image adjustment parameters to enhance image quality.

[0066] It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments without departing from the scope or spirit of the present invention. Given the foregoing, it is intended that the present invention cover modifications and variations that fall within the scope of the appended claims and their equivalents.

Claims

1. An image processing apparatus, comprising: an image processing circuit configured to receive a first image signal and an image adjustment parameter, and process the first image signal according to the image adjustment parameter to output a second image signal; and an information marking circuit coupled to the image processing circuit and configured to receive the second image signal of the image processing circuit and internal information, and mark at least a part of the second image signal according to the internal information to generate a marked output image having at least one mark indicating the internal information, for real-time presenting the adjustment result generated by the image adjustment parameter to facilitate modifying the image adjustment parameter, wherein the at least one mark is used to distinguish the movement conditions of different objects or the image edge intensities presented by different objects in the second image signal.

2. The image processing apparatus according to claim 1, wherein the information marking circuit comprises: a receiving unit configured to receive the second image signal of the image processing circuit and the internal information; a mark generating unit configured to generate the at least one mark according to the second image signal of the image processing circuit and the internal information; and an overlapping unit configured to overlap at least a part of the second image signal with the at least one mark to generate the marked output image.

3. The image processing apparatus according to claim 2, wherein the image processing circuit is further configured to classify at least a part of the second image signal into different categories and the internal information indicates the result of the classification, and the mark generating unit is configured to generate different marks for the different categories according to the internal information.

4. The image processing apparatus according to claim 2, wherein at least a part of the second image signal comprises a first part belonging to a first category among different categories and a second part belonging to a second category among the different categories, and the overlapping unit is configured to overlap the first part with a first mark indicating the first category and overlap the second part with a second mark indicating the second category.

5. The image processing apparatus according to claim 3, wherein the different marks have different colors for the different categories.

6. The image processing apparatus according to claim 2, wherein the image processing circuit comprises a spatio-temporal filter configured to filter the first image signal according to the image adjustment parameter.

7. The image processing apparatus according to claim 6, wherein the mark generating unit is configured to generate different marks for different categories and the different categories correspond to different movement conditions, wherein the different movement conditions include at least one of a stationary state, a moving state, and a transitional state.

8. The image processing apparatus according to claim 2, wherein the image processing circuit comprises an edge enhancement unit or a sharpening unit configured to enhance or sharpen the edge of the first image signal according to the image adjustment parameter.

9. The image processing apparatus according to claim 8, wherein the marker generation unit is configured to generate different markers of different categories according to the internal information, and the different categories correspond to different edge intensities.

10. The image processing apparatus according to claim 1, wherein the second image signal is a YUV signal, and the information marking circuit is configured to perform a marking operation by respectively modifying the U value and the V value of the YUV signal according to the internal information while maintaining the Y value of the YUV signal.

11. The image processing apparatus according to claim 1, wherein the second image signal is a YUV signal, and the information marking circuit is configured to perform a marking operation by modifying the Y value of the YUV signal according to the internal information while modifying each of the U value and the V value of the YUV signal to a predetermined value.

12. The image processing apparatus according to claim 1, wherein the second image signal is an RGB signal, and the information marking circuit is configured to perform a marking operation by respectively modifying the R value, the G value, and the B value of the RGB signal to predetermined values.

13. The image processing apparatus according to claim 1, wherein the information marking circuit further includes: a multiplexer circuit configured to receive the marked output image and the second image signal and output the marked output image or the second image signal according to a mode selection signal.

14. The image processing apparatus according to claim 2, wherein the image adjustment parameter can be modified by a user by observing the marked output image.

15. An image processing system, comprising: an image input device configured to acquire an original image signal and output a first image signal; a parameter adjustment device configured to modify an image adjustment parameter; an image processing device configured to receive the first image signal and the image adjustment parameter and process the first image signal according to the image adjustment parameter to generate a second image signal, and the image processing device further generates a marked output image by marking the second image signal for real-time display of an adjustment result generated by the image adjustment parameter to facilitate modification of the image adjustment parameter; and an image output device coupled to the image processing device and configured to output the marked output image having at least one marker indicating internal information of the image processing device wherein the at least one marker is used to distinguish movement conditions of different objects or image edge intensities presented by different objects in the second image signal.

16. The image processing system according to claim 15, wherein the image processing device includes: an image processing circuit configured to receive the first image signal and the image adjustment parameter and adjust the first image signal according to the image adjustment parameter to output a second image signal; and an information marking circuit coupled to the image processing circuit and configured to receive the second image signal and the internal information and mark at least a part of the second image signal according to the internal information to generate the marked output image.

17. The image processing system according to claim 15, wherein the image input device includes an image sensor.

18. The image processing system according to claim 17, wherein the image input device is configured to read the original image signal from a storage device.

19. An image processing method, comprising: receiving a first image signal and image adjustment parameters and processing the first image signal according to the image adjustment parameters by an image processing circuit to output a second image signal; and receiving the second image signal of the image processing circuit and internal information and marking at least one part of the second image signal according to the internal information to generate a marked output image having at least one mark indicating the internal information for real-time presenting an adjustment result generated by the image adjustment parameters to facilitate modifying the image adjustment parameters, wherein the at least one mark is used to distinguish movement conditions of different objects or image edge intensities presented by different objects in the second image signal.

20. The image processing method according to claim 19, further comprising: generating at least one mark according to the second image signal and the internal information; and causing the at least one part of the second image signal to overlap with the at least one mark to generate the marked output image.

21. The image processing method according to claim 19, wherein the second image signal is a YUV signal, and a marking operation is performed by respectively modifying U values and V values of the YUV signal according to the internal information while maintaining a Y value of the YUV signal.

22. The image processing method according to claim 19, wherein the second image signal is a YUV signal, and the information marking circuit is configured to perform a marking operation by modifying a Y value of the YUV signal according to the internal information while modifying each of U values and V values of the YUV signal into a predetermined value.

23. The image processing method according to claim 19, wherein the second image signal is an RGB signal, and a marking operation is performed by respectively modifying an R value, a G value, and a B value of the RGB signal into a predetermined value.

24. The image processing method according to claim 19, further comprising: receiving the marked output image and the second image signal and outputting the marked output image or the second image signal according to a mode selection signal.

Images