Method for editing image

Abstract

A method of editing an image includes capturing the image by an image capturing device to generate display data, reading the display data of the image by an image input unit, the display data including a plurality of angle features of an object in the image, and editing the image by the image capturing device. The plurality of angle features comprise color information of the object corresponding to the plurality of visual angles, modifying the plurality of angle features of the object according to at least one editing instruction through the processor to generate edited display data, and outputting the edited display data through the image output unit.

Description

[0001] This application is a divisional application of the invention patent application with application date of December 17, 2020, application number 202011496216.3, and invention name “Method for editing images and image editing system”. Technical Field

[0002] The present disclosure relates to a method and an image editing system for editing an image, and more particularly to a method and an image editing system for editing the angle characteristics of an image. Background Art

[0003] As display devices are used in more and more fields, the market's requirements for visual quality are also increasing. For example, High Dynamic Range (HDR) technology has been developed to present images with high contrast, so that both bright details and dark details in the image can be clearly presented at the same time. Although high dynamic range displays can present high-contrast images and thus present better visual effects than traditional displays, high dynamic range displays still have difficulty presenting the gloss effect (kirameki effect) presented when real light shines on an object.

[0004] For example, some objects, such as butterfly wings or bubbles, scatter light to produce structural colors. In this case, if the display device is to be able to present the aforementioned effect, it must display different light configurations in different directions, and when people view the object from different positions, they will see different colors and / or brightness. However, traditional displays can only present the light reflection of static scenes at fixed angles, so it is difficult to present a realistic gloss effect. In addition, even if the display can present a realistic gloss effect, the light configuration of the image needs to be edited or optimized to provide a satisfactory visual effect. Summary of the invention

[0005] An embodiment of the present disclosure provides a method for editing an image by an image editing system. The image editing system includes an image input unit, a processor, and an image output unit. The method includes reading display data of the image through the image input unit, modifying multiple angle features of an object according to at least one editing instruction by the processor to generate edited display data, and outputting the edited display data through the image output unit.

[0006] The display data includes multiple angle features of an object in the image, and the multiple angle features include color information of the object corresponding to multiple viewing angles.

[0007] Another embodiment of the present disclosure provides an image editing system, which includes an image input unit, a processor, and an image output unit.

[0008] The image input unit reads display data of an image, wherein the display data includes multiple angle features of an object in the image, and the multiple angle features include color information of the object corresponding to multiple viewing angles. The processor receives the display data from the image input unit, and modifies the multiple angle features of the object according to at least one editing instruction to generate edited display data. The image output unit outputs the edited display data. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of an image editing system according to an embodiment.

[0010] Figure 2 It is a schematic diagram of image display data according to an embodiment.

[0011] Figure 3 yes Figure 2 A graph of the angular characteristics of an object in .

[0012] Figure 4 is a schematic diagram of display data according to an embodiment.

[0013] Figure 5 yes Figure 3 Discrete cosine transform of the angular characteristics and a schematic diagram showing the data.

[0014] Figure 6 It is a schematic diagram of the original angle characteristic curve and the transformed angle characteristic curve.

[0015] Figure 7 yes Figure 2 Schematic diagram of the angle characteristic curve of the object and the synthesized angle characteristic curve.

[0016] Figure 8 is a schematic diagram of an image editing system according to an embodiment.

[0017] Fig. 9 It is a flow chart of a method for editing an image according to an embodiment.

[0018] Explanation of reference numerals: 100, 200-image editing system; 110-image input unit; 120-processor; 130-image output unit; D1, D2, D3-display data; ED1 edited display data; IMG1-image; OB1, OB2-objects; RM1 to RMx, GM1 to GMx, BM1 to BM-color comparison chart; Itop-intensity peak; A0-peak viewing angle; Ibase-base intensity; DP-dispersion coefficient; PM1 to PMY-parameter pair According to the figure; CM1 to CMZ-discrete cosine transform coefficient comparison chart; C0, C1, C2, C3, C4, CA, CB, CC-angle characteristic curves; A1, A2, A3, A4, A5, A6, A7, A8-viewing angle; 240-image capture device; 242-multi-lens image sensor; 244-depth sensor; 246-fisheye sensor; 250-display device; 252-display panel; 254-optical modulator; 300-method; S310 to S330-steps. DETAILED DESCRIPTION

[0019] Figure 1 FIG. 1 is a schematic diagram of an image editing system 100 according to an embodiment of the present invention. The image editing system 100 includes an image input unit 110 , a processor 120 , and an image output unit 130 .

[0020] The image input unit 110 may read the display data D1 of the image to be edited. In some embodiments, the display data D1 may include multiple angular profiles of an object in the image, and the angular profiles may include color information of the object corresponding to multiple viewing angles. The processor 120 may receive the display data D1 from the image input unit 110, and may modify the angular profiles of the object in the display data according to the editing instruction to generate the edited display data ED1. The image output unit 130 may output the edited display data ED1.

[0021] Figure 2 FIG. 1 is a schematic diagram of display data D1 of an image IMG1 according to an embodiment. Figure 2 In the embodiment, the image IMG1 may include an object OB1 and an object OB2. Since the materials of the object OB1 and the object OB2 may be different, they may have different angle characteristics. In some embodiments, the angle characteristics may also be understood as the light characteristics presented by the object at different viewing angles. Figure 3 is a graph of the angle characteristics of a point on object OB1. Figure 3In the above diagram, object OB1 may be a metal ball. Therefore, the angle feature of the object OB1 at this point will have an intensity peak in the small area from viewing angle A1 to viewing angle A2. That is, when the observer views object OB1 from the range of viewing angle A1 to viewing angle A2, the observer will see bright reflected light. However, if the observer views object OB1 outside the range of viewing angle A1 to viewing angle A2, the observer will not see reflected light.

[0022] However, even though different parts of object OB1 may contain the same material, different parts of object OB1 may have different angular characteristics because of their different positions relative to the light source. In addition, some materials may have different angular characteristics for different colored lights. For example, when an observer is looking at soap bubbles or butterfly wings, the observer will see different colors depending on the viewing angle.

[0023] Therefore, in some embodiments, Figure 2 The display data D1 in may include multiple color comparison maps. Each color comparison map may correspond to a specific angle and a specific color. Figure 2 In the color comparison maps RM1 to RMx, the red light intensity corresponding to the pixels of the image IMG1 at X viewing angles may be recorded, where X is an integer greater than 1.

[0024] In some embodiments, X may be, for example but not limited to, 61. In this case, the color reference maps RM1 to RMx may correspond to the red light intensity that each pixel should present when the viewing angle is -30°, -29°, ..., -1°, 0°, 1°, ..., 29° and 30°, respectively.

[0025] Similarly, the display data D1 may also include color comparison maps GM1 to GM x and BM1 to BM x . Color comparison chart GM1 to GM x The green light intensity corresponding to each pixel at X viewing angles can be recorded, and the color comparison graphs BM1 to BM x The blue light intensity corresponding to each pixel at X viewing angles can be recorded.

[0026] In this case, the display data D1 can be displayed using the red color maps RM1 to RM x , Green color comparison chart GM1 to GM x And blue color comparison chart BM1 to BM xTo record the angle characteristics of the object in the image IMG. However, in some embodiments, the color comparison map can also be established according to the definition of other color spaces. For example, according to the definition of the YUV color space, the display data D1 can also use a color comparison map to record the corresponding values ​​of luminance and chrominance at different viewing angles.

[0027] Furthermore, in some embodiments, in order to reduce the amount of display data, the display data may also record the angle characteristics through a fitting parameter used to describe the angle characteristics. For example, Figure 2 The angular characteristics of each color can also be used Figure 3 The peak intensity I top , peak intensity I top The corresponding peak viewing angle A0, basic intensity I base And the dispersion coefficient DP are used as fitting parameters to describe the angle characteristics.

[0028] Figure 4 FIG. 1 is a schematic diagram of display data D2 of an embodiment. In some embodiments, the display data D2 may be used to store angle features of objects in the image IMG1. Figure 4 In the example, the display data D2 may include multiple parameter comparison graphs PM1 to PM Y , where Y is an integer greater than 1. Each parameter comparison chart PM1 to PM Y A specific fitting parameter that can correspond to the angular feature. For example, the parameter map PM1 can record the peak viewing angle, the parameter map PM2 can record the intensity peak, and the parameter map PM3 can record the basic intensity. However, since objects of different materials may have different angular features, different fitting parameters may be used to describe their angular features. Therefore, in some embodiments, the image input unit 110 can also receive a header file, and the header file can record the fitting parameters used for the angular feature, so that the processor 120 can analyze and edit the display data D2 according to the content of the header file.

[0029] In some embodiments, a discrete cosine transform (DCT) may generate a series of coefficients for describing the angle feature, so that different angle features may also be represented using a common format. Figure 5 yes Figure 3 The discrete cosine transform of the angle characteristics and the schematic diagram of the display data D3. Figure 5In the embodiment, the angle feature can be converted to the frequency dimension by discrete cosine transform. Since the curve of the angle feature is usually very simple and has fewer turning points, the number of non-zero items or items with larger coefficient values ​​in the discrete cosine transform coefficients will also be less. In some embodiments, the display data D3 may include discrete cosine transform coefficient comparison maps CM1 to CM2 corresponding to the angle feature. Z , where Z is an integer greater than 1. For example, each discrete cosine transform coefficient comparison map CM1 to CM Z It can correspond to the coefficient of a specific frequency of the angle feature in the frequency dimension. In this way, the angle feature can be recorded in the form of frequency, and the amount of data required for recording the angle feature can be reduced.

[0030] After the processor 120 receives the display data D1 (or one of D2 and D3) from the image input unit 110, the processor 120 may modify the angular characteristics of the object according to the editing instruction to generate the edited display data ED1. In some embodiments, the image input unit 110 may provide a user interface for the user to control the processor 120 to execute the required editing instructions. For example, the image editing system 100 may display the image to be edited through the user interface, and may display the tools or instruction list for editing the image. However, unlike other image editing software today, the editing instructions provided by the image editing system 100 allow the user to edit the angular characteristics of each object in the image. The editing instructions may include, for example but not limited to, generation instructions, transformation instructions, selection instructions, copy and paste instructions, and / or synthesis instructions.

[0031] For example, when the processor 120 is required to execute the generation instruction, the processor 120 can generate new angle features for the specified object accordingly. For example, the image IMG1 may include objects with fixed angle features. That is to say, when observing these objects, even if the observer changes the viewing angle, the light intensity and color presented by these objects will not change, so the visual effects received by the observer at different viewing angles are the same. In this case, in order to allow these objects to present a realistic gloss effect, the generation instruction can be used to generate new angle features for these objects. Alternatively, the user of the image editing system 100 may also want to change the material of the object to enhance the visual effect of the image IMG1. In all of the above cases, the generation instruction may be executed. In some embodiments, the processor 120 may include a database to record the angle features of different materials, such as the angle features of metal, wood and glass, so that the user can select the angle features of the required material and apply the selected angle features to the specified object. In some embodiments, the processor 120 may also need to analyze the depth information of the surface of the specified object and the position of the light source before it can properly adjust the selected angle feature and apply it to the specified object, so that the angle feature of the specified object can be compatible with the environment in which the specified object is located and other objects in the image IMG1. Furthermore, in some embodiments, the processor 120 may allow the user to create a new angle feature instead of directly applying the ready-made angle feature in the database.

[0032] In addition, when the processor 120 is required to execute a transformation instruction, the processor 120 may, based on the content of the instruction, for example but not limited to, make the angular feature of the object more concentrated, make the angular feature of the object wider, translate the angle of the angular feature of the object in the angular dimension, translate the intensity of the angular feature of the object in the intensity dimension, reverse the angle of the angular feature in the angular dimension, reverse the intensity of the angular feature in the intensity dimension, scale the angular feature, copy the angular feature, remove noise from the angular feature, or add noise to the angular feature.

[0033] For example, the processor 120 may increase the intensity peak of the angle feature and narrow the visible angle range where the reflected light can be seen, so as to make the angle feature more concentrated. In this case, the edited display data ED1 may present a concentrated specular reflection effect on the specified object. Conversely, the processor 120 may also reduce the intensity peak and widen the visible angle range where the reflected light can be seen, so as to make the angle feature wider. In this case, the edited display data ED1 may present a wider specular reflection effect on the specified object.

[0034] In another embodiment, the processor 120 may translate the visible angle corresponding to the intensity peak to achieve the effect of the translated angle feature. Figure 6is a schematic diagram of the original angle characteristic curve and the transformed angle characteristic curve. Figure 6 In the figure, curve C0 is the original angle characteristic curve, and curve C1 is the angle characteristic curve after translation transformation in the angle dimension. The intensity peak of the original angle characteristic curve C0 mainly appears between viewing angle A1 and viewing angle A2, but after the translation transformation, the intensity peak of the transformed angle characteristic curve C1 mainly appears between viewing angle A3 and viewing angle A4. In some embodiments, translation transformation in the angle dimension can create a visual effect of rotating an object, but in some embodiments, the processor 120 can also translate the intensity peak and / or basic intensity of the angle characteristic in the intensity dimension to perform the translation transformation and achieve the effect of strengthening or weakening the mirror reflection.

[0035] In addition, the processor 120 may invert the angle feature in the angle dimension and / or the intensity dimension according to the requirements of the transformation instruction. Figure 6 In the figure, curve C2 is the angle characteristic curve after the angle characteristic curve C0 is reversed in the angle dimension, and curve C3 is the angle characteristic curve after the angle characteristic curve C0 is reversed in the intensity dimension.

[0036] Furthermore, the processor 120 may replicate or scale the angular features in the angular dimension. Figure 6 In the figure, curve C4 is the angle characteristic curve after the angle characteristic curve C0 is copied and scaled in the angle dimension. In this case, when the observer's viewing angle falls within the range of viewing angle A5 to viewing angle A6 and viewing angle A7 to viewing angle A8, the mirror reflection phenomenon of the object can be observed.

[0037] In addition, the processor 120 may add noise to the angle feature so that the surface of the object appears to be rougher. Conversely, the processor 120 may also remove the noise from the angle feature so that the surface of the object appears to be smoother.

[0038] In some embodiments, the processor 120 may modify the angular characteristics of different colors according to the transformation instructions. For example, the processor 120 may shift the hue or color purity in the angular dimension so that the observer can see different colors at different viewing angles, creating different realistic gloss effects.

[0039] By providing different types of transformation instructions, the image editing system 100 can enable users to create different visual effects, thereby enriching the observer's visual experience and improving the quality of the image.

[0040] In some embodiments, in order to accurately execute the transformation instruction on the selected object, the processor 120 also needs to be able to define the boundary range of the selected object. For example, the user can click a point (i.e., the click point) on the object to be selected through the user interface of the image editing system 100. In this case, the processor 120 can first analyze the angle characteristics of the selected position in the object, that is, the click point, to determine the material type of the object. After the processor 120 determines the material type of the object, the processor 120 can combine the area adjacent to the selected position and whose angle characteristics have the same material type as the angle characteristics of the object to define the range of the object. For example, if the click point is determined to have an angle characteristic similar to that of metal, the processor 120 will analyze the angle characteristics of the area adjacent to the click point, and link the areas with angle characteristics similar to metal to define the complete range of the selected object. In other words, by analyzing the angle characteristics of adjacent parts, the boundary of the object can be defined.

[0041] After selecting an object, the processor 120 may execute a transformation command accordingly. In addition, in some embodiments, after an object is selected, the processor 120 may execute a copy and paste command to copy the angle feature of a selected object and apply it to another selected object. For example, the user may select Figure 2 Select object OB1 in the image and copy the angle feature of object OB1. Then select object OB2 and apply the angle feature of object OB1 to object OB2. In this case, object OB2 will have the same angle feature of metal material as object OB1.

[0042] In addition, the processor 120 may synthesize at least one angle feature into the selected object in the image according to the synthesis instruction. Figure 7 Schematic diagram of the angle characteristic curves of object OB1 and object OB2 and the synthesized angle characteristic curve. Figure 7 In the figure, curve CA is the angle characteristic curve of object OB1, curve CB is the angle characteristic curve of object OB2, and curve CC is the composite angle characteristic curve of object OB1 and object OB2. By composite the angle characteristics of object OB1 and object OB2, the selected area can present two different textures at the same time, simulating the visual effect of a semi-transparent object.

[0043] Through editing instructions, such as generating instructions, transforming instructions, selecting instructions, copying and pasting instructions, and synthesizing instructions, the user can use the image editing system 100 to edit the display data D1 of the image IMG1 to generate edited display data ED1 and present the desired visual effect.

[0044] Figure 82 is a schematic diagram of an image editing system 200 according to an embodiment. The image editing system 200 and the image editing system 100 may operate according to similar principles. However, the image editing system 200 may further include an image capture device 240 and a display device 250 .

[0045] The image capture device 240 can capture images to generate display data D1. In some embodiments, the image capture device 240 can include a multi-lens image sensor 242 to capture the angle characteristics of the object. However, in some embodiments, if there is software that can establish the angle characteristics based on the image of a single perspective, the image capture device 240 can also include a single-lens image sensor instead of a multi-lens image sensor.

[0046] In addition, the image capture device 240 may also include a depth sensor 244 and a fisheye sensor 246. The depth sensor 244 can be used to obtain the depth information of the object, so that the angle characteristics generated by the image capture device 240 can be more accurate. The fisheye sensor 246 can be used to establish a light source position map of the scene. For example, the fisheye sensor 246 can record the position of the light source in the scene, so that the angle characteristics are closer to the actual situation. However, the depth sensor 244 and the fisheye sensor 246 can also be omitted according to the needs of the system. In other words, the image capture device 240 can selectively be equipped with the depth sensor 244 and the fisheye sensor 246. In some other embodiments, the image capture device 240 may also include other suitable sensors, and the present disclosure is not limited to these sensors. Figure 8 The image capture device 240 is shown as a limitation.

[0047] Furthermore, the display device 250 may include a display panel 252 and an optical modulator 254. The display panel 252 may be, for example but not limited to, a liquid crystal display panel or a light emitting diode display panel, and may simultaneously display multiple images corresponding to multiple viewing angles to present the edited display data ED1, and the optical modulator 254 may guide the multiple images displayed by the display panel 252 to different directions. The optical modulator 254 may be a lenticular lens, a liquid crystal gradient-index (GRIN) lens, a parallax barrier, or a liquid crystal barrier. In this way, when an observer views the display device 250 from different viewing angles, the observer can see images presented under different light configurations. In this case, the user can directly see the effect of the angle feature edited by the image editing system 200, so that the user can make further adjustments to the image IMG1 according to the display result.

[0048] Fig. 93 is a flowchart of a method 300 for editing an image IMG1 according to an embodiment. The method 300 may be executed by using the image editing system 100 or 200. The method 300 may include steps S310 to S330.

[0049] S310: Reading display data of the image IMG1 through the image input unit 110;

[0050] S320: Modifying the angle feature of the object according to at least one editing instruction by the processor 120 to generate edited display data ED1; and

[0051] S330 : Outputting the edited display data ED1 via the image output unit 130 .

[0052] In some embodiments, the image input unit 110 may read the display data D1 in step S310. The display data D1 may include color comparison maps RM1 to RM x GM1 to GM x and BM1 to BM x To record the angle characteristics of each object in the image IMG1. However, in some other embodiments, the image input unit 110 can also read the aforementioned display data D2 or display data D3 according to system requirements. The display data D2 can include parameter comparison maps PM1 to PM2. Y , and each parameter comparison chart PM1 to PM Y The display data D3 may include discrete cosine transform coefficient comparison maps CM1 to CM Z , and each discrete cosine transform coefficient comparison chart CM1 to CM Z It can correspond to the coefficient of a specific frequency of the angle feature in the frequency dimension.

[0053] In step S320 , the processor 120 may change the angle feature of the object according to different editing instructions to generate edited display data ED1 . The editing instructions may include generation instructions, transformation instructions, selection instructions, copy and paste instructions, and synthesis instructions.

[0054] According to the generation instruction, the processor 120 can generate a new angle feature for the selected object. According to different transformation instructions, the processor 120 can, for example but not limited to, make the angle feature of the object more concentrated, make the angle feature of the object wider, translate the angle of the angle feature of the object in the angle dimension, translate the intensity of the angle feature of the object in the intensity dimension, reverse the angle of the angle feature in the angle dimension, reverse the intensity of the angle feature in the intensity dimension, scale the angle feature, copy the angle feature, remove noise from the angle feature, or add noise to the angle feature.

[0055] In addition, according to the selection instruction, the processor 120 can analyze the angle characteristics of the selected position in the object to determine the material type of the object, and combine the area adjacent to the selected position and whose angle characteristics have the same material type as the angle characteristics of the object to define the range of the object.

[0056] After the object is selected, the processor 120 can process subsequent instructions. For example, the processor 120 can execute a copy and paste instruction. In this case, the processor 120 can copy the angle feature of the selected object and apply the angle feature of the selected object to another selected object.

[0057] In some embodiments, Figure 7 As shown, the processor 120 can synthesize at least one angle feature to a selected position in the image according to the synthesis instruction to create a semi-transparent visual effect.

[0058] In summary, the image editing system and image editing method provided by the embodiments of the present disclosure allow users to edit the angle characteristics of objects in an image, thereby improving the image quality. By providing various types of transformation instructions, the image editing system also helps users create unique visual effects and improve the effect of the display presenting realistic gloss.

[0059] The above description is only an embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for editing an image, characterized in that: The method comprises: Capturing the image by an image capturing device to generate display data; Reading the display data of the image through an image input unit, wherein the display data includes a plurality of angular features of an object in the image and a plurality of discrete cosine transform coefficient comparison maps, the plurality of angular features including color information of the object corresponding to a plurality of viewing angles, and the plurality of discrete cosine transform coefficient comparison maps corresponding to the plurality of angular features; modifying the plurality of angle features of the object according to at least one editing instruction by a processor to generate edited display data; and The edited display data is outputted through an image output unit.

2. The method according to claim 1, characterized in that The display data includes a plurality of parameter comparison graphs, and each of the plurality of parameter comparison graphs is a fitting parameter corresponding to the plurality of angle features.

3. The method according to claim 2, characterized in that Also includes: A header file is received through the image input unit, wherein the header file records the fitting parameters for describing the plurality of angle features.

4. The method according to claim 1, characterized in that The step of modifying the plurality of angle features of the object according to the at least one editing instruction by the processor to generate the edited display data comprises: The processor generates a new angle feature for the object according to the generation instruction.

5. The method according to claim 1, characterized in that The step of modifying the plurality of angle features of the object according to the at least one editing instruction by the processor to generate the edited display data comprises: The processor may, according to transformation instructions, make the angular feature of the object more concentrated, make the angular feature wider, translate the angle of the angular feature, translate the intensity of the angular feature, reverse the angle of the angular feature, reverse the intensity of the angular feature, scale the angular feature, copy the angular feature, remove noise from the angular feature, or add noise to the angular feature.

6. The method according to claim 1, characterized in that The step of modifying the plurality of angle features of the object according to the at least one editing instruction by the processor to generate the edited display data comprises: The processor selects an instruction according to: Analyzing the angle characteristics of the selected position in the object to determine the material type of the object; and Regions adjacent to the selected position and having the same angular characteristics as the material type of the object are combined to define the range of the object.

7. The method according to claim 1, characterized in that The step of modifying the plurality of angle features of the object according to the at least one editing instruction by the processor to generate the edited display data comprises: Copy and paste the instructions through the processor according to: Copy the angular characteristics of the selected object; and Apply the angle feature of the selected object to another selected object.

8. The method according to claim 1, characterized in that The step of modifying the plurality of angle features of the object according to the at least one editing instruction by the processor to generate the edited display data comprises: The processor synthesizes at least one angle feature into a selected area in the image according to a synthesis instruction.

9. The method according to claim 1, characterized in that Also includes: The edited display data is presented by simultaneously displaying a plurality of images corresponding to a plurality of viewing angles through a display panel.

10. The method according to claim 9, characterized in that Also includes: The multiple images displayed by the display panel are guided to different directions through an optical modulator.

Images