Visualization of Gloss

Abstract

Methods, computer systems, and computer program products for gloss visualization are provided. The invention may include recognizing an object in a digital image loaded on a user device. The invention may include assigning a gloss index to the recognized object. The invention may include determining a light direction relative to the recognized object based on a plurality of pixel values ​​corresponding to the recognized object. The invention may include tracking an eye position of a user viewing the digital image on the user device. The invention may include applying at least one filter in real time to the recognized object to simulate a gloss effect of the recognized object in the digital image in response to detecting a movement of the user's eye position.

Description

Technical Field

[0001] The present invention generally relates to the field of computing, and more particularly to digital image processing.

Background Art

[0002] Color may have various levels of gloss, including a matte finish. Such effects of color can directly affect the way color appears to the human eye. For example, a color with a matte finish may appear flat. However, the appearance of the same color with a glossy finish reflects more light and can produce more color variations. This difference can be easily confirmed in video or 3D moving images because a glossy color can change depending on the position of the camera or the eye.

Summary of the Invention

[0003] Embodiments of the present invention disclose a method, a computer system, and a computer program product for visualizing gloss. The present invention may include recognizing an object in a digital image loaded on a user device. The present invention may include assigning a shine index to the recognized object. The present invention may include determining a direction of light with respect to the recognized object based on a plurality of pixel values corresponding to the recognized object. The present invention may include tracking the position of the user's eye while the user is viewing the digital image on the user device. The present invention may include applying at least one filter to the recognized object in real time to simulate a gloss effect of the recognized object in the digital image in response to detecting movement of the position of the user's eye.

[0004] According to one aspect of the present invention, recognizing at least one object in a digital image loaded on a user device, assigning a gloss index to the recognized at least one object, determining a direction of light with respect to the recognized at least one object based on a plurality of pixel values corresponding to the recognized at least one object, tracking a position of a user's eye while viewing the digital image on the user device, and in response to detecting movement of the position of the user's eye, applying at least one filter to the recognized at least one object in the digital image in real time to simulate a gloss effect of the recognized at least one object, a computer-implemented method is provided.

[0005] According to another aspect of the present invention, a computer system for visualization of gloss, comprising one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions, the program instructions being stored in at least one of the one or more computer-readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more memories, the computer system being capable of executing a method, the method comprising recognizing at least one object in a digital image loaded on a user device, assigning a gloss index to the recognized at least one object, determining a direction of light with respect to the recognized at least one object based on a plurality of pixel values corresponding to the recognized at least one object, tracking a position of a user's eye while viewing the digital image on the user device, and in response to detecting movement of the position of the user's eye, applying at least one filter to the recognized at least one object in the digital image in real time to simulate a gloss effect of the recognized at least one object, a computer system is provided.

[0006] According to another aspect of the present invention, there is provided a computer program product for visualizing gloss, comprising one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media, the program instructions being executable by a processor to cause the processor to execute a method, the method comprising: recognizing at least one object in a digital image loaded into a user device; assigning a gloss index to the recognized at least one object; determining a direction of light with respect to the recognized at least one object based on a plurality of pixel values corresponding to the recognized at least one object; tracking the position of a user's eyes while viewing the digital image on the user device; and in response to detecting movement of the position of the user's eyes, applying at least one filter to the recognized at least one object in the digital image in real time to simulate a gloss effect of the recognized at least one object.

[0007] Here, preferred embodiments of the present invention will be described by way of example only with reference to the following drawings.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

[0009] The figures are provided to clarify the invention for those skilled in the art to understand the invention in conjunction with the detailed description, so the various features of the drawings are not to scale.

[0010] Although detailed embodiments of the claimed structures and methods are disclosed herein, it will be understood that the disclosed embodiments are merely illustrative of the claimed structures and methods that may be embodied in various forms. However, the present invention may be embodied in many different forms and should not be construed as limited to the exemplary embodiments described herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In this description, well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0011] The present invention can be a system, a method, or a computer program product, or a combination thereof, in the integration at any possible technical detail level. The computer program product can include a computer-readable storage medium (or media) having thereon computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0012] The computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of the computer-readable storage medium includes a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick (registered trademark), a floppy (registered trademark) disk, a mechanically encoded device such as a punched card or raised structures in a groove having instructions recorded thereon, and any suitable combination thereof. The computer-readable storage medium should not be construed to be a transitory signal itself, such as, for example, a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted through a wire.

[0013] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices or to an external computer or external storage device via a network, such as, for example, the Internet, a local area network, a wide area network, or a wireless network, or combinations thereof. The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or combinations thereof. The network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0014] The computer-readable program instructions for carrying out the operations of the present invention may be source code or object code written in any combination of one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or object-oriented programming languages such as Smalltalk®, Python, C++, and procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the last scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may be customized for an individual by utilizing the state information of the computer-readable program instructions to execute the computer-readable program instructions to carry out aspects of the present invention.

[0015] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0016] These computer-readable program instructions, when executed via the processor of a computer or other programmable data processing apparatus, may provide means for causing a general purpose computer, a special purpose computer, or other programmable data processing apparatus to produce a machine that implements the functions / acts specified in one or more blocks of a flowchart, a block diagram, or both. These computer-readable program instructions may also be stored in a computer-readable storage medium that, when stored, causes a manufacture comprising instructions that implement the aspects of the functions / acts specified in one or more blocks of a flowchart, a block diagram, or both, to be configured to function in a particular manner on a computer, a programmable data processing apparatus, or other device, or a combination thereof.

[0017] Also, these computer-readable program instructions may be loaded onto a computer, other programmable apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process that, when executed, causes the computer, other programmable apparatus, or other device to implement the functions / acts specified in one or more blocks of a flowchart, a block diagram, or both.

[0018] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may be performed out of the order illustrated. For example, depending on the functionality involved, two blocks shown in succession may in fact be implemented as one step, executed simultaneously, substantially simultaneously, partially or fully temporally overlapping, or the blocks may be executed in the reverse order, as the case may be. It will also be noted that each block of the block diagrams or flowchart diagrams, or combinations of blocks in the block diagrams or flowchart diagrams or both, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or by a combination of dedicated hardware and computer instructions.

[0019] The exemplary embodiments described below provide a system, method, and program product for visualization of gloss. Therefore, the present embodiments have the ability to improve the technical field of digital image processing by visualizing shiny objects in a still digital image. More specifically, a gloss visualization program may recognize an object in a digital image and determine a relevant gloss index corresponding to the recognized object. Then, in response to the determined gloss index satisfying a threshold gloss index, the gloss visualization program may extract pixel values of a plurality of pixels associated with the recognized object in the digital image. Next, the gloss visualization program may detect a change in the extracted pixel values of the plurality of pixels associated with the recognized object, and the detected change in the extracted pixel values may be related to light reflected from the recognized object in the digital image. Then, the gloss visualization program may determine the position of a light source with respect to the recognized object in the digital image based on the detected change in the extracted color values. Next, the gloss visualization program may track the position of the eyes of a user viewing the digital image on a user device. Thereafter, in response to detecting movement of the position of the user's eyes, the gloss visualization program may apply a filter to a subset of the plurality of pixels of the recognized object to simulate a gloss effect on the recognized object in the digital image.

[0020] As described above, colors may have various levels of gloss, or may include a matte finish. Such effects of color can directly affect how the color appears to the human eye. For example, a color with a matte finish may appear flat. However, the appearance of the same color with a glossy finish reflects more light and can therefore produce more color variations. This difference can be easily confirmed in a video or 3D moving image because a shiny color can change depending on the position of the camera or the eyes. However, in a fixed digital image, due to the static nature of the medium, it is impossible to see the gloss effect of an object.

[0021] Therefore, it can be advantageous to provide a method for detecting whether a user viewing a digital image on a user device (having a camera) is moving, and, when movement is detected, simulating a gloss effect of an object within the digital image. Thus, the disclosed embodiments can improve the user experience and provide more information to a user interacting with a digital image on a user device.

[0022] In real life, a viewer can observe the brightness of an object as a result of the position of the viewer and the position of the sun or other light source. According to at least one embodiment of the present disclosure, by adding an additional level of data to a static or fixed digital image, an image reader can be enabled to display the reflection of light on various shiny objects within the image. In one embodiment, image recognition can be implemented to detect and recognize objects within the digital image. Based on the material of the objects recognized within the image, a gloss level or index can be assigned to each object. Objects including a gloss index that meets a threshold gloss level can be further processed. In one embodiment, the further processing can include detecting a change in the color of an object due to the reflection of light and determining the direction of the light with respect to the object within the digital image. According to one embodiment, when an image is being displayed on a user device by a reader, the camera of the user device can track the movement of the user (e.g., based on the user's eyes or line of sight). If the user is stable, the reader may not need to modify the image. However, if movement of the user or the device is detected, the reader can apply one or more filters to the image to simulate a gloss effect on various objects within the image. In one embodiment, when the user moves their head away from a light source, the reader can apply a filter to simulate a mirror effect by the light source by enhancing the brightness of the object. In other embodiments, when the user moves their head closer to a light source (e.g., between the light source and the object), the reader can apply a filter to reduce the brightness of the object.

[0023] Referring to FIG. 1, an exemplary networked computer environment 100 according to one embodiment is shown. The networked computer environment 100 may include a computer 102 having a processor 104 and a data storage device 106 enabled to execute a software program 108 and a luster visualization program 110a. The networked computer environment 100 may also include a server 112 enabled to execute a luster visualization program 110b that can interact with a database 114 and a communication network 116. The networked computer environment 100 may include a plurality of computers 102 and servers 112, and only one of them is shown. The communication network 116 may include various types of communication networks such as, for example, a wide area network (WAN), a local area network (LAN), a telecommunications network, a wireless network, a public switched telephone network, or a satellite network, or a combination thereof. It should be understood that FIG. 1 provides only an illustration of one implementation and is not intended to impose any limitations on the environments in which various embodiments may be implemented. Many changes may be made to the illustrated environment based on design and implementation requirements.

[0024] Client computer 102 can communicate with server computer 112 via communication network 116. Communication network 116 can include connections such as, for example, wired, wireless communication links, or fiber optic cables. As will be described with reference to FIG. 10, server computer 112 can include internal components 902a and external components 904a, respectively, and client computer 102 can include internal components 902b and external components 904b, respectively. Server computer 112 can operate in a cloud computing service model such as, for example, software as a service (SaaS), platform as a service (PaaS), or infrastructure as a service (IaaS). Server 112 can be deployed in a cloud computing deployment model such as, for example, a private cloud, a community cloud, a public cloud, or a hybrid cloud. Client computer 102 can be, for example, a mobile device, a telephone, a personal digital assistant, a netbook, a laptop computer, a tablet computer, a desktop computer, or any type of computing device capable of executing a program, accessing a network, and accessing database 114. According to various implementations of embodiments of the present invention, the gloss visualization programs 110a, 110b can interact with database 114, which can be embodied in various storage devices such as, but not limited to, computer / mobile device 102, networked server 112, or cloud storage service.

[0025] According to embodiments of the present invention, a user using client computer 102 or server computer 112 can use gloss visualization programs 110a, 110b, respectively, to animate a still digital image and simulate a gloss effect on the image object. Embodiments of the present disclosure will be described in more detail below with respect to FIGS. 2-9.

[0026] Referring now to FIG. 2, a schematic block diagram of a digital image processing environment 200 implementing gloss visualization programs 110a, 110b according to at least one embodiment is shown. According to one embodiment, the digital image processing environment 200 may include one or more components of the computer environment 100 described above with reference to FIG. 1 (e.g., client computer 102, server computer 112, communication network 116).

[0027] According to one embodiment, the digital image processing environment 200 may include a computing device capable of using a camera, which may be referred to as a user device 202. In various embodiments, the user device 202 may include a workstation, a personal computing device, a laptop computer, a desktop computer, a tablet computer, a smart phone, or other suitable electronic device. According to one embodiment, the user device 202 may include a tangible storage device (e.g., data storage device 106) and a processor enabled to execute gloss visualization programs 110a, 110b.

[0028] In one embodiment, the gloss visualization programs 110a, 110b may include a single computer program, a plurality of program modules, or a set of instructions that are executed by a processor of the user device 202. The gloss visualization programs 110a, 110b may include routines, objects, components, units, logic, data structures, and actions that may perform specific tasks or implement specific abstract data types. The gloss visualization programs 110a, 110b may be implemented in a distributed cloud computing environment where tasks may be executed by remote processing devices linked via a communication network 116. In one embodiment, the gloss visualization programs 110a, 110b may include program instructions that may be collectively stored on one or more computer-readable storage media. As shown in the illustrated embodiment, the gloss visualization programs 110a, 110b may include an image recognition module 204, a gloss assignment module 206, a color extraction module 208, a light detection module 210, a motion detection module 212, and an image rendering module 214.

[0029] According to one embodiment, the digital image processing environment 200 may include a digital image 216 (e.g., a still or stationary digital image) loaded onto the user device 202. Exemplary file types of the digital image 216 may include, for example, Joint Photographic Experts Group (JPEG), Portable Network Graphics (PNG), Raw Image Format (RAW), or any other suitable file type for storing still images. In one embodiment, the gloss visualization programs 110a, 110b may be implemented to simulate a gloss effect on any object within the digital image 216 that may have a natural tendency to shine based on the material or color or both of the object.

[0030] According to one embodiment, the gloss visualization programs 110a, 110b may implement an image recognition module 204 for detecting, classifying, or recognizing any object captured in the digital image 216. In one embodiment, the image recognition module 204 receives the digital image 216 as an input layer and outputs a label corresponding to each detected or recognized object or element in the digital image 216 based on pixel features, for example, a trained machine learning model such as a trained convolutional neural network (CNN). In various embodiments, the image recognition module 204 may also be enabled to determine the color and surface material or characteristics of each recognized object. For example, the image recognition module 204 may recognize a lake as one object in the digital image 216. In one embodiment, the image recognition module 204 may also indicate that the color of the lake is blue and the surface material of the lake is water.

[0031] According to one embodiment, the gloss visualization programs 110a, 110b may implement a gloss assignment module 206 for assigning a gloss level or gloss index to each object recognized by the image recognition module 204 in the digital image 216. In at least one embodiment, the gloss index of an object may be related to the amount of light that the object can reflect (e.g., based on the color, material, or finish of the object, or a combination thereof). In one embodiment, the gloss index may range from 0 to 1, where a gloss index of 0 may indicate that the object does not reflect light (e.g., a wall painted with a matte finish), and a gloss index of 1 may indicate that the object is very glossy (e.g., a mirror, gold, water). In other embodiments, the gloss index may range from 0 to any number (e.g., 100). In some embodiments, the range of the gloss index may be reversed so that a gloss index of 0 may indicate the highest level of gloss.

[0032] According to one embodiment, the digital image processing environment 200 may include an object gloss index repository 218 that can store gloss indices corresponding to various objects and materials that can be captured in the digital image 216. In one embodiment, the gloss assignment module 206 may communicate with the object gloss index repository 218 to retrieve the gloss index of each object recognized by the image recognition module 204.

[0033] According to one embodiment, if none of the gloss indices assigned to the objects recognized in the digital image 216 satisfy a minimum gloss index (e.g., a threshold gloss index), the gloss visualization programs 110a, 110b may display the digital image 216 as a still image. In one embodiment, if the gloss indices assigned to some of the objects recognized in the digital image 216 satisfy the minimum gloss index, the gloss visualization programs 110a, 110b may further process the gloss of those objects to animate them and display the other objects that did not satisfy the minimum gloss index as they are. In one embodiment, any gloss index greater than 0 may satisfy the minimum gloss index requirement. In one embodiment, the gloss visualization programs 110a, 110b may include a default minimum gloss index and may also allow the user to input a user-defined minimum gloss index.

[0034] According to one embodiment, the gloss visualization programs 110a, 110b may implement a color extraction module 208 and a light detection module 210 for determining the position of the light source and the direction of light captured in the digital image 216. In one embodiment, since color is directly related to the reflection of light, the direction of light can be determined by the color of the object. For example, even an object that is actually a single color or a uniform color, such as a red car, can be perceived as having a color change based on the way light hits the car. In one embodiment, the color change can be indicated by a tone value, and the tone value of the color can be changed by adding white, black, or gray to the original color. All colors may include a tone value associated with the lightness or darkness of the color. In one embodiment, the tone value of the high tone of the color may indicate a brighter change in the color, and the tone value of the low tone of the color may indicate a darker change in the color. In one embodiment, the color extraction module 208 may extract and compare pixel values corresponding to the pixels of each object to detect a change in the color of the object (e.g., a tone change). In one embodiment, the color extraction module 208 may compare pixel values using the additive color mixing model of red, green, and blue (RGB). In other embodiments, the color extraction module 208 may also be enabled to use a color model of hue, saturation, lightness (HSL) or hue, saturation, value (HSV) when comparing pixel values.

[0035] According to one embodiment, the light detection module 210 may calculate one or more luminosity levels of an object using the tonal changes of the color measured by the color extraction module 208. In one embodiment, luminosity may refer to a measurement of the amount of light hitting the surface of an object. In one embodiment, luminosity may indicate a relative value (e.g., relative luminosity) in a range from 0 to 1, a range from 0 to 100, or any other suitable range, and a higher luminosity level associated with a surface may indicate that more light is hitting that surface. In various embodiments, the light detection module 210 may calculate multiple luminosity levels for each object, and the surface of the object receiving the most light may include the highest luminosity level (e.g., 1), and the surface of the object receiving the least light may include the lowest luminosity level (e.g., 0.2). According to one embodiment, a portion of an object that includes a relatively high-tonal tonal value (e.g., a brighter change in color) determined by the color extraction module 208 may be determined by the light detection module 210 to include a relatively high luminosity level. In one embodiment, the luminosity level calculated by the light detection module 210 may indicate a luminosity gradient from light to dark (e.g., from high relative luminosity to low relative luminosity). In various embodiments, the calculated luminosity gradient may indicate the position of the light source and the direction of the light. More specifically, the light detection module 210 may determine that the position of the light source and the direction of the light are on the side of the object having the highest luminosity level.

[0036] According to one embodiment, the processes described above with reference to the image recognition module 204, the gloss assignment module 206, the color extraction module 208, and the light detection module 210 (e.g., object recognition, gloss index assignment, and light direction identification) can be executed in real time as soon as the digital image 216 is loaded into the user device 202. The metadata generated by the above processes can be stored in the main memory of the user device 202 associated with the gloss visualization programs 110a, 110b. In one embodiment, the motion detection module 212 and the image rendering module 214 can utilize the stored metadata to simulate the gloss effect on one or more objects of the digital image 216, as further described below.

[0037] According to one embodiment, the gloss visualization programs 110a, 110b may implement a motion detection module 212 for tracking the movement of a user looking at a digital image 216 on the user device 202 in interaction with the camera of the user device 202. In one embodiment, the motion detection module 212 may track either the movement of the user's head or the movement of the device, because in either case the position of the user's eyes is different with respect to the camera. In at least one embodiment, the motion detection module 212 may track the movement of the user's eyes with respect to the position of the light source determined by the light detection module 210. According to one embodiment, if the motion detection module 212 determines that the user may move away from the position of the light source, the image rendering module 214 may be implemented to simulate an increase in the mirror effect by the light source by applying a filter in real time to increase the brightness of the object. In at least one embodiment, the image rendering module 214 may apply any suitable filter (e.g., brightness filter, contrast filter, highlight filter, tone value filter) to at least a subset of the pixels associated with the object to simulate a gloss effect. Similarly, if the motion detection module 212 determines that the user may move towards the position of the light source, or between the light source and the object, or both, the image rendering module 214 may be implemented to apply one or more filters in real time to decrease the brightness of the object. According to one embodiment, the image rendering module 214 may use the gloss index assigned to the object as a multiplier with the filter. Thus, the image rendering module 214 may apply the filter more strongly to objects with a higher gloss index.

[0038] Referring now to FIG. 3, an operational flowchart is shown illustrating an exemplary gloss visualization process 300 used by the gloss visualization programs 110a, 110b, according to at least one embodiment.

[0039] At 302, objects are recognized within the digital image. According to one embodiment, the image recognition modules 204 of the gloss visualization programs 110a, 110b may implement an image recognition function, such as a trained CNN model, to detect and recognize various objects captured in the digital image loaded on the user device.

[0040] Next, at 304, a gloss index is assigned to the recognized object. According to one embodiment, the gloss assignment module 206 of the gloss visualization programs 110a, 110b may assign a gloss index to each object recognized by the image recognition module 204 within the digital image. As described above with reference to FIG. 2, the gloss assignment module 206 may be accessible to an object gloss index repository 218 that may store gloss indices corresponding to various objects and materials.

[0041] Next, at 306, in response to the assigned gloss index meeting a threshold gloss index, a plurality of pixel values of the recognized object are extracted. According to one embodiment, the gloss visualization programs 110a, 110b may implement a threshold gloss index as the minimum gloss index that an object recognized within the digital image must meet before further processing. As described above with reference to FIG. 2, the gloss visualization programs 110a, 110b may include a default minimum gloss index (e.g., a default threshold gloss index), and may also allow a user to input a user-defined minimum gloss index (e.g., a user-defined threshold gloss index). In one embodiment, the color extraction module 208 may extract and compare pixel values corresponding to the pixels of each object, as described above with reference to FIG. 2.

[0042] Next, at 308, a tonal change in the extracted plurality of pixel values of the recognized object is detected. According to one embodiment, the color extraction module 208 can detect the tonal change by comparing pixel values using a color model, as described above with reference to FIG. 2. The tonal change in the color of the object can be the result of the amount of light reflected from the object.

[0043] Next, at 310, the position of the light source with respect to the object recognized based on the extracted plurality of pixel values is determined. According to one embodiment, the light detection modules 210 of the gloss visualization programs 110a, 110b can calculate one or more photometric levels of the object using the tonal change in color measured by the color extraction module 208. According to one embodiment, a portion of the object determined by the color extraction module 208 that includes a relatively high-tone tonal value (e.g., a brighter change in color) can be determined by the light detection module 210 to include a relatively high photometric level. In one embodiment, the photometric level calculated by the light detection module 210 can indicate a photometric gradient from bright to dark (e.g., from a high relative photometric level to a low relative photometric level). In various embodiments, the photometric gradient can indicate the position of the light source and the direction of the light. More specifically, the light detection module 210 can determine that the position of the light source and the direction of the light are on the side of the object having the highest photometric level.

[0044] Next, at 312, when the user is viewing a digital image on the user device, the user's eyes are tracked. According to one embodiment, the motion detection modules 212 of the gloss visualization programs 110a, 110b can communicate with the camera of the user device to track the motion of the user viewing the digital image on the user device. In one embodiment, the motion detection module 212 can track either the movement of the user's head or the movement of the device, because in either case the position of the user's eyes relative to the camera is different. In at least one embodiment, the motion detection module 212 can also, at 312, track the movement of the user's eyes relative to the position of the light source determined by the light detection module 210.

[0045] Thereafter, at 314, in response to detecting movement of the user's eye position, one or more filters are applied to the recognized object to simulate a gloss effect of the recognized object in the digital image. In one embodiment, the image rendering module 214 may apply various filters in real time to increase or decrease the gloss effect of the object based on the movement of the user's eyes. According to one embodiment, when the motion detection module 212 determines that the user may move away from the position of the light source, the image rendering module 214 may be implemented to simulate an increase in the mirror effect by the light source by applying one or more filters to increase the brightness of the object. In at least one embodiment, the image rendering module 214 may apply any suitable filter (e.g., brightness filter, contrast filter, highlight filter, tone value filter) to at least a subset of the pixels associated with the object to simulate the gloss effect. Similarly, when the motion detection module 212 determines that the user may move towards the position of the light source, or between the light source and the object, or both, the image rendering module 214 may be implemented to apply one or more filters in real time to decrease the brightness of the object. According to one embodiment, the image rendering module 214 may use the gloss index assigned to the object as a multiplier with the filter. Thus, the image rendering module 214 may apply the filter more strongly to objects with a higher gloss index.

[0046] Referring now to FIGS. 4-9, there is shown a block diagram illustrating an example of the gloss visualization process 300 of FIG. 3 used by the gloss visualization programs 110a, 110b according to at least one embodiment.

[0047] According to one embodiment, FIG. 4 shows event 400. In event 400, a digital image 216 can be loaded into the user device, and the gloss visualization programs 110a, 110b can execute an image recognition module 204 to discover various elements or objects within the digital image 216. In this example, the image recognition module 204 can recognize and label the following three objects within the digital image 216, namely, the first object 402 (hand; color "light skin color"), the second object 404 (color of the ring "gold"), and the third object 406 (background color "light gray"). As shown in FIG. 4, the image recognition module 204 can also recognize the color or material or both of the recognized objects.

[0048] Continuing with event 400, when various objects within the digital image 216 are detected and recognized, the gloss assignment module 206 can determine a gloss index corresponding to each object. The gloss index of an object can depend on the ability of the object to reflect light. As previously described with reference to FIG. 2, the gloss visualization programs 110a, 110b can access an object gloss index repository 218 to look up the gloss index of the corresponding object or material. Based on the object gloss index repository 218, the gloss assignment module 206 can determine that the gloss index of the first object 402 (e.g., hand; skin color) is 0.2, the gloss index of the second object 404 (e.g., ring; gold) is 0.8, and the gloss index of the third object 406 (e.g., background; gray) is 0. In this example, the threshold gloss index can be set to 0.1, which can be satisfied by the first and second objects 402, 404. Since the first object 406 (e.g., background) is associated with gloss = 0 (e.g., this object does not shine at all), the gloss visualization programs 110a, 110b may not need to process the third object 406 to simulate the gloss effect. Further processing can continue with the first and second objects 402, 404.

[0049] According to one embodiment, FIG. 5 shows event 500. In event 500, color extraction module 208 may measure and compare pixel values corresponding to the second object 404 to detect a tonal change in the color of the object. In one embodiment, color extraction module 208 may detect any number of tonal changes 502 for each object.

[0050] According to one embodiment, FIG. 6 shows event 600. In event 600, light detection module 210 may use the tonal change determined by color extraction module 208 in event 500 to calculate one or more light intensity levels of the second object 404. As described above, light intensity may refer to a measurement of the amount of light hitting the surface of an object. In this example, the light intensity may indicate a relative value in the range from 0 to 1 (e.g., relative light intensity). In event 600, light detection module 210 may calculate two light intensity levels for the second object 404. The surface of the object receiving the most light may include the highest light intensity level (e.g., light intensity = 1), and the surface of the object receiving the least light may include the lowest light intensity level (e.g., light intensity = 0.2). Based on these light intensity levels, light detection module 210 may determine that the position 602 of the light source and the direction of the light 604 are on the side of the object having the highest light intensity level (e.g., light intensity = 1).

[0051] According to one embodiment, FIG. 7 shows event 700. Similar to event 500, in event 700, color extraction module 208 may measure and compare pixel values corresponding to the first object 402 to detect a tonal change in the color of the object.

[0052] According to one embodiment, FIG. 8 shows event 800. Similar to event 600, in event 800, the light detection module 210 may calculate one or more photometric levels of the first object 402 using the tone change determined by the color extraction module 208 in event 700. In event 800, the light detection module 210 may calculate three photometric levels for the first object 402. The surface of the object that receives the most light may include the highest photometric level (e.g., photometric value = 1), the surface of the object that receives the least light may include the lowest photometric level (e.g., photometric value = 0.2), and the surface of the object that receives an intermediate amount of light may include an intermediate photometric level (e.g., photometric value = 0.5). Based on these photometric levels, the light detection module 210 may maintain the determination (e.g., from event 600) that the light source is at position 602 and the direction of light 604 is on the side of the object having the highest photometric level (e.g., photometric value = 1).

[0053] According to one embodiment, FIG. 9 shows event 850. In event 850, the gloss visualization programs 110a, 110b may execute the motion detection module 212 to track the motion of the user 804 who is viewing the digital image 216 on the user device display 806 by interacting with the camera 802 of the user device.

[0054] For example, at time 1 (T1)-808, if the user 804 is stable (e.g., not moving), the image rendering module 214 of the gloss visualization programs 110a, 110b may render the default (e.g., original) version of the digital image 216 on the user device display 806.

[0055] For example, at time 2 (T2)-810, if the user 804 moves their head to the left with respect to the camera 802, the image rendering module 214 is executed to apply one or more filters in real time to increase the brightness of the first and second objects 402, 404, thereby simulating an increase in the mirror effect by the light source. As described above, the image rendering module 214 can use the specular index assigned to the first and second objects 402, 404 as a multiplier with the filter. Thus, since the second object 404 included a specular index of 0.8, the image rendering module 214 can apply the filter more strongly to the pixels of the second object 404 compared to the first object 402, which had a specular index of 0.2.

[0056] Thereafter, for example, at time 3 (T3)-812, if the user 804 moves their head to the right with respect to the camera 802, the image rendering module 214 is executed to apply one or more filters in real time to decrease the brightness of the first and second objects 402, 404.

[0057] The specular visualization programs 110a, 110b can improve the computer's functionality because they enable the computer to detect whether a user viewing a digital image on the computer (having a camera) is moving, and to simulate the specular effect of objects in the digital image when movement is detected. Thus, the specular visualization programs 110a, 110b can improve the user experience and provide more information to the user interacting with the digital image on the computer.

[0058] It will be understood that FIGS. 2-9 provide only an illustration of one embodiment and are not intended to impose any limitation as to how various embodiments may be implemented. Many changes may be made to the illustrated embodiments based on design and implementation requirements.

[0059] FIG. 10 is a block diagram 900 of the internal and external components of the computer shown in FIG. 1, according to an exemplary embodiment of the present invention. It should be understood that FIG. 10 provides only an illustration of one implementation and is not intended to impose any limitation with respect to the environments in which various embodiments may be implemented. Many modifications to the illustrated environments may be made based on design and implementation requirements.

[0060] Data processing systems 902, 904 represent any electronic device capable of executing machine-readable program instructions. The data processing systems 902, 904 may represent a smart phone, a computer system, a PDA, or other electronic device. Examples of computing systems, environments, or configurations, or combinations thereof, that may be represented by the data processing systems 902, 904 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0061] User client computer 102 and network server 112 may each include a respective set of internal components 902a, b and external components 904a, b shown in FIG. 10. Each set of internal components 902a, b includes one or more processors 906, one or more computer-readable RAMs 908, and one or more computer-readable ROMs 910 on one or more buses 912, one or more operating systems 914, and one or more computer-readable tangible storage devices 916. One or more operating systems 914, software programs 108, and specular visualization programs 110a in client computer 102, as well as specular visualization programs 110b in network server 112, may be stored on one or more computer-readable tangible storage devices 916 for execution by one or more processors 906 via one or more RAMs 908 (typically including cache memory). In the embodiment shown in FIG. 10, each of the computer-readable tangible storage devices 916 is a magnetic disk storage device of an internal hard drive. Alternatively, each of the computer-readable tangible storage devices 916 is a semiconductor storage device, such as ROM 910, EPROM, flash memory, or any other computer-readable tangible storage device capable of storing computer programs and digital information.

[0062] Each set of internal components 902a, b also includes an R / W drive or interface 918 for reading from and writing to one or more portable computer-readable tangible storage devices 920, such as, for example, CD-ROMs, DVDs, memory sticks, magnetic tapes, magnetic disks, optical disks, or semiconductor storage devices. Software programs 108 and software programs such as gloss visualization programs 110a and 110b can be stored on one or more of the respective portable computer-readable tangible storage devices 920, read through the respective R / W drives or interfaces 918, and loaded onto the respective hard drives 916.

[0063] Each set of internal components 902a, b may also include a network adapter (or switch port card) or interface 922, such as, for example, a TCP / IP adapter card, a wireless wi-fi interface card, or a 3G or 4G wireless interface card, or other wired or wireless communication links. The software program 108 and the gloss visualization program 110a within the client computer 102, as well as the gloss visualization program 110b within the network server computer 112, can be downloaded from an external computer (such as a server) via a network (such as the Internet, a local area network, or other wide area network) and the respective network adapters or interfaces 922. From the network adapter (or switch port adapter) or interface 922, the software program 108 and the gloss visualization program 110a within the client computer 102, as well as the gloss visualization program 110b within the network server computer 112, are loaded onto the respective hard drives 916. The network may include copper wires, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or combinations thereof.

[0064] Each set of external components 904a, b can include a computer display monitor 924, a keyboard 926, and a computer mouse 928. The external components 904a, b can also include a touch screen, a virtual keyboard, a touch pad, a pointing device, and other human interface devices. Each set of internal components 902a, b also includes a device driver 930 for interfacing with a computer display monitor 924, a keyboard 926, and a computer mouse 928. The device driver 930, the R / W drive or interface 918, and the network adapter or interface 922 include hardware and software (stored in storage device 916 or ROM 910 or both).

[0065] Although this disclosure includes a detailed description regarding cloud computing, it should be understood in advance that the implementation forms of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention can be implemented with any other type of computing environment known currently or developed in the future.

[0066] Cloud computing is a service delivery model that enables convenient on-demand network access to a shared pool of configurable computing resources (such as networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0067] The characteristics are as follows. On-demand self-service: Cloud consumers can unilaterally provision computing capabilities, such as server time and network storage, automatically as needed, without the need for human interaction with the service provider. Broad network access: The capabilities are available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). Resource pooling: The provider's computing resources are pooled and multiple consumers are served using a multi-tenant model in which various physical and virtual resources are dynamically assigned and reassigned according to demand. In general, the consumer has no control over, and no knowledge of, the exact location of the provided resources, but there is a sense of location independence in that the location can be specified at a higher level of abstraction (e.g., country, state, or data center). Rapid elasticity: Capabilities can be provisioned rapidly and elastically, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time. Measured service: Cloud systems leverage a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts) to automatically control and optimize resource use. Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service.

[0068] The service model is as follows. Software as a Service (SaaS): The ability provided to the consumer is to use the provider's application operating on cloud infrastructure. The application is accessible from various client devices via a thin-client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings. Platform as a Service (PaaS): The ability provided to the consumer is to deploy consumer-created or acquired applications onto cloud infrastructure created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but controls the deployed applications and, in some cases, the application hosting environment configuration. Infrastructure as a Service (IaaS): The ability provided to the consumer is to provision processing, storage, networks, and other basic computing resources that the consumer can deploy and run any software that can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but controls the operating systems, storage, deployed applications, and, in some cases, limited control over selected networking components (e.g., host firewalls).

[0069] The deployment model is as follows. Private Cloud: The cloud infrastructure is operated exclusively for an organization. It can be managed by the organization or a third party and can exist on - premise or off - premise. Community Cloud: The cloud infrastructure is shared by several organizations and supports a specific community with common concerns (e.g., mission, security requirements, policies, and compliance considerations, etc.). It can be managed by the organization or a third party and can exist on - premise or off - premise. Public Cloud: The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization that sells cloud services. Hybrid Cloud: The cloud infrastructure remains a distinct entity but is composed of two or more clouds (private, community, or public) combined by standardized or proprietary technologies (e.g., cloud bursting for load distribution between clouds) that enable data and application portability.

[0070] Cloud computing environments are service - oriented, emphasizing statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0071] Referring now to FIG. 11, an exemplary cloud computing environment 1000 is shown. As illustrated, cloud computing environment 1000 includes one or more cloud computing nodes 100 that are used to communicate with local computing devices used by cloud consumers such as, for example, a personal digital assistant (PDA) or cellular telephone 1000A, a desktop computer 1000B, a laptop computer 1000C, or an automotive computer system 1000N, or a combination thereof. Nodes 100 can communicate with one another. These can be physically or virtually grouped (not shown) in one or more networks such as, for example, the private, community, public, or hybrid clouds described above, or a combination thereof. This allows cloud computing environment 1000 to provide infrastructure as a service, platform as a service, or software as a service, or a combination thereof, such that a cloud consumer does not need to maintain resources on a local computing device. It is intended that the types of computing devices 1000A - N shown in FIG. 11 are exemplary only, and that computing nodes 100 and cloud computing environment 1000 can communicate with any type of computerized device via any type of network or network addressable connection (such as using a web browser), or both.

[0072] Referring now to FIG. 12, a set of functional abstraction layers provided by cloud computing environment 1000 is shown. It is intended that the components, layers, and functions shown in FIG. 12 are exemplary only and that embodiments of the invention are not limited thereto. As illustrated, the following layers and corresponding functions are provided.

[0073] The hardware and software layer 1102 includes hardware components and software components. Examples of hardware components include mainframe 1104, RISC (Reduced Instruction Set Computer) architecture-based server 1106, server 1108, blade server 1110, storage device 1112, and network and networking components 1114. In some embodiments, the software components include network application server software 1116 and database software 1118.

[0074] The virtualization layer 1120 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 1122, virtual storage 1124, virtual network 1126 including a virtual private network, virtual applications and operating systems 1128, and virtual client 1130.

[0075] In one example, the management layer 1132 may provide the following functions. Resource provisioning 1134 provides for the dynamic procurement of computing resources and other resources utilized to execute tasks within a cloud computing environment. Metering and pricing 1136 provides for expense tracking when resources are utilized within a cloud computing environment and accounting or billing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides not only identity verification of cloud consumers and tasks, but also protection for data and other resources. The user portal 1138 provides access to the cloud computing environment for consumers and system administrators. Service level management 1140 provides for the allocation and management of cloud computing resources such that the requested service level is met. Planning and fulfillment of service level agreements (SLAs) 1142 provides for the advance arrangement and procurement of cloud computing resources expected to be required in the future in accordance with the SLA.

[0076] The workload layer 1144 provides examples of functionality that a cloud computing environment may utilize. Examples of workloads and functions that may be provided from this layer include mapping and navigation 1146, software development and lifecycle management 1148, virtual classroom education delivery 1150, data analysis processing 1152, transaction processing 1154, as well as gloss visualization 1156. The gloss visualization programs 110a, 110b provide a method for animating a still digital image to simulate a gloss effect on an image object.

[0077] The descriptions of various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terms used herein are chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

A method for execution by computer information processing, comprising: recognizing at least one object in a digital image loaded on a user device; assigning a gloss index to the recognized at least one object; determining a direction of light with respect to the recognized at least one object based on a plurality of pixel values corresponding to the recognized at least one object; tracking a position of a user's eye while the user is viewing the digital image on the user device; applying at least one filter to the recognized at least one object in real time to simulate a gloss effect of the recognized at least one object in the digital image in response to detecting movement of the position of the user's eye; A method comprising the steps of: Claim 2 further processing the recognized at least one object to simulate the gloss effect of the recognized at least one object in the digital image in response to the assigned gloss index associated with the recognized at least one object satisfying a threshold gloss index; The method according to claim 1, further comprising: Claim 3 displaying an original version of the recognized at least one object in the digital image in response to the assigned gloss index associated with the recognized at least one object being below a threshold gloss index; The method according to claim 1 or 2, further comprising: Claim 4 comparing the plurality of pixel values corresponding to the recognized at least one object; detecting at least one tone change in the compared plurality of pixel values, wherein the detected at least one tone change is related to light reflected from the recognized at least one object; The method according to any one of claims 1 to 3, further comprising: Claim 5 calculating at least one light intensity level associated with the recognized at least one object based on the detected at least one tone change in the compared plurality of pixel values; The method according to claim 4, further comprising: Claim 6 Calculating a photometric gradient associated with the at least one recognized object based on the plurality of pixel values corresponding to the at least one recognized object, the calculated photometric gradient including a highest photometric level and a lowest photometric level, the calculating; Determining that the direction of the light is on a side of the at least one recognized object having the highest photometric level; The method according to any one of claims 1 to 5, further comprising.

7. Utilizing with the at least one filter to which the assigned gloss index is applied as a multiplier, wherein a higher gloss index can increase the intensity of the at least one filter applied, the utilizing; The method according to any one of claims 1 to 6, further comprising.

8. A computer system for visualization of gloss, One or more processors; One or more computer-readable memories; Including, the one or more processors being configured to: Recognize at least one object in a digital image loaded on a user device; Assign a gloss index to the at least one recognized object; Determine a direction of light with respect to the at least one recognized object based on a plurality of pixel values corresponding to the at least one recognized object; Track the position of the eyes of a user viewing the digital image on the user device; In response to detecting movement of the position of the user's eyes, apply at least one filter to the at least one recognized object in the digital image in real time to simulate a gloss effect of the at least one recognized object; A computer system configured to perform.

9. The one or more processors are configured to: Further process the at least one recognized object in the digital image to simulate the gloss effect of the at least one recognized object in response to the assigned gloss index associated with the at least one recognized object satisfying a threshold gloss index; The computer system according to claim 8, further configured to perform.

10. The one or more processors, in response to the assigned glossiness index associated with the at least one recognized object being below a threshold glossiness index, display an original version of the at least one recognized object in the digital image The computer system according to claim 8 or 9, further configured to perform.

11. The one or more processors, compare the plurality of pixel values corresponding to the at least one recognized object; detecting at least one tonal change in the plurality of compared pixel values, wherein the detected at least one tonal change is related to light reflected from the at least one recognized object, said detecting The computer system according to claim 8, 9, or 10, further configured to perform.

12. The one or more processors, calculate at least one light intensity level associated with the at least one recognized object based on the at least one detected tonal change in the plurality of compared pixel values The computer system according to claim 11, further configured to perform.

13. The one or more processors, calculate a light intensity gradient associated with the at least one recognized object based on the plurality of pixel values corresponding to the at least one recognized object, the calculated light intensity gradient including a maximum light intensity level and a minimum light intensity level, said calculating; determining that the direction of the light is on the side of the at least one recognized object having the maximum light intensity level The computer system according to any one of claims 8 to 12, further configured to perform.

14. The one or more processors, utilize with the at least one filter to which the assigned glossiness index is applied as a multiplier, wherein a higher glossiness index can increase the intensity of the at least one filter applied, said utilizing The computer system according to any one of claims 8 to 13, further configured to perform.

15. A computer program, comprising program code means adapted to execute the method according to any one of Claims 1 to 7 when the program is executed on a computer.

16. A computer-readable storage medium having recorded thereon the computer program according to Claim 15.

Images