Stylized image drawing

Abstract

A photo filter (e.g., artistic / stylized painting) light field effect system includes an eye-mounted device with a frame, a temple connected to a side of the frame, and a depth capture camera. A programmed configuration stylized image drawing effect system is executed by a processor, and photo filter selection is applied to (i) the left side original image or the left side processed image to create a left side photo filter image, and (ii) the right side original image or the right side processed image to create a right side photo filter image. A stylized image drawing effect system may generate a photo filter stylized drawing effect image with a spatial rotation or movement representation by mixing a left photo filter image and a right photo filter image based on a left image disparity map and a right image disparity map, respectively.

Description

This application is a divisional application of the invention patent with the application number 202080068344.X, the invention title "Stylized Image Painting", and the application date August 27, 2020. Cross - reference to related applications

[0001] This application claims the priority of U.S. Utility Patent Application 16 / 587,015, titled "Stylized Image Painting", filed on September 29, 2019, the content of which is incorporated herein by reference in its entirety. Background Art

[0002] Current available computing devices, such as wearable devices, include portable eye - worn devices (e.g., smart glasses, headsets, and headgear); mobile devices (e.g., tablets, smartphones, and laptops); and personal computers, which integrate image displays and cameras. Currently, computing device users can use photo filters to create special effects on images. A variety of photo - decoration applications provide some tool features, such as stickers, emojis, and text, for editing images. Drawings

[0003] The drawings are intended to illustrate one or more embodiments, which are intended to provide examples and are not restrictive. In the drawings, the same reference numerals represent the same or similar elements.

[0004] Figure 1A is a right - hand view of an example hardware configuration of an eye - worn device used in a stylized image painting system with a photo filter (e.g., artistic / stylized painting), where the style of a selected image is transferred to a designated part of a target image to create a stylized image.

[0005] Figure 1B is Figure 1A a top - down cross - sectional view of the right - hand group of blocks of the eye - worn device shown, which shows the visible - light camera on the right side of the depth - capture camera and the circuit board.

[0006] Figure 1C is Figure 1A a left - hand view of an example hardware configuration of the eye - worn device shown, which shows the visible - light camera on the left side of the depth - capture camera.

[0007] Figure 1D is Figure 1C a top - down cross - sectional view of the left - hand group of blocks of the eye - worn device shown, which shows the visible - light camera on the left side of the depth - capture camera and the circuit board.

[0008] Figure 2A is a left - hand view of another example hardware configuration of the eye - worn device utilized in a stylized image painting system with a photo filter (e.g., artistic / stylized painting), which shows the visible - light camera on the right side and the depth sensor of the depth - capture camera to generate a depth image.

[0009] Figure 2B and 2C is a rear view of an exemplary hardware configuration of an eye - worn device, including two different types of image displays.

[0010] Figure 3 shows Figure 2A a rear perspective cross - sectional view of the eye - worn device shown, which shows the infrared camera of the depth sensor, the front of the frame, the rear of the frame, and the circuit board.

[0011] Figure 4 is a cross - sectional view taken through Figure 3 the infrared camera and the frame of the eye - worn device shown.

[0012] Figure 5 shows Figure 2A a rear perspective view of the eye - worn device, which shows the infrared emitter of the depth sensor, the infrared camera of the depth sensor, the front of the frame, the rear of the frame, and the circuit board.

[0013] Figure 6 is a cross - sectional view taken through Figure 5 the infrared emitter and the frame of the eye - worn device shown.

[0014] Figure 7 shows an example of an infrared light pattern emitted by the infrared emitter of the eye - worn device depth sensor, and the reflected change of the infrared light emission pattern captured by the depth sensor infrared camera, to measure the pixel depth in the original image and thereby generate a depth image.

[0015] Figure 8A is shown as the infrared light captured by the depth sensor infrared camera as an infrared image, and the visible light captured by the visible light camera as the original image, for an example of generating a depth image of a three - dimensional scene.

[0016] Figure 8B is shown as the visible light captured by the left - hand visible light camera as the left - hand original image, and the visible light captured by the right - hand visible light camera as the right - hand original image, for an example of generating a depth image of a three - dimensional scene.

[0017] Figure 9 is a high - level functional block diagram of an example filter (e.g., artistic / stylized painting) light field effect system, including an eye - worn device (equipped with a depth - capture camera to generate a filter - stylized painting effect image) and a user input device (e.g., a touch sensor), a mobile device, and a server system connected via various networks.

[0018] Figure 10 shows Figure 9Example of the hardware configuration of a photo filter (e.g., artistic / stylized painting) light field effect system for a mobile device, which includes a user input device (such as a touch screen device) for receiving marked inputs and an image selection for migrating to the original image or the processed image to generate a filtered effect image.

[0019] Figure 11A Is a flowchart of a method that can be implemented in a photo filter (e.g., artistic / stylized painting) light field effect system, applied to an original image or a processed image to generate a photo filter stylized painting effect image.

[0020] Figure 11B Is a flowchart of the method used to create a stylized painting image.

[0021] Figure 12A Shows an example of the first presentation of the original image, which is a processed (e.g., corrected) image.

[0022] Figure 12B Shows an example of the first presentation of the original image with user markings.

[0023] Figure 12C Shows from Figure 12A An example of a filtered (e.g., stylized painting effect) image created from the first presentation of the original image.

[0024] Figure 12D Shows from Figure 12B An example of the first filtered (e.g., artistic / stylized painting) light field effect image generated from a photo filter image, with its spatial movement or rotation tilted to the left.

[0025] Figure 12E Shows from Figure 12B An example of the second filtered (e.g., artistic / stylized painting) light field effect image generated from a photo filter image, with its spatial movement or rotation tilted to the right.

[0026] Figure 13 Shows an example of a photo filter (e.g., artistic / stylized painting) image mixing a stylized painting with the original image within a marked area. Details

[0027] The examples described herein relate to migrating the style of a selected image for marking a subject image. This enables the user to "paint" (selectively apply) the style of the subject image, thereby providing more artistic freedom and improving the user experience.

[0028] The following detailed description sets forth numerous specific details by way of example in order to provide a thorough understanding of the relevant teachings. However, it should be apparent to those skilled in the art that the teachings may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have been described in considerable detail without detailed explanation in order to avoid unnecessarily obscuring aspects of the teachings.

[0029] As used herein, the term "photo filter" refers to a graphic effect that edits, alters, or transforms a photo or picture to change specific pixels, by applying a style approach such as pop art (e.g., numerous paintings such as "Marilyn Monroe" by Andy Warhol and "The Scream" by Edvard Munch, 1893, etc.), or by using a deep neural network (such as a neural style transfer algorithm (NST)) to capture style from elements such as paintings; graphics (e.g., hats, beards, jewels, picture frames, stickers, and graphic overlays); textures; light saturation; color exposure; colors; sharpness; themes (sepia, dramatic, nostalgic, grayscale, black and white, vintage, disco, color fantasy, and vignette); and image quality enhancement (brightness, contrast, saturation, blur, etc.). The term "artistic effect" refers to editing or altering a photo or picture by applying a type of style transfer of pop art or a filter. The term "stylized painting effect" refers to an area of a photo or picture (i.e., an area of the photo or picture identified by the user, e.g., by marking) that is edited or altered by applying a type of style transfer of pop art or a filter.

[0030] Generally speaking, the term "light field" refers to the luminous intensity at a point in a given direction. The term "light field effect" refers to rendering different views of an image scene to provide an appearance of spatial movement or rotation as if the observer is viewing the scene from different angles or perspectives. The term "photo filter light field effect" refers to rendering different filter scene views of a photo filter image to provide an appearance of spatial movement or rotation as if the observer is viewing the photo filter scene from different angles or perspectives. The term "artistic light field effect" refers to rendering different artistic effect scene views of an artistic effect image to provide an appearance of spatial movement or rotation as if the observer is viewing the artistic effect scene from different angles or perspectives. The term "stylized painting light field effect" refers to rendering different artistic / stylized painting effect scene views of an artistic / stylized painting effect image (including a stylized painting) to provide an appearance of spatial movement or rotation as if the observer is viewing the artistic effect scene with a stylized painting from different angles or perspectives.

[0031] A light field effect camera can capture light from different directions and move around to create a three - dimensional or four - dimensional scene (e.g., using multiple lenses). However, such processing in three - dimensional space (X, Y, and Z) and four - dimensional space (X, Y, Z, and time) is relatively complex and may require a large amount of computation. As described herein, two visible - light cameras 114A - B can be used to create a simplified light field effect from two images by operating only in two - dimensional space, with less computational intensity.

[0032] As used herein, the terms "coupled" or "connected" refer to any logical, optical, physical, or electrical connection, link, etc., by which a signal or light generated or provided by one system element is passed to another coupled or connected element. Unless otherwise specified, the coupled or connected elements or devices are not necessarily directly connected to each other and may be separated by intermediate components, elements, or communication media that can modify, manipulate, or carry light or signals.

[0033] The orientation of the eye - worn device, related components, and any set of devices incorporating the depth - capture camera shown in the figures is provided only as an example for illustrative and discussion purposes. In actual operation, for photo filters (e.g., artistic / stylized paintings) light field effects, the eye - worn device can be oriented in any other direction suitable for a particular application of the eye - worn device, such as up, down, sideways, or any other direction. Additionally, within the scope of this document, any directional terms, such as front, back, inner, outer, direction, left, right, lateral, longitudinal, up, down, upward, downward, top, bottom, side, horizontal, vertical, and diagonal, are used only as examples and do not limit the orientation or orientation of any depth - capture camera or depth - capture camera components otherwise described herein.

[0034] The following description will set forth in part other purposes, advantages, and novel features of the examples. Such other purposes, advantages, and novel features will be apparent to those skilled in the art upon review of the following and the figures, or may be learned and understood by making or operating the examples. The purposes and advantages of the subject matter can be realized and obtained by the methods, means, and combinations particularly pointed out in the appended claims.

[0035] Reference is now made in detail to the examples shown in the figures and discussed below.

[0036] Figure 1AIs a right view of an example hardware configuration of an eye-wear device 100 used in a photo filter (e.g., artistic / stylized painting) light field effect system, showing the right visible light camera 114B of the depth capture camera for generating a depth image. As further described below, in a photo filter (e.g., artistic / stylized painting) light field effect system, a filter selection input (e.g., including the stylized artistic aspects within the image region identified by the user) is applied to the original image or the processed image to create a photo filter image with a photo filter scene. The photo filter images can be blended based on the disparity map to create a photo filter light field effect image including artistic / stylized painting aspects. The photo filter light field effect image can provide an appearance of spatial movement or rotation around the photo filter scene of the photo filter image. In one example, the photo filter type is a stylized painting effect. Thus, in this example, the stylized painting effect selection input from the user is applied to the original image or the processed image identified by the user to create an artistic / stylized painting effect image, and then these images are blended to generate a stylized painting light field effect image with a stylized painting effect scene. The stylized painting light field effect image can provide an appearance of spatial movement or rotation around the stylized painting effect scene of the artistic effect image.

[0037] The eye-wear device 100 includes a right optical component 180B that presents an image using an image display, such as based on the left original image, the processed left image, the right original image, the processed right image, the photo filter image (e.g., stylized painting effect image), or the photo filter light field effect image (e.g., stylized painting light field effect image). As Figure 1A shown in -B, the eye-wear device 100 includes the right visible light camera 114B. The eye-wear device 100 may include a plurality of visible light cameras 114A-B that form a passive type depth capture camera, such as a stereo camera, where the right visible light camera 114B is located on the right chunk 110B. As Figure 1C shown in -D, the eye-wear device 100 may also include the left visible light camera 114A. Alternatively, in the Figure 2A example of, the depth capture camera can be an active type depth capture camera that includes a single visible light camera 114B and a depth sensor (see component 213 in Figure 2A ).

[0038] The left and right visible light cameras 114A-B are sensitive to the visible light wavelength range. In the visible light cameras 114A-B, each camera has a different forward field of view, which overlaps with each other to allow the generation of a three-dimensional depth image. For example, the right visible light camera 114B has a shown right field of view 111B. Generally speaking, the "field of view" is a part of the scene that can be seen by the camera at a specific position and orientation in space. When the visible light camera captures an image, the objects or object features outside the field of view 111A-B will not be recorded in the original image (e.g., a photo or a picture). The field of view describes the angular range or extent within which the image sensor of the visible light cameras 114A-B picks up electromagnetic radiation in the captured image of a given scene. The field of view can be represented as the angular size of a visual cone, i.e., the viewing angle. The viewing angle can be measured horizontally, vertically, or diagonally.

[0039] In one example, the visible light cameras 114A-B have a field of view with a viewing angle between 15° and 30° (e.g., 24°) and a resolution of 480 x 480 pixels. The "coverage angle" describes the angular range within which the lens of the visible light cameras 114A-B or the infrared cameras 220 (see Figure 2A ) can effectively image. Generally, the size of the imaging circle produced by the camera lens is large enough to cover the film or sensor, and may include some vignetting (i.e., the brightness or saturation of the image decreases towards the edge compared to the center of the image). If the coverage angle of the camera lens does not fill the sensor, the imaging circle will become visible, usually with strong vignetting towards the edge, and the effective viewing angle will be limited by the coverage angle.

[0040] Examples of such visible light cameras 114A-B include high-resolution complementary metal oxide semiconductor (CMOS) image sensors and video graphics array (VGA) cameras, such as 640p (e.g., 640 x 480 pixels, a total of 0.3 megapixels), 720p, or 1080p. When referring to the field of view, the term "overlap" as used herein means that the pixel matrices overlap by 30% or more in the original image of the generated scene or the infrared image. When referring to the field of view, the term "substantial overlap" as used herein means that the pixel matrices overlap by 50% or more in the original image of the generated scene or the infrared image.

[0041] Image sensor data from the visible light cameras 114A-B is captured together with geographical location data, digitized by an image processor, and then stored in a memory. The left and right raw images captured by the respective visible light cameras 114A-B are in a two-dimensional spatial domain and include a pixel matrix on a two-dimensional coordinate system that includes an X-axis representing horizontal position and a Y-axis representing vertical position. Each pixel includes a color attribute (e.g., red pixel light value, green pixel light value, and / or blue pixel light value); and a position attribute (e.g., X-position coordinate and Y-position coordinate).

[0042] To provide stereoscopic vision, the visible light cameras 114A-B can be coupled to an image processor ( Figure 9 element 912 as shown) for digital processing and for capturing a timestamp of the scene image. The image processor 912 includes circuitry for receiving signals from the visible light cameras 114A-B and processing the signals from the visible light cameras 114 into a format suitable for storage in a memory. The timestamp can be added by the image processor or another processor that is responsible for controlling the operation of the visible light cameras 114A-B. The visible light cameras 114A-B allow a depth capture camera to simulate human binocular vision. The depth capture camera has the ability to reproduce a three-dimensional image based on two captured images from the visible light cameras 114A-B with the same timestamp. Such three-dimensional images can provide an immersive and realistic experience, e.g., virtual reality or video games.

[0043] For stereovision, at a given moment, a pair of raw red, green, and blue (RGB) images are captured from the scene - one image corresponding to each of the left and right visible light cameras 114A - B (e.g., a stereo pair). When processing (e.g., by an image processor) a pair of captured raw images from the forward left and right fields of view 111A - B of the left and right visible light cameras 114A - B, a depth image is generated. The depth image can be based on a 3D model, which may include a 3D mesh (e.g., a triangular mesh) and textures, and these textures are uploaded to the graphics processing unit (GPU) as vertices along with texture mapping. Generally, its depth is actually not visible, but the effect of depth can be seen in the rendered and displayed 2D image. By converting to 2D images for displaying various viewpoints, the user can perceive the generated depth image on the optical components 180A - B or other image displays (e.g., the display of a mobile device). The generated depth image is within a 3D spatial domain and may include a vertex matrix on a 3D position coordinate system, which includes an X - axis representing the horizontal position (e.g., length), a Y - axis representing the vertical position (e.g., height), and a Z - axis representing the depth (e.g., distance). Each vertex includes a color attribute (e.g., red pixel light value, green pixel light value, and / or blue pixel light value); a position attribute (e.g., X - position coordinate, Y - position coordinate, and Z - axis coordinate); a texture attribute, and / or a reflectivity attribute. The texture attribute can quantify the perceived texture of the depth image, such as the spatial arrangement of colors or intensities in the vertex region of the depth image.

[0044] Generally, depth perception originates from the disparity of a given 3D point in the left and right raw images captured by the visible light cameras 114A - B. Disparity is the difference in the imaging positions of the same 3D point when projected under the perspective of the visible light cameras 114A - B (d = x 左 -x 右 ). For example, the correlation of left and right pixels in each of the left and right raw images can be achieved through semi - global block matching (SGBM). For visible light cameras 114A - B with parallel optical axes, focal length f, baseline b, and corresponding imaging points (x 左 , y 左 ) and (x 右 , y 右 ), triangulation can be used to deduce the position (Z - axis position coordinate) of the 3D point, and triangulation determines the depth through disparity. Generally, the depth of a 3D point is inversely proportional to the disparity. Many other techniques can also be applied. The generation of 3D depth images and photo filter (e.g., artistic / stylized painting) light field effect images will be described in more detail below.

[0045] In one example, a photo filter (e.g., artistic / stylized painting) light field effect system includes an eyewear device 100. The eyewear device 100 includes a frame 105, a left temple 110A extending from a left side surface 170A of the frame 105, and a right temple 110B extending from a right side surface 170B of the frame 105. The eyewear device 100 also includes a depth capture camera. The depth capture camera includes: (i) at least two visible light cameras with overlapping fields of view; or (ii) at least one visible light camera 114A-B and a depth sensor ( Figure 2A shown element 213). In one example, the depth capture camera includes a left visible light camera 114A, whose left field of view 111A is connected to the frame 105 or the left temple 110A to capture a left image of the scene. The eyewear device 100 also includes a right visible light camera 114B connected to the frame 105 or a right temple 110B with a right field of view 111B to capture a right image of the scene that partially overlaps the left image (e.g., simultaneously with the left visible light camera 114A).

[0046] The photo filter (e.g., artistic / stylized painting) light field effect system also includes a computing device coupled to the eyewear device 100 via a network, such as a host (e.g., Figure 9 - 10 shown mobile device 990). The photo filter (e.g., artistic / stylized painting) light field effect system also includes an image display for presenting (e.g., displaying) an image sequence (the optical components 180A-B of the eyewear device; Figure 10 the image display 1080 of the shown mobile device 990). The image sequence includes an original image, an original image in two-dimensional space or a processed original image (e.g., corrected), a photo filter (e.g., artistic / stylized effect) image, and a photo filter (e.g., artistic / stylized painting) light field effect image. The photo filter (e.g., artistic / stylized painting) light field effect system also includes an image display driver ( Figure 9 the element 942 of the shown eyewear device 100; Figure 10 the element 1090 of the shown mobile device 990), which is coupled to the image display (the optical components 180A-B of the eyewear device; Figure 10 the image display 1080 of the shown mobile device 990), and controls the image display to display the image sequence. The image sequence may include an original image, an original image in two-dimensional space or a processed original image (e.g., corrected), a photo filter (e.g., artistic / stylized effect) image, and a photo filter (e.g., artistic / stylized painting) light field effect image.

[0047] The photo filter (e.g., artistic / stylized effect) light field effect system also includes a user input device for receiving a two-dimensional input selection from a user. Examples of user input devices include touch sensors ( Figure 9Element 991) shown for the eye - worn device 100, touch - screen display( Figure 10 Element 1091) shown for the mobile device 1090, and a computer mouse for a personal computer or laptop. The photo filter (e.g., art / stylized painting) light field effect system also includes a processor coupled to the eye - worn device 100 and the depth - capture camera( Figure 9 Element 932 of the eye - worn device 100 shown; Figure 10 Element 1030) of the mobile device 990 shown. The photo filter (e.g., art / stylized painting) light field effect system also includes a memory accessible to the processor( Figure 9 Element 934 of the eye - worn device 100 shown; Figure 10 Elements 1040A - B) of the mobile device 990 shown, and photo filter (e.g., art / stylized painting) light field effect programming in the memory( Figure 9 Element 945 of the eye - worn device 100 shown; Figure 10 Element 945) of the mobile device 990 shown), e.g., in the eye - worn device 100 itself, the mobile device( Figure 9 Element 990) shown) or another part of the photo filter (e.g., art / stylized painting) light field effect system (e.g., Figure 9 Server system 998) shown).

[0048] As described below, the photo filter (e.g., art / stylized painting) light field effect system takes the left - hand image and the right - hand image as input viewpoints, but there is no image between the viewpoints. To generate an art / stylized painting effect (e.g., in the case where the camera rotates around one face of the image and the user makes stylized marks at different angles to freeze the moment in the time dimension), interpolation is performed between the left - and right - hand images captured by the left - and right - hand cameras 114A - B. The art / stylized painting effect images can be stitched together from several different viewpoints as a series of images in a video to provide spatial motion.

[0049] Interpolate two non-raw RGB (modified / unreal) left and right images to generate a photo filter (e.g., artistic / stylized painting) light field effect image, and the interpolation is based on a disparity map generated from two raw RGB images. By rotating even the unreal images, this provides an illusion of a 3D world, but only two modified 2D images (frames) are needed to produce a stylized painting effect. The disparity map can determine how many pixels need to be moved between pixels in the left image to obtain the corresponding pixels in the right image, and vice versa. Calculate the disparity between stereo pixel pairs corresponding to depth for interpolation between the two non-raw RGB images. In some examples, the left image can be a black-and-white blend, while the right image can be in color. In another example, simulate an artistic / stylized painting style in one image (such as the left image), while simulate a raw RGB image in another image (such as the right image), and interpolate between a raw RGB image and a modified image based on left and right disparities.

[0050] Figure 1B Yes Figure 1A A top cross-sectional view of the right block 110B of the eye-wear device 100 is shown, showing the right visible light camera 114B of the depth capture camera and the circuit board. Figure 1C Yes Figure 1A A left view of an example hardware configuration of the eye-wear device 100 is shown, showing the left visible light camera 114A of the depth capture camera. Figure 1D Yes Figure 1C A top cross-sectional view of the left block 110A of the eye-wear device is shown, schematically showing the left visible light camera 114A of the depth capture camera and the circuit board. The configuration and mounting position of the left visible light camera 114A are substantially similar to those of the right visible light camera 114B, except that it is connected and coupled to the left side 170A. As Figure 1B As an example shows, the eye-wear device 100 includes a right visible light camera 114B and a circuit board, which can be a flexible printed circuit board (PCB) 140B. The right hinge 126B connects the right block 110B to the right temple 125B of the eye-wear device 100. In some examples, the components of the right visible light camera 114B, the flexible printed circuit board 140B, or other electrical connectors or contacts can be located on the right temple 125B or the right hinge 126B.

[0051] The right block 110B includes a block body 211 and a block cap, and the block cap is omitted in the Figure 1B shown cross-section. Arranged inside the right block 110B are various interconnected printed circuit boards (PCBs), such as conventional or flexible printed circuit boards, which include controller circuits for the right visible light camera 114B, microphones, low-power wireless circuits (e.g., for Bluetooth TMWireless short - range network communication), high - speed wireless circuits (e.g., for wireless local area network communication via WiFi).

[0052] The right - hand visible - light camera 114B is coupled or disposed on the flexible printed circuit board 240 and covered by a visible - light camera cover lens, which is aligned through an opening formed in the frame 105. For example, the right - hand frame 107B of the frame 105 is connected to the right - hand block 110B and includes an opening for the visible - light camera cover lens. The frame 105 includes a front side configured to face away from the user's eyes. The visible - light camera cover - lens opening is formed on and passes through the front - facing side. In this example, the right - hand visible - light camera 114B has a field of view 111B facing outward, with the line of sight or viewing angle of the right eye of the user wearing the eye - wear device 100. The visible - light camera cover lens can also be adhered to the outer surface of the right - hand block 110B, with an opening formed thereon having an outward - covering angle, but in a different outward direction. Indirect coupling via an intermediate component can also be employed.

[0053] The left - hand (first) visible - light camera 114A is connected to the left - hand image display of the left - hand optical assembly 180A to capture the left - eye scene observed by the wearer of the eye - wear device 100 in the left - hand original image. The right - hand (second) visible - light camera 114B is connected to the right - hand image display of the right - hand optical assembly 180B to capture the right - eye scene observed by the wearer of the eye - wear device 100 in the right - hand original image. The left - hand original image and the right - hand original image partially overlap to present a three - dimensional observable space of the generated depth image.

[0054] The flexible printed circuit board 140B is disposed within the right - hand block 110B and is coupled to one or more other components within the right - hand block 110B. Although shown as being formed on the circuit board of the right - hand block 110B, the right - hand visible - light camera 114B can also be formed on the circuit board of the left - hand block 110A, the temple 125A - B, or the frame 105.

[0055] Figure 2A is a left - hand view of another example hardware configuration of the eye - wear device 100 used in a photo - filter (e.g., art / stylized painting) light - field effect system. As shown, the depth - capture camera includes a left - hand visible - light camera 114A and a depth sensor 213 located on the frame 105 for generating a depth image. Here, instead of using at least two visible - light cameras 114A - B, a single visible - light camera 114A and a depth sensor 213 are used to generate the depth image. As Figure 1AAs shown in the -D example, the photo filter selection from the user should be applied to the original image to create a photo filter image and subsequently generate a photo filter art / stylized painting effect image. The infrared camera 220 of the depth sensor 213 has an outward field of view that essentially overlaps with the left visible light camera 114A for the user's line of sight. As shown, the infrared emitter 215 and the infrared camera 220 are both located in the upper part of the left frame 107A together with the left visible light camera 114A.

[0056] In Figure 2A In the example shown, the depth sensor 213 of the eye - worn device 100 includes an infrared emitter 215 and an infrared camera 220 for capturing infrared images. The visible light cameras 114A - B typically include blue filters to block infrared light detection. In one example, the infrared camera 220 is a visible light camera, such as a low - resolution video graphics array (VGA) camera (e.g., 640x480 pixels, total pixels of 0.3 megapixels) with its blue filter removed. The infrared emitter 215 and the infrared camera 220 are both located on the frame 105. For example, both are shown as connected to the upper part of the left frame 107A. As further detailed below, the frame 105 or one or more of the left and right chunks 110A - B include a circuit board for the infrared emitter 215 and the infrared camera 220. For example, the infrared emitter 215 and the infrared camera 220 can be connected to the circuit board by soldering.

[0057] Other arrangements of the infrared emitter 215 and the infrared camera 220 can be implemented, including arrangements where both the infrared emitter 215 and the infrared camera 220 are located on the right frame 107A, or arrangements at different positions on the frame 105. For example, the infrared emitter 215 is located on the left frame 107B and the infrared camera 220 is located on the right frame 107B. Nevertheless, typically, at least one visible light camera 114A and the depth sensor 213 have a substantially overlapping field of view for generating a three - dimensional depth image. In another example, the infrared emitter 215 is located on the frame 105 and the infrared camera 220 is located on one of the chunks 110A - B, and vice versa. Basically, the infrared emitter 215 can be connected to any position on the frame 105, the left chunk 110A, or the right chunk 110B for emitting an infrared pattern within the user's line of sight. Similarly, the infrared camera 220 can be connected to any position on the frame 105, the left chunk 110A, or the right chunk 110B for capturing at least one reflection change in the infrared light emission pattern of the three - dimensional scene within the user's line of sight.

[0058] The infrared emitter 215 and the infrared camera 220 are arranged facing outward to pick up an infrared image of the scene, which image carries an object or object feature observed by a user wearing the eyewear device 100. For example, the infrared emitter 215 and the infrared camera 220 are directly in front of the eyes, in the upper part of the frame 105 or in the blocks 110A-B at both ends of the frame 105, having a forward field of view to capture an image of the scene the user is looking at for measuring object depth and object features.

[0059] In one example, the infrared emitter 215 of the depth sensor 213 emits infrared light illumination within the forward field of view of the scene, which can be near-infrared light or other short-wavelength light beams of low-energy radiation. As an alternative or in addition, the depth sensor 213 can include other emitters for emitting light of wavelengths other than infrared light, and the depth sensor 213 also includes a camera sensitive to that wavelength for receiving and capturing an image having that wavelength. As described above, the eyewear device 100 is coupled to a processor and a memory, for example, a processor and a memory within the body of the eyewear device 100 or within other components of a filter (e.g., an artistic / stylized painting) light field effect system. The eyewear device 100 or the photo filter (e.g., an artistic / stylized painting) light field effect system can then process the captured infrared image during the generation of a three-dimensional depth image, such as a depth image.

[0060] Figure 2B -C is a rear view of an example hardware configuration of the eyewear device 100, including two different types of image displays. The eyewear device 100 is in a configuration form for a user to wear, which is glasses in this example. The eyewear device 100 can take other forms and can incorporate other types of frames, such as a headband, headphones, or a helmet.

[0061] In the example of glasses, the eyewear device 100 includes a frame 105, where the left lens frame 107A is connected to the right lens frame 107B through a nose bridge 106 adapted to the user's nose. The left and right lens frames 107A-B each include respective openings 175A-B for accommodating respective optical elements 180A-B, such as lenses and display devices. The term "lens" as used herein refers to a curved and / or flat transparent or translucent glass or plastic piece for covering, with or without light converging / diverging action.

[0062] Although shown as equipped with two optical elements 180A-B, depending on the specific use or intended user of the eyewear device 100, the eyewear device 100 may also include other arrangements, such as a single optical element, or may not include any optical elements 180A-B. As further shown, the eyewear device 100 includes a left module 110A adjacent to the left side surface 170A of the frame 105 and a right module 110B adjacent to the right side surface 170B of the frame 105. The modules 110A-B can be integrated into the frame 105 on the respective side surfaces 170A-B (as shown), or can be implemented as separate components connected to the frame 105 on the respective side surfaces 170A-B. As an alternative, the modules 110A-B can also be integrated into the temple arms (not shown) connecting the frame 105.

[0063] In one example, the image display of the optical assemblies 180A-B includes an integrated image display. As Figure 2B shown, the optical assemblies 180A-B include a suitable display matrix 170 of any appropriate type, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or any other such display. The optical assemblies 180A-B also include one or more optical layers 176, which may include lenses, optical coatings, prisms, mirrors, waveguides, light strips, and any combination of other optical components. The optical layers 176A-N can include a prism, such a prism having a suitable size and configuration, and being provided with a first surface for receiving light from the display matrix and a second surface for emitting light towards the user's eye. The prism of the optical layer 176A-N can extend into all or at least part of the respective openings 175A-B formed in the left and right frames 107A-B, so that when the user views through the corresponding left and right frames 107A-B with the eyes, the second surface of the prism can be seen. The first surface of the prism of the optical layer 176A-N is arranged to face upward along the frame 105, and the display matrix covers the prism, ensuring that photons and light emitted from the display matrix irradiate onto the first surface. The size and shape of the prism can ensure that the light is refracted within the prism and is directed by the second surface of the prism of the optical layer 176A-N towards the user's eye. In this regard, the second surface of the prism of the optical layer 176A-N can be convex to direct the light towards the center of the eye. The size and shape of the prism can be selectively adjusted to magnify the image projected by the display matrix 170 and to allow the light to pass through the prism, so that the image viewed from the second surface is larger than the image emitted from the display matrix 170 in one or more dimensions.

[0064] In another example, the image display device of the optical assemblies 180A-B includes as Figure 2CThe projection image display shown. The optical components 180A-B include a laser projector 150, which is a three-color laser projector using a scanning mirror or a galvanometer. During operation, a light source such as the laser projector 150 is arranged within or on any of the temple arms 125A-B of the eyewear device 100. The optical components 180A-B include one or more light bands 155A-N, which are spaced apart along the lens width of the optical components 180A-B or along the lens depth between the front and rear surfaces of the lens.

[0065] When the photons projected by the laser projector 150 pass through the lens of the optical components 180A-B, the photons encounter the light bands 155A-N. When a specific photon encounters a specific light band, the photon is either redirected towards the user's eye or transmitted to the next light band. Specific photons or light beams can be controlled by combining laser projector 150 modulation and light band modulation. In one example, the processor controls the light bands 155A-N by activating mechanical, acoustic, or electromagnetic signals. Although shown as equipped with two optical components 180A-B, the eyewear device 100 can also adopt other arrangements, such as a single or three optical components, or different arrangements of the optical components 180A-B can be adopted according to the application of the eyewear device 100 or the specific situation of the intended user.

[0066] As Figure 2B -C further shows, the eyewear device 100 includes a left block 110A adjacent to the left side surface 170A of the frame 105 and a right block 110B adjacent to the right side surface 170B of the frame 105. The blocks 110A-B can be integrated into the frame 105 on the respective side surfaces 170A-B (as shown in the figure), or implemented as independent components connected to the frame 105 on the respective side surfaces 170A-B. Alternatively, the blocks 110A-B can also be integrated into the temple arms 125A-B connecting the frame 105. The blocks 110A-B used herein can include a housing that encapsulates a combination of a processing unit, a camera, sensors, etc. (for example, with differences for the right and left sides).

[0067] In one example, the image display includes a first (left) image display and a second (right) image display. The eyewear device 100 includes first and second apertures 175A-B, which respectively accommodate the first and second optical components 180A-B. The first optical component 180A includes a first image display (for example, Figure 2B the display matrix 170A shown; or Figure 2C the light bands 155A-N’ and the projector 150 shown). The second optical component 180B includes a second image display, for example, Figure 2B the display matrix 170B shown; or Figure 2C the light bands 155A-N” and the projector 150B shown).

[0068] Figure 3 Shows Figure 2A A rear perspective cross-sectional view of the eye-wear device shown, showing the infrared camera 220, the front portion 330 of the frame, the rear portion 335 of the frame, and the circuit board. As can be seen, the upper portion of the left lens frame 107A of the frame 105 of the eye-wear device 100 includes the front portion 330 of the frame and the rear portion 335 of the frame. The front portion 330 of the frame includes a front-facing side configured to face away from the user's eyes outwardly. The rear portion 335 of the frame includes a rear-facing side configured to face toward the user's eyes inwardly. An opening for the infrared camera 220 is formed in the front portion 330 of the frame.

[0069] As shown in the circumferential cross-section 4-4 of the upper middle portion of the left lens frame 107A of the frame 105, the circuit board (flexible printed circuit board (PCB) 340) is sandwiched between the front portion 330 of the frame and the rear portion 335 of the frame. In addition, the connection of the left module 110A to the left temple 325A through the left hinge 126A is further shown in detail. In some examples, components of the depth sensor 213, including the infrared camera 220, the flexible printed circuit board 340, or other electrical connectors or contacts, may be located on the left temple 325A or the left hinge 126A.

[0070] In one example, the left module 110A includes a module body 311, a module cap 312, an inward surface 391, and an outward surface 392 (marked but not visible). Arranged within the left module 110A are various interconnected circuit boards, such as printed circuit boards or flexible printed circuit boards, which include controller circuits for charging the battery, inward-facing light-emitting diodes (LEDs), and outward-facing (forward) LEDs. Although shown as being formed on the circuit board of the left lens frame 107A, the depth sensor 213 (including the infrared emitter 215 and the infrared camera 220) may also be formed on the circuit board of the right lens frame 107B for capturing infrared images used in generating three-dimensional depth images, for example, in combination with the right visible light camera 114B.

[0071] Figure 4 Is through the infrared camera 220 and corresponding to Figure 3 A cross-sectional view of the frame of the eye-wear device shown in the circumferential cross-section 4-4. In Figure 4The layers of the eyewear device 100 can be seen in the cross-section shown. As shown, the flexible printed circuit board 340 is disposed on the rear portion 335 of the frame and is connected to the front portion 330 of the frame. The infrared camera 220 is disposed on the flexible printed circuit board 340 and is covered by the infrared camera cover lens 445. For example, the infrared camera 220 is soldered to the back surface of the flexible printed circuit board 340 by reflow soldering. By subjecting the flexible printed circuit board 340 to controlled heat, the infrared camera 220 is reflow soldered to the electrical contact pads formed on the back surface of the flexible printed circuit board 340, and the controlled heat is used to melt the solder paste to achieve the connection between the two components. In one example, reflow soldering is used to surface mount the infrared camera 220 onto the flexible printed circuit board 340 and to achieve the electrical connection between the two components. However, it should be understood that through holes (for example) can be used to connect the leads extending from the infrared camera 220 to the flexible printed circuit board 340 by way of interconnection.

[0072] The front portion 330 of the frame includes an infrared camera opening 450 for the infrared camera cover lens 445. The infrared camera opening 450 is formed on the front surface of the front portion 330 of the frame, and the front portion 330 of the frame is configured to face away from the user's eyes and towards the scene observed by the user. In an example, the flexible printed circuit board 340 can be connected to the rear portion 335 of the frame by the flexible printed circuit board adhesive 460. The infrared camera cover lens 445 can be connected to the front portion 330 of the frame by the infrared camera cover lens adhesive 455. The connection can be made indirectly through an intermediate component.

[0073] Figure 5 Shows Figure 2A A rear perspective cross-sectional view of the eyewear device shown. The eyewear device 100 includes an infrared emitter 215, an infrared camera 220, a front portion 330 of the frame, a rear portion 335 of the frame, and a circuit board 340. As Figure 3 Shown, it can be seen Figure 5 In the upper part of the left frame of the eyewear device 100 shown, it includes a front portion 330 of the frame and a rear portion 335 of the frame. The opening for the infrared emitter 215 is formed on the front portion 330 of the frame.

[0074] As shown in the circumferential cross-section 6-6 of the upper middle part of the left frame of the frame, the circuit board (flexible printed circuit board 340) is sandwiched between the front portion 330 of the frame and the rear portion 335 of the frame. In addition, it is further shown in detail that the left module 110A is connected to the left temple 325A through the left hinge 126A. In some examples, components of the depth sensor 213, including the infrared emitter 215, the flexible printed circuit board 340, or other electrical connectors or contacts, can be located on the left temple 325A or the left hinge 126A.

[0075] Figure 6 Is through the infrared emitter 215 and corresponding toFigure 5 Cross-sectional view of the frame of the eye-wear device around the horizontal cross-section 6-6. As Figure 6 As shown in the multiple layers of the eye-wear device 100 in the cross-section, the frame 105 includes a front frame portion 330 and a rear frame portion 335. The flexible printed circuit board 340 is disposed on the rear frame portion 335 and is connected to the front frame portion 330. The infrared emitter 215 is disposed on the flexible printed circuit board 340 and is covered by the infrared emitter cover lens 645. For example, the infrared emitter 215 is soldered to the back surface of the flexible printed circuit board 340 by reflow soldering. By subjecting the flexible printed circuit board 340 to a controlled heat, the infrared emitter 215 is reflow soldered to the electrical contact pads formed on the back surface of the flexible printed circuit board 340, and the controlled heat is used to melt the solder paste to achieve the connection between the two components. In one example, the infrared emitter 215 is surface-mounted on the flexible printed circuit board 340 by reflow soldering and the two components are electrically connected. However, it should be understood that through-holes (for example) can be used to connect the leads led out from the infrared emitter 215 to the flexible printed circuit board 340 by means of interconnection.

[0076] The front frame portion 330 includes an infrared emitter opening 650 for the infrared emitter cover lens 645. The infrared emitter opening 650 is formed on the front surface of the front frame portion 330, and the front frame portion 330 is configured to face away from the user's eyes outward and toward the scene observed by the user. In an example, the flexible printed circuit board 340 can be connected to the rear frame portion 335 by the flexible printed circuit board adhesive 460. The infrared emitter cover lens 645 can be connected to the front frame portion 330 by the infrared emitter cover lens adhesive 655. Indirect coupling via an intermediate component can also be used.

[0077] Figure 7 An example of the emission pattern of the infrared light 781 emitted by the infrared emitter 215 of the depth sensor 213 is shown. As shown in the figure, the reflected change of the infrared light 782 emission pattern is captured as an infrared image by the infrared camera 220 of the depth sensor 213 of the eye-wear device 100. The reflected change of the infrared light 782 emission pattern is used to measure the depth of the pixels in the original image (for example, the left original image) to generate a three-dimensional depth image including the depth image.

[0078] The depth sensor 213 in this example includes an infrared emitter 215 for projecting an infrared light pattern, and an infrared camera 220 (for capturing an infrared image of an object or object feature in space that is distorted by the projected infrared light), as shown in the scene 715 observed by the wearer of the eye-mounted device 100. For example, the infrared emitter 215 can emit infrared light 781, which falls on an object, or on an object feature within the scene 715, like a collection of a large number of points. In some examples, the infrared light is emitted in a pattern such as a line, a spiral, or concentric rings. Infrared light is generally invisible to the naked eye. The infrared camera 220 is similar to a standard red, green, and blue (RGB) camera, except that it receives and captures images of light in the infrared wavelength range. For depth sensing, the infrared camera 220 is coupled to an image processor ( Figure 9 912 in the image processing system) and photo filter (e.g., artistic / stylized painting) light field effects programming or application (element 945) that determines the time of flight based on the infrared image of the captured infrared light. For example, the distorted dot pattern 782 in the captured infrared image can then be processed by the image processor to determine the depth based on the displacement of the dots. Typically, nearby targets or target features will appear as a pattern with a sparser distribution of dots, while distant objects will appear as a denser pattern of dots. It should be understood that the aforementioned functionality can be embodied in the programming instructions of the photo filter (e.g., artistic / stylized painting) light field effects programming or application (element 945) that can be found in one or more system components.

[0079] Figure 8A An example of infrared light captured by the infrared camera 220 of the depth sensor 213 using the left infrared camera field of view 812 is shown. The infrared camera 220 captures the reflected changes in the emission pattern of the infrared light 782 in the three-dimensional scene 715 as an infrared image 859. As further shown, the left visible light camera 114A captures visible light using the left visible light camera field of view 111A as a left raw image 858A. Based on the infrared image 859 and the left raw image 858A, a three-dimensional depth image of the three-dimensional scene 715 can be generated.

[0080] Figure 8B An example of visible light captured by the left visible light camera 114A and visible light captured by the right visible light camera 114B is shown. The visible light captured by the left visible light camera 114A using the left visible light camera field of view 111A is used as the left raw image 858A. The visible light captured by the right visible light camera 114B using the right visible light camera field of view 111B is used as the right raw image 858B. Based on the left raw image 858A and the right raw image 858B, a three-dimensional depth image of the three-dimensional scene 715 can be generated.

[0081] Figure 9is a high-level functional block diagram of an example photo filter (e.g., artistic / stylized painting) light field effect system 900, which includes a wearable device (e.g., an eyewear device 100), a mobile device 990, and a server system 998 connected via various networks. The eyewear device 100 includes depth capture cameras, such as at least one visible light camera 114A-B; and a depth sensor 213, shown as an infrared emitter 215 and an infrared camera 220. The depth capture cameras may alternatively include at least two visible light cameras 114A-B (one associated with the left side 170A and the other associated with the right side 170B), in which case a depth sensor is not required. The depth capture cameras can generate depth images 961A-N, which are rendered three-dimensional (3D) models and are texture-mapped images of red, green, and blue (RGB) imaging scenes, e.g., derived from the original images 858A-N and processed (e.g., corrected) images 965A-N.

[0082] The mobile device 990 can be a smartphone, a tablet computer, a laptop computer, an access point, or any other such device capable of connecting to the eyewear device 100 using both a low-power wireless connection 925 and a high-speed wireless connection 937. The mobile device 990 accesses the server system 998 and the network 995. The network 995 can include any combination of wired and wireless connections

[0083] The eyewear device 100 also includes two image displays of the optical components 180A-B (one associated with the left side 170A and the other associated with the right side 170B). The eyewear device 100 also includes an image display driver 942, an image processor 912, a low-power circuit 920, and a high-speed circuit 930. The image displays of the optical components 180A-B are used to present images, such as the original images 957A-N (e.g., the original images 858A-N and the processed images 965A-N), photo filter (e.g., stylized painting effect) images 963A-N, and photo filter (e.g., stylized painting) light field effect images 964A-N. The image display driver 942 is coupled to the image displays of the optical components 180A-B to control the image presentation of the optical components 180A-B image displays. The eyewear device 100 also includes a user input device 991 (e.g., a touch sensor) for receiving photo filter (e.g., stylized painting effect) selection 962a input (based on marker 962b input); and can receive two-dimensional (2D) input selection 973 from the user.

[0084] Figure 9The components of the eyewear device 100 shown, such as a printed circuit board or a flexible printed circuit board, are located on one or more circuit boards in the frame or temple. As an alternative or in addition, the components shown may be located in a chunk, frame, hinge, or bridge of the eyewear device 100. The left and right visible light cameras 114A - 114B may include digital camera elements, such as complementary metal oxide semiconductor (CMOS) image sensors, charge - coupled devices, lenses, or any other corresponding visible light or light - capturing elements that can be used to capture data, including images of scenes with unknown objects.

[0085] The eyewear device 100 includes a memory 934 that contains photo - filter (e.g., stylized painting) light - field effect programming 945 for performing a subset or all of the functions of the photo - filter (e.g., stylized painting) light - field effects described herein, where a photo - filter selection 962A from the user (based on a user marker 962B) is applied to an original image 858A - B or a processed image 965A - B to generate a photo - filter (e.g., stylized painting) light - field effect image 964A - N. As shown, the memory 934 also includes a left - hand original image 858A captured by the left visible light camera 114A, a right - hand original image 858B captured by the right visible light camera 114B, and an infrared image 859 captured by the infrared camera 220 of the depth sensor 213. The memory 934 also includes a plurality of depth images 961A - N generated via the depth - capture camera.

[0086] Figure 11A and 11B An overview flowchart showing functions that can be implemented in the photo - filter (e.g., stylized painting) light - field effect programming 945 is shown. The memory 934 also includes a two - dimensional input selection 962a (e.g., an initial contact point and a final contact point) and a two - dimensional marker 962b received by the user input device 991. The memory 934 also includes: a left - hand image disparity map 960A, a right - hand image disparity map 960B, photo - filter (e.g., stylized painting effect) images 963A - N, a horizontal position parameter 966, a left - hand interpolated pixel matrix 967A including left - hand moving X - axis position coordinates 968A - N, a right - hand interpolated pixel matrix 967B with right - hand moving X - axis position coordinates 969A - N, and a left - hand processed (e.g., corrected) image 965A and a right - hand processed (e.g., corrected) image 965B (e.g., removing vignetting towards the lens end). As further shown, the memory 934 includes a vertex matrix 970 and a rotation matrix 974. Some or all of the information stored in the memory 934 may be generated during the processing of the original images 858A - B for generating the corresponding photo - filter (e.g., art / stylized painting) light - field effect images 964A - N.

[0087] As Figure 9As shown, the high-speed circuit 930 includes a high-speed processor 932, a memory 934, and a high-speed wireless circuit 936. In an example, the image display driver 942 is coupled to the high-speed circuit 930 and is operated by the high-speed processor 932 to drive the left and right image displays of the optical components 180A-B. The high-speed processor 932 can be any processor capable of managing the high-speed communications required for the eyewear device 100 and any general computing system operations. The high-speed processor 932 includes the processing resources required to manage high-speed data transmission to a wireless local area network (WLAN) over the high-speed wireless connection 937 using the high-speed wireless circuit 936. In a particular embodiment, the high-speed processor 932 may execute an operating system of the eyewear device 100, such as the LINUX operating system or other such operating systems, and the operating system is stored in the memory 934 for execution. In addition to any other tasks, the high-speed processor 932 that executes the software architecture of the eyewear device 100 is used to manage data transmission with the high-speed wireless circuit 936. In some embodiments, the high-speed wireless circuit 936 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, also known herein as Wi-Fi. In other embodiments, other high-speed communication standards may be implemented by the high-speed wireless circuit 936.

[0088] The low-power wireless circuit 924 and the high-speed wireless circuit 936 of the eyewear device 100 may include short-range transceivers (Bluetooth TM ), as well as wireless wide area, local area, or wide area network transceivers (e.g., cellular or WiFi). The mobile device 990 includes transceivers that communicate via the low-power wireless connection 925 and the high-speed wireless connection 937, and may be implemented using details of the architecture of the eyewear device 100, as may other elements of the network 995.

[0089] The memory 934 includes any storage device capable of storing various data and applications, in addition to camera data generated by the left and right visible light cameras 114A-B, the infrared camera 220, and the image processor 912, and images generated by the image display driver 942 for display on the image displays of the optical components 180A-B. Although the memory 934 is shown as integrated with the high-speed circuit 930, in other embodiments, it may be a separate element of the eyewear device 100. In some such embodiments, electrical routing lines may provide a connection to the memory 934 through a chip that includes the high-speed processor 932 or the low-power processor 922 from the image processor 912. In other embodiments, the high-speed processor 932 may manage the addressing of the memory 934 such that the low-power processor 922 will direct the high-speed processor 932 whenever any read or write operations involving the memory 934 are required.

[0090] As Figure 9As shown, the processor 932 of the head - mounted device 100 can be coupled to a depth - capture camera (visible - light cameras 114A - B; or visible - light camera 114A, infrared emitter 215, and infrared camera 220), an image - display driver 942, a user - input device 991, and a memory 934. As Figure 10 shown, the processor 1030 of the mobile device 990 can be coupled to a depth - capture camera 1070, an image - display driver 1090, a user - input device 1091, and a memory 1040A. Since the processor 932 of the head - mounted device 100 executes photo - filter (e.g., artistic / stylized painting) light - field effect programming 945 in the memory 934, the head - mounted device 100 can perform all or a subset of any of the following functions. As a result of the processor 1030 of the mobile device 990 executing photo - filter (e.g., artistic / stylized painting) light - field effect programming 945 in the memory 1040A, the mobile device 990 can perform all or a subset of any of the following functions. The functions can be divided in a photo - filter (e.g., artistic / stylized painting) light - field effect system 900 such that, on the one hand, the head - mounted device 100 generates the original images 858A - B, and on the other hand, the mobile device 990 performs the remaining part of the image processing on the original images 858A - B to generate photo - filter (e.g., artistic / stylized painting) light - field effect images 964A - N.

[0091] In one example, the depth - capture camera of the head - mounted device 100 includes at least two visible - light cameras, which are composed of a left - hand visible - light camera 114A with a left field of view 111A and a right - hand visible - light camera 114B with a right field of view 111B. The left field of view 111A and the right field of view 111B have an overlapping field of view 813 (see Figure 8B ). The depth - capture camera 1070 of the mobile device 990 can have a similar construction.

[0092] The execution of photo - filter (e.g., artistic / stylized painting) light - field effect programming 945 by the processors 932, 1030 configures the photo - filter (e.g., artistic / stylized painting) light - field effect system 900 to perform functions, including capture functions, via the depth - capture camera, the left - hand original image 858A, and the right - hand original image 858B. The photo - filter (e.g., artistic / stylized painting) light - field effect system 900 is used to calculate: (i) a left - hand image disparity map 960A between the left - hand pixel matrix and the right - hand pixel matrix, and (ii) a right - hand image disparity map 960B between the right - hand pixel matrix and the left - hand pixel matrix. The left - hand original image 858A or the left - hand processed image 965A includes the left - hand pixel matrix, while the right - hand original image 858B or the right - hand processed image 965B includes the right - hand pixel matrix.

[0093] The photo filter (e.g., artistic / stylized painting) light field effect system 900 presents the original image 957A through the image displays 180A-B, 1080. The photo filter (e.g., artistic / stylized painting) light field effect system 900 receives the artistic effect selection 962a (based on the marking 962b from the user) through the user input devices 991, 1091 to be applied to the presented original image 957A. The photo filter (e.g., artistic / stylized painting) light field effect system 900 can create at least one stylized painting effect image with a stylized painting effect scene by applying the stylized painting effect selection 962 (based on the marking 962b from the user) to: (i) the left original image 858A or the left processed image 965A to create the left stylized painting effect image 963A, (ii) the right original image 858B or the right processed image 965B to create the right stylized painting effect image 963B, or (iii) a combination of the above two.

[0094] The photo filter (e.g., artistic / stylized painting) light field effect system 900 can generate the stylized painting effect image 964A with an appearance of spatial movement or rotation around the stylized painting effect scene of at least one stylized painting effect image. This function is achieved by mixing the left stylized painting effect image 963A and the right stylized painting effect image 963B based on the left image disparity map 960A and the right image disparity map 960B. The photo filter (e.g., artistic / stylized painting) light field effect system 900 presents the stylized painting effect image 964A through the image displays 180A-B, 1080.

[0095] The function for calculating the left image disparity map 960A and the right image disparity map 960B includes the following. First, create the left corrected image 965A as the left processed image 965A from the left original image 858A, and create the right corrected image 965B as the right processed image 965B from the right original image 858B to align the left and right original images 858A-B and remove the distortion in the lenses of the left and right visible light cameras 114A-B respectively. Second, extract the left image disparity map 960A and the right image disparity map 960B by correlating the pixels in the left corrected image 965A and the right corrected image 965B, and vice versa, to calculate the disparity of each correlated pixel.

[0096] The function for generating the stylized painting effect image 964A includes the following functions. First, determine the horizontal position movement parameter 966 along the X-axis of the left and right pixel matrices. Second, fill the left interpolated pixel matrix 967A by moving the pixels in the left pixel matrix along the X-axis based on the horizontal movement parameter 966. Third, fill the right interpolated pixel matrix 967B by moving the pixels in the right pixel matrix along the X-axis based on the horizontal movement parameter 966. Finally, create the stylized painting effect image 964A by blending the left interpolated pixel matrix 967A and the right interpolated pixel matrix 967B.

[0097] The function for filling the left interpolated pixel matrix 967A includes the following functions. First, multiply the left image disparity corresponding to the left image disparity map 960A of each corresponding pixel in the left pixel matrix by the horizontal movement parameter 966 to derive the corresponding left moving X-axis position coordinates 968A-N. Second, in the left interpolated pixel matrix 967A, move each corresponding pixel to the corresponding left moving X-axis position coordinates 968A-N.

[0098] The function for filling the right interpolated pixel matrix 967B includes the following functions. First, multiply the right image disparity corresponding to the right image disparity map 960B of each corresponding pixel in the right pixel matrix by the complement of the horizontal movement parameter 966 to derive the corresponding right moving X-axis position coordinates 969A-N. For example, the complement of the horizontal movement parameter 966 is the number 1 minus the horizontal movement parameter 966 (i.e., 1 - x). Second, move each corresponding pixel to the corresponding right moving X-axis position coordinates 969A-N in the right interpolated pixel matrix 967B.

[0099] The function for generating the stylized painting effect image 964A by blending the left interpolated pixel matrix 967A and the right interpolated pixel matrix 967B can be based on the disparity confidence level, gradient, or a combination of both in the left image disparity map 960A and the right image disparity map 960B. For example, the disparity confidence level value is based on the magnitude of the correlation between the left and right pixels.

[0100] The function for determining the horizontal position movement parameter 966 includes the following functions. First, a two-dimensional input selection 973 of the presented original image 957A is received from the user via the user input devices 991, 1091. Second, the movement of the two-dimensional input selection 973 from the initial contact point to the final contact point on the presented original image 957A is tracked via the user input devices 991, 1091. Third, a rotation matrix 974 describing the rotation from the initial contact point to the final contact point is determined, and the horizontal position movement parameter 966 is derived. Nevertheless, it should be understood that the rotation matrix 974 is not required in the stylized painting effect unless the data uses 3D vertices to represent. In some examples, the horizontal position movement parameter 966 can also be determined by IMU 972 measurements, for example, using the tilt angle of the mobile device 990 or the eyewear device 100.

[0101] In one example, the user input devices 991, 1091 include a touch sensor, which consists of an input interface and a sensor array and is used to be coupled to the input interface to at least be able to receive a single-finger contact from a user input. The user input devices 991, 1091 also include a sensing circuit integrated into or connected to the touch sensor or connected to the processors 932, 1030. The sensing circuit is configured to measure voltage to at least be able to track a single-finger contact on the user input interface. The function for receiving the stylized painting effect selection 962a and the mark 962b from the user via the user input devices 991, 1091 includes being able to receive at least a single-finger contact from a user input on the input interface of the touch sensor. The function for tracking the movement of the two-dimensional input selection 973 from the initial contact point to the final contact point via the user input devices 991, 1091 includes tracking at least a single-finger contact from the input interface via the sensing circuit, dragging from the initial contact point to the final contact point on the input surface of the touch sensor.

[0102] The touch-based user input device 991 can be integrated into the eyewear device 100. As described above, the eyewear device 100 includes chunks 110A-B integrated into or connected to the frame 105 on the sides 170A-B of the eyewear device 100. The frame 105, the temple arms 125A-B or the chunks 110A-B include a circuit board with a touch sensor. The circuit board includes a flexible printed circuit board. The touch sensor is disposed on the flexible printed circuit board. The sensor array is a capacitive array or a resistive array. The capacitive array or the resistive array includes a grid forming a two-dimensional rectangular coordinate system for tracking the X-axis and Y-axis position coordinates.

[0103] In an example of a photo filter (e.g., artistic / stylized effect) light field effect system 900, the processor consists of a first processor 932 and a second processor 1030. The memory consists of a first memory 934 and a second memory 1040A. The eye - worn device 100 includes a first network communication interface 924 or 936 for communicating via a network 925 or 937 (e.g., a wireless short - range network or a wireless local area network). The first processor 932 is coupled to the first network communication interface 924 or 936. The first memory 934 is accessible to the first processor 932. The eye - worn device 100 also includes a photo filter (e.g., artistic / stylized painting) light field effect program 945 in the first memory 934. The photo filter (e.g., artistic / stylized painting) light field effect program 945, executed by the first processor 932, configures the eye - worn device 100 to perform functions of capturing a left - hand raw image 858A and a right - hand raw image 858B via a depth - capture camera.

[0104] The photo filter (e.g., artistic / stylized painting) light field effect system 900 also includes a host, such as a mobile device 990, coupled to the eye - worn device 100 via a network 925 or 937. The host includes a second network communication interface 1010 or 1020 for communicating via a network 925 or 937. The second processor 1030 is coupled to the second network communication interface 1010 or 1020. The second memory 1040A is accessible to the second processor 1030. The host also includes a photo filter (e.g., artistic / stylized painting) light field effect program 945 in the second memory 1040A.

[0105] The photo filter (e.g., artistic / stylized painting) light field effect programming 945 is executed by the second processor 1030 to configure the host to execute functions for receiving the original image 957A from the eyewear device 100 via the network 925 or 937 through the second network communication interface 1010 or 1020. The photo filter (e.g., artistic / stylized painting) light field effect programming 945 is executed by the second processor 1030 to configure the host to calculate: (i) the left image disparity map 960A, and (ii) the right image disparity map 960B. The photo filter (e.g., artistic / stylized painting) light field effect programming 945 is executed by the second processor 1030 to configure the host to present the original image 957A through the image display 1080. The photo filter (e.g., artistic / stylized painting) light field effect programming 945 is executed by the second processor 1030 to configure the host to receive from the user a stylized painting effect selection 962a and a marker 962b through the user input device 1091 (e.g., touch screen or computer mouse). The photo filter (e.g., stylized painting) light field effect programming 945 is executed by the second processor 1030 to configure the host to create a stylized painting effect image 963A-B by applying the style of the selected image to the marker. The photo filter (e.g., stylized painting) light field effect programming 945 is executed by the second processor 1030 to configure the host to generate a stylized painting effect image 964A with a rotating or spatial motion appearance. The photo filter (e.g., stylized painting) light field effect programming 945 is executed by the second processor 1030 to configure the host to present the stylized painting effect image 964A through the image display 1080.

[0106] Each depth image 961A-N is formed by a vertex matrix 970. Each pixel of the two-dimensional images 858A-B, 963A-B, 964A-N, 965A-B can be associated with respective vertices of the vertex matrix 970. Each vertex has a position attribute. The position attribute of each vertex is based on a three-dimensional position coordinate system, including an X position coordinate on the X-axis (representing a horizontal position), a Y position coordinate on the Y-axis (representing a vertical position), and a Z position coordinate on the Z-axis (representing a depth position). Each vertex also includes one or more of a color attribute, a texture attribute, or a reflection attribute. Thus, the function of applying the stylized painting effect selection 962a (based on the mark 962b from the user and the style transfer to the marked area) to: (i) the left original image 858A or the left processed image 965A to create the left stylized painting effect image 963A, (ii) the right original image 858B or the right processed image 965B to create the right stylized painting effect image 963B, or (iii) a combination of both, is based on the Z position coordinate to change the filter effect intensity of the stylized painting effect function 971 according to the depth position of each vertex associated with each pixel for transforming each pixel. On the Z-axis, the intensity of applying the filter effect is stronger on the vertices with Z position coordinates of a deeper depth position compared to those with a shallower depth position.

[0107] The server system 998 can be one or more computing devices that are part of a service or network computing system (e.g., including a processor, a memory, and a network communication interface) to communicate with the mobile device 990 and the eye-wear device 100 via the network 995. The eye-wear device 100 is connected to a host. For example, the eye-wear device 100 is paired with the mobile device 990 via a high-speed wireless connection 937, or is connected to the server system 998 via the network 995.

[0108] By generating multiple (e.g., ten 964A-J) views between the left and right images 858A-B, 965A-B for a particular moment, a raster print can be made from the generated photo filter (e.g., artistic / stylized painting) light field effect images 964A-N. The multi-views (each view corresponding to the generated photo filter stylized painting effect images 964A-J) are printed in strips. The raster is made of plastic and is glued to the raster printing device with a half-tube, acting like a lens. When the observer views with both eyes, each eye will see a different set of stripes and thus different images 964A-J. By forming multiple (ten) views 964A-J and gluing the raster on top, a stereoscopic raster print can be created to provide a 3D appearance. Moving the raster print provides the effect of different viewpoints, thus obtaining a short animation within the raster print. A photo printing service (implemented by a host, such as server system 998 or mobile device 990) can receive multiple generated photo filter (e.g., artistic / stylized painting) light field effect images 964A-N via networks 925, 937, 995, which can be printed as a raster print (e.g., using a 3D printer). In some examples, the raster print can splice the photo filter (e.g., artistic / stylized painting) light field effect images 964A-N together in sequence to form a short video.

[0109] For example, create N viewpoints, 0.1, 0.2, up to 1, to generate ten views corresponding to ten photo filter (e.g., artistic / stylized painting) light field effect images 964A-J. The printer sequence is as follows: the first column of the first view 964A, the first column of the second view 964B, and so on until 964J; then the next pixel column, i.e., the second column of the first view 964A, the second column of the second view 964B, and so on until 964J. Glue a raster on the printed matter, whereby each image up to 964A-J can be seen in different angular directions. When the user rotates the raster print, the ten different views rotate relative to each other until 964A-J.

[0110] The output component of the eye-worn device 100 includes visual components, such as Figure 2B- The left and right image displays of the optical component 180A-B shown in FIG. C (e.g., a liquid crystal display (LCD), a plasma display panel (PDP), a light emitting diode (LED) display, a projector, or a waveguide, etc.). The image displays of the optical component 180A-B are driven by an image display driver 942. The output components of the eyewear device 100 further include an acoustic component (e.g., a speaker), a haptic component (e.g., a vibration motor), other signal generators, etc. The input components of the eyewear device 100, the mobile device 990, and the server system 998 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), a point-based input component (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing devices), a haptic input component (e.g., a physical button, a touch screen that provides touch position and force or touch gestures, or other haptic input components), an audio input component (e.g., a microphone), and so on.

[0111] The eyewear device 100 may optionally include additional peripheral device elements. Such peripheral device elements may include biometric sensors, additional sensors, or display elements integrated with the eyewear device 100. For example, the peripheral device elements may include any input / output components, composed of output components, motion components, position components, indicators, or any other such elements described herein.

[0112] For example, the biometric components include components that can be used to detect expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measure biological signals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identify people (e.g., voice recognition, retina recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. The motion components include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. The position components include a position sensor component for generating position coordinates (e.g., a global positioning system (GPS) receiver component), a WiFi or Bluetooth TM transceiver, a height sensor component (e.g., an altimeter or a barometer that detects air pressure from which height can be derived), a direction sensor component (e.g., a magnetometer), etc. Such positioning system coordinates can also be received from the mobile device 990 via the low-power wireless circuit 924 or the high-speed wireless circuit 936 through the wireless connections 925 and 937.

[0113] The inertial measurement unit (IMU) 972 is an electronic device that measures and reports specific forces, angular rates of the body, and sometimes also measures and reports the magnetic field around the body, using a combination of accelerometers and gyroscopes, and sometimes also a magnetometer. If it is a magnetometer, the magnetic field can be used as an input to detect specific postures that rely on the Earth's magnetic field or an artificial magnetic field. In this example, the inertial measurement unit is used to determine the rotational acceleration of the eye-wear device 100. The inertial measurement unit 972 works by using one or more accelerometers to detect linear acceleration and one or more gyroscopes to detect rotational speed. A typical configuration of an inertial measurement unit includes an accelerometer, a gyroscope, and a magnetometer for each of the three axes: a horizontal axis (X) for left-right movement, a vertical axis (Y) for top-bottom movement, and a depth axis or distance axis (Z) for up-down movement. The gyroscope detects the rotational speed around the three axes (X, Y, and Z). The magnetometer is used to detect the magnetic field (e.g., southward, northward, etc.), similar to a compass, and its principle is to generate a heading reference, which is a mixture of the Earth's magnetic field and other artificial magnetic fields (such as the magnetic field generated by a wire). The three accelerometers detect acceleration along the horizontal (X), vertical (Y), and depth or distance (Z) axes defined above, and these axes can be defined relative to the ground, the eye-wear device 100, the depth capture camera, or the user wearing the eye-wear device 100. Therefore, the accelerometers are used to detect a three-axis acceleration vector, which can then be used to detect the Earth's gravity vector.

[0114] Figure 10 is a high-level functional block diagram of an example of the mobile device 990, which communicates through Figure 9 the photo filter (e.g., stylized painting) light field effect system 900 shown in the figure. The mobile device 990 includes a user input device 1091 for receiving a photo filter (e.g., stylized painting) selection 962a, a marker 962b, or a two-dimensional input selection 973, which is applied to the original image 957A to generate a photo filter (e.g., stylized painting) light field effect image 964A.

[0115] The mobile device 990 includes a flash memory 1040A, which contains photo filter (e.g., art / stylized painting) light field effect programming 945 for performing all or a subset of the functions of generating a photo filter (e.g., art / stylized painting) light field effect described herein, where the photo filter selection 962a (based on the marker 962b from the user and the style transfer of the region) is applied to the original image 858A-B or the processed image 965A-B to generate a photo filter stylized painting effect image 964A-N.

[0116] As shown in the figure, the memory 1040A further includes a left original image 858A captured by the left visible light camera 114A, a right original image 858B captured by the right visible light camera 114B, and an infrared image 859 captured by the infrared camera 220 of the depth sensor 213. The mobile device 1090 may include a depth capture camera 1070, which is composed of at least two visible light cameras (a first and a second visible light camera with overlapping fields of view), or at least one visible light camera, and a depth sensor with a substantially overlapping field of view (such as the eyewear device 100). When the mobile device 990 includes components such as the eyewear device 100 (e.g., a depth capture camera), the left original image 858A, the right original image 858B, and the infrared image 859 can be captured by the depth capture camera 1070 of the mobile device 990.

[0117] The memory 1040A further includes a plurality of depth images 961A-N, which are generated by the depth capture camera of the eyewear device 100 or by the depth capture camera 1070 of the mobile device 990 itself. Figure 11A and 11B The figure shows an overview flowchart of functions that can be implemented in the photo filter (e.g., artistic / stylized painting) light field effect programming 945. The memory 1040A further includes a two-dimensional input selection 973, such as an initial contact point and a final contact point received by the user input device 1091. The memory 1040A further includes: a left image disparity map 960A, a right image disparity map 960B, photo filter (e.g., stylized painting effect) images 963A-N, a horizontal position parameter 966, a left interpolation pixel matrix 967A including left moving X-axis position coordinates 968A-N, a right interpolation pixel matrix 967B of right moving X-axis position coordinates 969A-N, and a left processed (e.g., corrected) image 965A and a right processed (e.g., corrected) image 965B (e.g., removing vignetting towards the lens end). As further shown, the memory 1040A includes a vertex matrix 970 and a rotation matrix 974. During the image processing of the original images 858A-B, some or all of the information stored in the memory 1040A can be generated to generate corresponding photo filter (e.g., artistic / stylized painting) light field effect images 964A-N.

[0118] As shown in the figure, the mobile device 990 includes an image display 1080, an image display driver 1090 for controlling the image display, and a user input device 1091 similar to the eyewear device 100. In Figure 10 the example shown, the image display 1080 and the user input device 1091 are integrated into a touch screen display.

[0119] Examples of touchscreen-enabled mobile devices include, but are not limited to, smartphones, personal digital assistants (PDAs), tablet computers, laptop computers, or other portable devices. However, the structure and operation of touchscreen-enabled devices are provided by way of example, and the subject technology described herein is not limited thereto. Thus, for the purposes of this discussion, Figure 10 A block diagram illustration of an exemplary mobile device 990 is provided, the exemplary mobile device 990 being equipped with a touchscreen display for displaying content and receiving user input as a user interface (or as part thereof).

[0120] The key activities discussed herein typically involve applying a photo filter selection 962a from a user (based on a user marker 962b and style transfer to the marked area) to an original image 858A-B or a processed image 965A-B to generate data for a photo filter stylized painting effect image 964A-N for a data communication portable eyewear device 100 or a mobile device 990. As Figure 10 shown, the mobile device 990 includes at least one digital transceiver (XCVR) 1010, shown as a WWAN XCVR, for digital wireless communication via a wide area wireless mobile communication network. The mobile device 990 also includes additional digital or analog transceivers, such as a short-range XCVR 1020 for short-range network communication, for example by means of NFC, VLC, DECT, ZigBee, Bluetooth TM , or WiFi. Taking the short-range XCVR 1020 as an example, it may take the form of any available bi-directional wireless local area network (WLAN) transceiver, the type of which is compatible with one or more standard communication protocols implemented in the wireless local area network, such as a Wi-Fi standard according to IEEE 802.11 and WiMAX.

[0121] To generate location coordinates for positioning the mobile device 990, the mobile device 990 may include a Global Positioning System (GPS) receiver. As an alternative or in addition, the mobile device 990 may utilize one or both of the short-range XCVR 1020 and the WWAN XCVR 1010 to generate location coordinates for positioning. For example, a GPS-based positioning system, a cellular network, WiFi, or Bluetooth TM can generate very accurate location coordinates, especially when used in combination. Such location coordinates can be transmitted to the eyewear device via one or more network connections utilizing XCVR1010, 1020.

[0122] The transceivers 1010, 1020 (network communication interfaces) conform to one or more of the various digital wireless communication standards used in modern mobile networks. Examples of the WWAN transceiver 1010 include (but are not limited to) transceivers configured to operate according to Code Division Multiple Access (CDMA) and Third Generation Partnership Project (3GPP) network technologies, including but not limited to 3GPP2 type (or 3GPP2) and LTE, sometimes referred to as "4G". Taking the transceivers 1010, 1020 as an example, they can provide two-way wireless communication of information, including digital audio signals, static images and video signals, web page information for display and input related to web pages, as well as various types of mobile message communication between the mobile device 990 for photo filters (e.g., artistic / stylized paintings) light field effects.

[0123] As mentioned above, several types of communication through the transceivers 1010, 1020 and the network involve protocols and processes that support communication between the eye-wear device 100 or the server system 998 that generate photo filter (e.g., artistic / stylized painting) light field effect images 964A-N, such as sending the left original image 858A, the right original image 858B, the infrared image 859, the depth images 961A-N, the photo filter images 963A-B and the processed (e.g., corrected) images 965A-B. Taking these communications as an example, packet data can be transmitted to the eye-wear device 100 or from the eye-wear device 100 via the short-range XCVR 1020 through the wireless connections 925 and 937, as Figure 9 shown. In addition, such communications can also be transmitted via the network (e.g., the Internet) 995 shown in Figure 9 using IP packet data transmission via the WWAN XCVR 1010. Both the WWAN XCVR 1010 and the short-range XCVR 1020 are connected to the relevant antennas (not shown) through radio frequency (RF) transmit and receive amplifiers (not shown).

[0124] The mobile device 990 also includes a microprocessor, shown in the figure as CPU 1030, which is sometimes referred to as the host controller in this article. The processor is a circuit whose elements are constructed and arranged to perform one or more processing functions, usually various data processing functions. Although discrete logic components can be used, components that make up a programmable CPU are used in the example. For example, the microprocessor includes one or more integrated circuit (IC) chips that integrate electronic elements for performing CPU functions. Taking the processor 1030 as an example, it can be based on any known or available microprocessor architecture, such as Reduced Instruction Set Computing (RISC) using the ARM architecture, which is very commonly used in today's mobile devices and other portable electronic devices. Other processor circuits can be used to form the CPU 1030 or the processor hardware in smartphones, laptops and tablets.

[0125] The microprocessor 1030 acts as a programmable host controller for the mobile device 990 by configuring the mobile device 990 to perform various operations (e.g., according to instructions or programming executable by the processor 1030). For example, such operations can include various general operations of the mobile device, as well as operations related to photo filter (e.g., artistic / stylized painting) light field effect programming 945 and communication with the eyewear device 100 and the server system 998. Although the processor can be configured by employing hardwired logic, typical processors in mobile devices are general processing circuits configured by executing programming.

[0126] The mobile device 990 includes a memory or storage device system for storing data and programming. In an example, the memory system can include a flash memory 1040A and a random access memory (RAM) 1040B. The RAM 1040B serves as a short-term memory for instructions and data processed by the processor 1030, e.g., as a working data processing memory. The flash memory 1040A generally provides longer-term storage.

[0127] Thus, in the example of the mobile device 990, the flash memory 1040A is used to store the programming or instructions executed by the processor 1030. Depending on the device type, the mobile device 990 can store and run a mobile operating system, through which specific applications are executed, including photo filter (e.g., artistic / stylized painting) light field effect programming 945. Taking the photo filter (e.g., artistic / stylized painting) light field effect programming 945 application as an example, it can be a native application, a hybrid application, or a web application running on the mobile device 990 (e.g., a dynamic web page executed by a web browser) to generate photo filter (e.g., stylized painting) light field effect images 964A-N according to the received photo filter (e.g., stylized painting) selections 962a and markings 962b. Examples of mobile operating systems include Google Apple ( or device), Windows Amazon OS, RIM operating systems, etc.

[0128] The mobile device 990 should be understood as just one type of host in the photo filter (e.g., artistic / stylized painting) light field effect system 900, and other arrangements can also be adopted. Taking Figure 9 the shown server system 998 as an example, after generating the original images 858A-B, it can generate photo filter (e.g., artistic / stylized painting) light field effect images 964A through the depth capture camera of the eyewear device 100.

[0129] Figure 11A It is a flowchart of a method with specific steps that can be implemented in a photo filter (e.g., artistic / stylized painting) light field effect system 900 for applying a photo filter selection 962a from a user (based on user markings 962b and style transfer of the marked area) to an original image 858A - B or a processed image 965A - B to generate a photo filter (e.g., artistic / stylized painting) light field effect image 964A - N. In an example of a depth capture camera with two visible light cameras, the left and right original images are used. In an example of a depth capture camera with one visible light camera and one infrared camera, the left or right original image from the visible light camera (depending on which side the visible light camera is on) is used, and a right or left processed image is generated from the left or right visible original image using an offset based on distance information obtained from the infrared camera, and the process is performed using the original image and the processed image.

[0130] Taking the method shown in block 1100 as an example, it includes capturing a left original image 858A and a right original image 858B through a depth capture camera. For example, a left visible light camera 114A and a right visible light camera 114B respectively capture the left original image 858A and the right original image 858B. In another example, the method used includes capturing a left original image 858A or a right original image 858B through a depth capture camera, and generating a right processed image 965B or a left processed image 965A using an offset based on distance information obtained from the infrared camera. According to this example, in the following steps, such an offset is used to generate a processed image on one side from the original image on the other side instead of the original image.

[0131] Continuing the discussion of block 1110, the method shown also includes calculating: (i) a left image disparity map 960A between a left pixel matrix and a right pixel matrix, and (ii) a right image disparity map 960B between a right pixel matrix and a left pixel matrix. The left pixel matrix is based on the left original image 858A or the left processed image 965A. The right pixel matrix is based on the right original image 858B or the right processed image 965B.

[0132] Calculating the left image disparity map 960A and the right image disparity map 960B includes the following steps. First, a left corrected image 965A is created from the left original image 858A as the left processed image 965A, and a right corrected image 965B is created from the right original image 858B as the right processed image 965B to align the left and right original images 858A - B and remove the distortion in the lenses of the left and right visible light cameras 114A - B respectively. Second, the left image disparity map 960A and the right image disparity map 960B are extracted by correlating the pixels in the left corrected image 965A and the right corrected image 965B, and vice versa, to calculate the disparity of each correlated pixel (e.g., using SGBM).

[0133] Apply correction to modify each captured image or video so that the corresponding pixels are on the same raster line (row). Once done, an image disparity calculation algorithm such as SGBM can be applied. The disparity calculation algorithm finds the corresponding pixel for each pixel in the right image and the left image. For each pixel in the right image, the corresponding pixel is found in the left image. For non - occluded pixels (pixels visible from both cameras), the same disparity is usually found in the order from left to right and from right to left; on the other hand, occluded pixels need to be processed separately, usually using adjacent pixel blending techniques.

[0134] The method shown in block 1120 also includes presenting the original image 957A through the image displays 180A - B, 1080. The original image 957A is based on the left original image 858A, the left processed image 965A, the right original image 858B, the right processed image 965B, or a combination thereof.

[0135] The method shown in block 1130 also includes receiving a photo filter selection 962a from the user through the user input devices 991, 1091 and applying it to the presented original image 957A.

[0136] The method shown in block 1135 also includes creating a photo filter (e.g., a stylized painting) image with a photo filter effect scene. In one example, marks on the original image 957A are received through the user input devices 991, 1091 (block 1135a; Figure 11B ). The marks define the image regions to which the image style is applied, and they can be used to blank the image and store it in the memory. Subsequently, the style of the image is applied to the marks (block 1135b; Figure 11B ). The user can select a target image from a pre - selected list of images, an image found on the Internet, an image stored on the mobile device, or an image captured by the mobile device through the user input devices 991, 1091. The style of the image can be transferred to the marks through NST.

[0137] The method shown in block 1140 also includes creating photo filter images in response to a user's selection of: (i) the left original image 858A or the left processed image 965A for creating the left photo filter image 963A, (ii) the right original image 858B or the right processed image 965B for creating the right photo filter image 963B, or (iii) a combination of both. The photo filter images can be overlaid on the original image to replace the marked area (see Figure 12C ), or can be blended with the pixels from the original image in the marked area so that features such as shadows pass through (see Figure 13 ).

[0138] Continuing with block 1150, the method shown also includes generating a photo filter stylized painting effect image 964A having an appearance of spatial movement or rotation around at least one photo filter scene in the photo filter stylized painting effect images 963A - B. This can be achieved by blending the left photo filter image 963A and the right photo filter image 963A, based on the left image disparity map 960A and the right image disparity map 960B. Generating the photo filter stylized painting effect image 964A includes the following steps. First, determine the horizontal position movement parameter 966 along the X-axis of the left and right pixel matrices. Second, fill the left interpolated pixel matrix 967A by moving the pixels in the left pixel matrix along the X-axis based on the horizontal movement parameter 966. Third, fill the right interpolated pixel matrix 967B by moving the pixels in the right pixel matrix along the X-axis based on the horizontal movement parameter 966. Finally, create the photo filter stylized painting effect image 964A by blending the left interpolated pixel matrix 967A and the right interpolated pixel matrix 967B.

[0139] The step of filling the left interpolated pixel matrix 967A includes the following functions. First, multiply the left image disparity corresponding to the left image disparity map 960A of each corresponding pixel in the left pixel matrix by the horizontal movement parameter 966 to derive the corresponding left moving X-axis position coordinates 968A - N. Second, in the left interpolated pixel matrix 967A, move each corresponding pixel to the corresponding left moving X-axis position coordinates 968A - N.

[0140] The operation of filling the right interpolated pixel matrix 967B includes the following steps. First, multiply the right image disparity in the right image disparity map 960B of each corresponding pixel in the right pixel matrix by the complement of the horizontal movement parameter 966 (e.g., subtract the horizontal movement parameter 966 from the number 1; i.e., 1 - x) to derive the corresponding right moving x-axis position coordinates 969A - N. Second, move each corresponding pixel to the corresponding right moving x-axis position coordinates 969A - N in the right interpolated pixel matrix 967B.

[0141] Once two disparity maps (a left image disparity map 960A and a right image disparity map 960B) are created, the horizontal movement parameter 966 will move between 0 and 1 to set or tilt the spatial movement or rotation of the generated photo filter (e.g., artistic / stylized painting) light field effect image 964A. Assuming that when the horizontal movement parameter 966 is set to 0, it is tilted completely towards the left image, then when the horizontal movement parameter 966 is set to 1, it will be tilted completely towards the right image. If the horizontal movement parameter 966 is set to 0, the weight is set to output the left image as the photo filter (e.g., artistic / stylized painting) light field effect image 964A. If the horizontal movement parameter 966 is set to 1, the weight is set to output the right image as the photo filter (e.g., artistic / stylized painting) light field effect image 964A. When the photo filter (e.g., artistic / stylized painting) light field effect image 964A is not equal to 0 or 1 (intermediate value), the spatial movement or rotation is between the left and right images. For the horizontal movement parameter 966 set to 0.5, the empty interpolation pixel matrix 967A-B is filled with RGB values to derive the intermediate photo filter (e.g., artistic / stylized painting) light field effect image 964A-N. For the left interpolation pixel matrix 967A, since the horizontal movement parameter 966 is set to 0.5, according to the corresponding disparity values from the left image disparity map 960A, the pixels in the left image are moved to half of the corresponding pixels in the right image. For example, each disparity value from the left image disparity map 960A is multiplied by 0.5 and added to the X-axis position coordinate to derive the left-shifted X-axis position coordinate 968A. The right interpolation pixel matrix 967B is filled in the same way by moving the pixels in the right image to half of the corresponding pixels in the left image according to the corresponding disparity values from the right image disparity map 960B. For example, each disparity value from the right image disparity map 960B is multiplied by 0.5 and added to the X-axis position coordinate to derive the right-shifted X-axis position coordinate 969A. Therefore, for each pixel, the color value remains unchanged, but the X-axis position coordinate is moved by half of the disparity value on the X-axis. If a pixel has no value (is occluded), but the adjacent pixels have values, the pixel value of the occluded pixel is calculated based on the weighted adjacent pixels and the disparity confidence level.

[0142] In another example, assume that the horizontal movement parameter 966 is set to 0.1. To fill the left interpolation pixel matrix 967A, the following calculation is used: for each left pixel in the left image, multiply the corresponding disparity value from the left image disparity map 960A by 0.1 to derive the corresponding left movement X-axis position coordinates 968A-N. To fill the right interpolation pixel matrix 967B, the following calculation is adopted: for each right pixel in the right image, multiply the corresponding disparity value in the right image disparity map 960B by 0.9 to obtain the corresponding right movement X-axis position coordinates 969A-N. This will create a new view between the left image and the right image.

[0143] The step of generating a photo filter (e.g., artistic / stylized painting) light field effect image 964A is achieved by blending the left interpolation pixel matrix 967A and the right interpolation pixel matrix 967B. This blending is based on the disparity confidence levels (e.g., by weighting the contribution values on each side), gradients, or a combination thereof in the left image disparity map 960A and the right image disparity map 960B. Taking the disparity confidence level value as an example, it is based on the magnitude of the correlation between the left and right pixels. Although it may be desirable to obtain the same image, due to differences in reflections, lighting, etc. caused by different perspectives in the left image and the right image, the combined photo filter (e.g., artistic / stylized painting) light field effect image 964A is not the same (hence, the name stylized painting effect). This will create a photo filter (e.g., artistic / stylized painting) light field effect image 964A using the new view.

[0144] When generating a photo filter (e.g., artistic / stylized painting) light field effect image 964A, actual distance or depth rotation is not used, nor are 3D vertices employed. Instead, disparity is used, which is related to depth, but disparity does not directly replace depth. In fact, disparity is the movement of pixels, which means that image processing can be completed in a 2D space to speed up the running speed and reduce memory requirements. Without any 3D transformation, only the interpolation between corresponding pixels and corresponding pixels is provided. Although the correspondence (disparity) can be converted to depth (distance), the photo filter (e.g., artistic / stylized painting) light field effect does not require depth. Whether the depth on the Z-axis is 10 meters or 20 meters is not important because the pixels are moved to different X-axis position coordinates based on the horizontal movement parameter 966.

[0145] Turning to block 1160, the method further includes presenting, via image displays 180A-B, 1080, a photo filter stylized painting effect image 964A. In some examples, the steps of capturing a left original image 858A and a right original image 858B via a depth capture camera are implemented on the eye-worn device 100. The computing steps include: (i) a left image disparity map 960A, and (ii) a right image disparity map 960B; presenting the original image 957A via the image display 1080; receiving a photo filter effect selection 962a via the user input device 1091; creating a photo filter image 963A-B; generating a photo filter stylized painting effect image 964A; and presenting, via the image display 1080, the photo filter stylized painting effect image 964A on the hosts 990, 998.

[0146] Applying the stylized painting effect selection 962a from the user to: (i) the left original image 858A or the left processed image 965A to create a left stylized painting effect image 963a, (ii) the right original image 858B or the right processed image 965B to create a right stylized painting effect image 963B, or (iii) a combination of both, which may be based on the Z position coordinates. This can vary the filter effect intensity of the stylized painting effect function 971 to transform each pixel according to the depth position of the respective vertices associated with each pixel. Along the Z-axis, the intensity of the filter effect applied to the respective vertices with a deeper depth position Z coordinate is more intense compared to those with a shallower depth position.

[0147] Having completed the discussion of the present - complete box 1170, the method may further include generating a raster print from views of light - field effect images 964A - N of multiple photo filters (e.g., artistic / stylized paintings). The raster print is used to create two 3D images from the left - hand image and the right - hand image and all the intermediate images. In one example, 15 different views 964A - O can be combined together. Then, when the raster print is moved around, a hologram - like image appears. Using the stylized - painting effect images, various views of the stylized - painting effect images can be printed to provide a holographic (moving - image) experience. To achieve the migration of the raster print, for example, by generating and then printing fifteen end - views 964A - O, each pixel of the raster print can take the first pixel of the first stylized - painting effect image, the first pixel of the second stylized - painting effect image, until the first pixel of the nth stylized - painting effect image. Next, the second pixel of the first stylized - painting effect image, the second pixel of the second stylized - painting effect image, until the second pixel of the nth stylized - painting effect image are printed. This provides the stripes of all N images. When making the raster print, the lens directs the light of each stripe directly into the viewer's eyes. When viewing each image through the raster at the top, the complete stylized - painting effect image from that single view can be observed. But when the raster print is moved, the lens causes each eye to see a different image because the light rays point in different directions. When the viewer looks with both eyes, two different views appear, thus providing a 3D experience.

[0148] Figure 12A An example of the first presented original image 957A is shown, which is the processed (e.g., corrected) image 965A. The first presented original image 957A includes various two - dimensional pixels having X and Y position coordinates on the X - axis 1205 and Y - axis 1210.

[0149] Figure 12B Shown in Figure 12A is an example of an image 1202 having a marker 962b on the first presented original image 957A. The marker 962b determines the area where the stylized painting is applied in the image 1202. Through the user - input device 991, the user can input a marker (e.g., draw with a finger on the display). The marker 962b can be recorded on a blank image and stored in the memory.

[0150] Figure 12C The example of Figure 12A shows the first presented original image 957A and Figure 12BAn image 1204 of a photo filter (e.g., a stylized painting effect) created by the mark 962b. As shown, applying the user's selection 962a of the stylized painting effect to the area of the first presented original image 957A defined by the user mark 962b is based on the first photo filter (e.g., stylized effect) function 971A, which converts each pixel of the first presented original image 957A in the area defined by the mark 962b to create a stylized painting effect scene. The photo filter (e.g., stylized painting effect) function 971 transfers the style from paintings such as Edvard Munch's "The Scream" in 1883 to the area as a stylized painting effect scene.

[0151] Figure 12D An example shows a first photo filter (e.g., stylized painting effect) light field effect image 1206 generated from Figure 12C the photo filter (e.g., stylized painting effect) image 1204, where the spatial movement or rotation is tilted to the left.

[0152] Figure 12E An example shows a first photo filter (e.g., stylized painting effect) light field effect image 1208 generated from Figure 12C the photo filter (e.g., stylized painting effect) image 1204, where the spatial movement or rotation is tilted to the right.

[0153] The left and right disparity maps are calculated from the original RGB image. To obtain a stylized painting effect that rotates around the stylized painting image to have a spatial movement, two modified images can be blended, or a modified and an unmodified RGB image can be blended. When modifying two or more corresponding pixels in the left and right images, the disparity of the unmodified image, i.e., the disparity pre-calculated based on the unmodified image, is used to achieve the stylized painting effect.

[0154] Another example in FIG. 12 shows a photo filter (e.g., stylized painting effect) image 1300 created according to the first presented original image and the mark with the stylized painting 963a. The mark includes two arrows (a straight arrow and a curved arrow). The photo filter image 1300 is blended with the background such that the shadows 103 from, for example, a person 1304 and a structure 1306 are retained.

[0155] As previously described, any of the photo filter (e.g., stylized painting) light field effect functions described herein for the eye-mounted device 100, the mobile device 990, and the server system 998 can be embodied in one or more applications. According to some embodiments, "function (one)", "function (multiple)", "application (one)", "application (multiple)", "instruction (one)", "instruction (multiple)", or "programming" is a program that executes the functions defined in the program. Various programming languages ​​can be used to create one or more applications constructed in various ways, such as an object-oriented programming language (e.g., Objective-C, Java, or C++) or a programming language (e.g., C or assembly language). In a specific example, a third-party application (e.g., an application created by an entity other than a vendor of a specific platform using ANDROID TM or IOS TM Software Development Kit (SDK) applications can be developed on IOS TM ANDROID TM , Mobile software running on a mobile operating system such as iOS or other mobile operating systems such as Android phones. In this example, the third-party application can call API calls provided by the operating system to facilitate the functions described in this article.

[0156] Therefore, machine-readable media can take the form of tangible storage media in many forms. Taking non-volatile storage media as an example, it includes optical or magnetic disks, such as all storage devices in any computer, etc., which can be used to implement the client devices, media gateways, transcoders, etc. shown in the figure. Volatile storage media include dynamic memory, such as the main memory of a certain computer platform. Tangible transmission media include coaxial cables; copper wires and optical fibers, including wires that constitute the bus within the computer system. The form of carrier transmission media can take the form of electrical or electromagnetic signals, or sound or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Therefore, common forms of computer-readable media include: floppy disks, floppy disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs or DVD-ROMs, any other optical media, punch cards, paper tapes, any other physical storage media with hole patterns, RAM, PROMs and EPROMs, FLASH-EPROMs, any other memory chips or cassettes, carriers that transmit data or instructions, cables or links that transmit such carriers, or any other media from which a computer can read programming code and / or data. Among other things, computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.

[0157] The scope of protection is defined solely by the appended claims. That scope is intended to and shall be construed to be as broad as is reasonable when interpreted in accordance with this specification, and the ordinary meaning of the language used in the claims is to be construed consistent with the following prosecution history and to cover all equivalents of structure and function. No claim is intended to cover, nor should any claim be construed to cover, subject matter that fails to meet the requirements of Sections 101, 102, or 103 of the Patent Act, nor should they be construed in such a way. No inadvertent coverage of such subject matter is claimed herein.

[0158] Except as otherwise noted above, nothing stated or shown is intended or should be construed to result in dedication of any component, step, function, object, benefit, advantage, or equivalent to the public, whether or not recited in the claims.

[0159] It should be understood that, unless otherwise specifically defined herein, the terms and expressions used herein have the ordinary meaning given to such terms and expressions respective to their corresponding fields of investigation. Relative terms such as "first" and "second" may be used solely to distinguish one entity or action from another entity or action, and do not necessarily require or imply any actual such relationship or order between such entities or actions. The terms "comprises," "comprising," "includes," "including," "contains," "containing," or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, includes, or contains a series of elements or steps does not include only those elements or steps but may include other elements or steps not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "a" does not preclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0160] Unless otherwise specified, any and all measurements, values, ratings, positions, magnitudes, dimensions, and the like specifications set forth in this specification (including the following claims) are approximations and not exact values. Such quantities are intended to have a reasonable range consistent with the functions to which they pertain and consistent with the practices in the art to which they belong. For example, unless expressly stated otherwise, parameter values etc. may vary from the stated amounts by ±10%.

[0161] In addition, in the foregoing detailed description, it can be seen that, for the purposes of simplifying the disclosure, various features are combined in various examples. This method of disclosure should not be interpreted as reflecting an intention that the examples claimed require more features than are expressly recited in each claim. On the contrary, as reflected in the following claims, the subject matter to be protected is not in the full features of any single disclosed example. Accordingly, the following claims are hereby incorporated into the detailed description, with each claim standing on its own as a separately claimed subject matter.

[0162] Although the foregoing describes what is considered to be the best mode and other examples, it is understood that various modifications may be made in such modes and examples, that the subject matter disclosed herein may be implemented in various forms and examples, and that such modes and examples may be applied to a variety of applications, only some of which are described herein. The following claims are intended to claim any and all modifications and variations that fall within the true scope of this concept.

Claims

1. A stylized painting effect system for creating an image with a stylized painting effect, the system comprising: An image display for presenting an image, including an original image, wherein the original image is based on a left original image, a left processed image, a right original image, a right processed image, or a combination thereof; A user input device for receiving markings of the original image and a style selection from the user; A processor coupled to the user input device, the processor being configured to perform: Display the original image through the image display; Receive markings and a style selection from the user through the user input device; Create at least one stylized painting effect image using a stylized painting effect scenario; Apply the stylized painting effect image to the following markings: (i) the left original image or the left processed image to create a left stylized painting effect image, (ii) the right original image or the right post-processed image to create a right stylized painting effect image, or (iii) a combination thereof; Generate a stylized painting effect image by blending the left stylized painting effect image and the right stylized painting effect image; and Present the stylized painting effect image through the image display.

2. The system according to claim 1, wherein in order to receive a style selection, the processor is configured to perform: Receive an image selection from the user; and Among them, In order to create at least one stylized painting effect image using a stylized painting effect scenario, the processor is configured to: Apply the image selection to the markings using neural style transfer.

3. The system according to claim 1, wherein in order to create at least one stylized painting effect image, the processor is configured to perform: Blend the stylized painting effect image with the original image in the marked area.

4. The system according to claim 1, wherein the stylized painting effect image has an appearance of spatial movement or rotation of a stylized painting effect scenario around the at least one stylized painting effect image, and wherein the user input device includes: An inertial measurement unit configured to generate spatial movement or rotation around a stylized painting effect scenario of the at least one stylized painting effect image.

5. The system according to claim 1, further comprising: Calculating a left image disparity map and a right image disparity map by: Creating a left corrected image from the left original image as the left processed image, creating a right corrected image from the right original image as the right processed image, aligning the left and right original images, and eliminating distortion of the lenses of the left and right visible light cameras respectively; And Extracting the left image disparity map and the right image disparity map by correlating pixels in the left corrected image with the right corrected image, and vice versa, to calculate the disparity of each correlated pixel.

6. The system according to claim 5, wherein: In order to generate a stylized painting effect image, the processor is configured to perform: Determine a horizontal position movement parameter along the X-axis of the left pixel matrix and the right pixel matrix; Fill the left interpolated pixel matrix by moving pixels in the left pixel matrix along the X-axis based on the horizontal movement parameter; Filling the right interpolated pixel matrix by moving pixels in the right pixel matrix along the X-axis based on a horizontal movement parameter; and Creating a stylized painting effect image by blending the left interpolated pixel matrix and the right interpolated pixel matrix.

7. The system according to claim 6, wherein: Generating a stylized painting effect image by blending the left interpolated pixel matrix and the right interpolated pixel matrix is based on the disparity confidence level, gradient, or a combination of both in the left image disparity map and the right image disparity map.

8. The system according to claim 6, wherein: To determine the horizontal position movement parameter, the processor is configured to perform: Receiving, via a user input device, a two-dimensional input selection from the user presenting the original image; and Tracking, via the user input device, the movement of the two-dimensional input selection from an initial contact point presenting the original image to a final contact point.

9. The system according to claim 8, wherein: The user input device includes: A touch sensor composed of an input interface and a sensor array, used to be coupled to the input interface to receive at least one finger contact from user input; and A sensing circuit integrated in or connected to the touch sensor and connected to the processor, the sensing circuit being configured to measure voltage to track at least one finger contact on the input interface; Wherein to track the movement of the two-dimensional input selection from the initial contact point to the final contact point via the user input device to derive the horizontal position movement parameter, the processor is configured to track, via the sensing circuit, the drag of at least one finger on the input interface of the touch sensor from the initial contact point to the final contact point.

10. A method comprising the following steps: Calculate: (i) a left image disparity map between a left pixel matrix and a right pixel matrix, and (ii) a right image disparity map between the right pixel matrix and the left pixel matrix, wherein, The left pixel matrix is based on the left original image or the left processed image, and the right pixel matrix is based on the right original image or the right processed image; Presenting the original image via an image display, wherein the original image is based on the left original image, the left processed image, the right original image, the right processed image, or a combination thereof; Receiving, via a user input device, a marker and style selection from the user; Creating at least one stylized painting effect image using a stylized painting effect scenario; Applying the at least one stylized painting effect image to the following markers: (i) the left original image or the left processed image to create a left stylized painting effect image, (ii) the right original image or the right processed image to create a right stylized painting effect image, or (iii) a combination of both; Generating a stylized painting effect image by blending the left stylized painting effect image and the right stylized painting effect image; and Presenting the stylized painting effect image via an image display.

11. The method according to claim 10, wherein the receiving the style selection includes: Receiving an image selection from the user; and wherein, creating includes: Applying the image selection to the marker using neural style transfer.

12. The method according to claim 10, wherein the stylized painting effect image has an appearance of spatial movement or rotation of a stylized painting effect scene around the at least one stylized painting effect image, and the method further includes: Determining an inertial motion of an image display to generate spatial movement or rotation around a stylized painting effect scene of the at least one stylized painting effect image.

13. The method according to claim 10, wherein creating the at least one stylized painting effect image includes: Mixing the stylized painting effect image with the original image in a marked area.

14. The method according to claim 10, wherein: The steps of calculating a left image disparity map and a right image disparity map include: Creating a left corrected image from the left original image as a left processed image, creating a right corrected image from the right original image as a right processed image, aligning the left and right original images, and eliminating distortion of respective lenses of the left and right visible light cameras; and Extracting the left image disparity map and the right image disparity map by correlating pixels in the left corrected image with the right corrected image, and vice versa, to calculate the disparity of each correlated pixel.

15. The method according to claim 14, wherein the step of generating a stylized painting effect image includes: Determining a horizontal position movement parameter along the X-axis of a left pixel matrix and a right pixel matrix; Filling a left interpolated pixel matrix by moving pixels in the left pixel matrix along the X-axis based on the horizontal movement parameter; Filling a right interpolated pixel matrix by moving pixels in the right pixel matrix along the X-axis based on the horizontal movement parameter; And Creating a stylized painting effect image by mixing the left interpolated pixel matrix and the right interpolated pixel matrix.

16. The method according to claim 15, wherein: The step of generating a stylized painting effect image by mixing the left interpolated pixel matrix and the right interpolated pixel matrix is based on a disparity confidence level, a gradient, or a combination of both in the left image disparity map and the right image disparity map.

17. The method according to claim 15, further includes: Capturing the left original image and the right original image via a depth capture camera using an eye-worn device; And Performing calculations on a host: (i) the left image disparity map, and (ii) the right image disparity map; presenting the original image via an image display; receiving a style selection via a user input device; creating a stylized painting effect image; generating a stylized painting effect image; And presenting the stylized painting effect image via the image display.

18. The method according to claim 17, wherein: Each pixel is associated with respective vertices of a vertex matrix; Each vertex has a position attribute; The position attribute of each vertex is based on a three-dimensional position coordinate system, including an X position coordinate on the X-axis representing a horizontal position, a Y position coordinate on the Y-axis representing a vertical position, and a Z position coordinate on the Z-axis representing a depth position; and Steps for applying a user's selection of a stylized painting effect: (i) the left original image or the left processed image for creating a left stylized painting effect image, (ii) the right original image or the right processed image for creating a right stylized painting effect image, or (iii) a combination of both, which is to change the filter effect intensity of a photo filter function based on the Z position coordinate to transform each pixel according to the depth position of the vertices associated with each pixel.

19. A non-transitory computer-readable medium storing program code that, when executed, causes an electronic processor to perform the following steps: Calculate: (i) a left image disparity map between a left pixel matrix and a right pixel matrix, and (ii) a right image disparity map between the right pixel matrix and the left pixel matrix, wherein, The left pixel matrix is based on the left original image or the left processed image, and the right pixel matrix is based on the right original image or the right processed image; Via an image display, present the original image, where the original image is based on the left original image, the left processed image, the right original image, the right processed image, or a combination thereof; Via a user input device, receive a marker and a style selection from the user; Create at least one stylized painting effect image using a stylized painting effect scenario; Apply the at least one stylized painting effect image to the following markers: (i) the left original image or the left processed image for creating a left stylized painting effect image, (ii) the right original image or the right processed image for creating a right stylized painting effect image, or (iii) a combination of both; Generate a stylized painting effect image by blending the left stylized painting effect image and the right stylized painting effect image; and Via the image display, present the stylized painting effect image.

20. The non-transitory computer-readable medium according to claim 19, wherein the receiving the style selection includes: Receiving an image selection from the user; And wherein, creating includes: Applying the image selection to the marker using neural style transfer.

Images