Selective Image Blur Using Machine Learning

A machine learning-based approach estimates depth and adjusts focus tables to apply selective blur, addressing the lack of focus and depth metadata in images, resulting in high-quality bokeh effects by ensuring the main subject is in focus and maintaining image quality.

JP7583924B2Active Publication Date: 2024-11-14GOOGLE LLC

Patent Information

Application Number
JP2023519574
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-03-16
Filing Date
2022-08-01
Publication Date
2024-11-14
Estimated Expiration
2042-08-01

AI Technical Summary

Technical Problem

Existing methods for generating a bokeh effect in images without focus and depth metadata rely on camera information, which is not available for scanned images or images with metadata removed, leading to inconsistent and unsatisfactory results.

Method used

A method using a machine learning-based focus table prediction model to estimate depth and apply selective blur, adjusting the focus table to ensure the main subject is in focus and applying appropriate blur based on detected faces and image texture, utilizing a trained model to generate a bokeh effect in images lacking focus and depth information.

Benefits of technology

Enables the generation of high-quality bokeh effects in images without original focus and depth metadata by accurately determining focal regions and applying appropriate blur, ensuring the main subject is in focus and maintaining image quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007583924000001
    Figure 0007583924000001
  • Figure 0007583924000002
    Figure 0007583924000002
  • Figure 0007583924000003
    Figure 0007583924000003
Patent Text Reader

Abstract

Implementations described herein relate to methods, computing devices, and non-transitory computer-readable media for generating an output image. In some implementations, the method includes obtaining depth by estimating a depth of the image. The method further includes generating a focus table for the image, the focus table including parameters indicative of a focus range and at least one of a forward gradient or a backward gradient. The method further includes determining whether one or more faces are detected from the image. If one or more faces are detected from the image, the method further includes identifying a face bounding box corresponding to each face and adjusting the focus table to include the face bounding box. If no faces are detected from the image, the method further includes scaling the focus table. The method further includes generating an output image by applying blur to the image using the focus table and the depth map.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Related Applications This application claims the benefit of U.S. Provisional Application No. 63 / 320,349, filed March 16, 2022, and entitled "Selective Image Blur Using Machine Learning," the entirety of which is incorporated herein by reference for all purposes. [Background technology]

[0002] background In photography, bokeh refers to the blurring that occurs in the out-of-focus parts of an image. Different lens aberrations and aperture shapes cause different bokeh effects. The bokeh effect is a common photographic effect, used, for example, to create portrait blur in an image, such that the subject of the image (e.g., one or more people or objects) is in focus (sharp) while other parts of the image, which may be in the foreground or background, are blurred. Summary of the Invention [Problem to be solved by the invention]

[0003] A depth map indicating the depths corresponding to various pixels in an image is utilized to generate the bokeh effect. A focus table based on the depth map indicates the amount of blur to be applied to each pixel. The focus table indicates the location and depth of the focal plane in the image. The focus table also determines the amount of blur to be applied to pixels in the background and foreground. Typically, the focus table is calculated based on information captured by the camera, e.g., autofocus calculations, user selection events (e.g., selection of a focus region). However, such information is not available for a particular image. For example, scanned images, images with metadata removed, etc. do not contain such information.

[0004] The discussion of the background art provided herein is intended to provide a general context for the present disclosure. To the extent that it is described in this background section, the work of the presently named inventors is not expressly or impliedly admitted as prior art to the present disclosure, as well as descriptions that do not qualify as prior art at the time of filing. [Means for solving the problem]

[0005] overview Implementations described herein relate to a method, a computing device, and a non-transitory computer-readable medium for generating an output image. Some implementations include a method. The method includes: obtaining a depth map indicating a depth of each pixel of the image by estimating a depth of the image; and generating a focus table for the image, the focus table including parameters indicating a focus range and at least one of a forward gradient or a backward gradient; the method further includes determining whether one or more faces are detected from the image; if it is determined that one or more faces are detected from the image, identifying a face bounding box corresponding to each face of the one or more faces, each face bounding box including an area of ​​the image corresponding to the face, and adjusting the focus table to include each of the face bounding boxes; if it is determined that no face is detected from the image, generating an output image by scaling the focus table and applying blur to the image using the focus table and the depth map, the output image including an in-focus region and one or more blurred regions.

[0006] In some implementations, adjusting the focus table includes expanding the range of focus depth values ​​until the pixels of each face bounding box are in the focus range. In some implementations, the focus table excludes a front gradient when foreground regions in front of the subject are not present in the image and excludes a rear gradient when background regions behind the subject are not present in the image. In some implementations, the in-focus regions in the output image include pixels associated with depth values ​​that correspond to a blur radius of zero in the depth map.

[0007] In some implementations, the image does not include information about focus and depth. In some implementations, the image is a scanned photograph, an image with metadata removed, an image taken with a camera that does not store focus and depth information, or a frame of a video. In some implementations, the method further includes displaying the output image.

[0008] In some implementations, generating the focus table includes using a focus table prediction model, the focus table prediction model being a trained machine learning model. In some implementations, the method further includes training the focus table prediction model, the training includes providing a plurality of training images as input to the focus table prediction model, each training image having an associated depth map and an associated ground truth blur radius image, and for each training image, generating a predicted focus table using the focus table prediction model, obtaining a predicted blur radius image using the predicted focus table and depth map associated with the training image, calculating a loss value based on the predicted blur radius image and ground truth blur radius image associated with the training image, and adjusting one or more parameters of the focus table prediction model using the loss value. In some implementations, the depth map associated with each training image is one of a ground truth depth map obtained at the time of image capture or an estimated depth map obtained using the depth prediction model. In some implementations, training the focus table prediction model further includes weighting the loss value using an image gradient of the training image before adjusting the one or more parameters of the focus table prediction model. Some implementations include a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform any of the methods described herein. Some implementations include a computing device comprising a processor and a memory coupled to the processor. The memory stores instructions that, when executed by the processor, cause the processor to perform any of the methods described herein. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating an example network environment that may be used in one or more implementations described herein. [Figure 2A] 1A-1C show examples of focus tables with different parameterizations. [Figure 2B] 1A-1C show examples of focus tables with different parameterizations. [Figure 2C] 1A-1C show examples of focus tables with different parameterizations. [Diagram 3] 1 illustrates an example method for generating an output image with selective blurring, according to some implementations. [Figure 4] FIG. 2 illustrates an example input image, a ground-truth depth image corresponding to the input image, and a predicted depth image corresponding to the input image. [Figure 6A] FIG. 2 illustrates an exemplary input image and first and second output images corresponding to the input image. [Figure 6B] FIG. 6B illustrates raw and scaled focus table predictions corresponding to the exemplary input image of FIG. 6A. [Figure 7] 1 illustrates an example method for generating an output image with selective blurring, according to some implementations. [Figure 8] FIG. 1 illustrates an exemplary computing device in accordance with some implementations. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Detailed Description This document describes techniques for generating a bokeh effect by applying blur to an image, even when the image is acquired by a camera and does not contain information indicating the focal region and / or depth associated with various regions / pixels of the image (e.g., based on an autofocus calculation or a user-selected event) (e.g., a depth map). The techniques include estimating depths of pixels of the image based on image data, e.g., a monocular depth estimator that generates depth values ​​based on the RGB (or other color values) of each pixel of the image. The techniques further include determining a focal table for the image. For example, the focal table may be determined using a suitably trained machine learning model. The depth values ​​of the pixels of the image estimated by the monocular depth estimator may be provided as input to the machine learning model. The machine learning model may be referred to as a "post-capture focal table prediction model."

[0011] A focus table is utilized to apply blur to an image. The focus table indicates the location and depth of the focal plane. The focus table also indicates the amount of blur to be applied to background and / or foreground regions of the image, i.e., out-of-focus regions in the output image. The focus table maps depth within the scene depicted in the image to the amount of blur to be applied to create the bokeh effect.

[0012] Example Network Environment 1 shows a block diagram of an example network environment 100 that may be used in some implementations described herein. In some implementations, the network environment 100 includes one or more server systems (e.g., server system 102 in the example of FIG. 1) and a number of client devices (e.g., client devices 120-126, each associated with a respective user U1-U4). Each of the server system 102 and the client devices 120-126 may be configured to communicate with a network 130.

[0013] The server system 102 can include a server device 104. In some implementations, the server device 104 can provide an image application 106a. In FIG. 1 and other figures, a letter following a reference number, e.g., "160a," denotes a reference to the element having that particular reference number. A reference number in the text without a letter following it, e.g., "160," denotes a general reference to an embodiment of the element having that reference number.

[0014] An image as referred to herein may include a digital image having pixels with one or more pixel values ​​(e.g., color values, brightness values, etc.). An image may be a still image (e.g., a still image, a single frame image, etc.), a dynamic image (e.g., a video, animated GIF, a cinemagraph in which some parts of the image include motion while other parts are still, etc.), or a video (e.g., a series of images or image frames that may include audio). It should be noted that an image as used herein may be any of the above. For example, implementations described herein may be used with still images (e.g., photographs or other images), video, or dynamic images.

[0015] The network environment 100 may also include one or more client devices, such as client devices 120, 122, 124, and 126. The client devices 120, 122, 124, and 126 may communicate with each other and / or with the server system 102 via the network 130. The network 130 may be any type of communication network including one or more of the Internet, a local area network (LAN), a wireless network, a switched or hub connection, and the like. In some implementations, the network 130 may include peer-to-peer communication between devices using, for example, a peer-to-peer wireless protocol (e.g., Bluetooth, Wi-Fi Direct), and the like. An example of peer-to-peer communication between two client devices 120 and 122 is illustrated by arrow 132.

[0016] In various implementations, users U1, U2, U3, and U4 can communicate with server system 102 and / or with each other using respective client devices 120, 122, 124, and 126. In some examples, users U1, U2, U3, and U4 can interact with each other via applications running on respective client devices and / or server system 102, and / or via network services implemented on server system 102, e.g., social networking services or other types of network services. For example, each client device 120, 122, 124, and 126 can communicate data to or from one or more server systems, e.g., server system 102.

[0017] In some implementations, the server system 102 can provide appropriate data to the client devices, so that each client device can receive communication content, shared content, and / or network services uploaded to the server system 102. In some examples, the users U1-U4 can interact via image sharing, audio or video conferencing, voice, video, text chat, or other communication modes or applications. The network services implemented by the server system 102 may include systems that allow users to perform various communications, form links and associations, upload and post shared content, e.g., images, text, audio, and other types of content, and / or perform other functions. For example, a client device may display received data, such as content posts sent or streamed to the client device, content posts originating from different client devices (or directly from different client devices) via a server and / or network service, or content posts originating from a server system and / or network service. In some implementations, client devices may communicate directly with each other, for example, using peer-to-peer communication between client devices as described above. In some implementations, a "user" may include one or more programs or virtual entities and may include humans interfacing with a system or network.

[0018] In some implementations, client devices 120, 122, 124, and / or 126 may each host one or more applications. For example, as shown in FIG. 1, client device 120 may host an image application 106b. Client devices 122-126 may host similar applications. Image application 106a may be implemented using hardware and / or software on client device 120. In different implementations, image application 106a may be a standalone client application running on any of client devices 120-124, for example, or may operate in conjunction with image application 106b hosted on server system 102.

[0019] The image application 106 may be implemented with user permission to provide a variety of image-related functions. For example, such functions may include one or more of capturing an image using a camera, modifying an image, determining image quality (e.g., based on factors such as face size, number of faces, image composition, lighting, exposure, etc.), storing an image or video, automatically applying a bokeh effect to an image, adjusting the bokeh effect, providing a user interface for viewing an image or a work based on an image, etc. In some implementations, the functionality provided by the image application 106 with user permission may include detecting one or more people depicted in an image by analyzing the image (e.g., using one or more user-permitted techniques such as face detection).

[0020] It should be noted that although various features of the image application 106 have been described above, in various implementations the image application 106 may provide fewer or more features, and each user may be provided with the option to enable and / or disable certain features. The functionality of the image application 106 may be specifically implemented with user permission.

[0021] In some implementations, the image application 106 allows a user to manage an image library. For example, a user may use a backup function of the image application 106b on a client device (e.g., any of the client devices 120-126) to back up local images on the client device to a server device, such as server device 104. For example, the user may manually select one or more images to be backed up or may specify backup settings that identify the images to be backed up. Backing up images to the server device may include, for example, in cooperation with the image application 106a on the server device 104, transmitting the images to the server for storage on the server.

[0022] In different implementations, client device 120 and / or server system 102 may include other applications (not shown), which may be applications providing various types of functionality, such as calendars, address books, email, web browsers, shopping, transportation (e.g., taxi, train, airline reservations), entertainment (e.g., music players, video players, gaming applications), social networking (e.g., messaging or chat, audio / video calling, image / video sharing), etc. In some implementations, the one or more other applications may be standalone applications executing on client device 120. In some implementations, the one or more other applications may access a server system, such as server system 102, that provides data and / or functionality for the other applications.

[0023] User interfaces on client devices 120, 122, 124, and / or 126 can enable the display of user and other content, including images, image-based works, data, other content, communications, privacy settings, notifications, and other data. Such user interfaces may be displayed using software on the client devices, software on the server device, and / or a combination of client and server software (e.g., application software or client software in communication with server system 102) running on server device 104. The user interfaces may be displayed by a display device (e.g., a touch screen or other display screen, a projector, etc.) of the client or server device. In some implementations, application programs running on the server system can receive user input at the client devices and output data, such as visual data, audio data, etc., at the client devices by communicating with the client devices.

[0024] For ease of illustration, FIG. 1 illustrates the server system 102 and the server device 104 as one block, and the client devices 120, 122, 124, and 126 as four blocks. The server blocks 102 and / or 104 may represent multiple systems, server devices, and network databases. These blocks may be provided in configurations different than those illustrated. For example, the server system 102 may represent multiple server systems that may communicate with other server systems via the network 130. In some implementations, the server system 102 may include, for example, a cloud hosting server.

[0025] Also, there may be any number of client devices. Each client device may be any type of electronic device, such as a desktop computer, a laptop computer, a portable or mobile device, a mobile phone, a smartphone, a tablet computer, a television, a television set-top box or entertainment device, a wearable device (e.g., display glasses or goggles, a watch, a headset, an armband, jewelry, etc.), a personal digital assistant (PDA), a media player, a gaming device, etc. In some implementations, the network environment 100 may not include all of the components illustrated and / or may include other elements, including other types of elements, instead of or in addition to the elements described herein.

[0026] Other implementations of the features described herein can use any type of system and / or service. For example, other network services (e.g., connected to the Internet) can be used instead of or in addition to a social networking service. Any type of electronic device can utilize the features described herein. Some implementations can provide one or more features described herein on one or more client or server devices that are disconnected or intermittently connected to a computer network. In some examples, a client device that includes or is connected to a display device can display content posts that have been previously received, for example, via a communication network and stored in a local storage device of the client device.

[0027] focus table In some implementations, the focus table may indicate a (back) focus depth value (dof_back) indicating the depth behind (further from the camera) the in-focus region of the image, an in-focus range of the image indicating the range of depth values ​​that are in focus, a back slope (back_slope) indicating the blur radius applied to each depth value farther from the camera than the back focus depth value, and a front slope (front_slope) indicating the blur radius applied to each depth value closer to the camera but not in the focus range. Figure 2A shows the structure of such a focus table.

[0028] In other implementations, the focus table can indicate the in focus disparity, the half depth of focus behind the focus disparity, the half depth of focus in front of the focus disparity, the back gradient, and the front gradient. Figure 2B shows the structure of such a focus table.

[0029] In yet another implementation, the focus table may indicate a back focal depth (dof_back), a front focal depth (dof_front), a back gradient, and a front gradient. Figure 2C shows the structure of such a focus table.

[0030] Exemplary Methods FIG. 3 illustrates an exemplary method for automatically generating an output image with selective blur (bokeh effect) from an input image (without focus and depth related metadata). As shown in FIG. 3, the input image is provided to a depth predictor. The depth predictor may be, for example, a monocular depth predictor for predicting the depth of each pixel in the input image. FIG. 1 illustrates an exemplary input image (cans on a shelf) and a depth image showing the depth of each pixel of the input image. The estimated depth of the image determined by the depth predictor is provided to a focus table estimator.

[0031] The focus table estimator generates a focus table based on the estimated depth and the input image (e.g., an RGB image). An exemplary focus table is shown in FIG. 3. The focus table is a function for specifying the amount of blur to be applied to each pixel of the input image for each depth value (0-255) of the depth map. The input image, the depth map, and the focus table are provided to the blur renderer. The focus table may be, for example, any type of focus table described with reference to FIGS. 2A-2C or a different type of focus table.

[0032] The blur renderer generates an output image based on the input image, the depth map, and the focus table. For example, the blur renderer applies each blur amount to various pixels of the image that are in the background or foreground regions, as indicated by the focus table. The blur can be applied using a blur radius that corresponds to the depth. In the example of Figure 3, the blur radius keeps the depth map values ​​from about 155 to 255 sharp. Beyond this, the blur radius gradually increases. Note that the actual rendering radius is the absolute value of the "blur radius". To perform front-to-back compositing, the rendering function determines that negative values ​​indicate the rear of the focus range.

[0033] focus table estimator In some implementations, the focus table estimator can include a machine learning model trained to generate a focus table based on the estimated depth of the input image, also referred to herein as a “post-capture focus table prediction model” or simply a “focus table prediction model.”

[0034] Training a Focus Table Prediction Model - Supervised Learning In some implementations, a machine learning model may be trained to predict (estimate) the focus table using supervised learning. In such implementations, a training dataset may include a training image set with associated depth maps and a ground truth focus table for each training image in the training image set. For example, the training images may be captured using a camera that captures depth information (ground truth depth map) and focus information and stores this information with the image as image metadata. A ground truth focus table for each image may be generated and stored at the time of capture, for example, based on the focus and depth information.

[0035] During training, images (captured by a camera) and a ground-truth depth map are provided as input to the model under training. The model is trained to generate a predicted focus table as an output. A loss value is determined based on the predicted focus table and the ground-truth focus table. For example, the loss value may be a mean squared error (MSE) value. The loss value is utilized as feedback to adjust one or more parameters of the model under training. By utilizing a sufficiently large training dataset and training until a threshold level of accuracy is reached, the model can be trained to generate a predicted focus table that is close to the ground-truth focus table of any input image and can be used to blur the image to generate a bokeh effect.

[0036] The trained model can then be utilized to predict a focus table for an image that lacks focus information. During prediction of a focus table for an input image (referred to as the inference phase), a ground truth depth map may not be available, for example, if the input image is a scanned image or if depth metadata is missing. In such cases, a depth predictor can be utilized to generate a depth map for the image, and the depth map can be provided as an input to the trained model. The depth predictor may be configured to predict the depth map based on the input image. For example, the depth predictor may be implemented using a machine learning depth prediction model trained using supervised learning. This allows the trained depth prediction model to generate a depth map that is close to the ground truth depth map from the camera. However, for example, when training different depth prediction models based on different data sources (e.g., different cameras with their respective depth maps), a domain gap may occur.

[0037] Due to these differences, there may also be differences between depth maps generated by different depth prediction models, and between the ground truth and output of a depth prediction model. Due to such domain gaps (differences in depth maps), using the same output focus table generated by a model trained on depth maps generated by different depth predictor models of an image may produce different blurry outputs.

[0038] 4 is a diagram showing an example of the difference between a ground truth depth map and an estimated depth map. As can be seen, when using the same focus table to generate an output blurred image based on the ground truth depth map and the estimated depth map, different input depth maps will result in different blurred output images, since different blur amounts are applied to the same pixels at different depths.

[0039] The problem caused by the domain gap can be addressed by training a custom focus table prediction model for each depth predictor and using the appropriate trained model in the inference stage. However, this is inefficient and may produce inconsistent and blurry output images, as multiple depth predictor models and custom focus table prediction models need to be trained and stored.

[0040] Training a Focus Table Prediction Model - Semi-Supervised Learning 5 illustrates an exemplary method for training a custom focus table prediction model using semi-supervised learning. As shown in FIG. 5, input training images are provided to, for example, a depth predictor that implements a depth prediction model. Estimated depth maps obtained from the depth predictor are provided to a focus table prediction model (MUT) during training to generate a predicted focus table.

[0041] The blur radius image is a single-channel image in which each pixel is the blur radius at that pixel. Its calculation is differentiable. The predicted focus table and the estimated depth map are used to generate a predicted blur radius image. The ground truth depth map and the ground truth focus table are also used to generate a ground truth blur radius image (also called a target blur radius image). A loss function (e.g., mean square error or other suitable function) is used to determine a loss value based on the predicted blur radius image and the ground truth blur radius image. The loss determined by the loss function is used to train a focus table prediction model (e.g., adjust one or more parameters of the focus table prediction model). For example, if the model is a neural network, such adjustments can include adjusting the weights of one or more nodes in one or more layers of the neural network, or the connectivity between different nodes of the neural network.

[0042] This technique is semi-supervised, and neither the ground truth depth map nor the ground truth focus table is directly provided to the focus table prediction model during training. Rather, the blur radius image used to generate the final output, i.e., the output blurred image, is used for training. By training in this manner, the model can be robust even when a different depth prediction model is used or when the ground truth focus table has different parameters (e.g., different parameterizations described with reference to Figures 2A-2C or other parameterizations) than the predicted focus table generated by the model.

[0043] Loss weighting based on image gradients Many images can contain different regions with different levels of texture. For example, an image containing a region showing a sky may have approximately the same pixel values ​​(color and depth) in the sky region, while another region showing scenery such as nearby trees and more distant mountains may have different pixel values ​​(color and depth). In some implementations, the loss may be weighted by the image gradient of the input image (which indicates the image texture). This can account for the fact that different blur radii in texture-free regions produce similar blur results, but higher accuracy in textured regions.

[0044] Ensuring your subject is in focus In some cases, the focus table estimator may generate a focus table that does not focus on the main subject of the image. For example, this can occur when there is another object closer to the camera than the main subject. An example of this situation would be when the main subject (e.g., a person) faces the camera, but another person or object is closer to the camera but not closer to the main subject (e.g., a person standing with their back to the camera). Other types of errors in the focus table are also possible. For example, such errors can occur when the training data is not sufficiently representative of real-world images that show the subject in arbitrary poses and at various depths from the camera.

[0045] If the focus table predicted by the focus table estimator is not focused on the main subject, for example if the parameter in_focus_range excludes a depth that captures the main subject, then the output blurred image may be unsatisfactory because blur has been applied to the main subject.

[0046] To reduce the likelihood of such unsatisfactory output images, in some implementations, the input image is analyzed using, for example, any suitable face detection technique to generate bounding boxes for subjects in the image, for example, one or more faces having at least a threshold size (number of pixels) in the image. The face bounding boxes indicate the pixels of the image that contain the primary subject.

[0047] In these implementations, the main subject is focused (unblurred) by adjusting the focus table output by the focus table estimator before applying blur. This is achieved by extending the focus range (e.g., parameter in_focus_range) to include the main subject in the scene. For example, if the focus range includes depth values ​​d1 to d2 and the main subject is at a depth value d3 that is greater than d2, then the focus range is adjusted to include value d3. For example, the focus range is updated to d1 to d3. By adjusting the focus table before applying blur, the output image is in focus on the main subject as long as the bounding box is accurate.

[0048] Limiting the amount of blur for images that do not contain faces In some cases, a user may wish to apply a bokeh effect to an image that does not contain a face. For such images that do not contain a human subject, it is preferable to apply less blur to generate a beautiful image. In such a case, if no face is detected from the input image by applying face detection techniques, the parameter dof_scale (which controls the amount of blur applied) is limited to a predefined threshold. FIG. 6A illustrates such a scenario. As shown in FIG. 6A(ii), applying blur to the input image of FIG. 6A(i) without any limitation based on the predicted focus table produces a blurry image that is too blurry and appears cut out from the background. FIG. 6A(iii) illustrates a blurred image with limited amount of blur applied, producing a more beautiful image.

[0049] Figure 6B(i) shows the raw focus table prediction used to generate Figure 6A(ii), and Figure 6B(ii) shows the scaled focus table prediction used to generate Figure 6A(iii). As shown, the raw focus table has a blur radius that ranges from 0 to over -6, while the scaled focus table limits the blur radius for the same range of depth values ​​to a range of 0 to -3.

[0050] Exemplary Methods 7 illustrates an example method 700 for generating a blurred image, for example an image having selective blur applied to generate a bokeh effect.

[0051] At block 710, an input image is received. For example, the input image may be any image that does not include information (e.g., metadata) regarding focus (a focal plane corresponding to a subject of the image) and depth (e.g., a depth map indicating the depth of each pixel of the image). Such an image may be a scanned photograph, an image from which metadata has been removed, an image taken with a camera that does not generate or store focus and depth information, etc. In some implementations, the input image may be a frame of a video. Method 700 may be performed for multiple frames of the video to generate a video with a bokeh effect. Block 720 may be performed after block 710.

[0052] In block 720, the depth of the input image is predicted. For example, a depth prediction model can be used to perform the depth prediction. A depth map indicating the depth of each pixel of the input image can be obtained. After block 720, block 730 can be performed.

[0053] In block 730, a focus table for the input image is generated. In some implementations, the focus table may be generated using a trained machine learning model. The focus table may include parameters indicating a focus range (indicating the depth values ​​to focus on) and a front gradient and / or a rear gradient (indicating the depth values ​​to blur and the respective blur radii). For example, if the input image does not include a foreground region located in front of the object, the front gradient may be absent (or may be zero). For example, if the input image does not include a background region located behind the image object, the rear gradient may be absent (or may be zero). Block 740 may be performed after block 730.

[0054] In block 740, it is determined whether one or more faces are detected in the input image. Any suitable face detection technique may be utilized to detect faces in the input image. If one or more faces are detected, block 740 may be followed by block 750. If no faces are detected, block 740 may be followed by block 770.

[0055] In block 750, a face bounding box for the detected face is identified. The bounding box may include the area of ​​the input image (pixels) that corresponds to the detected face. Block 760 may be executed after block 750.

[0056] In block 760, the focus table is adjusted to include an area corresponding to the face bounding box. Adjusting the focus table may include expanding the range of focus depth values ​​until the pixels of the face bounding box are in the focus range. Block 780 may be performed after block 760.

[0057] If no face is detected in block 740, then block 770 is executed after block 740. In block 770, the focus table is scaled to limit the amount of blur applied to the image. The scaling may include adjusting the front gradient and / or back gradient of the focus table. Block 770 may be executed after block 780.

[0058] In block 780, an output image is generated by applying blur to the image using the focus table and the depth map. The output image includes in-focus regions (pixels of the image having depth values ​​corresponding to a blur radius of zero) and one or more blurred regions (pixels of the image having depth values ​​corresponding to a non-zero blur radius). The application of blur may be performed using an appropriate blur kernel. The output image has a blur effect.

[0059] In some implementations, one or more blocks of method 700 may be merged. For example, blocks 740 and 750 may be merged to simultaneously perform face detection and bounding box generation using face detection techniques. In some implementations, one or more blocks of the method may not be performed. For example, in some implementations, if no face is detected in the image, block 770 may not be performed and block 780 may be performed immediately after block 740. In some implementations, blocks 740-770 may not be performed and block 780 may be performed immediately after block 730.

[0060] In some implementations, method 700 can be performed on multiple input images to generate corresponding multiple output images. In some implementations, method 700 can be performed on one or more frames (still images) of a video, and the output images can be arranged in the same sequence as the input frames to provide a blurred video.

[0061] In some implementations, the blurred output image may be displayed via a display device such as a monitor, a wearable device, a virtual reality device, etc. In some implementations, a user interface may be provided that allows a user to edit the output image.

[0062] In addition to the above, the systems, programs or features described herein may give the user control to select whether and when to enable collection of user information (e.g., information about the user's image and / or video, social network, social behavior or activity, occupation, user preferences, or the user's current location) and to transmit content or information from the server. Furthermore, certain data may be processed to remove identifiable personal information in one or more ways before being stored or used. For example, the user's identity may be processed so that the user's personal information cannot be identified. Also, when obtaining location information (e.g., to the city, zip code, or state level), the user's geographic location may be generalized so that the user's location cannot be identified. Thus, the user may control the user information that is collected, the use of the information, and the information provided to the user.

[0063] Exemplary Computing Device FIG. 8 is a block diagram illustrating an example device 800 that may be used to implement one or more features described herein. In one example, device 800 may be used to implement a client device, such as any of client devices 120-126 shown in FIG. 1. Alternatively, device 800 may be used to implement a server device, such as server 102 or 104. In some implementations, device 800 may be used to implement a client device, a server device, or both a client device and a server device. As discussed above, device 800 may be any suitable computer system, server, or other electronic or hardware device.

[0064] One or more methods described herein may be implemented as a standalone program running on any type of computing device, a program running on a web browser, or a mobile application (app) running on a mobile computing device (e.g., a mobile phone, a smartphone, a tablet computer, a wearable device (e.g., a watch, an armband, jewelry, headwear, virtual reality goggles or glasses, augmented reality goggles or glasses, head mounted displays), a laptop computer). In one example, a client / server configuration may be used. For example, a mobile computing device (client device) may send input data from a user to a server device and receive and output (e.g., display) final output data from the server. In another example, all computations may be performed in a mobile app (and / or other apps) on the mobile computing device. In another example, computations may be shared between the mobile computing device and one or more server devices.

[0065] In some implementations, the device 800 includes a processor 802, a memory 804, and an input / output (I / O) interface 806. The processor 802 may be one or more processors and / or processing circuits for executing program code and controlling basic operations of the device 800. A "processor" includes any suitable hardware system, mechanism, or component for processing data, signals, or other information. A processor may include a general-purpose central processing unit (CPU) having one or more cores (e.g., single-core, dual-core, or multi-core configurations), multiple processing units (e.g., having a multi-processor configuration), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a complex programmable logic device (CPLD), a dedicated circuit for achieving functionality, a dedicated processor for performing neural network model-based processing, a neural circuit, a system having a processor optimized for performing matrix calculations (e.g., matrix multiplication), or other systems. In some implementations, the processor 802 may include one or more co-processors for performing neural network processing. In some implementations, the processor 802 may be a processor that generates a probabilistic output by processing data. For example, the output generated by the processor 802 may be inaccurate or accurate within a range of output expectations. The processing need not be limited to a particular geographic location, nor limited in time. For example, the processor may perform functions in "real-time," "offline," or "batch mode." Parts of the processing may be performed by different (or the same) processing systems at different times and in different locations. A computer may be any processor in communication with a memory.

[0066] The memory 804 is typically provided within the device 800 for use by the processor 802 and may be any suitable processor-readable storage medium for storing instructions executed by the processor 802, such as random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory. The memory 804 may be located separately from the processor 802 and / or integrated therewith. The memory 804 may store an operating system 808, software executed on the server device 800 by the processor 802, such as machine learning applications 830, other applications 812, and application data 814. The other applications 812 may include applications such as a data display engine, a web hosting engine, an image display engine, an image editing application, an image management application, a notification engine, a social networking engine, and the like. In some implementations, the machine learning applications 830 and the other applications 812 may each include instructions that enable the processor 802 to perform the functions described herein, such as the method of FIG. 8 or the machine learning model training method described with reference to the focus table estimator.

[0067] Other applications 812 may include, for example, an image editing application, a media viewing application, a communication application, a web hosting engine or application, a mapping application, a media sharing application, etc. One or more methods disclosed herein may operate in a number of environments and platforms, for example, as a standalone computer program that can run on any type of computing device, as a web application having a web page, as a mobile application (app) running on a mobile computing device, etc.

[0068] In various implementations, the machine learning application may utilize a Bayesian classifier, a support vector machine, a neural network, or other learning techniques. In some implementations, the machine learning application 830 may include a trained model 834, an inference engine 836, and data 832. In some implementations, the data 832 may include training data, e.g., data used to generate the trained model 834. For example, the training data may include any type of data, such as text, images, audio, video, etc. For example, the training images may include images captured by a camera and including stored focus information (e.g., focus depth) and depth information (e.g., depth of each pixel of the image) as image metadata. If the trained model 834 is a focus table estimator, the training data may include training images.

[0069] The training data may be obtained from any source, e.g., a data repository specified for training, data for which permission has been given for use as training data for machine learning. In implementations where one or more users authorize the use of their data to train a machine learning model, e.g., trained model 834, the training data may include these user data. In implementations where one or more users authorize the use of their data, data 832 may include authorization data, such as images (e.g., photographs or other user-generated images).

[0070] In some implementations, the training data may include synthetic data generated for training purposes, e.g., data that is not based on user input or activity in the situation being trained, data obtained from simulated photographs or other computer-generated images. In some implementations, the machine learning application 830 excludes the data 832. For example, in these implementations, the trained model 834 may be generated, for example, on a different device and provided as part of the machine learning application 830. In various implementations, the trained model 834 may be provided as a data file that includes the model structure or form and associated weights. The inference engine 836 can read the data file for the trained model 834 and can implement a neural network with node connectivity, layers and weights based on the model structure or form specified in the trained model 834.

[0071] In some implementations, the trained model 834 may include one or more model forms or structures. For example, the model form or structure may include any type of neural network, such as a linear network, a deep neural network implementing multiple layers (e.g., a "hidden layer" between an input layer and an output layer, where each layer is a linear network), a convolutional neural network (e.g., a network that splits or partitions input data into multiple portions or tiles, processes each tile separately with one or more neural network layers, and aggregates the results from the processing of each tile), a sequence-to-sequence neural network (e.g., a network that receives sequential data as input, such as words in a sentence or frames in a video, and produces a sequence of results as output). The model form or structure may specify the connections between the various nodes and the organization of the nodes organized into layers.

[0072] For example, nodes in a first layer (e.g., input layer) may receive data as input data 832 or application data 814. For example, when using a trained model to analyze or generate an image or to apply an effect, such as a bokeh effect, the input data may include, for example, one or more pixels of each node. Subsequent intermediate layers may receive as input the output of nodes in the previous layer according to connections specified in the model format or structure. These layers are also referred to as hidden or latent layers.

[0073] The final layer (e.g., output layer) generates the output of the machine learning application. For example, the output may be a blurred image with a bokeh effect. In some implementations, the model form or structure specifies the number and / or type of nodes in each layer.

[0074] In different implementations, the trained model 834 may include multiple nodes arranged in layers of each model structure or form. In some implementations, the node may be a computational node without memory, configured to process, for example, one unit of input and generate one unit of output. The computation performed by the node may include, for example, multiplying each of the multiple node inputs by a weight, obtaining a weighted sum, and generating the node output by adjusting the weighted sum with a bias value or an intercept value. Also, in some implementations, the computation performed by the node may include applying a step / activation function to the adjusted weighted sum. In some implementations, the step / activation function may be a nonlinear function. In various implementations, such computations may include operations such as matrix multiplication. In some implementations, the computations of multiple nodes may be performed in parallel, for example, using multiple processor cores of a multi-core processor, or using individual processing units of a GPU or dedicated neural circuit. In some implementations, the node may include memory, for example, to store one or more previous inputs and use one or more previous inputs in processing subsequent inputs. For example, a node with memory can include a long short-term memory (LSTM) node. The LSTM node can use memory to maintain a "state" that allows the node to operate like a finite state machine (FSM). Models with such nodes may be useful in processing sequential data, such as multiple words in a sentence or paragraph, multiple frames in a video, conversation or other audio, etc.

[0075] In some implementations, the trained model 834 may include embeddings or weights for individual nodes. For example, the model may be initialized as a number of nodes organized into layers as specified by the model topology or structure. At initialization, a respective weight may be applied to the connection between each pair of nodes connected according to the model topology, e.g., each pair of nodes in successive layers of a neural network. For example, each weight may be randomly assigned or initialized to a default value. The model may then be trained, e.g., using the data 832, to generate results.

[0076] For example, the training step can include applying a supervised learning technique. In supervised learning, the training data can include multiple inputs (e.g., a set of grayscale images) and expected outputs corresponding to each input (e.g., a set of ground-truth images or other colorized images corresponding to the grayscale images). For example, the values ​​of the weights are automatically adjusted based on a comparison of the model's outputs and the expected outputs to increase the probability that the model will generate the expected outputs given similar inputs.

[0077] In some implementations, the training step may include applying semi-supervised or unsupervised learning techniques. In unsupervised learning, only input data may be provided and a model may be trained to distinguish between the data, for example to cluster the input data into multiple groups, each group containing input data that is similar in some way.

[0078] In some implementations, unsupervised learning can be used to generate a knowledge representation that can be used, for example, by the machine learning application 830. In various implementations, the trained model includes a set of weights or embeddings that correspond to the model structure. In implementations that omit the data 832, the machine learning application 830 can include a trained model 834 based on prior training, for example, by a developer of the machine learning application 830 or a third party. In some implementations, the trained model 834 can include a set of fixed weights, for example, downloaded from a server that provides the weights.

[0079] The machine learning application 830 also includes an inference engine 836. The inference engine 836 is configured to provide inferences by applying the trained model 834 to data, such as the application data 814. In some implementations, the inference engine 836 may include software code executed by the processor 802. In some implementations, the inference engine 836 may specify a circuit configuration (e.g., of a programmable processor, a field programmable gate array (FPGA)) that enables the processor 802 to apply the trained model. In some implementations, the inference engine 836 may include software instructions, hardware instructions, or a combination thereof. In some implementations, the inference engine 836 may provide an application programming interface (API). The operating system 808 and / or other applications 812 may utilize this API to call the inference engine 836 to generate inferences, such as by applying the trained model 834 to the application data 814. For example, the inference of the focus table estimator model may be a focus table. In another example, the inference of the depth prediction model may be a predicted depth value for various pixels of an image.

[0080] The machine learning application 830 can provide several technical advantages. For example, if the trained model 834 was generated based on unsupervised learning, the inference engine 836 can apply the trained model 834 to generate a knowledge representation (e.g., a numerical representation) from input data, e.g., the application data 814. For example, a model trained for image analysis can generate an image representation (e.g., 1 KB) with a smaller data size than the input image (e.g., 10 MB). In some implementations, such a representation is useful to reduce the processing cost (e.g., computational cost, memory usage, etc.) for generating an output (e.g., a label, a classification, a description of the image, a colorized image obtained from a grayscale image).

[0081] In some implementations, such representations may be provided as input to a different machine learning application that generates an output from the output of the inference engine 836. In some implementations, the knowledge representations generated by the machine learning application 830 may be provided, for example, over a network, to a different device that performs further processing. In such implementations, providing the knowledge representations rather than images may provide technical benefits, for example, allowing faster data transmission at lower cost. In another example, a model trained to cluster documents may generate document clusters from the input documents. The document clusters may be suitable for further processing (e.g., determining whether a document is related to a topic, determining a taxonomic category of the document) without needing to access the original documents, thus saving computational costs.

[0082] In some implementations, the machine learning application 830 may be implemented offline. In these implementations, the trained model 834 may be generated in a first stage and provided as part of the machine learning application 830. In some implementations, the machine learning application 830 may be implemented online. For example, in such implementations, an application that invokes the machine learning application 830 (e.g., the operating system 808, one or more other applications 812) may utilize the inferences generated by the machine learning application 830 (e.g., provide the inferences to a user) and generate a system log (e.g., actions taken by the user based on the inferences, if permitted by the user, or results of further processing, if utilized as input for further processing). The system log may be generated periodically, e.g., hourly, monthly, or quarterly, and may be used to update the trained model 834, e.g., to update the embeddings of the trained model 834, if permitted by the user.

[0083] In some implementations, the machine learning application 830 may be implemented to adapt to the particular configuration of the device 800 on which it executes. For example, the machine learning application 830 may determine a computation graph that utilizes available computational resources, e.g., the processor 802. For example, the machine learning application 830 may determine the computations to be performed on each device to optimize the computations when implemented as a distributed application on multiple devices. In another example, the machine learning application 830 may implement an inference engine (e.g., as 1000 separate processes or threads) if it determines that the processor 802 includes a GPU with a particular number (e.g., 1000) of GPU cores.

[0084] In some implementations, the machine learning application 830 can implement a set of trained models. For example, the trained models 834 can include multiple trained models, each applicable to the same input data. In these implementations, the machine learning application 830 can select a particular trained model based on, for example, available computational resources, success rate using previous inferences, etc. In some implementations, the machine learning application 830 can execute the inference engine 836 to apply the multiple trained models. In these implementations, the machine learning application 830 can combine the outputs, for example, using a majority vote to score the outputs obtained by applying each trained model, or by selecting one or more particular outputs. Furthermore, in these implementations, the machine learning application can apply a time threshold (e.g., 0.5 ms) for applying the individual trained models and utilize only the individual outputs available within the time threshold. Outputs not received within the time threshold may not be utilized, e.g., discarded. For example, such an approach may be appropriate when there is a time limit specified between invoking the machine learning application, for example, by the operating system 808 or one or more applications 812.

[0085] In different implementations, the machine learning application 830 can generate different types of output. For example, the machine learning application 830 can provide a representation or cluster (e.g., a numerical representation of the input data), a label (e.g., of the input data including images, documents, etc.), a phrase or sentence (e.g., used as a response to the input sentence and appropriately representing the image or video), an image (e.g., a colorized image, an image with a bokeh effect, or an image in other styles generated by the machine learning application in response to an input image, e.g., a grayscale image), an audio, or a video. For example, the machine learning application 830 can generate an output video with a particular effect applied in response to an input video. For example, if the trained model 834 was trained using training data from a comic book or a particular artist, the output video is rendered in the style of, for example, a comic book or a particular artist. In some implementations, the machine learning application 830 can generate an output based on a format specified by a calling application, e.g., the operating system 808 or one or more applications 812. In some implementations, the application being called may be another machine learning application. Such a configuration may be used, for example, for a generative adversarial network in which the machine learning application being called is trained using output from machine learning application 830, or in which machine learning application 830 is trained using output from a machine learning application being called.

[0086] Alternatively, any of the software in memory 804 may be stored on any other suitable memory location or computer readable medium. Memory 804 (and / or other connected storage devices) may also store one or more messages, one or more classification criteria, an electronic encyclopedia, a dictionary, a thesaurus, a knowledge base, message data, grammars, user preferences, and / or other instructions and data used in the features described herein. Memory 804 and any other type of storage (e.g., magnetic disks, optical disks, magnetic tapes, or other tangible media) may be considered "storage" or "storage devices."

[0087] The I / O interface 806 can provide functionality that allows the server device 800 to connect to other systems and devices. The connected devices may be included as part of the device 800 or may be separate from the device 800 and communicate with the device 800. For example, network communication devices, storage devices (e.g., memory and / or database 106), and input / output devices can communicate through the I / O interface 806. In some implementations, the I / O interface can connect to interface devices such as input devices (keyboards, pointing devices, touch screens, microphones, cameras, scanners, sensors, etc.) and / or output devices (display devices, speaker devices, printers, motors, etc.).

[0088] Some examples of devices that can be connected to the I / O interface 806 include one or more display devices 820. The display devices 820 can be used to display content, such as images, videos, and / or user interfaces of output applications described herein. The display devices 820 may be connected to the device 800 via a local connection (e.g., a display bus) and / or via a network connection and may be any suitable display device. The display devices 820 may include any suitable display device, such as an LCD, LED, plasma display screen, CRT, television, monitor, touch screen, 3D display screen, or other visual display device. For example, the display devices 820 may be a flat display screen provided on a mobile device, multiple display screens provided on goggles, a headset device, or a monitor screen for a computing device.

[0089] The I / O interface 806 can be connected to other input and output devices. Some examples include one or more cameras that can take images. Some implementations can provide a microphone for capturing sound (e.g., as part of a captured image, voice commands, etc.), an audio speaker device for outputting sound, or other input / output devices.

[0090] For ease of illustration, FIG. 7 illustrates each of the processor 802, memory 804, I / O interface 806, and software blocks 808, 812, and 830 with one block. These blocks may represent one or more processors or processing circuits, operating systems, memories, I / O interfaces, applications, and / or software modules. In other implementations, the device 800 may not have all of the components shown, and / or may have other elements, including other types of elements instead of or in addition to the elements described herein. Although some components are described as performing the blocks and operations described in some implementations herein, any suitable component or combination of components of the environment 100, the device 800, a similar system, or any suitable processor associated with such a system, may perform the blocks and operations described herein.

[0091] The methods described herein may be implemented by computer program instructions or code executable on a computer. For example, the code may be implemented by one or more digital processors (e.g., microprocessors or other processing circuits) and stored on a computer program product. Computer program products include magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media including semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), flash memory, rigid magnetic disks, optical disks, solid-state memory drives, and the like. The program instructions may also be contained in and provided as electronic signals, for example, in a software as a service (SaaS) format delivered from a server (e.g., distributed systems and / or cloud computing systems). Alternatively, one or more methods may be implemented in hardware (e.g., logic gates) or a combination of hardware and software. Exemplary hardware may be programmable processors (e.g., field programmable gate arrays (FPGAs), complex programmable logic devices), general-purpose processors, graphic processors, application-specific integrated circuits (ASICs), and the like. One or more of the methods may be implemented as part of or as a component of an application executing on the system, or as an application or software running in conjunction with other applications and the operating system.

[0092] Although the present disclosure has been described with reference to particular implementations, these particular implementations are exemplary only and not limiting, and the concepts illustrated in these examples may be applied to other examples and implementations.

[0093] It should be noted that, as known to those skilled in the art, the functional blocks, operations, features, methods, devices, and systems described in this disclosure may be integrated or divided into different combinations of systems, devices, and functional blocks. Any suitable programming language and programming techniques may be used to implement the routines of a particular implementation. Different programming techniques (e.g., procedural or object-oriented programming techniques) may be employed. The routines may be executed on a single processing device or on multiple processors. Although steps, operations, or computations are shown in a particular order, this order may be changed in different particular implementations. In some implementations, multiple steps or operations shown as sequential in this specification may be executed simultaneously.

Claims

1. 1. A computer-implemented method, comprising: - estimating the depth of an image to obtain a depth map indicating the depth of each pixel of said image; generating a focus table for the image, the focus table including parameters indicative of a focus range and at least one of a front gradient or a rear gradient; The computer-implemented method further comprises: determining whether one or more faces are detected from the image; if it is determined that one or more faces have been detected in the image, identifying a respective face bounding box corresponding to each of the one or more faces, each of the face bounding boxes including an area of ​​the image corresponding to the face, and adjusting the focus table to include each of the face bounding boxes by expanding a range of focus depth values ​​until pixels of each face bounding box are within the focus range; if it is determined that no face has been detected in the image, scaling the focus table by adjusting at least one of the forward gradient and the backward gradient to limit an amount of blurring applied to the image; and generating an output image by applying blur to the image using the focus table and the depth map, the output image including in-focus regions and one or more blurred regions.

2. 2. The computer-implemented method of claim 1, wherein the focus table excludes the front gradients if foreground regions in front of an object are not present in the image and excludes the rear gradients if background regions behind an object are not present in the image.

3. The computer-implemented method of claim 1 , wherein the in-focus regions in the output image include pixels associated with depth values ​​in the depth map that correspond to a blur radius of zero.

4. generating the focus table includes using a focus table prediction model, the focus table prediction model being a trained machine learning model; The method further includes training the focus table prediction model; The training includes: providing a plurality of training images as input to the focus table prediction model, each training image having an associated depth map and an associated ground-truth blur radius image; For each training image, generating a predicted focus table using the focus table prediction model; and obtaining a predicted blur radius image using the predicted focus table and the depth map associated with the training images; calculating a loss value based on the predicted blur radius image and the ground truth blur radius image associated with the training images; and adjusting one or more parameters of the focus table predictive model using the loss value.

5. 5. The computer-implemented method of claim 4, wherein the depth map associated with each training image is one of a ground-truth depth map obtained at the time of image capture or an estimated depth map obtained using a depth prediction model.

6. 5. The computer-implemented method of claim 4, wherein training the focus table predictive model further comprises weighting the loss values ​​with image gradients of the training images before adjusting the one or more parameters of the focus table predictive model.

7. The computer-implemented method of claim 1 , wherein the image does not include information regarding focus and depth.

8. 10. The computer-implemented method of claim 1, wherein the image is a scanned photograph, an image with metadata removed, an image taken with a camera that does not store focus and depth information, or a frame of a video.

9. The computer-implemented method of claim 1 , further comprising displaying the output image.

10. A computer program which, when executed by a processor, causes the processor to carry out the method according to any one of claims 1 to 9.

11. A processor; A computing device comprising: a memory coupled to the processor and storing the computer program of claim 10.

Citation Information

Patent Citations

  • Image processing apparatus, image processing method and program

    JP2011066827A

  • Image processing device, image processing method, image processing program, and storage medium

    JP2012044551A

  • Imaging control device and control method therefor

    JP2017022560A

  • Electronic apparatus

    JP2021145209A

  • Method and apparatus for performing a blur rendering process on an image

    US20120219236A1

Cited By

  • Information processing device, control method for information processing device, and program

    JP2025180461A