Image processing method, apparatus, device, readable storage medium, and product

Abstract

Embodiments of the present disclosure provide an image processing method, an apparatus, a device, a readable storage medium, and a product. The image processing method comprises: acquiring an image to be processed, and determining depth information corresponding to said image; on the basis of the depth information, constructing three-dimensional data matching said image, and determining texture information corresponding to the three-dimensional data; on the basis of a preset trajectory and the depth information, performing a blurring operation on a plurality of pixels corresponding to the texture information, to obtain a blurred image; and performing a rendering operation on triangle meshes in the blurred image on the basis of at least one virtual camera in a preset virtual three-dimensional space, to obtain a blurred camera movement effect. According to the embodiments of the present disclosure, a camera movement effect can be presented while blurring is achieved, thereby effectively improving the image processing effect.

Description

Image processing method, device, equipment, readable storage medium and product

[0001] This application claims priority to Chinese Patent Application No. 202311657504.6 filed on December 5, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] Embodiments of the present disclosure relate to an image processing method, apparatus, device, readable storage medium, and product. Background Art

[0003] With the continuous development of image processing technology and terminal devices, more and more image processing applications are gradually entering users' lives. Users can perform image processing operations based on image processing applications on terminal devices according to actual needs, so that the processed images are more personalized to the user's needs. For example, to improve image quality or achieve the effect of image-to-video camera movement, users can blur specified areas in the image. Alternatively, users can blur the foreground or background of the image.

[0004] However, related image blurring solutions are generally only able to perform blurring operations based on a user-specified area, and the image blurring effect is often relatively simple, resulting in poor image processing effects. Summary of the Invention

[0005] Embodiments of the present disclosure provide an image processing method, apparatus, device, readable storage medium, and product.

[0006] In a first aspect, an embodiment of the present disclosure provides an image processing method, comprising:

[0007] Acquire an image to be processed, and determine depth information corresponding to the image to be processed;

[0008] constructing three-dimensional data matching the image to be processed based on the depth information, and determining texture information corresponding to the three-dimensional data;

[0009] Performing a blurring operation on a plurality of pixels corresponding to the texture information according to a preset trajectory and the depth information to obtain a blurred image;

[0010] The three-dimensional data in the blurred image is rendered based on at least one virtual camera in a preset virtual three-dimensional space to obtain a blurred camera effect.

[0011] In a second aspect, an embodiment of the present disclosure provides an image blurring device, comprising:

[0012] an acquisition module, configured to acquire an image to be processed and determine depth information corresponding to the image to be processed;

[0013] a construction module configured to construct three-dimensional data matching the image to be processed based on the depth information, and determine texture information corresponding to the three-dimensional data;

[0014] a blurring module configured to perform a blurring operation on a plurality of pixels corresponding to the texture information according to a preset trajectory and the depth information to obtain a blurred image;

[0015] The rendering module is configured to render the three-dimensional data in the blurred image based on at least one virtual camera in a preset virtual three-dimensional space to obtain a blurred camera effect.

[0016] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0017] The memory stores computer-executable instructions;

[0018] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the image processing method described in the first aspect and various possible designs of the first aspect.

[0019] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the image processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0020] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method described in the first aspect and various possible designs of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] FIG1 is a schematic diagram of a flow chart of an image processing method provided by an embodiment of the present disclosure;

[0023] FIG2 is a schematic diagram of image processing provided by an embodiment of the present disclosure;

[0024] FIG3 shows depth information determined based on an image to be processed;

[0025] FIG4 is a schematic diagram of a three-dimensional triangular mesh constructed based on depth information;

[0026] FIG5 is a schematic diagram of a flow chart of an image blurring method according to another embodiment of the present disclosure;

[0027] FIG6 is a schematic flow chart of an image blurring method according to another embodiment of the present disclosure;

[0028] FIG7 is a schematic flow chart of an image blurring method according to another embodiment of the present disclosure;

[0029] FIG8 is a schematic structural diagram of an image blurring mirror movement device provided by another embodiment of the present disclosure; and

[0030] FIG9 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0032] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0033] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0034] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0035] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0036] In order to solve the technical problem that the image blur processing method has a relatively single effect, the present disclosure provides an image processing method, device, equipment, readable storage medium and product.

[0037] It should be noted that the image processing method, apparatus, device, readable storage medium and product provided by the present disclosure can be applied in any image processing application scenario.

[0038] Current image blurring algorithms can only blur a user-specified image area, creating a static blur effect. However, the image processing effect is often relatively simple, resulting in poor image quality.

[0039] In the process of solving the above technical problems, the inventors of the present disclosure discovered through research that in order to improve the graphics processing effect, a preset trajectory can be set in advance, and a dynamic blurring effect can be achieved based on the preset trajectory and depth information.

[0040] Furthermore, the image blurring and dynamic three-dimensional camera movement can be combined to present the processing effect of dynamic three-dimensional camera movement while blurring the image.

[0041] Optionally, after acquiring an image to be processed, three-dimensional data corresponding to the image to be processed can be constructed based on depth information corresponding to the image to be processed. A blurring operation is performed on multiple pixels associated with texture information corresponding to the three-dimensional data based on a preset trajectory and depth information. Furthermore, by presetting at least one virtual camera, a rendering operation is performed on the triangular mesh in the blurred image based on the position of the virtual camera. This allows for dynamic blurring and a dynamic three-dimensional camera movement effect to be presented, effectively improving the image processing effect.

[0042] The system architecture underlying this disclosure includes at least a terminal device and a server. A user can trigger an image processing request on the terminal device. Accordingly, the server can receive the image processing request sent by the terminal device, which can include the image to be processed. The server can then perform blurring and camera movement on the image to be processed, and then feed the blurring and camera movement results back to the terminal device for display.

[0043] FIG1 is a flow chart of an image processing method provided by an embodiment of the present disclosure. As shown in FIG1 , the method includes:

[0044] Step 101: Acquire an image to be processed and determine depth information corresponding to the image to be processed.

[0045] The execution subject of this embodiment is an image processing device. The image processing device can be coupled to a server, which can communicate with a terminal device to obtain an image to be processed sent by the terminal device, blur the image and perform camera movement processing on the image to be processed, and feed the blur and camera movement results back to the terminal device for display.

[0046] Alternatively, the image processing device may be coupled to a terminal device, so as to perform blurring and camera movement processing based on the image to be processed acquired by the terminal device, and feed back the blurring and camera movement results to the terminal device for display.

[0047] In this embodiment, in order to realize the image blurring and mirror movement processing, the image to be processed can be first obtained. The image to be processed can be currently collected in real time, or can be uploaded by the user in a preset storage path, and this disclosure does not impose any restrictions on this.

[0048] Furthermore, in order to present a blur effect, the depth information corresponding to the image to be processed may be determined. The grayscale information of the image to be processed may be determined in any manner, and this disclosure does not limit this. The grayscale information may be a grayscale image corresponding to the image to be processed.

[0049] Blur is essentially an image blurring technique. Unlike standard blurring, blurring varies the blur level based on the distance from the camera lens to the scene represented by the image pixel. For example, in a blurred distant scene, the closer the image is to the camera lens, the less blurry it is, while the farther the image is from the camera lens, the more blurry it is. This distance is the depth information, allowing for accurate blurring of the processed image based on this depth information.

[0050] Step 102: construct three-dimensional data matching the image to be processed based on the depth information, and determine texture information corresponding to the three-dimensional data.

[0051] In this embodiment, in order to accurately blur each area in the image to be processed, after obtaining the depth information corresponding to the image to be processed, three-dimensional data matching the image to be processed can be constructed based on the depth information. The three-dimensional data includes, but is not limited to, a three-dimensional triangular mesh, which includes multiple triangular meshes and can represent the shape and depth relationship of the content in the image to be processed.

[0052] In order to accurately achieve blurring and camera movement processing, the texture information corresponding to each triangular mesh in the three-dimensional data can also be determined.

[0053] Optionally, foreground texture information and background texture information can be pre-set. The foreground texture information can be the image to be processed, while the background texture information can include edge regions within the image to be processed. Thus, after obtaining the three-dimensional data, the texture information corresponding to each triangular mesh can be determined based on the foreground and background texture information.

[0054] Step 103: performing a blurring operation on a plurality of pixels corresponding to the texture information according to the preset trajectory and the depth information to obtain a blurred image.

[0055] In this embodiment, a preset trajectory can be pre-set to tailor the blur effect to the user's individual needs. This trajectory includes the degree of foreground blur and background blur corresponding to each time frame within a preset time range. For example, the user can set the preset trajectory to represent the transition from foreground blur to background blur, or alternatively, the user can set the preset trajectory to represent the transition from background blur to foreground blur, depending on actual needs.

[0056] Therefore, after determining the preset trajectory, a blurring operation may be performed on multiple pixels corresponding to the texture information based on the preset trajectory and the depth information to obtain a blurred image, so that the blurred image presents a blurring effect consistent with the preset trajectory.

[0057] Step 104: Render the three-dimensional data in the blurred image based on at least one virtual camera in a preset virtual three-dimensional space to obtain a blurred camera effect.

[0058] In this embodiment, to create a camera effect that allows the processed image to appear as if it were captured from different angles, at least one virtual camera can be pre-set within the virtual three-dimensional space. A virtual camera is set for each time frame, and different virtual cameras can correspond to different time frames. This allows the at least one virtual camera to be used to render the triangular mesh in the blurred image, achieving a blurred camera effect.

[0059] Optionally, the blurring camera movement processing effect may be video media content of a preset time length, which can present a dynamic blurring effect and camera movement effects shot from different angles in the video media content.

[0060] FIG2 is a schematic diagram of image processing provided by an embodiment of the present disclosure. As shown in FIG2 , after obtaining an image 21 to be processed, three-dimensional data corresponding to the image 21 to be processed can be constructed based on the depth information corresponding to the image 21 to be processed. A blurring operation is performed on multiple pixels associated with the texture information corresponding to the three-dimensional data based on a preset trajectory and depth information. The triangular mesh in the blurred image is rendered using the preset position and angle of at least one virtual camera, thereby simultaneously presenting a camera movement effect while blurring, resulting in a blurred camera movement effect 22. In the blurred camera movement effect 22, the house area 23 is blurred and is closer to the preset lens, presenting the visual effect of the preset lens gradually approaching the house area 23.

[0061] FIG3 shows depth information determined based on an image to be processed, wherein the image to be processed may be an image of a flower, and the depth information map may represent the distance between each position corresponding to the flower and a preset lens.

[0062] FIG4 is a schematic diagram of a three-dimensional triangular mesh constructed based on depth information. As shown in FIG4 , the three-dimensional triangular mesh 41 includes multiple triangular meshes 42 . The three-dimensional triangular mesh can represent the shape and depth relationship of the flower in the flower image.

[0063] The image processing method provided in this embodiment constructs three-dimensional data corresponding to the image to be processed based on its depth information after acquiring it. Then, based on a preset trajectory and depth information, it performs a blurring operation on multiple pixels associated with the texture information corresponding to the three-dimensional data. This allows the image to present a dynamic blurring effect that matches the preset trajectory. Furthermore, by pre-setting virtual cameras at different positions and angles for each time frame, and rendering the triangular mesh in the blurred image based on the virtual camera's position and angle, this method can simultaneously present a camera movement effect while blurring, effectively improving the image processing performance.

[0064] FIG5 is a flowchart of an image processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG3 , step 102 includes:

[0065] Step 501: construct a first-level depth image with an undirected graph structure based on the image to be processed and depth information, wherein non-edge pixels in the first-level depth image are connected by preset edges.

[0066] Step 502: Fill a preset range of edge pixels in the first-level depth image to obtain a second-level depth image with overlapping nodes in the image space.

[0067] Step 503 : performing a block operation on the second-level depth image by a flood filling method to obtain a third-level depth image after block division, wherein no overlapping points in the image space exist in the image blocks in the third-level depth image.

[0068] Step 504: perform a gridding operation on the third-level depth image to obtain three-dimensional data.

[0069] In this embodiment, after obtaining the image to be processed and the depth information corresponding to the image to be processed, a first-level depth image with an undirected graph structure can be constructed based on the image to be processed and the depth information. In the first-level depth image, non-edge adjacent pixels are connected by preset edges, while there is no edge between two adjacent pixels that meet the edge conditions.

[0070] Furthermore, after obtaining the first-level depth image, the pixel points without preset edge connections can be diffusely filled with an appropriate range to obtain a second-level depth image with overlapping nodes in the image space. Furthermore, the second-level depth image can be divided into blocks using a flood fill method to obtain a third-level depth image after division, wherein the image blocks in the third-level depth image do not have overlapping points in the image space. Optionally, for the boundaries and endpoints in the second-level depth image, the endpoints of each set of boundaries can be extended to the image boundary, so that the image can be completely segmented through each set of boundaries. Starting from any node, a breadth-first search traversal is performed towards adjacent nodes with edges. Within a certain range centered on the starting point (e.g., a 50x50 pixel block area), the traversal process is ensured to not exceed the range, not cross the boundary, and not cross the extension line of the boundary with endpoints within the range. This ensures that the image blocks obtained in each traversal do not overlap in space. After a traversal is completed, the above process is repeated until all nodes are traversed to obtain the third-level depth image.

[0071] Furthermore, after obtaining the third-level depth image, the third-level depth image can be meshed to obtain three-dimensional data. The boundary of each image block is expanded downward and rightward by one pixel to ensure that there are no cracks at the image block boundary after meshing. Vertices are selected from the boundary at appropriate intervals, and the shared boundary vertices of adjacent pixel blocks are kept the same. A constrained triangulation algorithm is then applied by uniformly selecting points in the image block to obtain the triangulation result of the image block. Finally, the textures of all image blocks are mapped onto the same high-resolution texture image, and this is used together with the obtained triangulation result as the meshing output to obtain three-dimensional data.

[0072] The image processing method provided in this embodiment constructs a first-level depth image of an undirected graph structure based on the image to be processed and depth information, diffusely fills the edge pixels of the first-level depth image to obtain a second-level depth image with overlapping nodes in the image space, and blocks the second-level depth image using a flood fill method to obtain a third-level depth image after block division, and then grids the third-level depth image. In this way, a virtual three-dimensional scene can be obtained based on a single image to be processed, and then dynamic blurring and camera movement effects can be performed based on the virtual three-dimensional scene. In addition, blurring and camera movement operations on the image to be processed can be implemented on a graphics processor, thereby effectively improving the processing efficiency of dynamic blurring and three-dimensional camera movement.

[0073] FIG6 is a flow chart of an image processing method provided by another embodiment of the present disclosure. Texture information includes foreground texture information and background texture information. Based on any of the above embodiments, as shown in FIG6 , step 102 includes:

[0074] Step 601: Determine the mapping relationship between three-dimensional data and pixels in the image to be processed.

[0075] Step 602: Determine texture information corresponding to the three-dimensional data based on the mapping relationship and the foreground texture information and the background texture information.

[0076] In this embodiment, in order to achieve a dynamic blur effect, foreground texture information and background texture information can be pre-set. During the blurring process, the foreground blur can be switched to the background blur based on a preset trajectory, or the background blur can be switched to the foreground blur based on a preset trajectory.

[0077] Therefore, after constructing the 3D data, since the 3D data is constructed based on the depth information corresponding to the image to be processed, a mapping relationship exists between the 3D data and the image to be processed. The mapping relationship between the 3D data and pixels in the image to be processed can be determined. Based on this mapping relationship and the foreground texture information and background texture information, the texture information corresponding to the 3D data is determined.

[0078] The image processing method provided in this embodiment, by separately setting foreground and background texture information, can subsequently accurately achieve a dynamic blurring effect, transitioning from foreground blur to background blurring. Furthermore, by determining the texture information corresponding to the three-dimensional data based on the mapping relationship between the three-dimensional data and the pixels in the image to be processed, the accuracy of the blurring process can be improved, thereby enhancing the image processing effect.

[0079] Furthermore, based on any of the above embodiments, before step 602, the method further includes:

[0080] The image to be processed is determined as foreground texture information.

[0081] The edge mask corresponding to the image to be processed is determined by a preset edge detection operator, and a filling operation is performed on the image to be processed based on the edge mask to obtain background texture information.

[0082] In this embodiment, before blurring the image to be processed, it is first necessary to determine the foreground texture information and the background texture information.

[0083] Optionally, the image to be processed can be determined as foreground texture information. An edge mask corresponding to the image to be processed is determined using a preset edge detection algorithm. This edge mask can describe the edge structure of the content displayed in the image to be processed. Based on this edge mask, a filling operation is performed on the image to obtain background texture information.

[0084] The image processing method provided in this embodiment can achieve a dynamic blur effect of transforming foreground blur to background blur based on a preset trajectory set by the user by presetting foreground texture information and background texture information.

[0085] Furthermore, based on any of the above embodiments, step 103 includes:

[0086] A convolution operation is performed on multiple pixels corresponding to the texture information according to the preset trajectory and depth information to obtain a blurred image.

[0087] In this embodiment, to achieve dynamic image blurring, the user can set a preset trajectory based on actual needs. This preset trajectory includes the degree of foreground blur and background blur corresponding to each time frame within a preset time range. For example, the user can set the preset trajectory to represent the transition from foreground blur to background blur. Alternatively, the user can set the preset trajectory to represent the transition from background blur to foreground blur based on actual needs.

[0088] Optionally, after determining the preset trajectory, a convolution operation may be performed on a plurality of pixels corresponding to the texture information based on the preset trajectory and the depth information to obtain a blurred image.

[0089] Furthermore, based on any of the above embodiments, the preset trajectory includes the foreground blur degree and the background blur degree corresponding to each time frame within a preset time range. A convolution operation is performed on multiple pixels corresponding to the texture information according to the preset trajectory and the depth information to obtain a blurred image, including:

[0090] Determine the blur level corresponding to the current time frame based on the preset trajectory.

[0091] Target depth information matching the blur degree is determined according to the blur degree and the depth information.

[0092] A convolution operation is performed on a plurality of corresponding pixels that match the preset target depth information in the texture information to obtain a blurred image.

[0093] In this embodiment, after determining the preset trajectory, for each time frame, the blur degree corresponding to the time frame can be determined. The blur degree can include the foreground blur degree and the background blur degree.

[0094] Defocusing is essentially an image blurring technique. Unlike standard blurring, defocusing varies the degree of blur based on the distance of the scene represented by the image pixel from the camera lens. For example, when defocusing a distant scene, the closer it is to the camera lens, the less blurry it is, while the farther away, the more blurry it is. This distance is reflected in the depth data.

[0095] Therefore, after determining the blur level of the current frame, target depth information matching the blur level can be determined based on the blur level and depth information. A convolution operation is performed on corresponding pixels in the texture information that match the preset target depth information to obtain a blurred image.

[0096] The image processing method provided in this embodiment pre-sets a preset trajectory, thereby determining the degree of blurring for each frame based on the preset trajectory, and further determining the number of pixels to be blurred based on the blurring degree. This allows for a different blurring effect to be presented in each time frame, resulting in a blurring effect consistent with the preset trajectory set by the user.

[0097] FIG7 is a flowchart of an image processing method provided by another embodiment of the present disclosure. Based on any of the above embodiments, as shown in FIG7 , step 104 includes:

[0098] Step 701: Determine the position information and angle information of the virtual camera in the current time frame.

[0099] Step 702: Rendering operation is performed on the triangular mesh in the blurred image by at least one virtual camera according to the position information and the angle information to obtain a blurred camera effect.

[0100] In this embodiment, in order to present a three-dimensional camera effect, a virtual camera can be set up in the virtual space in advance. In each time frame, the display position and display angle of the virtual camera are different, so that different image effects can be presented by changing the position and angle of the virtual camera.

[0101] Optionally, the position and angle information of the virtual camera in the current time frame can be determined. Thus, during texture rendering, a rendering operation can be performed on the triangular mesh in the blurred image using at least one virtual camera based on the position and angle information, thereby achieving a blurred camera effect. For example, in the current frame, the virtual camera can be located at a first position in the virtual world, and in the next frame, the virtual camera can be located at a second position in the virtual world. By performing rendering operations based on the position and angle information, a visual effect of a camera angle changing from the first position to the second position can be presented.

[0102] The image processing method provided in this embodiment pre-sets a virtual camera corresponding to each time frame in a virtual space, thereby rendering a triangular mesh based on the position information and angle information corresponding to the virtual camera in each frame to present the image effect captured during the camera movement, thereby achieving an image processing effect that combines dynamic blur and three-dimensional camera movement.

[0103] Furthermore, based on any of the above embodiments, after step 104, the following steps may be further included:

[0104] At least one pixel having a missing pixel value during at least one virtual camera switching process is determined.

[0105] Perform a filling operation on at least one pixel using a preset filling algorithm.

[0106] In this embodiment, since the position of the virtual camera is different in each frame, the shooting angle may change, and the change of the angle may cause the loss of pixels, affecting the image processing effect.

[0107] Optionally, at least one pixel whose pixel value is missing during at least one virtual camera switching process can be determined. Any pixel recognition algorithm can be used to determine the missing pixel, and this disclosure does not limit this. Furthermore, a filling operation can be performed on the at least one pixel using a preset filling algorithm. The pixel filling operation can be implemented using an inpainting algorithm, or any other pixel filling algorithm can be used to fill the missing pixel, and this disclosure does not limit this.

[0108] The image processing method provided in this embodiment can further improve the image processing effect and enhance the quality of the processed media content by identifying and filling in the missing pixels during the perspective conversion process.

[0109] Figure 8 is a structural diagram of an image processing device provided by another embodiment of the present disclosure. As shown in Figure 8, the device includes: an acquisition module 81, a construction module 82, a blurring module 83 and a rendering module 84. Among them, the acquisition module 81 is configured to acquire the image to be processed and determine the depth information corresponding to the image to be processed. The construction module 82 is configured to construct three-dimensional data matching the image to be processed based on the depth information and determine the texture information corresponding to the three-dimensional data. The blurring module 83 is configured to perform a blurring operation on multiple pixels corresponding to the texture information according to a preset trajectory and depth information to obtain a blurred image. The rendering module 84 is configured to render the three-dimensional data in the blurred image based on at least one virtual camera in a preset virtual three-dimensional space to obtain a blurred camera effect.

[0110] Furthermore, based on any of the above embodiments, the construction module is configured to: construct a first-level depth image of an undirected graph structure based on the image to be processed and the depth information, and the non-edge pixels in the first-level depth image are connected by preset edges. The edge pixels in the first-level depth image are filled with a preset range to obtain a second-level depth image with overlapping nodes in the image space. The second-level depth image is divided into blocks by a flood filling method to obtain a third-level depth image after division, wherein there are no overlapping points in the image space in the image blocks in the third-level depth image. The third-level depth image is gridded to obtain three-dimensional data.

[0111] Furthermore, based on any of the above embodiments, the construction module is configured to: determine a mapping relationship between the 3D data and pixels in the image to be processed, and determine texture information corresponding to the 3D data based on the mapping relationship and the foreground texture information and the background texture information.

[0112] Furthermore, based on any of the above embodiments, the image processing apparatus further includes: a determination module configured to determine the image to be processed as foreground texture information; a processing module configured to determine an edge mask corresponding to the image to be processed using a preset edge detection operator, and perform a fill operation on the image to be processed based on the edge mask to obtain background texture information.

[0113] Furthermore, based on any of the above embodiments, the blurring module is configured to: perform a convolution operation on a plurality of pixels corresponding to the texture information according to a preset trajectory and depth information to obtain a blurred image.

[0114] Furthermore, based on any of the above embodiments, the preset trajectory includes a foreground blur degree and a background blur degree corresponding to each time frame within a preset time range. The blur module is configured to: determine a blur degree corresponding to the current time frame based on the preset trajectory; determine target depth information that matches the blur degree based on the blur degree and the depth information; and perform a convolution operation on a plurality of pixels corresponding to the preset target depth information in the texture information to obtain a blurred image.

[0115] Furthermore, based on any of the above embodiments, the rendering module is configured to determine position information and angle information of a virtual camera within a current time frame, and perform a rendering operation on a triangle mesh in the blurred image for at least one virtual camera based on the position information and angle information to achieve a blurred camera effect.

[0116] Furthermore, based on any of the above embodiments, the image processing apparatus further includes: a determination module configured to determine at least one pixel whose pixel value is missing during at least one virtual camera switching process; and a filling module configured to perform a filling operation on the at least one pixel using a preset filling algorithm.

[0117] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0118] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, an image processing method as described in any of the above embodiments is implemented.

[0119] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer program product, including a computer program. When the computer program is executed by a processor, the image processing method of any of the above embodiments is implemented.

[0120] In order to implement the above embodiment, the embodiment of the present disclosure further provides an electronic device: a processor and a memory;

[0121] Memory stores computer-executable instructions;

[0122] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the image processing method according to any one of the above embodiments.

[0123] FIG9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. The electronic device 900 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG9 is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0124] As shown in Figure 9, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0125] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although FIG9 shows an electronic device 900 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0126] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0127] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0128] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0129] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0130] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0132] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0133] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0134] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0136] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0137] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An image processing method, comprising: Acquire an image to be processed, and determine depth information corresponding to the image to be processed; constructing three-dimensional data matching the image to be processed based on the depth information, and determining texture information corresponding to the three-dimensional data; Performing a blurring operation on a plurality of pixels corresponding to the texture information according to a preset trajectory and the depth information to obtain a blurred image; The three-dimensional data in the blurred image is rendered based on at least one virtual camera in a preset virtual three-dimensional space to obtain a blurred camera effect.

2. The method according to claim 1, wherein: The step of constructing three-dimensional data matching the image to be processed based on the depth information includes: Constructing a first-level depth image of an undirected graph structure based on the image to be processed and the depth information, wherein non-edge pixels in the first-level depth image are connected by preset edges; Filling a preset range of edge pixels in the first-level depth image to obtain a second-level depth image with overlapping nodes in the image space; Performing a block operation on the second-level depth image by a flood filling method to obtain a third-level depth image after block division, wherein there are no overlapping points in the image space in the image blocks in the third-level depth image; A gridding operation is performed on the third-level depth image to obtain the three-dimensional data.

3. The method according to claim 1, wherein: The texture information includes foreground texture information and background texture information, and determining the texture information corresponding to the three-dimensional data includes: Determining a mapping relationship between the three-dimensional data and pixels in the image to be processed; Texture information corresponding to the three-dimensional data is determined based on the mapping relationship and the foreground texture information and the background texture information.

4. The method according to claim 3, wherein: Before determining the texture information corresponding to the three-dimensional data based on the mapping relationship and the foreground texture information and the background texture information, the method further includes: Determining the image to be processed as the foreground texture information; An edge mask corresponding to the image to be processed is determined by a preset edge detection operator, and a filling operation is performed on the image to be processed based on the edge mask to obtain the background texture information.

5. The method according to claim 1, wherein: The blurring operation is performed on a plurality of pixels corresponding to the texture information according to the preset trajectory and the depth information, comprising: A convolution operation is performed on a plurality of pixels corresponding to the texture information according to the preset trajectory and the depth information to obtain a blurred image.

6. The method according to claim 5, wherein: The preset trajectory includes the foreground blur degree and the background blur degree corresponding to each time frame within the preset time range; The step of performing a convolution operation on a plurality of pixels corresponding to the texture information according to the preset trajectory and the depth information to obtain a blurred image includes: Determining the blur degree corresponding to the current time frame according to the preset trajectory; Determine target depth information matching the blur degree according to the blur degree and the depth information; A convolution operation is performed on a plurality of corresponding pixels that match the preset target depth information in the texture information to obtain a blurred image.

7. The method according to any one of claims 1 to 6, wherein: The rendering of the three-dimensional data in the blurred image based on at least one virtual camera in a preset virtual three-dimensional space to obtain a blurred camera effect includes: Determine the position information and angle information of the virtual camera within the current time frame; A rendering operation is performed on the triangular mesh in the blurred image according to the position information and the angle information to obtain a blurred camera effect.

8. The method according to claim 7, wherein: After rendering the three-dimensional data in the blurred image based on at least one virtual camera in the preset virtual three-dimensional space to obtain the blurred camera effect, the method further includes: Determine at least one pixel whose pixel value is missing during the switching process of the at least one virtual camera; A filling operation is performed on the at least one pixel using a preset filling algorithm.

9. An image processing device, comprising: An acquisition module is configured to acquire an image to be processed and determine depth information corresponding to the image to be processed; A construction module, configured to construct three-dimensional data matching the image to be processed based on the depth information, and determine texture information corresponding to the three-dimensional data; A blur module, configured to perform a blur operation on a plurality of pixels corresponding to the texture information according to a preset trajectory and the depth information to obtain a blurred image; as well as The rendering module is configured to render the three-dimensional data in the blurred image based on at least one virtual camera in a preset virtual three-dimensional space to obtain a blurred camera effect.

10. An electronic device comprising: Processor and memory; Wherein, the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the image processing method according to any one of claims 1 to 8 is implemented.

Images