Picture processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a picture processing method and device, electronic equipment and storage medium, and relate to the technical field of games, the method comprises: displaying a real scene of an environment in which a player is located and a virtual window in a graphical user interface, and displaying a game scene in the virtual window; obtaining a first depth map corresponding to the real scene and a second depth map corresponding to the game scene; mixing the first depth map and the second depth map to obtain a target depth map; performing filter processing on the real scene according to the target depth map, and displaying the first real scene after processing in the graphical user interface based on the result of the filter processing, thereby ensuring the stability of the interface display content, and also reducing the discomfort caused by the inconsistency of brightness between the virtual window and the real scene, and enabling the player's attention to be effectively concentrated in the virtual window, thereby improving the player's visual experience.

Description

Technical Field

[0001] This invention relates to the field of game technology, and in particular to a method for processing images, an apparatus for processing images, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the development of science and technology, game developers have created different types of games for various hardware devices, such as mobile games, PC games, AR (Augmented Reality) games, and VR (Virtual Reality) games. In these games, players can play using the corresponding gaming devices. However, during the display of game visuals on these devices, issues such as unstable backgrounds, the need for players to repeatedly adapt to different lighting conditions, and a lack of visual focus severely impact the player's gaming experience. Summary of the Invention

[0003] The present invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing images, in order to solve or partially solve the problems of unstable backgrounds, the need for players to repeatedly adapt to different lighting conditions, and the lack of visual focus in game images.

[0004] This invention discloses a method for processing images, providing a graphical user interface via an electronic terminal. The method includes:

[0005] The graphical user interface displays the real-world scene of the player's environment and a virtual window, and displays the game scene in the virtual window;

[0006] Obtain the first depth map corresponding to the real-world scene and the second depth map corresponding to the game scene;

[0007] The first depth map and the second depth map are blended to obtain the target depth map;

[0008] The real-world scene is filtered based on the target depth map, and the processed first real-world scene is displayed in the graphical user interface based on the result of the filtering.

[0009] This invention also discloses a screen processing apparatus that provides a graphical user interface via an electronic terminal. The apparatus includes:

[0010] The scene display module is used to display the real scene of the player's environment and the virtual window in the graphical user interface, and to display the game scene in the virtual window;

[0011] The depth information acquisition module is used to acquire a first depth map corresponding to the real scene and a second depth map corresponding to the game scene.

[0012] A depth information fusion module is used to mix the first depth map and the second depth map to obtain a target depth map;

[0013] A filter processing module is used to perform filter processing on the real scene based on the target depth map, and display the processed first real scene in the graphical user interface based on the filter processing result.

[0014] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0015] The memory is used to store computer programs;

[0016] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0017] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0018] The embodiments of the present invention have the following advantages:

[0019] In this embodiment of the invention, for the real-world scene of the player's environment and the game scene displayed in the virtual window during the game, the terminal can obtain a first depth map corresponding to the real-world scene and a second depth map corresponding to the game scene, and mix the first depth map and the second depth map to obtain a target depth map. Then, the real-world scene is filtered according to the target depth map, and the processed first real-world scene is displayed in the graphical user interface based on the result of the filter processing. In this way, the virtual content is combined with the real-world scene through the depth map, which can ensure the stability of the interface display content without changing the layout of the real-world scene. At the same time, it can also reduce the discomfort caused by the brightness inconsistency between the virtual window and the real-world scene, and enable the player's attention to be effectively focused on the virtual window, thereby improving the player's visual experience. Attached Figure Description

[0020] Figure 1 This is a flowchart of the steps of a screen processing method provided in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the depth map provided in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the depth map provided in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the depth map provided in an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of the game interface provided in an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of the game interface provided in an embodiment of the present invention;

[0026] Figure 7 This is a structural block diagram of a screen processing device provided in an embodiment of the present invention. Detailed Implementation

[0027] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] As an example, in the corresponding games, players can play the games through corresponding gaming devices. However, during the process of displaying the game screen on the gaming device, there are problems such as unstable backgrounds, players needing to repeatedly adapt to different lighting conditions, and a lack of visual focus, which seriously affect the player's gaming experience.

[0029] In this invention, for the real-world scene of the player's environment and the game scene displayed in the virtual window during gameplay, the terminal can acquire a first depth map corresponding to the real-world scene and a second depth map corresponding to the game scene. The first depth map and the second depth map are then mixed to obtain a target depth map. The real-world scene is then filtered based on the target depth map. Based on the result of the filter processing, the processed first real-world scene is displayed in the graphical user interface. This combines virtual content with the real-world scene through the depth map, ensuring the stability of the displayed content without changing the layout of the real-world scene. It also reduces discomfort caused by the brightness inconsistency between the virtual window and the real-world scene, and allows the player's attention to be effectively focused on the virtual window, thus improving the player's visual experience.

[0030] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, some technical features involved in the embodiments of the present invention are explained and described below:

[0031] Overlay is a technique in image processing and graphic design. Its main function is to overlay one image or color layer onto another, creating different visual effects. The basic principles of overlay include: Color blending: Overlay generates new color values ​​by performing specific mathematical operations on the color values ​​of the underlying and top-level images. Typically, this blending effect enhances the brightness of the underlying image; Brightness value: The brightness value of the top-level image affects the final result. The higher the brightness value, the easier it is to see the details in the underlying image; Visual effects: Overlay can produce effects such as enhanced contrast and richer colors, making images more vivid and layered.

[0032] A depth map is a type of image data used to represent the distance or depth information of each pixel in a scene from the observer. It is typically presented as a grayscale image, where the grayscale value of each pixel corresponds to the depth of that point. Related concepts involved in depth maps include:

[0033] 1. Grayscale representation: In a depth map, white can represent near distance, black represents far distance, and gray represents intermediate distance. This method allows depth information to be represented and processed using a two-dimensional image.

[0034] 2. Pixel depth: The value of each pixel represents the depth information of that point, usually expressed in units (such as meters) or normalized values ​​(relative values).

[0035] 3. Acquisition Methods: Depth maps can be obtained in various ways, including:

[0036] Stereo vision: using images acquired by binocular cameras to perform depth calculations.

[0037] Laser scanning: using equipment such as lidar to directly measure depth.

[0038] Depth camera: Through specialized equipment, it can directly output depth maps.

[0039] Specifically, refer to Figure 1 This diagram illustrates a flowchart of a screen processing method provided in an embodiment of the present invention, which provides a graphical user interface via an electronic terminal and may specifically include the following steps:

[0040] Step 101: Display the real-world scene and virtual window of the player's environment in the graphical user interface, and display the game scene in the virtual window;

[0041] For electronic terminals (hereinafter referred to as terminals), these can be terminals running game applications. During gameplay, the terminal's graphical user interface can present the corresponding game scene to the player. In some games, in addition to the game scene, the terminal's graphical user interface can also display the real-world environment in which the player is located.

[0042] For example, assuming the electronic terminal is a head-mounted display device, the head-mounted display device can display the real scene corresponding to the player's environment in the graphical user interface, and at the same time display the corresponding virtual window. The virtual window can display the corresponding game scene, so that the player can see the real scene and the virtual screen at the same time inside the head-mounted display device.

[0043] Optionally, the head-mounted display device can be a spatial computing device that integrates technologies such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality) to provide players with a good visual experience.

[0044] It should be noted that when the terminal simultaneously displays both real-world and virtual scenes, users using the hovering window function of the head-mounted display may be in various environments. For some products, especially games requiring an immersive experience, constantly changing backgrounds can easily affect and interfere with the player's overall experience of the game. When the content displayed in the virtual window differs significantly from the real-world background, the user's eyes need to repeatedly move back and forth between the two lighting conditions, and the pupils need to adapt, which may cause discomfort. Furthermore, when displaying a virtual window, the difference between the game scene shown in the virtual window and the real-world scene can easily lead to a lack of visual focus, affecting the player's immersive experience. To address these potential issues, this application allows for the processing of the real-world scene during gameplay, based on the player's needs, to provide a better visual experience.

[0045] Step 102: Obtain the first depth map corresponding to the real-world scene and the second depth map corresponding to the game scene;

[0046] During gameplay, the terminal can acquire a first depth map of the real-world scene and a second depth map of the game scene based on preset trigger conditions or player actions. This allows the terminal to process the real-world scene using the first and second depth maps, and then combine the game scene with the real-world scene based on the processing results. This reduces the impact on other people in the same space without changing the layout of the real-world scene.

[0047] The depth map can be an image representing the distance of each pixel in the scene to the camera. The depth map can be labeled in the form of a grayscale image. Pixels closer to the camera have higher brightness values, while pixels farther from the camera have lower brightness values.

[0048] In some feasible implementations, when a player inputs an operation to add a filter to a real-world scene, it can trigger the processing of the real-world scene using a depth map. Specifically, the terminal can respond to the filter addition operation by obtaining a first depth map corresponding to the real-world scene and a second depth map corresponding to the virtual viewport. Optionally, if the graphical user interface displays functional controls, then when the electronic terminal is a head-mounted display device, the player can input a gaze operation targeting a functional space. The terminal, in response to the gaze operation targeting the functional controls, scans the environment to obtain a first depth map corresponding to the real-world scene and acquires depth information corresponding to the game scene through the virtual camera in the game, generating a second depth map corresponding to the depth information. This allows for the addition of appropriate filters to the real-world scene based on the first and second depth maps, thus combining the game scene with the real-world scene and ensuring the overall consistency of the image without changing the scene layout in the real-world scene.

[0049] It should be noted that, in addition to triggering filter addition through corresponding functional controls, filters can also be added to the real-world scene via voice commands, physical buttons, etc., allowing players to choose according to their actual needs. Filter processing for the real-world scene includes, but is not limited to, blurring, brightness adjustment, and texture addition. For example, in the case of blurring, combining a depth map can adjust the blur level of the real-world scene; in the case of brightness adjustment, combining a depth map can adjust the brightness of the real-world scene; and in the case of adding textures, combining a depth map can add corresponding texture images (such as static images corresponding to game scenes) to the real-world scene. This invention does not impose any limitations on these aspects.

[0050] Step 103: Mix the first depth map and the second depth map to obtain the target depth map;

[0051] After obtaining the first depth map corresponding to the real scene and the second depth map corresponding to the game scene, the terminal can mix the first depth map and the second depth map to obtain the corresponding target depth map. This allows for image processing of the real scene based on the target depth map, achieving the fusion of virtual content and real scene, and ensuring the overall consistency of the image without changing the scene layout in the real scene.

[0052] In some feasible implementations, the first depth map includes a first pixel, and the second depth map includes a second pixel. During the blending calculation, the first brightness value corresponding to each first pixel in the first depth map and the second brightness value corresponding to each second pixel in the second depth map can be obtained first. Then, multiplying is performed based on the first and second brightness values ​​to obtain the target depth map corresponding to the first and second depth maps. This allows for image processing of the real scene based on the target depth map, achieving the fusion between virtual content and the real scene, and ensuring the overall consistency of the image without changing the scene layout in the real scene.

[0053] In the process of performing mixed calculations on the first depth map and the second depth map, in order to ensure the accuracy of the mixed calculations, a multiply operation can be performed on the pixels corresponding to the same position in the first depth map and the second depth map, based on the same position. Specifically, a first target pixel can be extracted from the first pixel, and a second target pixel can be extracted from the second pixel. The first target pixel and the second target pixel correspond to the same position in the first depth map and the second depth map, respectively. Then, a multiply operation is performed using the first brightness value corresponding to the first target pixel, the second brightness value corresponding to the second target pixel, and a preset brightness range to obtain the corresponding target brightness value. Finally, the target depth map corresponding to the first depth map and the second depth map is constructed according to each target brightness value.

[0054] It should be noted that in the above process, if there is a misalignment of depth information between the first depth map and the second depth map, such as inconsistent image size, inconsistent coordinates of pixels, or inconsistent depth value ranges of pixels, the first depth map and the second depth map can be aligned first to obtain aligned first and second depth maps, and then the mixed calculation can be performed based on the aligned depth maps to ensure data consistency.

[0055] In some examples, size alignment can adjust two depth maps to the same size. This can be achieved through resizing (enlarging the smaller depth map or shrinking the larger one), cropping (cropping a region of the same size as the smaller depth map from the larger one), and interpolation (using interpolation algorithms such as bilinear interpolation to maintain the smoothness of depth information during scaling). If the two depth maps come from different viewpoints or cameras, their coordinate systems need to be aligned. This can be done through camera calibration (obtaining the camera's intrinsic parameters (focal length, principal point) and extrinsic parameters (rotation, translation)), or depth map projection (projecting one depth map onto the coordinate system of the other). If the depth values ​​of the two depth maps have inconsistent ranges, they can be normalized to the range [0, 1]. Alignment ensures the consistency and accuracy of depth information between different depth maps.

[0056] Furthermore, during the multiply calculation of the first and second depth maps, assuming that pixels at the same position in the first and second depth maps are pixel A and pixel B respectively (for example, by constructing a corresponding planar coordinate system and then locating based on the coordinates of each pixel), the brightness value 'a' corresponding to pixel A and the brightness value 'b' corresponding to pixel B are obtained. Simultaneously, the brightness range of the pixels is set to [0, 225]. The multiply calculation formula can be C = a*b / 255. This formula can be used to calculate the target brightness value after merging pixel A or pixel B. This calculation process is repeated to calculate the target brightness values ​​corresponding to all pixels. These values ​​are then combined according to their corresponding positions to obtain the target depth map, thus achieving the fusion of virtual content and real-world scene, ensuring the overall consistency of the image without altering the scene layout in the real-world scene.

[0057] For example, refer to Figure 2 , Figure 3 This illustrates a schematic diagram of a depth map provided in an embodiment of the present invention, wherein... Figure 2 It can be the first depth map corresponding to the real-world scene. Figure 3 This can be used to create a second depth map corresponding to the game scene, which can be achieved through multiplying. Figure 2 and Figure 3 The fusion process yields the target depth map, such as... Figure 4 As shown, this achieves the integration of virtual content with real-world scenes, ensuring overall consistency of the image without altering the scene layout in the real-world environment. Optionally, the first depth map can be used as the bottom layer image, and the second depth map can be used as the top overlay image for multiply blending calculations.

[0058] It should be noted that for the calculation formula of Multiply blending mode, C=a*b / 255, when a is 0, C is always 0 regardless of the value of b. That is, when a is completely black, no matter what kind of depth map b is, it will not affect a. When b=0, then C=0. When b=225, then C=a. That is, when the pixels of the upper image are pure white, the pixels of the original image a do not change after overlay. When 0<b<255, then b÷255<1, C<a. After Multiply blending mode, the pixels in the first depth map become darker. Thus, by fusing the depth information of virtual content and real scene, a target depth map can be obtained. Then, corresponding filter effects can be added to the real scene using the target depth map to enhance the visual effect of the real scene.

[0059] Step 104: Apply a filter to the real-world scene based on the target depth map, and display the processed first real-world scene in the graphical user interface based on the filter processing result.

[0060] In this embodiment of the invention, by fusing the depth map corresponding to the real scene with the depth map corresponding to the virtual scene in the game to obtain the target depth map, the real scene can be filtered according to the target depth map. Based on the result of the filter processing, the processed first real scene is displayed in the graphical user interface. Thus, by combining virtual content with the real scene through the depth map, the stability of the interface display content can be guaranteed without changing the layout of the real scene. At the same time, by utilizing the near-white and far-black characteristics of the depth map, the impact of the bright background in the real scene on the player is reduced, the discomfort caused by the brightness inconsistency between the virtual window and the real scene is reduced, and the player's attention can be effectively focused on the virtual window, thereby improving the player's visual experience.

[0061] In some feasible implementations, the terminal can sample the target depth map and the real scene separately to obtain a third target pixel and a fourth target pixel representing the same location. The third target pixel is located in the target depth map, and the fourth target pixel is located in the real scene. Then, the terminal obtains the first color value corresponding to the third target pixel and the second color value corresponding to the fourth target pixel. Then, the terminal performs a multiply operation using the first color value and the second color value to obtain the target color value corresponding to the fourth target pixel. Finally, the terminal adjusts the color of the real scene based on the target color value to obtain the first real scene corresponding to the real scene, and displays the first real scene in the graphical user interface.

[0062] It should be noted that when the target depth map and the real scene are not the same size, appropriate alignment processing can be used to make the pixels contained in the target depth map consistent with the pixels contained in the real scene, including the same position and the same number, so that each pixel in the real scene can be processed according to the target depth map, thereby adding the corresponding filter to the real scene.

[0063] In one example, suppose the target pixel C in the real scene has a high brightness, with a corresponding color value of #FDFEED and an RGB value of (253,254,237), while the corresponding pixel D in the generated target depth map has a color value of #161616 and an RGB value of (22,22,22). Then the calculation process for the Multiply blending mode can be: (real scene ①.rgb(253,254,237)* The depth map C.rgb(22,22,22))÷255=(21.8,21.9,20.4). Therefore, the color value obtained after adding a filter to the target pixel C through the target depth map can be (21.8,21.9,20.4). Similarly, after calculating all pixels in the real scene, the stability of the interface display content can be guaranteed without changing the layout of the real scene. At the same time, the characteristics of near white and far black in the depth map can be used to add corresponding depth fog to the real background, thereby reducing the impact of the bright background on the player's vision and allowing the player's attention to be effectively focused on the virtual window, thus improving the player's visual experience.

[0064] Optionally, during the filtering process of the real-world scene, the terminal can also acquire brightness adjustment information for the target depth map obtained through hybrid calculation, and then adjust the brightness of the target depth map using the brightness adjustment information. The brightness adjustment information can be preset by the terminal or input in real-time by the player during the filter processing. For example, the target depth map can be generated in real-time based on the content displayed in the current graphical user interface. After generation, the player can adjust the brightness of the target depth map to control the range of the fog effect, providing more freedom. Simultaneously, the terminal can also have corresponding preset adjustment information provided by the manufacturer at the factory. Players can directly use the preset parameters to effectively adjust the area of ​​the real-world scene covered by the filter by adjusting the brightness value of the target depth map. When combined with the game's storyline, this can greatly enhance the player's immersive experience. Optionally, players can adjust the brightness of the target depth map through corresponding interactive controls, etc., and this invention does not limit this.

[0065] Optionally, if the player desires it, or if the game progresses to a specific point in time or meets specific conditions, the real-world scene can be processed in conjunction with the game scene. Specifically, a game scene can be added to a first real-world scene to obtain a second real-world scene, which is then displayed in the graphical user interface. For example, after obtaining the target depth map, the terminal can extract the corresponding static scene from the game scene, adjust the brightness of the static scene according to the brightness information corresponding to the target depth map, and then overlay the adjusted static scene onto the real-world scene, thereby further enhancing the visual immersion during the game and providing the player with a better visual experience.

[0066] It should be noted that the embodiments of the present invention include, but are not limited to, the examples described above. It is understood that those skilled in the art can make further settings according to actual needs under the guidance of the ideas in the embodiments of the present invention, and the present invention does not limit such settings.

[0067] In this embodiment of the invention, for the real-world scene of the player's environment and the game scene displayed in the virtual window during the game, the terminal can obtain a first depth map corresponding to the real-world scene and a second depth map corresponding to the game scene, and mix the first depth map and the second depth map to obtain a target depth map. Then, the real-world scene is filtered according to the target depth map, and the processed first real-world scene is displayed in the graphical user interface based on the result of the filter processing. In this way, the virtual content is combined with the real-world scene through the depth map, which can ensure the stability of the interface display content without changing the layout of the real-world scene. At the same time, it can also reduce the discomfort caused by the brightness inconsistency between the virtual window and the real-world scene, and enable the player's attention to be effectively focused on the virtual window, thereby improving the player's visual experience.

[0068] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are provided for illustrative purposes:

[0069] As an example, suppose a player is playing a game through a head-mounted display (HUD). The HUD displays a real-world scene and a corresponding virtual window, while the virtual window displays the game scene. After collecting depth information from both the real-world and game scenes, a first depth map and a second depth map are obtained. Then, a multiply blending technique is used to combine the first and second depth maps to obtain a target depth map. After obtaining the target depth map, the real-world scene is sampled based on it. Areas in the target depth map with pure white RGB (255, 255, 255) fully display the real-world scene, while areas with pure black RGB (0, 0, 0) completely hide the real-world scene. Other values ​​are used for transition between the two effects.

[0070] Suppose that the target pixel A in the real-world scene displayed by the graphical user interface has a corresponding pixel position of pure black (0,0,0) on the depth map (i.e., the target depth map). Then, target pixel A and points farther away from it will not be displayed at all in the real-world scene. Conversely, depending on whether the real-world scene is fully displayed or displayed with transition effects, for example, refer to... Figure 5 This diagram illustrates a game interface provided in an embodiment of the present invention. Assuming target pixel A is connected to the bottommost point in the graphical user interface (the point closest to the camera), forming line segment AB, if the length of AB is 10 meters, then point A (0,0,0) and point B (255,255,255), the portion exceeding 10 meters from the camera, is completely hidden (the portion with brightness less than or equal to (0,0,0) can actually be displayed as pure black). For portions within 10 meters, the brightness or blur gradually decreases as the distance decreases until the corresponding real-world scene is fully displayed. By combining the depth map and utilizing its near-white and far-black characteristics, a corresponding depth fog is added to the real-world background, thereby reducing the impact of the bright background on the player's vision and allowing the player's attention to be effectively focused on the virtual window, enhancing the player's visual experience.

[0071] Furthermore, during gameplay, if players wish to, or if the game progresses to a specific point in time or meets certain conditions, they can also manipulate the real-world environment in conjunction with the game scenario, referring to... Figure 6 The terminal can extract the corresponding static scene from the game scene displayed in the virtual window 20, and then combine it with the target depth map to overlay the corresponding static scene in the real scene 10, thereby further enhancing the visual immersion during the game and bringing a good visual experience to the player.

[0072] Through the above process, virtual content can be combined with real-world scenes using depth maps. This ensures the stability of the displayed content without altering the layout of the real-world scene. It also reduces discomfort caused by brightness inconsistencies between the virtual window and the real-world scene, and allows players to effectively focus their attention on the virtual window, thus enhancing their visual experience.

[0073] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0074] Reference Figure 7This diagram illustrates a structural block diagram of a screen processing device provided in an embodiment of the present invention. It provides a graphical user interface via an electronic terminal and may specifically include the following modules:

[0075] The scene display module 701 is used to display the real scene of the player's environment and the virtual window in the graphical user interface, and to display the game scene in the virtual window;

[0076] The depth information acquisition module 702 is used to acquire a first depth map corresponding to the real scene and a second depth map corresponding to the game scene;

[0077] The depth information fusion module 703 is used to mix the first depth map and the second depth map to obtain a target depth map;

[0078] The filter processing module 704 is used to perform filter processing on the real scene according to the target depth map, and display the processed first real scene in the graphical user interface based on the result of the filter processing.

[0079] In some feasible implementations, the first depth map includes a first pixel, and the second depth map includes a second pixel. The depth information fusion module 703 is specifically used for:

[0080] Obtain the first brightness value corresponding to each of the first pixels;

[0081] Obtain the second brightness value corresponding to each of the second pixels;

[0082] Based on the first brightness value and the second brightness value, a positive overlay calculation is performed to obtain the target depth map corresponding to the first depth map and the second depth map.

[0083] In some feasible implementations, the deep information fusion module 703 is specifically used for:

[0084] Extract a first target pixel from the first pixel and extract a second target pixel from the second pixel, wherein the first target pixel and the second target pixel correspond to the same position in the first depth map and the second depth map, respectively.

[0085] The first brightness value corresponding to the first target pixel, the second brightness value corresponding to the second target pixel, and a preset brightness range are used to perform a multiply blending calculation to obtain the corresponding target brightness value.

[0086] Based on each of the target brightness values, construct the target depth map corresponding to the first depth map and the second depth map.

[0087] Among some feasible implementation methods are:

[0088] The alignment processing module is used to align the first depth map and the second depth map to obtain the aligned first depth map and second depth map.

[0089] In some feasible implementations, the filter processing module 704 is specifically used for:

[0090] The target depth map and the real scene are sampled respectively to obtain a third target pixel and a fourth target pixel representing the same location. The third target pixel is located in the target depth map, and the fourth target pixel is located in the real scene.

[0091] Obtain the first color value corresponding to the third target pixel;

[0092] Obtain the second color value corresponding to the fourth target pixel;

[0093] The target color value corresponding to the fourth target pixel is obtained by performing a multiplication calculation using the first color value and the second color value;

[0094] The real-world scene is color-adjusted based on the target color value to obtain a first real-world scene corresponding to the real-world scene, and the first real-world scene is displayed in the graphical user interface.

[0095] In some feasible implementations, the depth information acquisition module 702 is specifically used for:

[0096] In response to the filter addition operation for the real-world scene, a first depth map corresponding to the real-world scene and a second depth map corresponding to the virtual window are obtained.

[0097] In some feasible implementations, the graphical user interface displays functional controls, and the depth information acquisition module 702 is specifically used for:

[0098] In response to a gaze operation on the functional control, the environment is scanned to obtain a first depth map corresponding to the real scene;

[0099] Additionally, depth information corresponding to the game scene is obtained through a virtual camera in the game, and a second depth map corresponding to the depth information is generated.

[0100] In some feasible implementations, the apparatus further includes:

[0101] An adjustment information acquisition module is used to acquire brightness adjustment information for the target depth map;

[0102] An adjustment module is used to adjust the brightness information of the target depth map using the brightness adjustment information.

[0103] Among some feasible implementation methods are:

[0104] The scene processing module is used to add the game scene to the first real-world scene to obtain a second real-world scene;

[0105] A display module is used to display the second real-world scene in the graphical user interface.

[0106] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0107] In addition, this invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described screen processing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0108] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described image processing method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0110] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program code.

[0111] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0112] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0114] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0115] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0116] The above provides a detailed description of a method and apparatus for processing images provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for processing images, characterized in that, The method of providing a graphical user interface via an electronic terminal includes: The graphical user interface displays the real-world scene of the player's environment and a virtual window, and displays the game scene in the virtual window; Obtain the first depth map corresponding to the real-world scene and the second depth map corresponding to the game scene; The first depth map and the second depth map are blended to obtain the target depth map; The real-world scene is processed using a filter based on the target depth map, and the processed first real-world scene is displayed in the graphical user interface based on the result of the filter processing. Wherein, the first depth map includes a first pixel, and the second depth map includes a second pixel. The step of mixing the first depth map and the second depth map to obtain a target depth map includes: Obtain the first brightness value corresponding to each of the first pixels; Obtain the second brightness value corresponding to each of the second pixels; Based on the first brightness value and the second brightness value, a positive overlay calculation is performed to obtain the target depth map corresponding to the first depth map and the second depth map.

2. The method according to claim 1, characterized in that, The step of performing a positive blending calculation based on the first brightness value and the second brightness value to obtain a target depth map corresponding to the first depth map and the second depth map includes: Extract a first target pixel from the first pixel and extract a second target pixel from the second pixel, wherein the first target pixel and the second target pixel correspond to the same position in the first depth map and the second depth map, respectively. The first brightness value corresponding to the first target pixel, the second brightness value corresponding to the second target pixel, and a preset brightness range are used to perform a multiply blending calculation to obtain the corresponding target brightness value. Based on each of the target brightness values, construct the target depth map corresponding to the first depth map and the second depth map.

3. The method according to claim 1, characterized in that, Also includes: Align the first depth map and the second depth map to obtain the aligned first depth map and second depth map.

4. The method according to claim 1 or 2, characterized in that, The step of applying a filter to the real-world scene based on the target depth map, and displaying the processed first real-world scene in the graphical user interface based on the result of the filter processing, includes: The target depth map and the real scene are sampled respectively to obtain a third target pixel and a fourth target pixel representing the same location. The third target pixel is located in the target depth map, and the fourth target pixel is located in the real scene. Obtain the first color value corresponding to the third target pixel; Obtain the second color value corresponding to the fourth target pixel; The target color value corresponding to the fourth target pixel is obtained by performing a multiplication calculation using the first color value and the second color value; The real-world scene is color-adjusted based on the target color value to obtain a first real-world scene corresponding to the real-world scene, and the first real-world scene is displayed in the graphical user interface.

5. The method according to claim 1, characterized in that, The step of obtaining the first depth map corresponding to the real-world scene and the second depth map corresponding to the game scene includes: In response to the filter addition operation for the real-world scene, a first depth map corresponding to the real-world scene and a second depth map corresponding to the virtual window are obtained.

6. The method according to claim 5, characterized in that, The graphical user interface displays functional controls. The step of obtaining a first depth map corresponding to the real-world scene and a second depth map corresponding to the virtual window in response to a filter addition operation for the real-world scene includes: In response to a gaze operation on the functional control, the environment is scanned to obtain a first depth map corresponding to the real scene; Additionally, depth information corresponding to the game scene is obtained through a virtual camera in the game, and a second depth map corresponding to the depth information is generated.

7. The method according to claim 1 or 2, characterized in that, After mixing the first depth map and the second depth map to obtain the target depth map, the method further includes: Obtain brightness adjustment information for the target depth map; The brightness information of the target depth map is adjusted using the brightness adjustment information.

8. The method according to claim 4, characterized in that, Also includes: Add the game scene to the first real-world scene to obtain a second real-world scene; The second real-world scene is displayed in the graphical user interface.

9. A picture processing apparatus, characterized in that, The device provides a graphical user interface via an electronic terminal, comprising: The scene display module is used to display the real scene of the player's environment and the virtual window in the graphical user interface, and to display the game scene in the virtual window; The depth information acquisition module is used to acquire a first depth map corresponding to the real scene and a second depth map corresponding to the game scene. A depth information fusion module is used to mix the first depth map and the second depth map to obtain a target depth map; A filter processing module is used to perform filter processing on the real scene according to the target depth map, and display the processed first real scene in the graphical user interface based on the filter processing result; Wherein, the first depth map includes a first pixel, the second depth map includes a second pixel, and the depth information fusion module is specifically used for: Obtain the first brightness value corresponding to each of the first pixels; Obtain the second brightness value corresponding to each of the second pixels; Based on the first brightness value and the second brightness value, a positive overlay calculation is performed to obtain the target depth map corresponding to the first depth map and the second depth map.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-8.

11. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-8.

Images