Image generation method and apparatus

Abstract

The application discloses an image generation method and device, and belongs to the technical field of electronic equipment. The method comprises the following steps: acquiring a first image and first depth information corresponding to the first image; identifying editable objects in the first image, and establishing a mapping relationship between image areas corresponding to each editable object and the first depth information corresponding to each image area; in response to an editing operation of a target editing object by a user, determining the target editing object and spatial adjustment information corresponding to the target editing object; based on the spatial adjustment information and the mapping relationship, adjusting the first depth information of an image area corresponding to the target editing object to obtain second depth information of the image area corresponding to the target editing object; and generating a second image based on the second depth information of the image area corresponding to the target editing object.

Description

Technical Field

[0001] This application belongs to the field of electronic equipment technology, and specifically relates to an image generation method and apparatus. Background Technology

[0002] With the continuous improvement of the display and image processing capabilities of electronic devices, users have placed higher demands on the image viewing experience. Compared with traditional two-dimensional images, people increasingly hope that images can present a stronger sense of space and three-dimensionality, so as to obtain a more realistic and vivid browsing experience.

[0003] Currently, technologies for generating images with spatial visual effects from single 2D images are maturing. Specifically, this involves first automatically generating depth information for each object in the 2D image based on monocular depth estimation, and then combining this with the parallax between objects to render an image with spatial effects that change with the electronic device's posture. However, this approach relies entirely on algorithms to automatically determine the spatial hierarchy between people, objects, and the background in the image; that is, the same image corresponds to only one spatial effect. This results in relatively homogeneous spatial visual effects that cannot meet the personalized needs of different users. Summary of the Invention

[0004] The purpose of this application is to provide an image generation method and apparatus that enables users to actively adjust the spatial position of specific objects in an image, thereby improving the flexibility and controllability of spatial visual effect generation.

[0005] In a first aspect, embodiments of this application provide an image generation method, the method comprising: Obtain the first image and the first depth information corresponding to the first image; Identify editable objects in the first image and establish a mapping relationship between the image region corresponding to each editable object and the first depth information corresponding to each image region; In response to the user's editing operation on the target editing object, determine the target editing object and the corresponding space adjustment information of the target editing object; Based on the spatial adjustment information and the mapping relationship, the first depth information of the image region corresponding to the target editing object is adjusted to obtain the second depth information of the image region corresponding to the target editing object; A second image is generated based on the second depth information of the image region corresponding to the target editing object. Secondly, embodiments of this application provide an image generation apparatus, the apparatus comprising: The acquisition module is used to acquire a first image and first depth information corresponding to the first image; The recognition module is used to recognize editable objects in the first image and establish a mapping relationship between the image regions corresponding to each editable object and the first depth information corresponding to each image region. The determination module is used to determine the target editing object and the corresponding space adjustment information in response to the user's editing operation on the target editing object; An adjustment module is used to adjust the first depth information of the image region corresponding to the target editing object based on the spatial adjustment information and the mapping relationship, so as to obtain the second depth information of the image region corresponding to the target editing object; The generation module is used to generate a second image based on the second depth information of the image region corresponding to the target editing object.

[0006] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can run on the processor, and the programs or instructions, when executed by the processor, implement the method as described in the first aspect.

[0007] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0008] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0009] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0010] In this embodiment, by acquiring a first image and its corresponding first depth information, editable objects in the first image are identified, and a mapping relationship between the image region corresponding to each editable object and the depth information is established. This establishes an association between objects in the image and depth information. Based on this, in response to the user's editing operation on the target editable object, the target editable object and its spatial adjustment information can be determined. According to the spatial adjustment information and the mapping relationship, the first depth information of the image region corresponding to the target editable object is adjusted to obtain second depth information. Finally, a second image is generated based on the second depth information. Because a mapping relationship between editable objects and first depth information is established, the user's editing operation can directly affect the first depth information corresponding to the target editable object. This allows the image generation process to modify the depth information of specific objects in a targeted manner according to the user's spatial adjustment information. This solves the problem in existing solutions where spatial relationships are completely determined automatically by algorithms and the spatial visual effect result image is single and fixed. It enables the user to actively adjust the spatial position of specific objects in the image, improving the flexibility and controllability of spatial visual effect generation. Attached Figure Description

[0011] Figure 1 This is one of the schematic flowcharts of an image generation method provided in some embodiments of this application; Figure 2 This is a second schematic flowchart of an image generation method provided in some embodiments of this application; Figure 3 This is the third of the flowcharts illustrating the image generation method provided in some embodiments of this application; Figure 4 These are schematic diagrams illustrating the structure of an image generation apparatus according to some embodiments of this application; Figure 5 These are schematic diagrams illustrating the structure of an electronic device according to some embodiments of this application; Figure 6 These are schematic diagrams illustrating the hardware structure of an electronic device according to some embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or N objects. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] The terminology used in the embodiments of this invention will be explained below.

[0015] Depth information: used to characterize the distance of each region in an image relative to the observer, i.e., the scene depth value corresponding to each pixel or each image region. In this application, depth information can exist in the form of a depth map, where the value of each pixel in the depth map represents the distance between the 3D scene point corresponding to that pixel and the capturing device (or the observer). The magnitude of the depth value reflects the distance relationship; the smaller the depth value, the closer the region is to the observer; the larger the depth value, the farther the region is from the observer.

[0016] Spatial visual effects: These refer to the image display effect that creates a sense of spatial depth by combining depth information from the image with parallax rendering processing, allowing the electronic device's posture to change. When a user shakes the electronic device, objects at different depth positions in the image produce varying degrees of parallax motion, thus creating a viewing effect with spatial depth.

[0017] The technical solutions of this application can be applied to scenarios where spatial visual effects editing of the subjects in an image is required. For example, Xiaoming is browsing a group photo in the photo album of an electronic device. The group photo includes Xiaoming, Xiaohong, and Xiaoliang, with Xiaohong in the foreground, and Xiaoming and Xiaoliang in the background. Xiaoming is on the left and Xiaoliang is on the right. The background of the photo is a mountain. Now, the user wants to adjust himself to a more prominent position in the group photo to highlight Xiaoming. As another example, the user also wants to enhance the spatial depth between Xiaoming and the background mountain, so he wants to move the background mountain further back.

[0018] The image generation method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0019] Figure 1This is a flowchart illustrating an image generation method provided in an embodiment of this application. The subject executing the image generation method can be an electronic device, which can be, but is not limited to, a personal computer (PC), a smartphone, a tablet computer, or a personal digital assistant (PDA).

[0020] like Figure 1 As shown, the image generation method provided in this application embodiment may include steps 110-150.

[0021] Step 110: Obtain the first image and the first depth information corresponding to the first image.

[0022] The first image can be a previously captured image stored in an electronic device, or it can be an image captured in real time by the electronic device through a camera. The first image can be a two-dimensional image containing elements such as people, objects, and a background. This first image is the one to be processed for spatial visual effects editing; for example, the first image could be a group photo of multiple people as shown in the example above.

[0023] The first depth information can be the depth information corresponding to the first image, that is, the first depth information corresponds to the first image and is used to describe the distance of each region in the first image relative to the observer. The first depth information can be represented in the form of a depth map, where the depth value of each pixel in the depth map corresponds to the depth value of the corresponding pixel in the first image. The smaller the depth value, the closer it is to the observer, and the larger the depth value, the farther it is from the observer.

[0024] In some embodiments of this application, the first depth information can be obtained through monocular depth estimation, binocular depth estimation or other existing depth generation methods, and is not limited in the embodiments of this application.

[0025] In some embodiments of this application, the first image may be taken directly by the user using an image acquisition device, obtained by the user from the local storage of the electronic device, or downloaded from the network. The specific method of obtaining the first image and the first depth information is not limited in the embodiments of this application.

[0026] In some embodiments of this application, to improve the flexibility of acquiring the first image, step 110 may specifically include: Receive the user's first input; In response to the first input, the first image is acquired.

[0027] The first input can be user input to the editing interface of the image editing software. This first input is used to acquire a first image and can be a first operation. Exemplarily, the first input includes, but is not limited to: touch input from the user via a finger or stylus to the editing interface of the image editing software, voice commands input by the user, specific gestures input by the user, or other feasible inputs, which can be determined according to actual usage needs. This embodiment of the invention does not limit the specific gestures. Specific gestures in this application embodiment can be any one of a single-click gesture, a swipe gesture, a drag gesture, a pressure-recognition gesture, a long-press gesture, an area-change gesture, a double-press gesture, or a double-click gesture. Click input in this application embodiment can be a single-click input, a double-click input, or any number of clicks, and can also be a long-press input or a short-press input. For example, the first input can be user touch input to the editing interface of the image editing software, such as a click input from the user to the editing interface of the image editing software.

[0028] In some embodiments of this application, a first image can be obtained in response to a user's first input.

[0029] In the embodiments of this application, a first image can be obtained in response to a user's first input. This allows the acquisition of a first image that requires spatial visual effects editing based on the user's needs, thus improving the flexibility of obtaining the first image.

[0030] Step 120: Identify editable objects in the first image and establish a mapping relationship between the image region corresponding to each editable object and the first depth information corresponding to each image region.

[0031] The editable object can be an independent semantic unit in the first image that can be identified and selected by the user, including but not limited to people, objects or background areas. For example, in the above example, the editable object can be Xiaoming, Xiaohong and Xiaoliang in multiple group photos, as well as the mountain.

[0032] The editable objects mentioned above can be identified from the first image through methods such as instance segmentation, object detection, face detection, or salient object detection.

[0033] In some embodiments of this application, editable objects such as people, objects and background areas in the first image can be identified by performing instance segmentation, object detection, face detection or salient object detection on the first image. Then, the image area corresponding to each editable object can be determined. Combined with the first depth information corresponding to the first image, the first depth information corresponding to the image area corresponding to each editable object can be determined. In turn, a mapping relationship between the image area corresponding to each editable object and the first depth information corresponding to each image area can be established.

[0034] Continuing with the first example above, by performing instance segmentation on the first image, four editable objects—Xiaoming, Xiaohong, Xiaoliang, and the mountain—can be identified, and the corresponding image regions for each of them can be determined. Furthermore, by combining the first depth information, the depth values ​​of each pixel in the image region corresponding to Xiaoming, Xiaohong, Xiaoliang, and the mountain can be determined, and a mapping relationship can be established between the image regions corresponding to each editable object and the first depth information corresponding to each image region. Thus, in subsequent steps, when the user selects an editable object, the electronic device can quickly locate the corresponding image region and its depth information based on this mapping relationship.

[0035] Step 130: In response to the user's editing operation on the target editing object, determine the target editing object and the corresponding space adjustment information.

[0036] The target editing object can be one or more objects that the user selects from the editable objects and that require spatial adjustment.

[0037] Editing operations can be interactive inputs from users to the target editing object in the image editing interface. These operations are used to select the target editing object that needs to be adjusted and to express the intention to adjust the space of the target editing object.

[0038] Spatial adjustment information can be used to characterize the spatial adjustments a user expects to make to a target editing object. This information includes at least adjustment direction information and adjustment magnitude information. Adjustment direction information indicates the direction of movement of the target editing object in space, including moving it forward, backward, enhancing prominence, or weakening prominence. Adjustment magnitude information indicates the degree of spatial change the user wishes to make.

[0039] In some embodiments of this application, the target editing object and the corresponding space adjustment information can be determined in response to the user's editing operation.

[0040] In some embodiments of this application, step 130 may specifically include: In response to the user's selection of an editable object, the target editable object is determined based on the input location of the selection operation; In response to the user's adjustment operation on the target editing object, determine the spatial adjustment information corresponding to the target editing object.

[0041] The selection operation can be used to select the target object for spatial adjustment from the editable objects in the first image. Selection operations can include clicking, selecting by bounding, smearing, and long-pressing. Users can perform these selection operations using input devices such as fingers, styluses, or mice.

[0042] Adjustment operations can be used to input spatial adjustment intentions for a selected target editing object. Adjustment operations can include dragging, sliding, and pinching. Through these adjustment operations, users can obtain information about the direction and magnitude of adjustment to the target editing object.

[0043] In some embodiments of this application, the target editing object can be determined first by responding to the user's selection operation on the editable object and based on the input position of the selection operation, and then the spatial adjustment information corresponding to the target editing object can be determined in response to the user's adjustment operation on the target editing object.

[0044] Continuing with the example above, after identifying the four editable objects—Xiaoming, Xiaohong, Xiaoliang, and the mountain—if the user wants to move Xiaoming to a more prominent position to highlight his main role, the user first taps the area where Xiaoming is located in the first image. The electronic device, based on the coordinates of the user's tap location on the display interface and the established mapping relationship between the image areas corresponding to each editable object and the first depth information, determines that the editable object corresponding to the tap location is Xiaoming and sets Xiaoming as the target editable object.

[0045] After identifying Xiaoming as the target editing object, the user further performs adjustment operations on Xiaoming. For example, the user can slide two fingers upwards in the area where Xiaoming is located. The electronic device determines the adjustment direction information as "move forward" based on the type of the slide operation, and determines the adjustment magnitude information as "one level" based on the slide magnitude, which means that the user wants to move Xiaoming one level towards the observer.

[0046] For example, a user might want to move the mountains in the background further back to enhance the sense of spatial depth between the mountains and Xiaoming. The user can select the mountains again as the target object by clicking, and then slide two fingers upwards in the background area. The electronic device will determine the adjustment direction as moving them backwards and the adjustment level as one level based on this operation.

[0047] It should be noted that after the target editing object is determined, the target editing object can be highlighted or displayed in other visual ways in the first image to prompt the user that the editable object has been selected. For example, in the example above, after Xiaoming is selected in the first image, Xiaoming can be highlighted or displayed in other visual ways to prompt the user that he has been successfully selected.

[0048] In some embodiments of this application, the user's selection and adjustment operations can also be a continuous, unified operation. For example, the user can directly click and drag the target editing object. The electronic device determines the target editing object based on the click location, and simultaneously determines the corresponding adjustment direction and adjustment range information based on the dragging direction and distance. This method simplifies the user's operation process and improves editing efficiency.

[0049] In the embodiments of this application, by dividing the editing operation into two stages—selection and adjustment—users can first precisely select the target editing object to be adjusted from multiple editable objects, and then input specific spatial adjustment intentions for that target editing object. This avoids misselection or misoperation in multi-object scenarios, improving the accuracy of the editing operation and the user experience. Simultaneously, the phased design of the selection and adjustment operations allows the electronic device to respond to and process the selection and adjustment operations independently, reducing the complexity of a single interaction and improving interaction response efficiency.

[0050] In some embodiments of this application, editing operations may include touch operations, voice operations, or gesture operations. Touch operations can be contact-based interactive operations performed by a user on the display screen of an electronic device using a touch device such as a finger or stylus, and may include, but are not limited to, clicking, selecting, smearing, long-pressing, and dragging.

[0051] The aforementioned voice operation can be a voice command input by the user through the microphone of an electronic device. This voice command describes the target object to be edited and the desired spatial adjustment effect on the target object.

[0052] The aforementioned gesture operations can be commands input by the user to the electronic device through preset body movements or hand gestures. The electronic device can capture and recognize these gestures through a camera or sensors, converting them into corresponding editing intentions. For example, the electronic device can capture images of the user's hands or body through a camera, and use a gesture recognition algorithm to recognize the user's gestures and convert them into corresponding editing intentions. For instance, a user can use a forward-pushing gesture to indicate moving the target object forward, a backward-pushing gesture to indicate moving the target object backward, and an opening and closing gesture to indicate enhancing or weakening the highlighting effect.

[0053] When the editing operation includes voice operation, step 130 above may specifically include: In response to the user's voice operation, semantic recognition of the voice operation is performed to obtain the semantic information corresponding to the voice operation; Based on semantic information, the target editing object and the corresponding spatial adjustment information are determined.

[0054] The semantic information can be the information corresponding to the voice operation, such as the text information obtained after voice recognition of the voice command.

[0055] In some embodiments of this application, after receiving a user's voice operation, the electronic device can convert the user's voice command into text information using Automatic Speech Recognition (ASR) technology, and then perform semantic analysis on the text information using Natural Language Understanding (NLU) technology to extract information about the object to be edited, the direction of adjustment, and the degree of adjustment. The object to be edited information is used to determine the target object that needs to be adjusted; the direction of adjustment information is used to determine whether it is moved forward, backward, or changes in prominence; and the degree of adjustment information is used to determine the corresponding adjustment range.

[0056] Continuing with the example above: The user doesn't need to manually touch the screen; instead, they simply speak the voice command "Move the person on the left behind me a little forward" into the microphone of the electronic device. Upon receiving this voice command, the electronic device performs semantic recognition to obtain semantic information. This information is then parsed, extracting the editing object information as "the person on the left behind me," the adjustment direction information as "move forward," and the adjustment degree information as "slight adjustment." Based on this editing object information, and the established mapping relationship, the electronic device determines the target editing object as Xiaoming. Subsequently, based on "move forward" and "slight adjustment," the electronic device determines the corresponding spatial adjustment information as: the adjustment direction information is "move forward," and the adjustment magnitude information is "one level."

[0057] For example, a user can input the voice command "move the mountain behind you a little further away". The electronic device will interpret the command and determine that the target editing object is the mountain, the adjustment direction is "move backward", and the adjustment range is "slight adjustment".

[0058] It should be noted that when there are multiple candidate objects in the analysis results, the electronic device can highlight the candidate objects and receive further confirmation from the user.

[0059] In the embodiments of this application, in scenarios where users find it inconvenient to perform touch operations due to the distance of their handheld device or difficulty in touching the screen with their fingers, the target editing object and its corresponding spatial adjustment information can be determined through voice operation. In this way, users can determine the target editing object and its corresponding spatial adjustment information without manually touching the screen, which lowers the threshold for use and improves the naturalness and convenience of user interaction.

[0060] Step 140: Based on spatial adjustment information and mapping relationship, adjust the first depth information of the image region corresponding to the target editing object to obtain the second depth information of the image region corresponding to the target editing object.

[0061] The second depth information can be the depth information obtained by modifying the depth value of the image region corresponding to the target editing object in the first depth information based on spatial adjustment information.

[0062] In some embodiments of this application, when the spatial adjustment information includes adjustment direction information and adjustment magnitude information, step 140 above may specifically include: Based on the adjustment direction information and mapping relationship corresponding to the target editing object, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined. Based on the adjustment range information corresponding to the target editing object, determine the adjustment amount of the first depth information of the image region corresponding to the target editing object; Based on the adjustment trend and adjustment amount, the second depth information of the image region corresponding to the target editing object is obtained.

[0063] Among them, the adjustment trend can be used to characterize the trend of modifying the depth value of the image region corresponding to the target editing object, such as whether the depth value is increased or decreased.

[0064] The adjustment amount can be used to characterize the specific extent to which the depth value of the corresponding image region of the target editing object is modified. Different preset adjustment amounts can correspond to different adjustment range information. For example, when the adjustment range information is level one, it corresponds to a first preset adjustment amount; when the adjustment range information is level two, it corresponds to a second preset adjustment amount. The second preset adjustment amount is greater than the first preset adjustment amount, meaning that the adjustment range information and the adjustment amount are positively correlated. The aforementioned preset adjustment amounts can be set according to actual application needs, and this application embodiment does not limit this.

[0065] In some embodiments of this application, the adjustment trend of the first depth information of the image region corresponding to the target editing object can be determined first based on the adjustment direction information and mapping relationship corresponding to the target editing object. Then, the adjustment amount of the first depth information of the image region corresponding to the target editing object can be determined according to the adjustment magnitude information corresponding to the target editing object. Finally, the second depth information of the image region corresponding to the target editing object can be obtained according to the adjustment trend and the adjustment amount.

[0066] Continuing with the example above, the user determines the target object to be edited as Xiaoming through editing operations, and the spatial adjustment information is: adjustment direction information is "move forward," and adjustment magnitude information is one level. The electronic device first determines, based on the adjustment direction information "move forward" and the established mapping relationship, that the adjustment trend of the depth value of the corresponding image region of Xiaoming in the first depth information is "decreasing depth value." That is, the electronic device determines that the depth values ​​of each pixel in the corresponding image region of Xiaoming need to be adjusted in the direction of decreasing values.

[0067] Subsequently, the electronic device determines, based on the adjustment range information "Level 1", the adjustment amount of the depth value of Xiaoming's corresponding image region in the first depth information as the first preset adjustment amount. This first preset adjustment amount can be, for example, 30 depth value units.

[0068] Then, based on the adjustment trend of "reducing depth values" and the adjustment amount of "first preset adjustment," the electronic device modifies the first depth information of the image region corresponding to Xiaoming, that is, it reduces the depth value of each pixel in the image region corresponding to Xiaoming by 30, thus obtaining the second depth information of the image region corresponding to Xiaoming. At the same time, the electronic device keeps the relative depth relationship between each pixel within the image regions corresponding to Xiaohong and Xiaoliang unchanged, so as to maintain Xiaoming's own three-dimensional structure.

[0069] For example, a user determines the target editing object as a mountain in the background through editing operations. The spatial adjustment information is: the adjustment direction information is to move backward, and the adjustment range information is two levels. The electronic device determines the adjustment trend as "increase the depth value" based on "move backward" and determines the adjustment amount as the second preset adjustment amount based on "two levels", such as 60 depth value units. The electronic device increases the depth value of each pixel in the image area corresponding to the mountain by 60, thus obtaining the second depth information of the image area corresponding to the mountain.

[0070] In the embodiments of this application, by decomposing spatial adjustment information into adjustment direction information and adjustment magnitude information, and mapping them respectively to adjustment trends and adjustment amounts at the depth information level, the degree of change in the spatial position of the target editing object can be precisely controlled, so that the user's adjustment intention can be accurately quantified into specific modification operations of depth values, thereby improving the accuracy and controllability of spatial adjustment.

[0071] In some embodiments of this application, determining the adjustment trend of the first depth information of the image region corresponding to the target editing object based on the adjustment direction information and mapping relationship corresponding to the target editing object may specifically include: When the adjustment direction information corresponding to the target editing object is a forward movement operation, based on the forward movement operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to decrease the value of the first depth information. When the adjustment direction information corresponding to the target editing object is a backward operation, based on the backward operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to increase the value of the first depth information; When the adjustment direction information corresponding to the target editing object is to enhance the highlighting operation, based on the enhancement highlighting operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is to increase the depth difference between the image region corresponding to the target editing object and the surrounding image region.

[0072] The "move forward" operation can be used to indicate that the target object to be edited is moved closer to the observer. Since a smaller depth value indicates that the target object is closer to the observer, the adjustment trend corresponding to the "move forward" operation is to reduce the depth value of the image region corresponding to the target object.

[0073] The move-back operation can be used to indicate that the target object to be edited is moved away from the observer. Since a larger depth value indicates that the target object is farther away from the observer, the adjustment trend for the move-back operation is to increase the depth value of the image region corresponding to the target object.

[0074] The enhancement highlight operation can be used to indicate how much visual prominence the target editing object should be in the image.

[0075] In some embodiments of this application, when the adjustment direction information corresponding to the target editing object is a forward movement operation, based on the forward movement operation and the mapping relationship, it can be determined that the adjustment trend of the first depth information of the image region corresponding to the target editing object is to decrease the value of the first depth information.

[0076] Continuing with the example above, if the user wants to move Xiaoming to a more forward position, and the user inputs a forward movement direction, the electronic device, based on this forward movement operation and the established mapping relationship, determines that the adjustment trend of the depth value in the first depth information for the image region corresponding to Xiaoming is "decreasing depth value". That is, the electronic device determines that the depth values ​​of each pixel in the image region corresponding to Xiaoming need to be adjusted in a decreasing direction to make Xiaoming move closer to the observer in space.

[0077] In some embodiments of this application, when the adjustment direction information corresponding to the target editing object is a backward operation, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to increase the value of the first depth information based on the backward operation and the mapping relationship.

[0078] Continuing with the example above, the user wants to move the mountains in the background further away to enhance the spatial depth between the mountains and Xiaoming. When the user inputs a "move backward" adjustment direction, the system, based on this "move backward" operation and the established mapping relationship, determines that the adjustment trend for the depth value of the corresponding image region of the mountains in the first depth information is "increase depth value." That is, the electronic device determines that the depth values ​​of each pixel in the corresponding image region of the mountains need to be adjusted in the direction of increasing value, so that the mountains appear to move away from the observer in space.

[0079] In some embodiments of this application, when the adjustment direction information corresponding to the target editing object is an enhancement highlighting operation, based on the enhancement highlighting operation and mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object can be determined to be increasing the depth difference between the image region corresponding to the target editing object and the surrounding image region. Increasing the depth difference between the image region corresponding to the target editing object and the surrounding image region means adjusting the depth value of the image region corresponding to the target editing object towards the observer, while maintaining or increasing the depth value of the surrounding image region, thereby widening the depth distance between the target editing object and the surrounding region, thus making it more prominent in spatial hierarchy.

[0080] Continuing with the example above, the user wants Xiaoming to stand out more in the image. When the user inputs an "enhance prominence" adjustment direction, the electronic device, based on this enhancement prominence operation and the established mapping relationship, determines that the adjustment trend for the depth value of Xiaoming's corresponding image region in the first depth information is "increase the depth difference between Xiaoming's corresponding image region and the surrounding image regions." That is, the electronic device determines that the depth value of Xiaoming's corresponding image region needs to be adjusted to decrease, while maintaining or slightly increasing the depth value of the surrounding image regions, such as the ground behind Xiaoming or background buildings. This increases the depth difference between Xiaoming and the surrounding areas, resulting in greater parallax motion relative to the surrounding areas in the subsequently generated spatial visual effects, making Xiaoming visually more prominent.

[0081] In the embodiments of this application, by defining corresponding depth value adjustment trends for different adjustment direction information, each spatial adjustment intention of the user can be accurately responded to and mapped at the depth information level. Specifically, the forward movement operation corresponds to decreasing the depth value, the backward movement operation corresponds to increasing the depth value, and the enhancement and protrusion operation corresponds to increasing the depth difference with the surrounding area. The above mapping relationship is clear and explicit, which not only ensures the physical rationality of the depth adjustment, but also enables the user's spatial editing intention to be accurately converted into a calculable operation of depth information, thereby improving the accuracy and controllability of spatial adjustment.

[0082] In some embodiments of this application, there may be multiple target editing objects, that is, the user can simultaneously select two or more editable objects as target editing objects for collaborative adjustment. In the case of multiple target editing objects, the above step 140 may specifically include: Based on the spatial adjustment information and mapping relationship corresponding to each target editing object, determine the spatial constraint relationship between each target editing object; Based on spatial constraints, the first depth information of the image region corresponding to each target editing object is adjusted to obtain the second depth information of the image region corresponding to each target editing object.

[0083] Spatial constraints can be used to describe new spatial hierarchical relationships or relative positional relationships between multiple target editing objects.

[0084] In some embodiments of this application, spatial constraints between target editing objects can be determined based on the spatial adjustment information and mapping relationship corresponding to each target editing object. Then, according to these spatial constraints, the first depth information of the image region corresponding to each target editing object is adjusted to obtain the second depth information of the image region corresponding to each target editing object. In other words, when a user simultaneously inputs editing requirements for multiple target editing objects, the adjustments to each target editing object are not independent but need to be coordinated while satisfying the spatial constraints between them. That is, the second depth information of the image region corresponding to each target editing object satisfies the aforementioned spatial constraints.

[0085] Continuing with the example above, the user wants to adjust the spatial relationships between multiple objects simultaneously. Specifically, they want to move Xiaoming to a more prominent position and move the mountain in the background further away to enhance the spatial hierarchy between Xiaoming and the mountain. The user can select both Xiaoming and the mountain as target editing objects using a selection tool, and then input their respective adjustment requirements: Xiaoming's adjustment direction is forward, and the adjustment magnitude is one level; the mountain's adjustment direction is backward, and the adjustment magnitude is one level. After receiving the multiple target editing objects and their corresponding spatial adjustment information, the electronic device first determines the spatial constraints between the target editing objects based on their spatial adjustment information and the established mapping relationship: Xiaoming is located in front of the mountain, and the spatial distance between Xiaoming and the mountain needs to be increased.

[0086] Then, the electronic device can coordinately adjust the first depth information of the image region corresponding to Xiaoming and the first depth information of the image region corresponding to the mountain, respectively, according to the spatial constraint relationship. Specifically, the electronic device adjusts the depth value of the image region corresponding to Xiaoming in the decreasing direction (moving it forward) and adjusts the depth value of the image region corresponding to the mountain in the increasing direction (moving it backward), while ensuring that the adjusted depth value of Xiaoming is still less than the depth value of the mountain, that is, Xiaoming is still in front of the mountain, and the depth difference between the two meets the user's desired widening effect. Through the above coordinated adjustment, the second depth information of the image region corresponding to Xiaoming and the second depth information of the image region corresponding to the mountain are obtained.

[0087] In some embodiments of this application, when there is a conflict between the spatial adjustment information corresponding to each target editing object, such as when both objects require to be in the foreground, the electronic device can automatically coordinate according to the user's operation sequence, the relative size of the adjustment range information, or a preset priority rule, and prompt the user with the adjustment result.

[0088] In the embodiments of this application, by determining the spatial constraints between each target editing object in a multi-target editing scenario, and coordinating the adjustment of the depth information of each target editing object based on these spatial constraints, the spatial relationship adjustment between multiple objects is made consistent. This avoids spatial conflicts or unreasonable hierarchical results that may occur due to independent adjustments, such as the relative order being disordered after two editable objects are moved forward simultaneously. Furthermore, the introduction of spatial constraints allows users to complete the overall spatial layout adjustment of multiple objects at once, eliminating the need for multiple adjustments one by one, simplifying the operation process and improving editing efficiency in complex spatial editing scenarios.

[0089] Step 150: Generate a second image based on the second depth information of the image region corresponding to the target editing object.

[0090] The second image is generated after re-rendering the spatial visual effects based on the second depth information.

[0091] In some embodiments of this application, a second image can be generated based on the second depth information of the image region corresponding to the target editing object.

[0092] In some embodiments of this application, when there are multiple target editing objects, the spatial relationships between objects in the image may change after the depth information of the multiple target editing objects is adjusted. In order to accurately reflect these changes and generate a reasonable second image accordingly, after obtaining the second depth information of the image region corresponding to the target editing object, the method described above may further include: Based on the second depth information of the image region corresponding to each target editing object, determine the spatial change information of each target editing object in the first image; Correspondingly, step 150 above may specifically include: A second image is generated based on the second depth information and spatial variation information of the image region corresponding to each target editing object.

[0093] The spatial change information can be used to characterize the changes in spatial relationships in the first image caused by the adjustment of the depth information of each target editing object. This spatial change information may include, but is not limited to, the updated occlusion relationships between the target editing objects, the exposed areas in the first image caused by changes in depth values, and the changes in the relative depth order between the target editing objects.

[0094] In some embodiments of this application, after the depth information of each target editing object is adjusted, the spatial change information of each target editing object in the first image can be determined according to the second depth information of the image region corresponding to each target editing object, and then the second image can be generated based on the second depth information and spatial change information of the image region corresponding to the target editing object.

[0095] Continuing with the example above, the user selects both Xiaoming and the mountain as the target editing objects, moving Xiaoming forward one level and the mountain backward one level. After the electronic device adjusts the depth information of Xiaoming and the mountain, it obtains the second depth information of the image area corresponding to Xiaoming and the second depth information of the image area corresponding to the mountain. At this point, because Xiaoming has moved forward and the mountain has moved backward, the depth difference between the two has increased. Based on the adjusted second depth information of Xiaoming and the mountain, the electronic device recalculates the spatial hierarchy relationship between them and determines the spatial change information.

[0096] Specifically, spatial change information can include updated occlusion relationships. For example, after Xiaoming moves forward, the area of ​​the mountain behind him that was previously occluded by him may increase, and the system needs to redetermine the occlusion boundary between Xiaoming and the mountain based on the new depth value. As another example, spatial change information can include revealed areas resulting from changes in depth values. For instance, after Xiaoming moves forward, some background areas behind him that were previously occluded, such as part of the ground or trees behind him, may become visible again, and the system needs to determine the location and extent of these newly revealed areas.

[0097] After determining the spatial change information, the electronic device can generate a second image based on the second depth information and spatial change information of the image region corresponding to the target editing object. That is, when generating the second image, not only is the adjusted depth information used for parallax rendering, but the updated occlusion relationship and exposed area information are also combined to process the image content accordingly, so as to ensure that the generated spatial visual effect result conforms to the adjusted spatial relationship in terms of spatial hierarchy and image integrity.

[0098] In the embodiments of this application, by determining the spatial change information after the depth adjustment is completed, and taking into account the spatial change information when generating the second image, the spatial relationship changes caused by the depth adjustment of multiple objects can be fully captured by the electronic device and reflected in the final generated second image. This avoids the problem of unnatural images caused by relying solely on depth information adjustment while ignoring spatial change information, and improves the image quality and realism of spatial visual effects generated in multi-object collaborative editing scenarios.

[0099] In some embodiments of this application, the aforementioned spatial change information may include the occlusion relationship between target editing objects. This occlusion relationship can be used to describe the front-to-back overlapping relationship between multiple target editing objects due to differences in depth values. Specifically, when two target editing objects have overlapping areas on the image plane, the object with a smaller depth value (i.e., the object closer to the observer) will occlude the overlapping portion of the object with a larger depth value (i.e., the object farther from the observer). When the depth information of the target editing objects is adjusted, the front-to-back order between the target editing objects may change, and the corresponding occlusion relationship needs to be re-determined.

[0100] Correspondingly, when the spatial change information includes the occlusion relationship between each target editing object, the generation of the second image based on the second depth information and spatial change information of the image region corresponding to each target editing object may specifically include: Based on the occlusion relationship, update the boundary information of each target editing object; A second image is generated based on the second depth information of the image region corresponding to each target editing object, and the updated boundary information of each target editing object.

[0101] Boundary information can be used to characterize the contour boundaries between each target editing object and its surrounding area in the image, including the outer contour boundaries of each target editing object and the occlusion boundaries between each target editing object.

[0102] In some embodiments of this application, when the occlusion relationship between each target editing object changes, the boundary portion that was originally occluded by the foreground object may become visible, and the boundary portion that was originally visible may become occluded. Therefore, it is necessary to redetermine the boundary information of each target editing object according to the updated occlusion relationship, and then generate a second image based on the second depth information of the image region corresponding to each target editing object and the updated boundary information of each target editing object.

[0103] Continuing with the example above, the user selects both Xiaoming and the mountain as the target editing objects. Xiaoming is moved forward one level, and the mountain is moved backward one level. After the electronic device adjusts the depth information of Xiaoming and the mountain, it recalculates the spatial hierarchy between them based on the second depth information of both objects, determining the updated occlusion relationship. Specifically, after Xiaoming moves forward, its depth value decreases, while the mountain's depth value increases, widening the depth difference between them. Since Xiaoming is in front of the mountain, it will occlude a portion of the mountain. The electronic device then updates the boundary information between Xiaoming and the mountain based on this updated occlusion relationship.

[0104] Specifically, the electronic device can compare the second depth information of Xiaoming and the second depth information of the mountain. For the overlapping area of ​​Xiaoming and the mountain on the image plane, since Xiaoming's depth value is smaller in the first image, meaning Xiaoming is closer to the observer, the part of Xiaoming's boundary information that overlaps with the mountain is marked as the foreground boundary, while the part of the mountain's boundary information that overlaps with Xiaoming is marked as the occluded boundary. Based on this, the electronic device updates the boundary information of each target editing object, clarifying which contour edges are visible and which are occluded.

[0105] After updating the boundary information, the electronic device generates a second image based on the second depth information of the image region corresponding to Xiaoming, the second depth information of the image region corresponding to the mountain, and the updated boundary information. In the generated second image, the occlusion relationship between Xiaoming and the mountain conforms to the adjusted spatial hierarchy, Xiaoming's visible outline is clear and complete, and the part of the mountain occluded by Xiaoming is correctly processed during rendering, presenting a natural occlusion effect.

[0106] In the embodiments of this application, by determining the updated occlusion relationship in the spatial change information and updating the boundary information of each target editing object based on the occlusion relationship, the changes in the front-to-back coverage relationship between each target editing object caused by the depth adjustment can be accurately reflected in the final generated second image. This avoids the visual confusion caused by only adjusting the depth value and ignoring the occlusion boundary update, such as the boundary between the foreground object and the background object being intersected, or the area that should be occluded being displayed incorrectly. This improves the spatial realism and visual consistency of the image.

[0107] In some embodiments of this application, the aforementioned spatial change information may include exposed areas in the first image resulting from changes in the depth information of the image regions corresponding to each target editing object. Here, exposed areas refer to the areas where, after the depth information of the target editing object is adjusted, the image regions of the background or other objects that were originally obscured by the target editing object are re-exposed. Specifically, when the target editing object moves forward, a portion of the area behind its original position is no longer obscured by the object; these areas are the exposed areas. These exposed areas originally contained corresponding image content in the original first image, but were invisible due to being obscured by foreground objects. When the obscuring objects move, these areas are re-exposed.

[0108] Correspondingly, when the spatial change information includes the exposed areas in the first image caused by changes in the depth information of the image regions corresponding to each target editing object, the generation of the second image based on the second depth information and spatial change information of the image regions corresponding to each target editing object may specifically include: The exposed area is then image-completed to obtain the first image after image completion. A second image is generated based on the second depth information of the image region corresponding to each target editing object, and the first image after image completion processing.

[0109] In some embodiments of this application, image completion processing can be performed on the exposed area to obtain a first image after image completion processing. Here, image completion processing is used to fill the exposed area with reasonable image content so that the completed first image remains visually complete and natural.

[0110] The image completion processing described above can be implemented using generative completion, texture extension, background reconstruction, or other existing image completion schemes, and is not limited in the embodiments of this application.

[0111] Continuing with the example above, the user selects both Xiaoming and the mountain as the target editing objects. Xiaoming is moved forward one level, and the mountain is moved backward one level. After the electronic device adjusts Xiaoming's depth information, Xiaoming moves forward. The background area previously obscured by Xiaoming behind his original position is now exposed; these areas are the revealed areas. For example, some ground, grass, or trees behind Xiaoming will be exposed. The electronic device can compare the difference between the first depth information before and the second depth information after the adjustment to determine Xiaoming's movement direction and distance, thereby identifying the location and extent of the revealed areas caused by Xiaoming's forward movement.

[0112] After identifying the exposed area, the electronic device performs image completion processing on that area. Specifically, the electronic device can acquire the background texture information around the exposed area and use texture expansion or generative completion methods to generate image content for the exposed area that is coordinated with the surrounding background in terms of color, texture, and structure. For example, the electronic device can expand and fill the ground area behind Xiaoming that was originally obscured based on the visible ground texture around the exposed area, making the completed ground visually continuous and natural. Similarly, the electronic device can generatively complete the trees behind Xiaoming that were originally obscured based on the texture of the tree branches and leaves around the exposed area, making the completed trees blend naturally with the surrounding environment.

[0113] After completing the image completion process, the electronic device can generate a second image based on the second depth information of the image area corresponding to Xiaoming, the second depth information of the image area corresponding to the mountain, and the completed first image. In the generated second image, Xiaoming has moved to a more forward position, the mountain has moved to a more backward position, and the exposed area behind Xiaoming's original position has been properly filled, making the entire image visually complete and without any missing parts.

[0114] In some embodiments of this application, image completion processing can also be combined with depth information. For example, for different locations in the exposed area, a suitable reference texture source can be selected based on its corresponding depth value, so that the completed image content is consistent with the surrounding area in terms of depth.

[0115] In the embodiments of this application, by determining the exposed area caused by the change in depth value and performing image completion processing on it, the missing areas of the screen caused by the movement of the target editing object can be reasonably filled, avoiding the problem of screen holes or defects caused by the movement of the target editing object, and ensuring the visual integrity and naturalness of the generated image.

[0116] To better understand the image generation method provided in the embodiments of this application, the image generation method provided in the embodiments of this application will be described below with specific scenarios.

[0117] Figure 2 This is a flowchart illustrating an image generation method provided in an embodiment of this application. This image generation method is applicable to situations where the target editing object is a single entity, such as... Figure 2 As shown, the image generation method provided in this application embodiment may include steps 210-260.

[0118] Step 210: Obtain the first image and the first depth information corresponding to the first image.

[0119] Step 220: Identify editable objects in the first image and establish a mapping relationship between the image region corresponding to each editable object and the first depth information corresponding to each image region.

[0120] Steps 210-220 above are the same as steps 110-120 in the above embodiment, and will not be repeated here.

[0121] Step 230: In response to the user's selection operation on the editable object, determine the target editable object based on the input position of the selection operation.

[0122] Step 230 above is consistent with the process in the above embodiment of determining the target editable object based on the input position of the selection operation in response to the user's selection operation on the editable object, and will not be described again here.

[0123] Step 240: In response to the user's adjustment operation on the target editing object, determine the spatial adjustment information corresponding to the target editing object.

[0124] Step 240 above is consistent with the process in the above embodiment of determining the spatial adjustment information corresponding to the target editing object in response to the user's adjustment operation on the target editing object, and will not be described again here.

[0125] Step 250: Based on spatial adjustment information and mapping relationship, adjust the first depth information of the image region corresponding to the target editing object to obtain the second depth information of the image region corresponding to the target editing object.

[0126] Step 260: Generate a second image based on the second depth information of the image region corresponding to the target editing object.

[0127] Steps 250-260 above are the same as steps 140-150 in the above embodiment, and will not be repeated here.

[0128] It should be noted that after step 250, if the user wants to perform operations on other editable objects, they can return to step 230.

[0129] In this embodiment, the electronic device generates corresponding spatial adjustment information based on the user's adjustment requirements for the target editing object, modifies the spatial relationships in the first image based on the spatial adjustment information, and finally generates the corresponding spatial visual effect result, thereby realizing the user's interactive editing of the spatial effect.

[0130] Figure 3 This is a flowchart illustrating an image generation method provided in an embodiment of this application. This image generation method can be applied to situations where there are multiple target editing objects, such as... Figure 3 As shown, the image generation method provided in this application embodiment may include steps 301-311.

[0131] Step 301: Obtain the first image and the first depth information corresponding to the first image.

[0132] Step 302: Identify editable objects in the first image and establish a mapping relationship between the image region corresponding to each editable object and the first depth information corresponding to each image region.

[0133] Steps 301-302 described above are consistent with steps 110-120 in the above embodiments, and will not be repeated here.

[0134] Step 303: In response to the user's selection operation on the editable object, determine each target editable object according to the input position of the selection operation.

[0135] Step 303 above is consistent with the process in the above embodiment of determining the target editable object based on the input position of the selection operation in response to the user's selection operation on the editable object, and will not be described again here.

[0136] Step 304: In response to the user's adjustment operation on each target editing object, determine the spatial adjustment information corresponding to each target editing object.

[0137] Step 304 above is consistent with the process in the above embodiment of determining the spatial adjustment information corresponding to the target editing object in response to the user's adjustment operation on the target editing object, and will not be described again here.

[0138] Step 305: Based on the spatial adjustment information and mapping relationship corresponding to each target editing object, determine the spatial constraint relationship between each target editing object.

[0139] Step 305 above is consistent with the process in the above embodiment of determining the spatial constraint relationship between each target editing object based on the spatial adjustment information and mapping relationship corresponding to each target editing object, and will not be repeated here.

[0140] Step 306: According to the spatial constraint relationship, adjust the first depth information of the image region corresponding to each target editing object to obtain the second depth information of the image region corresponding to each target editing object.

[0141] Step 306 above is the same as the process in the above embodiment of adjusting the first depth information of the image region corresponding to each target editing object according to the spatial constraint relationship to obtain the second depth information of the image region corresponding to each target editing object, and will not be described again here.

[0142] Step 307: Determine the occlusion relationship between each target editing object in the first image based on the second depth information of the image region corresponding to each target editing object.

[0143] Step 307 above is consistent with the process in the above embodiment of determining the spatial change information of each target editing object in the first image based on the second depth information of the image region corresponding to each target editing object, and the spatial change information includes the occlusion relationship between each target editing object, and will not be described again here.

[0144] Step 308: Based on the occlusion relationship, update the boundary information of each target editing object.

[0145] Step 308 above is consistent with the process of updating the boundary information of each target editing object based on the occlusion relationship in the above embodiment, and will not be described again here.

[0146] Step 309: Based on the second depth information of the image region corresponding to each target editing object, determine the exposed area in the first image caused by the change in the depth information of the image region corresponding to each target editing object.

[0147] Step 307 above is consistent with the process in the above embodiment of determining the spatial change information of each target editing object in the first image based on the second depth information of the image region corresponding to each target editing object. The spatial change information includes the exposed area in the first image caused by the change of the depth information of the image region corresponding to each target editing object, and will not be described again here.

[0148] Step 310: Perform image completion processing on the exposed area to obtain the first image after image completion processing.

[0149] Step 310 above is the same as the process of performing image completion processing on the exposed area in the above embodiment to obtain the first image after image completion processing, and will not be described again here.

[0150] Step 311: Generate a second image based on the second depth information of the image region corresponding to each target editing object, the first image after image completion processing, and the updated boundary information of each target editing object.

[0151] Step 311 above is consistent with the process of generating a second image based on the second depth information and spatial change information of the image region corresponding to the target editing object in the above embodiment, and will not be repeated here.

[0152] In this embodiment, the electronic device not only supports spatial adjustment of a single target editing object, but also supports the reorganization of spatial relationships between multiple target editing objects. Through processes such as depth redistribution, occlusion relationship update, and local image completion, the adjusted spatial relationships are made more natural and reasonable.

[0153] The image generation method provided in this application can be executed by an image generation device. This application uses an image generation device executing the image generation method as an example to illustrate the image generation device provided in this application.

[0154] Figure 4 This is a schematic diagram illustrating the structure of an image generation apparatus according to an exemplary embodiment. For example... Figure 4 As shown, the image generation apparatus 400 may include: The acquisition module 410 is used to acquire a first image and first depth information corresponding to the first image; The recognition module 420 is used to recognize editable objects in the first image and establish a mapping relationship between the image regions corresponding to each editable object and the first depth information corresponding to each image region. The determining module 430 is used to determine the target editing object and the corresponding space adjustment information in response to the user's editing operation on the target editing object; The adjustment module 440 is used to adjust the first depth information of the image region corresponding to the target editing object based on the spatial adjustment information and the mapping relationship, so as to obtain the second depth information of the image region corresponding to the target editing object; The generation module 450 is used to generate a second image based on the second depth information of the image region corresponding to the target editing object.

[0155] In this embodiment, by acquiring a first image and its corresponding first depth information, editable objects in the first image are identified, and a mapping relationship between the image region corresponding to each editable object and the depth information is established. This establishes an association between objects in the image and depth information. Based on this, in response to the user's editing operation on the target editable object, the target editable object and its spatial adjustment information can be determined. According to the spatial adjustment information and the mapping relationship, the first depth information of the image region corresponding to the target editable object is adjusted to obtain second depth information. Finally, a second image is generated based on the second depth information. Because a mapping relationship between editable objects and first depth information is established, the user's editing operation can directly affect the first depth information corresponding to the target editable object. This allows the image generation process to modify the depth information of specific objects in a targeted manner according to the user's spatial adjustment information. This solves the problem in existing solutions where spatial relationships are completely determined automatically by algorithms and the spatial visual effect result image is single and fixed. It enables the user to actively adjust the spatial position of specific objects in the image, improving the flexibility and controllability of spatial visual effect generation.

[0156] In some embodiments of this application, the determining module is specifically used for: In response to a user's selection of an editable object, the target editable object is determined based on the input location of the selection operation; In response to the user's adjustment operation on the target editing object, determine the spatial adjustment information corresponding to the target editing object.

[0157] In some embodiments of this application, the spatial adjustment information includes adjustment direction information and adjustment magnitude information; The adjustment module is specifically used for: Based on the adjustment direction information and the mapping relationship corresponding to the target editing object, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined; Based on the adjustment magnitude information corresponding to the target editing object, determine the adjustment amount of the first depth information of the image region corresponding to the target editing object; Based on the adjustment trend and the adjustment amount, the second depth information of the image region corresponding to the target editing object is obtained.

[0158] In some embodiments of this application, the adjustment module is specifically used for: When the adjustment direction information corresponding to the target editing object is a forward movement operation, based on the forward movement operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to decrease the value of the first depth information; When the adjustment direction information corresponding to the target editing object is a backward operation, based on the backward operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to increase the value of the first depth information; When the adjustment direction information corresponding to the target editing object is an enhancement highlight operation, based on the enhancement highlight operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to increase the depth difference between the image region corresponding to the target editing object and the surrounding image region.

[0159] In some embodiments of this application, when there are multiple target editing objects, the adjustment module is specifically used for: Based on the spatial adjustment information and mapping relationship corresponding to each target editing object, the spatial constraint relationship between each target editing object is determined; According to the spatial constraint relationship, the first depth information of the image region corresponding to each of the target editing objects is adjusted to obtain the second depth information of the image region corresponding to each of the target editing objects; the second depth information of the image region corresponding to each of the target editing objects satisfies the spatial constraint relationship.

[0160] In some embodiments of this application, when there are multiple target editing objects, the determining module is further configured to determine the spatial change information of each target editing object in the first image based on the second depth information of the image region corresponding to each target editing object after obtaining the second depth information of the image region corresponding to each target editing object; The generation module is specifically used for: A second image is generated based on the second depth information and the spatial change information of the image regions corresponding to each of the target editing objects.

[0161] The image generation device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0162] The image generation device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0163] The image generation apparatus provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0164] Optionally, such as Figure 5As shown, this application embodiment also provides an electronic device 500, including a processor 501 and a memory 502. The memory 502 stores a program or instructions that can run on the processor 501. When the program or instructions are executed by the processor 501, they implement the various steps of the above-described image generation method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0165] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0166] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0167] The electronic device 600 includes, but is not limited to, components such as: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.

[0168] Those skilled in the art will understand that the electronic device 600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0169] The processor 610 is configured to: acquire a first image and first depth information corresponding to the first image; identify editable objects in the first image and establish a mapping relationship between the image regions corresponding to each editable object and the first depth information corresponding to each image region; in response to a user's editing operation on a target editable object, determine the target editable object and spatial adjustment information corresponding to the target editable object; adjust the first depth information of the image region corresponding to the target editable object based on the spatial adjustment information and the mapping relationship to obtain second depth information of the image region corresponding to the target editable object; and generate a second image based on the second depth information of the image region corresponding to the target editable object.

[0170] Thus, by acquiring the first image and its corresponding first depth information, editable objects in the first image are identified, and a mapping relationship is established between the image region corresponding to each editable object and the depth information. This establishes an association between objects in the image and depth information. Based on this, in response to the user's editing operation on the target editable object, the target editable object and its spatial adjustment information can be determined. According to this spatial adjustment information and the mapping relationship, the first depth information of the image region corresponding to the target editable object is adjusted to obtain the second depth information. Finally, the second image is generated based on the second depth information. Because a mapping relationship between editable objects and first depth information is established, the user's editing operation can directly affect the first depth information corresponding to the target editable object. This allows the image generation process to modify the depth information of specific objects in a targeted manner according to the user's spatial adjustment information. This solves the problem in existing solutions where spatial relationships are completely determined automatically by the algorithm and the spatial visual effect result image is single and fixed. It enables the user to actively adjust the spatial position of specific objects in the image, improving the flexibility and controllability of spatial visual effect generation.

[0171] Optionally, the processor 610 is further configured to, in response to a user's selection operation on an editable object, determine a target editable object based on the input position of the selection operation; and, in response to a user's adjustment operation on the target editable object, determine spatial adjustment information corresponding to the target editable object.

[0172] Optionally, the spatial adjustment information includes adjustment direction information and adjustment magnitude information; the processor 610 is further configured to determine the adjustment trend of the first depth information of the image region corresponding to the target editing object based on the adjustment direction information and the mapping relationship corresponding to the target editing object; determine the adjustment amount of the first depth information of the image region corresponding to the target editing object based on the adjustment magnitude information corresponding to the target editing object; and obtain the second depth information of the image region corresponding to the target editing object based on the adjustment trend and the adjustment amount.

[0173] Optionally, the processor 610 is further configured to, when the adjustment direction information corresponding to the target editing object is a forward movement operation, determine, based on the forward movement operation and the mapping relationship, that the adjustment trend of the first depth information of the image region corresponding to the target editing object is to decrease the value of the first depth information; when the adjustment direction information corresponding to the target editing object is a backward movement operation, determine, based on the backward movement operation and the mapping relationship, that the adjustment trend of the first depth information of the image region corresponding to the target editing object is to increase the value of the first depth information; and when the adjustment direction information corresponding to the target editing object is an enhancement / highlighting operation, determine, based on the enhancement / highlighting operation and the mapping relationship, that the adjustment trend of the first depth information of the image region corresponding to the target editing object is to increase the depth difference between the image region corresponding to the target editing object and the surrounding image region.

[0174] Optionally, when there are multiple target editing objects, the processor 610 is further configured to determine the spatial constraint relationship between each target editing object based on the spatial adjustment information and the mapping relationship corresponding to each target editing object; and adjust the first depth information of the image region corresponding to each target editing object according to the spatial constraint relationship to obtain the second depth information of the image region corresponding to each target editing object.

[0175] Optionally, when there are multiple target editing objects, the processor 610 is further configured to, after obtaining the second depth information of the image region corresponding to each target editing object, determine the spatial change information of each target editing object in the first image based on the second depth information of the image region corresponding to each target editing object; and generate a second image based on the second depth information of the image region corresponding to each target editing object and the spatial change information.

[0176] It should be understood that, in this embodiment, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042. The GPU 6041 processes image data of still images or videos obtained by an image capture device (such as a color camera) in video capture mode or image capture mode. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0177] The memory 609 can be used to store software programs and various data. The memory 609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 609 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0178] Processor 610 may include one or more processing units; optionally, processor 610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 610.

[0179] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image generation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0180] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0181] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image generation method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0182] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0183] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the image generation method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0184] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0186] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image generation method, characterized in that, The method includes: Obtain the first image and the first depth information corresponding to the first image; Identify editable objects in the first image and establish a mapping relationship between the image region corresponding to each editable object and the first depth information corresponding to each image region; In response to the user's editing operation on the target editing object, determine the target editing object and the corresponding space adjustment information of the target editing object; Based on the spatial adjustment information and the mapping relationship, the first depth information of the image region corresponding to the target editing object is adjusted to obtain the second depth information of the image region corresponding to the target editing object; A second image is generated based on the second depth information of the image region corresponding to the target editing object.

2. The method according to claim 1, characterized in that, The step of responding to a user's editing operation on a target editing object, determining the target editing object and the corresponding space adjustment information, includes: In response to a user's selection of the editable object, the target editable object is determined based on the input position of the selection operation; In response to the user's adjustment operation on the target editing object, determine the spatial adjustment information corresponding to the target editing object.

3. The method according to claim 1, characterized in that, The spatial adjustment information includes adjustment direction information and adjustment range information; The step of adjusting the first depth information of the image region corresponding to the target editing object based on the spatial adjustment information and the mapping relationship to obtain the second depth information of the image region corresponding to the target editing object includes: Based on the adjustment direction information and the mapping relationship corresponding to the target editing object, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined; Based on the adjustment magnitude information corresponding to the target editing object, determine the adjustment amount of the first depth information of the image region corresponding to the target editing object; Based on the adjustment trend and the adjustment amount, the second depth information of the image region corresponding to the target editing object is obtained.

4. The method according to claim 3, characterized in that, The step of determining the adjustment trend of the first depth information of the image region corresponding to the target editing object based on the adjustment direction information and the mapping relationship corresponding to the target editing object includes: When the adjustment direction information corresponding to the target editing object is a forward movement operation, based on the forward movement operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to decrease the value of the first depth information; When the adjustment direction information corresponding to the target editing object is a backward operation, based on the backward operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to increase the value of the first depth information; When the adjustment direction information corresponding to the target editing object is an enhancement highlight operation, based on the enhancement highlight operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to increase the depth difference between the image region corresponding to the target editing object and the surrounding image region.

5. The method according to claim 1, characterized in that, When there are multiple target editing objects, adjusting the first depth information of the image region corresponding to the target editing object based on the spatial adjustment information and the mapping relationship to obtain the second depth information of the image region corresponding to the target editing object includes: Based on the spatial adjustment information and mapping relationship corresponding to each of the target editing objects, the spatial constraint relationship between each of the target editing objects is determined; According to the spatial constraint relationship, the first depth information of the image region corresponding to each of the target editing objects is adjusted to obtain the second depth information of the image region corresponding to each of the target editing objects; the second depth information of the image region corresponding to each of the target editing objects satisfies the spatial constraint relationship.

6. The method according to claim 1, characterized in that, When there are multiple target editing objects, after obtaining the second depth information of the image region corresponding to the target editing object, the method further includes: Based on the second depth information of the image region corresponding to each of the target editing objects, determine the spatial change information of each of the target editing objects in the first image; The step of generating a second image based on the second depth information of the image region corresponding to each of the target editing objects includes: A second image is generated based on the second depth information and the spatial change information of the image regions corresponding to each of the target editing objects.

7. An image generation apparatus, characterized in that, The device includes: The acquisition module is used to acquire a first image and first depth information corresponding to the first image; The recognition module is used to recognize editable objects in the first image and establish a mapping relationship between the image regions corresponding to each editable object and the first depth information corresponding to each image region. The determination module is used to determine the target editing object and the corresponding space adjustment information in response to the user's editing operation on the target editing object; An adjustment module is used to adjust the first depth information of the image region corresponding to the target editing object based on the spatial adjustment information and the mapping relationship, so as to obtain the second depth information of the image region corresponding to the target editing object; The generation module is used to generate a second image based on the second depth information of the image region corresponding to the target editing object.

8. The apparatus according to claim 7, characterized in that, The determining module is specifically used for: In response to a user's selection of the editable object, the target editable object is determined based on the input position of the selection operation; In response to the user's adjustment operation on the target editing object, determine the spatial adjustment information corresponding to the target editing object.

9. The apparatus according to claim 7, characterized in that, The spatial adjustment information includes adjustment direction information and adjustment range information; The adjustment module is specifically used for: Based on the adjustment direction information and the mapping relationship corresponding to the target editing object, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined; Based on the adjustment magnitude information corresponding to the target editing object, determine the adjustment amount of the first depth information of the image region corresponding to the target editing object; Based on the adjustment trend and the adjustment amount, the second depth information of the image region corresponding to the target editing object is obtained.

10. The apparatus according to claim 9, characterized in that, The adjustment module is specifically used for: When the adjustment direction information corresponding to the target editing object is a forward movement operation, based on the forward movement operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to decrease the value of the first depth information; When the adjustment direction information corresponding to the target editing object is a backward operation, based on the backward operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to increase the value of the first depth information; When the adjustment direction information corresponding to the target editing object is an enhancement highlight operation, based on the enhancement highlight operation and the mapping relationship, the adjustment trend of the first depth information of the image region corresponding to the target editing object is determined to be to increase the depth difference between the image region corresponding to the target editing object and the surrounding image region.

11. The apparatus according to claim 7, characterized in that, When there are multiple target editing objects, the adjustment module is specifically used for: Based on the spatial adjustment information and mapping relationship corresponding to each of the target editing objects, the spatial constraint relationship between each of the target editing objects is determined; According to the spatial constraint relationship, the first depth information of the image region corresponding to each target editing object is adjusted to obtain the second depth information of the image region corresponding to each target editing object. The second depth information of the image region corresponding to each of the target editing objects satisfies the spatial constraint relationship.

12. The apparatus according to claim 7, characterized in that, When there are multiple target editing objects, the determining module is further configured to determine the spatial change information of each target editing object in the first image based on the second depth information of the image region corresponding to each target editing object after obtaining the second depth information of the image region corresponding to each target editing object; The generation module is specifically used for: A second image is generated based on the second depth information and the spatial change information of the image regions corresponding to each of the target editing objects.

Images