Shooting method and device

Abstract

The invention discloses a shooting method and device. The shooting method comprises the steps that in response to first input, N times of shooting are carried out, N image data sets are obtained, each image data set comprises a first static image and a first video associated with the first static image, and N is a positive integer larger than 1; and in response to the second input, performing synthesis processing on the first video in the N image data sets to generate a second video, performing synthesis processing on the first static image in the N image data sets to generate a second static image, the second video comprising a content image layer and a dynamic special effect image layer superposed with the content image layer, the content image layer comprises N first videos arranged in a grid layout, the dynamic special effect image layer comprises dynamic elements in a grid special effect template, and the second static image comprises N first static images arranged in a grid layout; and associating the second video with the second static image to generate a dynamic photo of the grid layout.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a shooting method and apparatus. Background Technology

[0002] With the widespread adoption of smartphone cameras, users are demanding greater creativity and convenience in photo shooting. Currently, various shooting modes exist, such as Live Photos and collages. Live Photos combine still images with associated short videos; users can long-press on a still image to play a short, approximately 3-second video, enhancing the photo's liveliness. However, existing Live Photos typically only support combinations of a single photo and a single video, resulting in limited style options and a lack of layout variation and visual creativity.

[0003] Four-grid photos allow users to create a composite image from four independent shots, arranged in a grid layout. However, this four-grid photo format cannot produce dynamic effects. To achieve a dynamic grid effect, users typically need to use third-party software for post-editing, manually combining multiple Live Photos into a grid-based dynamic video. However, this editing process often results in image quality degradation or content loss due to image cropping and compression, making it difficult to guarantee image quality, and the operation is cumbersome. Summary of the Invention

[0004] The purpose of this application is to provide a shooting method and apparatus that can solve the problems of cumbersome operation and poor image quality in generating dynamic photos with grid layouts in related technologies.

[0005] In a first aspect, embodiments of this application provide a shooting method, including:

[0006] In response to the first input, N shots are taken to obtain N image data groups, each of which includes a first still image and a first video associated with the first still image, where N is a positive integer greater than 1;

[0007] In response to the second input, the first video in the N groups of image data is synthesized to generate the second video, and the first static image in the N groups of image data is synthesized to generate the second static image. The second video includes a content layer and a dynamic effect layer superimposed on the content layer. The content layer includes N first videos arranged in a grid layout. The dynamic effect layer includes dynamic elements in a grid effect template. The second static image includes N first static images arranged in a grid layout.

[0008] The second video is associated with the second still image to generate a dynamic photo with a grid layout.

[0009] Secondly, a shooting device is provided, comprising:

[0010] A first processing module is configured to, in response to a first input, take N shots to obtain N image data groups, each image data group including a first still image and a first video associated with the first still image, where N is a positive integer greater than 1;

[0011] The second processing module is configured to, in response to the second input, perform compositing processing on the first video in the N image data groups to generate a second video, and perform compositing processing on the first static image in the N image data groups to generate a second static image, wherein the second video includes a content layer and a dynamic effect layer superimposed on the content layer, the content layer includes N first videos arranged in a grid layout, the dynamic effect layer includes dynamic elements in a grid effect template, and the second static image includes N first static images arranged in a grid layout;

[0012] The third processing module is used to associate the second video with the second static image to generate a dynamic photo with a grid layout.

[0013] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0014] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0015] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0016] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0017] In this embodiment, in response to a first input, N shots are taken to obtain N image data groups; in response to a second input, the first video in the N image data groups is synthesized to obtain a second video, and the first static image in the N image data groups is synthesized to generate a second static image. The second video includes a content layer and a dynamic effects layer superimposed on the content layer. The content layer includes N first videos arranged in a grid layout, and the second static image includes N first static images arranged in a grid layout. The second video and the second static image are associated to generate a grid-layout dynamic photo. This solution allows for the direct generation of grid-layout dynamic photos during the shooting stage, offering a simple and convenient operation that not only preserves the original high-quality image content but also improves the final image quality. Attached Figure Description

[0018] Figure 1 This is one of the schematic flowcharts of the shooting method according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram illustrating the association between the grid dynamic video and the grid static image in an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of entering the shooting interface of a film camera in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram illustrating the activation of the N-grid shooting function and the dynamic shooting function in the embodiments of this application;

[0022] Figure 5 This is a schematic diagram illustrating how the preview interface is accessed by clicking the floating button in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the preview interface in an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of taking a picture by clicking the photo button in an embodiment of this application;

[0025] Figure 8 This is one of the schematic diagrams illustrating the generation of a grid-like dynamic photograph in the embodiments of this application;

[0026] Figure 9 This is the second schematic diagram of generating dynamic grid photos in the embodiments of this application;

[0027] Figure 10 This is a schematic diagram of the frame buffer in an embodiment of this application;

[0028] Figure 11 This is a schematic diagram illustrating the playback of a grid of dynamic photos in an embodiment of this application;

[0029] Figure 12 This is a second schematic flowchart of the shooting method according to an embodiment of this application;

[0030] Figure 13 This is a schematic diagram of the camera device according to an embodiment of this application;

[0031] Figure 14 This is one of the structural block diagrams of the electronic device according to an embodiment of this application;

[0032] Figure 15 This is the second structural block diagram of the electronic device according to an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0034] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0035] The shooting method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0036] like Figure 1 As shown, this application provides a shooting method executed by an electronic device, which may be a terminal. The shooting method includes:

[0037] Step 101: In response to the first input, take N shots to obtain N image data groups, each of the image data groups including a first still image and a first video associated with the first still image, where N is a positive integer greater than 1.

[0038] In some embodiments of this application, during each shooting process, the system performs parallel shooting and video encoding processing to generate two cached data sets. One cached data set is the aforementioned first static image, and the other cached data set is the first video associated with the first static image.

[0039] Optionally, the first still image can be a high-resolution image processed by a multi-frame algorithm, which includes, but is not limited to, high dynamic range (HDR) processing and noise reduction. The first still image can be a JPEG format image, which can be cached as a still image bitmap after decoding. This high-resolution image provides the data foundation for subsequently providing high-quality grid-based dynamic photos.

[0040] Optionally, the first video associated with the first still image is a dynamic video segment associated with the shooting time of the first still image. For example, a sequence of video frames 1.5 seconds before and 1.5 seconds after the shooting time is extracted and encoded in real time using an encoder, such as MediaCodec, to generate an mp4 file.

[0041] For example, the first input mentioned above can be the user's action of continuously clicking the capture button on the shooting interface. For example, if N is 4, then the first input can be the user's action of clicking the capture button four times consecutively on the shooting interface. Each click of the capture button generates a group of image data.

[0042] Step 102: In response to the second input, the first video in the N groups of image data is synthesized to generate a second video, and the first static image in the N groups of image data is synthesized to generate a second static image. The second video includes a content layer and a dynamic effect layer superimposed on the content layer. The content layer includes N first videos arranged in a grid layout. The dynamic effect layer includes dynamic elements in a grid effect template. The second static image includes N first static images arranged in a grid layout.

[0043] Optionally, the second input mentioned above can be the input to the "Generate Photo" button in the preview interface.

[0044] Optionally, the grid effect template can be in PAG format, and the aforementioned dynamic elements include, but are not limited to, transition animations, text fly-in, and filter changes.

[0045] Optionally, the second video mentioned above can also be described as a grid-based dynamic video, and the second static image mentioned above can also be described as a grid-based static image.

[0046] Step 103: Associate the second video with the second static image to generate a dynamic photo with a grid layout.

[0047] Optionally, the above-mentioned grid layout of dynamic photos can be simply described as grid dynamic photos.

[0048] In one implementation, grid-based dynamic video and grid-based static images are encapsulated using a protocol that combines static images with associated dynamic media. For example, this encapsulation protocol could be a motion photo (or live photo) protocol. During encapsulation, Extensible Metadata Platform (XMP) information is written to the end of the grid-based dynamic video file to record the grid-based static image, grid-based dynamic video (or dynamic grid video), and motion format information. Private identification information is written to the end of the grid-based static image for recognition and parsing by photo album applications. Figure 2 As shown, the image content section describes and associates static images (such as the grid static images mentioned above). For example, the XMP information mentioned above can be encapsulated in this image content section. The Motion Photo Video Data (MPVD) section is used to tag and encapsulate the grid dynamic video. The media data box is used to store the compressed and encoded video frame data, such as the video frame data of the grid dynamic video mentioned above.

[0049] In the scheme of this application embodiment, in response to a first input, N shots are taken to obtain N image data groups; in response to a second input, the first video in the N image data groups is synthesized to obtain a second video, and the first static image in the N image data groups is synthesized to generate a second static image. The second video includes a content layer and a dynamic effect layer superimposed on the content layer. The content layer includes N first videos arranged in a grid layout, and the second static image includes N first static images arranged in a grid layout. The second video and the second static image are associated to generate a grid-layout dynamic photo. The above scheme can directly generate a grid-layout dynamic photo during the shooting stage, which is simple and convenient to operate, not only preserving the original high-quality image content, but also helping to improve the final image quality.

[0050] In some embodiments of this application, before performing step 101 above, the method of this application embodiment further includes: in response to the fourth input, enabling the grid dynamic photo shooting function.

[0051] Optionally, the fourth input mentioned above includes the user's input to enable the N-grid function control in the shooting interface and the input to enable the dynamic shooting function control. The first input can be a click input, a long press input, a swipe input, etc.

[0052] In one implementation, such as Figure 3 As shown, the user opens the camera and enters the OneShot film camera shooting interface by swiping up in the camera preview interface. Figure 4As shown, the N-grid shooting function, such as the four-grid shooting function, is enabled by the N-grid shooting function control 401 in the top bar of the shooting interface of the Yipao film camera. At the same time, the dynamic shooting function is enabled by the dynamic shooting function control 402 in the preview area of ​​the shooting interface. In this way, the purpose of supporting grid dynamic shooting is achieved.

[0053] In some embodiments of this application, the above-described shooting method further includes:

[0054] During N shots, display shooting progress information, and / or display the grid position information of the currently captured photo.

[0055] Once the dynamic shooting function and the N-grid shooting function are enabled, shooting progress information will be displayed. For example, this shooting progress information will include "Please take the i-th shot," where 1 ≤ i ≤ N, and i is a positive integer. This shooting progress information guides the user to complete N shots.

[0056] like Figure 4 As shown, a floating button 403 is displayed in the preview area of ​​the shooting interface. This floating button can display a grid layout thumbnail 4031 and numerical information 4032. The grid layout thumbnail 4031 displays the grid position information of the currently captured photo, and the numerical information 4032 displays the shooting progress information, i.e., the number of photos currently captured. For example, N is 4, and the grid layout thumbnail is a grid of characters. For example, when the numerical information 403 is 0 / N, it indicates that the number of photos captured out of N photos is 0; when the numerical information is 1 / N, it indicates that the number of photos captured out of N photos is 1; and when the numerical information 403 is 2 / N, it indicates that the number of photos captured out of N photos is 2. When the user is about to take the first photo, the first cell of the aforementioned grid layout thumbnail will be highlighted, flashed, or filled. After the first photo is taken, the first square may turn solid color or display a thumbnail of the photo, while the second square will start to highlight, indicating that the next photo will be placed in the second position in the final image.

[0057] The aforementioned floating button provides users with clear feedback on shooting status and shooting position, allowing users to intuitively understand the shooting progress and grid position information, effectively solving problems such as sequence confusion, counting burden, and unclear target that are prone to occur in continuous shooting.

[0058] In some embodiments of this application, prior to the response to the second input, the method further includes:

[0059] In response to a third input, the N image data groups are updated, the third input being used to edit the N image data groups.

[0060] For example, the above editing includes deletion operations, position adjustment operations, etc.

[0061] In the embodiments of this application, such as Figure 5 As shown, after each shot, you can click the floating button 403 to enter the preview interface to view the shooting effect. The preview interface will automatically appear after N shots are taken. Figure 6 As shown, in the preview interface, a delete button 501 is displayed at a predetermined display position for each captured photo (such as the first still image mentioned above), such as the upper right corner. Users can delete the captured grid photos using this delete button. Long-pressing and dragging a photo allows adjustment of its arrangement, thereby updating the first still image in N image data groups. Based on the updated first still image and the first video associated with it, the goal of updating the N image data groups is achieved.

[0062] In some implementations, such as Figure 7 As shown, clicking the photo button 701 takes a picture. During each photo capture, the system performs parallel photo capture and video encoding processing, generating two sets of cached data. For example, one set of cached data is a high-resolution main image bitmap, i.e., the first static image mentioned above. The other set of cached data is a video mp4 file containing a real-time dynamic image sequence aligned 1.5 seconds before and after the capture time, i.e., the first video associated with the first static image mentioned above. The main image bitmap is obtained by decoding JPEG format data processed by a multi-frame algorithm. The cached video frame sequence 1.5 seconds before and after the capture is sent to the encoder (MediaCodec) for encoding and outputting an mp4 file, which is then cached in the camera directory.

[0063] The cached Bitmap and MP4 files from N shots will be recorded in a list. Clicking the floating button 403 at the bottom will take you to the shooting preview interface, as shown below. Figure 6 As shown, clicking the button in the upper right corner deletes the captured grid photos, while long-pressing and dragging adjusts their arrangement. After each operation, the cached list data is updated synchronously. After four shots are taken, the user is automatically redirected to the photo preview interface. Clicking the "Generate Photo" button at the bottom triggers the generation of a grid-based dynamic video and a grid-based static image. This "Generate Photo" button is related to... Figure 6 The "Continue Taking Photos" button 601 can be the same button. After N photos have been taken, this button becomes the "Generate Photo" button; otherwise, it becomes the "Continue Taking Photos" button.

[0064] In this embodiment, the shooting effect can be viewed after each photo is taken, and after N shots are taken, the preview interface is automatically entered.

[0065] The proposed solution adds a process before the final compositing step where users can edit the first still image they captured (such as adjusting the order, deleting, or replacing it). This significantly enhances the user's sense of control and flexibility during the creative process, allowing them to correct mistakes or optimize the arrangement of materials in real time during the shooting stage. This ensures that the materials entering the compositing stage are high-quality materials that satisfy the user, guaranteeing both the efficiency of "one-click filmmaking" and the fine control over the quality of the final work. As a result, it increases the success rate of finished films and user satisfaction while reducing reliance on external post-editing.

[0066] In some embodiments of this application, the step of compositing a first video from N groups of image data to generate a second video, and compositing a first still image from the N groups of image data to generate a second still image, includes:

[0067] The first video and the grid effect template in the N image data groups are rendered and synthesized to generate the second video. The first static image in the N image data groups is stitched and synthesized according to the grid layout of the grid effect template to generate the second static image.

[0068] For example, N is 4, such as Figure 8 As shown, the video data of the four first videos, namely video data 1, video data 2, video data 3 and video data 4, and the four grid effect template files are synthesized and encoded to obtain the second video. The data of the four first static images, namely photo data 1, photo data 2, photo data 3 and photo data 4, are spliced ​​and synthesized to obtain the second static image. The grid dynamic photo is obtained based on the second video and the second static image.

[0069] By using the aforementioned grid effect template to generate the second video and the second static image, it is possible to ensure that the grid layout information (such as the number, position, and size ratio of the grids) on which the video rendering and image stitching are based is consistent. This enables the final generated second video and second static image to achieve pixel-level alignment in spatial composition, thereby effectively guaranteeing the image quality of the grid dynamic photo generated based on the second video and the second static image.

[0070] In some embodiments of this application, the step of rendering and compositing the first video and the grid effect template from the N image data groups to generate the second video includes:

[0071] The first video in each of the N image data groups is decoded to obtain the pixel data of the N first videos, and the pixel data of the N first videos is converted into N video textures;

[0072] The grid effect template is parsed to obtain the grid layout information and the dynamic effect layer;

[0073] In the frame buffer of the graphics processing unit (GPU), N video textures are rendered to the frame buffer according to the grid layout information to obtain the content layer;

[0074] The dynamic effects layer is overlaid and rendered on the content layer to obtain the second video.

[0075] In this embodiment, the grid effect template is parsed to extract the defined layer structure, keyframe animation, and various visual effects parameters. Based on the extracted information, the grid layout information and dynamic effect layers are obtained.

[0076] For example, the grid layout information mentioned above includes the number of rows and columns, the coordinates and size of each grid cell, and the spacing between grid cells.

[0077] For example, the aforementioned dynamic effects layer is obtained by the graphics engine calculating and rendering all dynamic elements (such as transitions, text animations, filter changes, etc.) in the grid effect template in real time according to the current compositing timeline. This dynamic effects layer can be understood as an overlay texture containing dynamic effects.

[0078] In some implementations of this application, the pixel data of the first video can be raw RGB or YUV pixel data. After obtaining the pixel data of N first videos, the pixel data of the N first videos is further uploaded to the graphics processing unit (GPU) to generate a GPU texture object that can be directly manipulated by the rendering pipeline, i.e., the aforementioned video texture. At the same time, the system loads a preset grid effect template file (assuming it is in PAG format), such as a four-grid effect template file. This template file is deconstructed by a parser to extract the layer structure, keyframe animation, and various visual effects parameters defined therein, and the grid layout information and dynamic effect layers are obtained based on the extracted information. The core rendering and compositing stage is performed in the GPU's framebuffer.

[0079] In this embodiment, on the one hand, the parallel computing power of the GPU significantly improves the rendering efficiency and compositing speed of grid dynamic videos, while reducing the load and power consumption of the central processing unit (CPU); on the other hand, through pixel-level precision layout control and flexible overlay of dynamic effect layers, while ensuring accurate alignment of the image, rich and smooth dynamic visual effects are achieved, thus technically supporting a high-quality, low-latency, and flexibly scalable "one-click generation" grid dynamic photo experience.

[0080] In some embodiments of this application, the second video further includes a background layer, which includes a photographic paper texture.

[0081] For example, the photo paper texture can be a vintage photo paper texture. This vintage photo paper texture can create a retro, nostalgic, or personalized visual style, effectively enhancing the finished product's feel and providing a unified visual base for the final image.

[0082] When a user triggers the "Generate Photo" command, the system initiates an integrated multimedia compositing process. This process is coordinated by the Encoding Manager, for example, as shown in... Figure 9 As shown, the photo and video data generated from four independent shots are packaged and processed. First, the video compositing pipeline is entered. The encoding manager retrieves the temporary Live video files corresponding to each of the four shots. These files are sequentially fed into the video decoder, which decodes the compressed video stream frame by frame into raw RGB or YUV pixel data, and then uploads it to the graphics processing unit (GPU) to generate GPU texture objects that can be directly manipulated by the rendering pipeline. At the same time, the system loads a preset four-grid effect template file (assuming it is in PAG format). This template file is deconstructed by the parser, extracting the defined layer structure, keyframe animation, and various visual effects parameters. The core rendering and compositing stage is performed in the GPU's framebuffer. The following uses the above-mentioned four-grid effect template as an example to illustrate the rendering and compositing process, specifically including:

[0083] (1) Video layout rendering: The four video textures obtained from decoding are precisely rendered into the specified areas a, b, c, and d of the frame buffer according to the four-grid layout information parsed from the template, such as... Figure 10 As shown.

[0084] (2) PAG animation rendering: The graphics engine calculates and renders all dynamic elements (such as transitions, text animations, filter changes, etc.) in the PAG template in real time according to the current composition timeline, and outputs an overlay texture containing dynamic effects.

[0085] (3) Layer compositing: Using the blending mode defined by the template, the static grid video layer (i.e., the content layer) generated in the first step is blended with the dynamic PAG effect layer (i.e., the dynamic effect layer mentioned above) generated in the second step at the pixel level.

[0086] (4) Photo paper material overlay: The specified retro photo paper material texture is used as the background or overlay layer and mixed with the above synthesis result to give the overall image a specific retro texture.

[0087] Finally, the synthesized frame sequence is fed into a video encoder (such as an H.264 / HEVC encoder) to generate a complete grid video file containing four frames and dynamic effects (i.e., a grid dynamic video). In parallel, a still image (i.e., the first still image) compositing pipeline is performed. The system acquires four high-resolution large-image data sets processed by independent algorithms. Using image processing algorithms, the four images are geometrically aligned and edge-blended according to a four-grid arrangement consistent with the video template. Subsequently, a retro photo paper simulation algorithm is applied to overlay a uniform tone, graininess, and retro elements such as borders onto the stitched overall image, ultimately synthesizing a high-resolution four-grid still image (i.e., the second still image). Finally, in the media binding stage, the encoding manager associates and encapsulates the four-grid video file and the four-grid still image produced in the above steps. Figure 2 As shown, LivePhotos are packaged using the Motion protocol, with XMP information appended to the end of the video file to record image, video, and Motion format information. Private tags are appended to the end of the image file for identification and LivePhoto parsing. This ensures a seamless transition between still photos and dynamic videos on supported platforms or devices through different interaction methods (such as long presses). Figure 11 As shown, long-press to play a grid of live photos in the gallery, such as playing a grid of dynamic videos associated with static images.

[0088] The solution in this application guides the user to take N shots. The N shots and live video are packaged together, and the video file is combined with an N-grid PAG effect file. Layer rendering and encoding are then performed, and the photos are stitched together using a grid algorithm. The resulting photos and live video are then combined and bound into a single N-grid live photo. This solution eliminates the tedious post-editing process, effectively improving the color and image quality of the N-grid dynamic photo. Furthermore, a single grid live photo can be generated using only the first and second inputs, increasing the success rate.

[0089] The following explanation uses N=4 as an example to illustrate the shooting method of this application.

[0090] like Figure 12 As shown, the shooting method includes:

[0091] Step 1201: Turn on the camera and enter the OneShot film camera mode.

[0092] For example, such as Figure 3 As shown, you can enter the OneShot film camera shooting interface by swiping up in the camera preview interface.

[0093] Step 1202: Enable the 4-grid shooting function through the 4-grid shooting function control 401 in the top bar of the OneShot film camera shooting interface.

[0094] Step 1203: Determine whether the dynamic shooting function is enabled.

[0095] Step 12041: If the dynamic shooting function is enabled, the first image data set is generated.

[0096] Step 12042: Generate the second image data set.

[0097] Step 12043: Generate the third image data set.

[0098] Step 12044: Generate the fourth image data set.

[0099] Step 12051: If the dynamic shooting function is not enabled, take the first photo;

[0100] Step 12052: Take the second photo;

[0101] Step 12053: Take the third photo;

[0102] Step 12054: Take the fourth photo;

[0103] Step 12055: Display the final image style;

[0104] Step 120561: After taking each photo, proceed to editing.

[0105] Step 120562: Delete or not?

[0106] If the photo is deleted, the current photo will be taken again; otherwise, the next photo will be taken.

[0107] Step 120563: Click Generate.

[0108] Step 1207: Synthesize the code using the grid effect template.

[0109] Step 1208: Generate the composite image.

[0110] It should be noted that when the dynamic shooting function is enabled, the composite image is a dynamic photo with the grid layout mentioned above; when the dynamic shooting function is not enabled, the composite image is a static photo with the grid layout.

[0111] Step 1209: Save to album.

[0112] The solution in this application example guides users to take N shots. By packaging the N shots of photos and live videos into a single data package, combining the video file with an N-grid effect PAG file, performing layer rendering, compositing, and encoding, and then stitching the photos together using a grid algorithm, the resulting photos and live videos are combined and bound to generate an N-grid live photo. This avoids the need for secondary editing when users shoot N-grid photos that lack dynamic elements, reduces issues such as image cropping and resolution reduction caused by post-editing, and generates grid dynamic photos in a WYSIWYG manner, thus improving the user experience.

[0113] The shooting method provided in this application can be executed by a shooting device. This application uses a shooting device executing the shooting method as an example to illustrate the shooting device provided in this application.

[0114] like Figure 13 As shown, this application embodiment also provides a shooting device 1300, including:

[0115] The first processing module 1301 is configured to respond to the first input by taking N shots to obtain N image data groups, each image data group including a first still image and a first video associated with the first still image, where N is a positive integer greater than 1;

[0116] The second processing module 1302 is configured to, in response to the second input, perform compositing processing on the first video in the N image data groups to generate a second video, and perform compositing processing on the first static image in the N image data groups to generate a second static image, wherein the second video includes a content layer and a dynamic effect layer superimposed on the content layer, the content layer includes N first videos arranged in a grid layout, the dynamic effect layer includes dynamic elements in a grid effect template, and the second static image includes N first static images arranged in a grid layout;

[0117] The third processing module 1303 is used to associate the second video with the second static image to generate a dynamic photo with a grid layout.

[0118] Optionally, the second processing module is used for:

[0119] The first video and the grid effect template in the N image data groups are rendered and synthesized to generate the second video. The first static image in the N image data groups is stitched and synthesized according to the grid layout of the grid effect template to generate the second static image.

[0120] Optionally, the second processing module is used for:

[0121] The first video in each of the N image data groups is decoded to obtain the pixel data of the N first videos, and the pixel data of the N first videos is converted into N video textures;

[0122] The grid effect template is parsed to obtain the grid layout information and the dynamic effect layer;

[0123] In the frame buffer of the graphics processor, N video textures are rendered to the frame buffer according to the grid layout information to obtain the content layer;

[0124] The dynamic effects layer is overlaid and rendered on the content layer to obtain the second video.

[0125] Optionally, the device further includes:

[0126] The fourth processing module is used to update the N image data groups in response to the third input, wherein the third input is used to edit the N image data groups.

[0127] Optionally, the imaging device in this application embodiment further includes:

[0128] The display module is used to display shooting progress information and / or the grid position information of the currently captured photo during N shooting sessions.

[0129] The apparatus of this application embodiment, in response to a first input, performs N shots to obtain N image data groups; in response to a second input, it synthesizes a first video from the N image data groups to obtain a second video, and synthesizes a first static image from the N image data groups to generate a second static image. The second video includes a content layer and a dynamic effects layer superimposed on the content layer. The content layer includes N first videos arranged in a grid layout, and the second static image includes N first static images arranged in a grid layout. The second video and the second static image are associated to generate a grid-layout dynamic photo. This solution allows for the direct generation of grid-layout dynamic photos during the shooting stage, offering simple and convenient operation. It not only preserves the original high-quality image content but also improves the final image quality.

[0130] The shooting device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. The embodiments of this application do not specifically limit the scope.

[0131] The shooting device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems, and this application embodiment does not specifically limit it.

[0132] The imaging device provided in this application embodiment can achieve... Figures 1 to 12 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0133] Optionally, such as Figure 14As shown, this application embodiment also provides an electronic device 1400, including a processor 1401 and a memory 1402. The memory 1402 stores a program or instructions that can run on the processor 1401. When the program or instructions are executed by the processor 1401, they implement the various steps of the above-described shooting method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0134] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0135] Figure 15 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0136] The electronic device 1500 includes, but is not limited to, components such as: radio frequency unit 1501, network module 1502, audio output unit 1503, input unit 1504, sensor 1505, display unit 1506, user input unit 1507, interface unit 1508, memory 1509, and processor 1510.

[0137] Those skilled in the art will understand that the electronic device 1500 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1510 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 15 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0138] In one embodiment of this application, the processor 1510 is configured to perform N shots in response to a first input to obtain N image data groups, each of the image data groups including a first still image and a first video associated with the first still image, where N is a positive integer greater than 1.

[0139] In response to the second input, the first video in the N groups of image data is synthesized to generate the second video, and the first static image in the N groups of image data is synthesized to generate the second static image. The second video includes a content layer and a dynamic effect layer superimposed on the content layer. The content layer includes N first videos arranged in a grid layout. The dynamic effect layer includes dynamic elements in a grid effect template. The second static image includes N first static images arranged in a grid layout.

[0140] The second video is associated with the second still image to generate a dynamic photo with a grid layout.

[0141] Optionally, the processor is further configured to: perform rendering and compositing processing on the first video and the grid effect template in the N image data groups to generate the second video, and perform splicing and compositing processing on the first static image in the N image data groups according to the grid layout of the grid effect template to generate the second static image.

[0142] Optionally, the processor is further configured to:

[0143] The first video in each of the N image data groups is decoded to obtain the pixel data of the N first videos, and the pixel data of the N first videos is converted into N video textures;

[0144] The grid effect template is parsed to obtain the grid layout information and the dynamic effect layer;

[0145] In the frame buffer of the graphics processor, N video textures are rendered to the frame buffer according to the grid layout information to obtain the content layer;

[0146] The dynamic effects layer is overlaid and rendered on the content layer to obtain the second video.

[0147] Optionally, the processor is further configured to:

[0148] In response to a third input, the N image data groups are updated, the third input being used to edit the N image data groups.

[0149] Optionally, the processor is further configured to:

[0150] During N shots, display shooting progress information, and / or display the grid position information of the currently captured photo.

[0151] It should be understood that, in this embodiment, the input unit 1504 may include a graphics processing unit (GPU) 15041 and a microphone 15042. The GPU 15041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1506 may include a display panel 15061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1507 includes at least one of a touch panel 15071 and other input devices 15072. The touch panel 15071 is also called a touch screen. The touch panel 15071 may include a touch detection device and a touch controller. Other input devices 15072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0152] The memory 1509 can be used to store software programs and various data. The memory 1509 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1509 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1509 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0153] Processor 1510 may include one or more processing units; optionally, processor 1510 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1510.

[0154] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described shooting method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0155] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0156] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described shooting method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0157] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0158] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described shooting method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0159] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0161] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A shooting method, characterized in that, include: In response to the first input, N shots are taken to obtain N image data groups, each of which includes a first still image and a first video associated with the first still image, where N is a positive integer greater than 1; In response to the second input, the first video in the N groups of image data is synthesized to generate the second video, and the first static image in the N groups of image data is synthesized to generate the second static image. The second video includes a content layer and a dynamic effect layer superimposed on the content layer. The content layer includes N first videos arranged in a grid layout. The dynamic effect layer includes dynamic elements in a grid effect template. The second static image includes N first static images arranged in a grid layout. The second video is associated with the second still image to generate a dynamic photo with a grid layout.

2. The shooting method according to claim 1, characterized in that, The step of compositing the first video from the N image data groups to generate the second video, and compositing the first still image from the N image data groups to generate the second still image, includes: The first video and the grid effect template in the N image data groups are rendered and synthesized to generate the second video. The first static image in the N image data groups is stitched and synthesized according to the grid layout of the grid effect template to generate the second static image.

3. The shooting method according to claim 2, characterized in that, The step of rendering and compositing the first video and the grid effect template from the N image data groups to generate the second video includes: The first video in each of the N image data groups is decoded to obtain the pixel data of the N first videos, and the pixel data of the N first videos is converted into N video textures; The grid effect template is parsed to obtain the grid layout information and the dynamic effect layer; In the frame buffer of the graphics processor, N video textures are rendered to the frame buffer according to the grid layout information to obtain the content layer; The dynamic effects layer is overlaid and rendered on the content layer to obtain the second video.

4. The shooting method according to claim 1, characterized in that, The method further includes responding before the second input: In response to a third input, the N image data groups are updated, the third input being used to edit the N image data groups.

5. The shooting method according to claim 1, characterized in that, Also includes: During N shots, display shooting progress information, and / or display the grid position information of the currently captured photo.

6. A shooting device, characterized in that, include: A first processing module is configured to, in response to a first input, take N shots to obtain N image data groups, each image data group including a first still image and a first video associated with the first still image, where N is a positive integer greater than 1; The second processing module is configured to, in response to the second input, perform compositing processing on the first video in the N image data groups to generate a second video, and perform compositing processing on the first static image in the N image data groups to generate a second static image, wherein the second video includes a content layer and a dynamic effect layer superimposed on the content layer, the content layer includes N first videos arranged in a grid layout, the dynamic effect layer includes dynamic elements in a grid effect template, and the second static image includes N first static images arranged in a grid layout; The third processing module is used to associate the second video with the second static image to generate a dynamic photo with a grid layout.

7. The shooting device according to claim 6, characterized in that, The second processing module is used for: The first video and the grid effect template in the N image data groups are rendered and synthesized to generate the second video. The first static image in the N image data groups is stitched and synthesized according to the grid layout of the grid effect template to generate the second static image.

8. The shooting device according to claim 7, characterized in that, The second processing module is used for: The first video in each of the N image data groups is decoded to obtain the pixel data of the N first videos, and the pixel data of the N first videos is converted into N video textures; The grid effect template is parsed to obtain the grid layout information and the dynamic effect layer; In the frame buffer of the graphics processor, N video textures are rendered to the frame buffer according to the grid layout information to obtain the content layer; The dynamic effects layer is overlaid and rendered on the content layer to obtain the second video.

9. The shooting device according to claim 6, characterized in that, The device further includes: The fourth processing module is used to update the N image data groups in response to the third input, wherein the third input is used to edit the N image data groups.

10. The shooting device according to claim 6, characterized in that, Also includes: The display module is used to display shooting progress information and / or the grid position information of the currently captured photo during N shooting sessions.

Images