Co-production methods, devices, electronic equipment and readable media
By extracting the outline information of template materials and using compositing algorithms, the problem of compositing complex scenes and body movements by users in duet applications has been solved, achieving high-quality duet effects and improving user experience.
Patent Information
- Application Number
- CN202111027906.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-02
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2041-09-02
AI Technical Summary
Existing duet apps struggle to achieve high-quality compositing when users mimic complex scenes and body movements, resulting in a disconnect and unnatural transition between the user's footage and the original material, thus impacting the user experience.
By extracting the outline information of the first instance from the template material, users are guided to import the co-production material. Then, semantic segmentation or instance segmentation algorithms are used to synthesize the second instance with the first instance region in the template material. Combined with color and lighting transfer technology, the consistency between the co-production material and the template material is improved.
It achieves high-quality co-production in complex scenes and with physical movements, improving the compositing effect and user experience, and ensuring a natural transition and visual consistency between co-production materials and template materials.
Smart Images

Figure CN115766972B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and readable medium for simultaneous shooting. Background Technology
[0002] With the development of social media, photography, and special effects software, a plethora of entertaining and engaging applications have emerged. Users can capture photos or video clips to imitate characters from classic photos or film scenes, recreating specific scenarios and plots, and experiencing the joy of performance and acting. These applications require the user's imitation to be combined with the original photo or clip—a process known as "composite shooting." For example, in movie character replacement, a user can film themselves, imitating the character's body language, expressions, and dialogue. Then, the user can replace the original character in the movie clip, making the user feel as if they are actually there, resulting in a more vivid and realistic portrayal of the film.
[0003] However, the complex and diverse shooting environments of users, the difficulty for users to accurately portray the original characters, and the variations in the distance and position of shooting equipment, as well as the differences in the movements imitated each time, all pose challenges to the compositing of the original video with the user's shooting content. Current co-production applications offer relatively simple characters or scenes for users to imitate, primarily involving only facial expressions or minor head movements. If the movements are larger or involve changes in body movements, high-quality co-production cannot be achieved, resulting in a disconnect between the user's shooting content and the original material, unnatural transitions, poor composite effects, and a negative impact on the user experience. Summary of the Invention
[0004] This disclosure provides a co-production method, apparatus, electronic device, and readable medium to improve the consistency between co-production material and template material, thereby increasing the accuracy of co-production.
[0005] In a first aspect, embodiments of this disclosure provide a method for simultaneous shooting, including:
[0006] Extract the outline information of the first instance from the template material, wherein the template material includes the first instance and the instance background;
[0007] Obtain the duet footage imported by the user based on the outline information, wherein the duet footage includes a second instance corresponding to the first instance;
[0008] The second instance is added to the area corresponding to the first instance in the template material to obtain a matching result, wherein the matching result includes the second instance and the instance background.
[0009] Secondly, this disclosure also provides a timing device, including:
[0010] The outline extraction module is used to extract the outline information of the first instance in the template material, wherein the template material includes the first instance and the instance background;
[0011] The material acquisition module is used to acquire the duet material imported by the user based on the outline information, wherein the duet material includes a second instance corresponding to the first instance;
[0012] The "co-op" module is used to add the second instance to the area corresponding to the first instance in the template material to obtain a co-op result, wherein the co-op result includes the second instance and the instance background.
[0013] Thirdly, embodiments of this disclosure also provide an electronic device, including:
[0014] One or more processors;
[0015] Storage device for storing one or more programs;
[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the synchronization method as described in the first aspect.
[0017] Fourthly, embodiments of this disclosure also provide a computer-readable medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the synchronization method as described in the first aspect.
[0018] The method of this disclosure guides the user to import duet material by utilizing the outline information of the first instance, thereby improving the consistency between the duet material and the template material, and thus achieving the synthesis of the background of the second instance and the template material instance, improving the accuracy of the duet. Furthermore, due to the guidance of the outline information, the method can be applied to more diverse duet scenarios. Attached Figure Description
[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0020] Figure 1 This is a flowchart of the synchronization method in Embodiment 1 of this disclosure;
[0021] Figure 2 This is a flowchart of the synchronization method in Embodiment 2 of this disclosure;
[0022] Figure 3 This is a schematic diagram of the first instance of the template material in Embodiment 2 of this disclosure;
[0023] Figure 4 This is a schematic diagram of the outline information of the first example in Embodiment 2 of this disclosure;
[0024] Figure 5 This is a schematic diagram of background completion for the area where the first instance was removed, as shown in Embodiment 2 of this disclosure;
[0025] Figure 6 This is a schematic diagram of the user shooting interface in Embodiment 2 of this disclosure;
[0026] Figure 7 This is a schematic diagram illustrating the determination of template materials in Embodiment 2 of this disclosure;
[0027] Figure 8 This is a flowchart of the synchronization method in Embodiment 3 of this disclosure;
[0028] Figure 9 This is a schematic diagram of the timing device in Embodiment 4 of this disclosure;
[0029] Figure 10 This is a schematic diagram of the hardware structure of the electronic device in Embodiment 5 of this disclosure. Detailed Implementation
[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.
[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.
[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0034] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0035] In the following embodiments, each embodiment provides optional features and examples. The various features described in the embodiments can be combined to form multiple optional solutions. Each numbered embodiment should not be regarded as only one technical solution. Furthermore, unless otherwise specified, the embodiments and features in the embodiments of this disclosure can be combined with each other.
[0036] Example 1
[0037] Figure 1 This is a flowchart of the co-shooting method in Embodiment 1 of this disclosure. This method is applicable to situations where users co-shoot using template materials. Specifically, by adding instances from the co-shooting materials to the template materials and compositing them with the instance background in the template materials, various scenes or plots can be imitated or interpreted. This method can be executed by a co-shooting device, which can be implemented by software and / or hardware and integrated into an electronic device. In this embodiment, the electronic device can be a computer, laptop, server, tablet computer, or smartphone, or other device with image processing capabilities.
[0038] like Figure 1 As shown in Embodiment 1 of this disclosure, a method for simultaneous shooting includes the following steps:
[0039] S110. Extract the outline information of the first instance in the template material, wherein the template material includes the first instance and the instance background.
[0040] In this embodiment, template materials can be understood as images or videos provided for users to reference for imitation or interpretation, such as famous paintings, classic movie clips, or special effects animations. Template materials used for co-production can be specified by the user and downloaded locally from a material library by an electronic device. Template materials include a first instance and an instance background. The first instance includes the object to be imitated or interpreted by the user. This object will not be displayed in the co-production result but will be replaced or covered by the content imitated or interpreted by the user. For example, the first instance can be a character in a movie clip, or it can include props held by the character. The instance background includes objects that do not need to be imitated or interpreted by the user but can be displayed in the co-production result, such as the environment around the character in the movie clip, such as walls, roads, and rivers.
[0041] Template materials can contain multiple instances. Semantic segmentation algorithms or instance segmentation algorithms can identify and segment all instances within the template materials. The principle can be understood as follows: bounding boxes are detected and located for each instance in the template materials. For each instance's bounding box, pixel-level foreground and background segmentation is performed within the bounding box, where the foreground is the instance and the background is considered the instance's background. The first instance can be one or more of multiple instances. The first instance can be determined by the electronic device based on the template materials' default configuration or specified by the user. It should be noted that semantic segmentation algorithms are primarily suitable for template materials with only one first instance, while instance segmentation algorithms are primarily suitable for template materials with at least two first instances.
[0042] The contour information of the first instance can be extracted using semantic segmentation or instance segmentation algorithms. This contour information describes the position and shape of the first instance. For example, if the first instance in a template is a little girl dancing, the contour information needs to indicate the little girl's position and dancing posture within the template to help users adjust the shooting angle when taking photos with electronic devices, ensuring the user is correctly positioned in the frame and guiding them to perform the same or similar actions as the little girl. Contour information can be represented through text, lines, symbols, simple drawings, or auxiliary lines.
[0043] Optionally, after determining the first instance in the template material, the outline information of the first instance is stored locally. When the user shoots a duet, the outline information is read and displayed in a visual form on the user's shooting interface to guide the user in positioning and completing the corresponding actions. Since the template material has been downloaded to the electronic device, the process of semantic segmentation or instance segmentation of the template material can be performed offline.
[0044] S120. Obtain the duet material imported by the user based on the outline information, wherein the duet material includes a second instance corresponding to the first instance.
[0045] Specifically, collaborative footage can be understood as images or videos captured by a user imitating or interpreting a first instance. The content imitated or interpreted by the user constitutes a second instance, which corresponds to the first instance. Specifically, the second instance shares the same or similar outline as the first instance. Collaborative footage can be footage captured in real-time by the user based on the outline information of the first instance, or it can be pre-captured footage imported from a gallery, including instances with outlines identical or similar to the first instance. This ensures consistency between the outlines of the second and first instances, improves the usability of the collaborative footage, and facilitates accurate compositing of the second instance with the instance background.
[0046] Optionally, the duet footage can also include the shooting background, that is, the environment in which the user is when shooting the duet footage. For example, if a user imitates the little girl in the template footage dancing in their bedroom and shoots the footage, the shot can be used as the duet footage, the user dancing in the shot is the second instance, and the bedroom environment in the shot is the shooting background.
[0047] In this embodiment, the user-imported duet footage needs to undergo semantic segmentation or instance segmentation to obtain a second instance and a shooting background. The second instance is used to replace or cover the first instance in the template footage, thereby compositing with the instance background to achieve a duet. If there is only one second instance in the duet footage (usually the user being photographed), the outline of the second instance can be extracted using a semantic segmentation algorithm, saving computation. If there may be multiple second instances in the duet footage, an instance segmentation algorithm can be used to identify all the second instances in the duet footage. In this case, the segmentation result of each second instance can be associated with the instance identifier. Based on the relative positional relationship between multiple second instances and / or the outline information of each second instance, the first instance associated with each second instance in the template footage can be determined, thereby enabling multi-user duets.
[0048] Optionally, since the styles of user-shot collaborative footage are varied and highly uncertain, semantic segmentation or instance segmentation of collaborative footage can be performed online, which facilitates flexible use of relevant algorithms and computing resources.
[0049] S130. Add the second instance to the area corresponding to the first instance in the template material to obtain a matching result, wherein the matching result includes the second instance and the instance background.
[0050] Specifically, the second instance can be used to replace or cover the first instance in the template material, thus compositing it with the instance background to achieve a harmonious result. Adding the second instance to the area corresponding to the first instance can mean removing the first instance from the template material (the area after removing the first instance can be empty or blank, or it can be filled according to the texture features of the instance background), and then displaying the second instance on the area corresponding to the first instance. In this case, the second instance blends better with the instance background. Alternatively, it can mean covering the first instance with the second instance. In this case, a high degree of consistency between the outlines of the first and second instances is required, ensuring that the second instance completely covers the first instance. Adding the second instance to the area corresponding to the first instance can also be understood as the compositing process of the second instance and the instance background.
[0051] Optionally, the process of adding a second instance to the template assets can be done online.
[0052] Optionally, the area corresponding to the first instance includes the outline of the first instance. In this case, the area corresponding to the first instance is relatively large compared to the first instance. When replacing or covering the first instance with the second instance, the second instance is added to the larger area. This avoids any gaps between the second instance and the instance background, allowing the second instance to exist more naturally on the instance background. This results in a more natural edge transition after replacement, improving the user's visual experience of the matching result.
[0053] Optionally, if there are multiple second instances in the collaborative video material, the relative positional relationship between the second instances and / or the outline information of each second instance can be used to determine which first instance in the template material each second instance corresponds to. Based on this, each second instance is added to the area corresponding to the first instance in the template material to achieve multi-user collaborative video.
[0054] This embodiment provides a method for creating a collaborative video, which uses the outline information of a first instance to guide the user to import collaborative video materials, thereby improving the consistency of the outlines of instances in the collaborative video materials and template materials. This enables the compositing of the second instance with the background of the template material instance, improving the accuracy of the collaborative video. Furthermore, due to the guidance of the outline information, this method can be applied to a wider variety of collaborative video scenarios. Even when the first instance has complex movements, large range of motion, or changes in body movements, it can improve the usability of the collaborative video materials and ensure the compositing effect and the quality of the collaborative video.
[0055] Example 2
[0056] Figure 2 This is a flowchart of the co-production method in Embodiment 2 of this disclosure. Embodiment 2 elaborates on the process of obtaining co-production material and adding the second instance to the template material, based on the above embodiments.
[0057] like Figure 2 As shown in Embodiment 2 of this disclosure, a method for simultaneous shooting includes the following steps:
[0058] S210. Extract the outline information of the first instance in the template material. The template material includes the first instance and the instance background.
[0059] Figure 3 This is a schematic diagram of the first instance of the template material in Embodiment 2 of this disclosure. The template material can be an image or a video. If the template material is a video, the outline information of the first instance needs to be extracted frame by frame. Figure 3 As shown, taking a frame of image in the template material as an example, the area circled in white contains the first instance, which is a person including a head and upper body; the background of the instance mainly includes the sea and railings.
[0060] Figure 4This is a schematic diagram of the outline information of the first example in Embodiment 2 of this disclosure. For example... Figure 4 As shown, the first instance and the instance background are obtained by segmenting the template material into instances. The black area is the area corresponding to the instance background in the template material, and the white area is the area corresponding to the first instance. The boundary line between the black and white areas is the outline of the first instance. The outline information can be recorded or stored in the form of text, lines, symbols, simple drawings or auxiliary lines.
[0061] S220. Generate the contour auxiliary lines of the first instance based on the contour information.
[0062] Specifically, outline guidelines are used to identify the position and shape of the first instance in the template material. An outline guide is a line drawn around the outer edge of the first instance and can be represented by a dashed or solid line. For example, for... Figure 4 For the first instance in the dataset, the process of generating contour auxiliary lines can be as follows: based on the instance segmentation results, ... Figure 4 Points on the boundary line between the black and white areas are sampled, and starting from a certain sampling point, each sampling point is connected in a clockwise or counterclockwise direction to obtain a contour auxiliary line.
[0063] S230. Display outline guide lines in the user shooting interface to guide the user to shoot the matching material according to the outline guide lines.
[0064] In this embodiment, the collaborative footage is captured by the user following the outline guide lines. The outline guide lines are displayed at a specific location on the user's shooting interface, theoretically corresponding to the position of the first instance in the template footage, allowing for a set range of error. For example, Figure 3 If the first instance is located in the center-right area of the template material, then in the user's shooting interface, the outline guide line is also located in the center-right area of the shooting frame. Based on this, the outline guide line can guide the user to adjust the shooting angle so that the captured second instance (e.g., the user themselves) is located within the outline guide line. This allows the electronic device to quickly extract the second instance from the center-right area of the shooting frame for a combined shot. The user's shooting interface can also display prompts such as text, lines, symbols, and / or simple drawings. The user can then begin acting out and shooting based on these prompts and the outline guide line.
[0065] Optionally, if the error between the outline of the second instance and the outline auxiliary line is within a set range in the user shooting interface, the captured image will be used as the material for the collaborative shoot.
[0066] Specifically, in the user's shooting interface, if the error between the outline of the second instance and the outline auxiliary line is within a set range, meaning the position and shape of the second instance are consistent with or close to the outline (or outline auxiliary line) of the first instance, it indicates that the user's shooting image can correspond to the template material and meets the compositing conditions. In this case, the shooting image can be used as the composite material. If the position and shape of the second instance are inconsistent with or too far from the outline (or outline auxiliary line) of the first instance, the shooting image cannot accurately correspond to the template material, and the second instance and the instance background cannot be accurately connected or composited. In this case, prompts can be used to guide the user to adjust the position and shape. The error between the outline of the second instance and the outline auxiliary line being within a set range can mean that the number of pixels outside the outline auxiliary line in the second instance is less than a first threshold, the overlap between the outline of the second instance and the outline auxiliary line is greater than a second threshold, or the furthest distance between the corresponding pixels of the outline of the second instance and the outline auxiliary line is less than a third threshold, etc. This embodiment does not limit this.
[0067] Optionally, after obtaining the collaborative footage imported by the user based on the contour information, the method further includes: performing semantic segmentation or instance segmentation on the collaborative footage to obtain a second instance.
[0068] This embodiment uses the shooting screen in the user's shooting interface as the collaborative shooting material as an example. Using semantic segmentation or instance segmentation algorithms, a second instance can be extracted from the shooting screen, and there may be one or more second instances. If there is only one second instance in the collaborative shooting material, the outline of the second instance can be extracted using the semantic segmentation algorithm. Based on this, it can be added to the area corresponding to the first instance in the template material. If there are multiple second instances in the collaborative shooting material, all second instances in the collaborative shooting material can be identified using the instance segmentation algorithm. In this case, each second instance can be used to replace or cover the associated first instance in the template material, thereby realizing multi-user collaborative shooting.
[0069] S240. Remove the first instance from the template material.
[0070] In this embodiment, the first instance is removed from the template material, specifically through a cutout algorithm. The area after removing the first instance can be empty or blank, or it can be filled based on the texture features of the instance's background.
[0071] In one embodiment, after removing the first instance, the method further includes: filling in the background of the missing area after removing the first instance based on the image features of the instance background.
[0072] Figure 5This is a schematic diagram illustrating background completion of the region where the first instance was removed, as shown in Embodiment 2 of this disclosure. Specifically, the background completion process can be understood as using an image restoration or completion algorithm to predict the features of pixels within the missing region based on the image features of the instance background, and then filling the missing region accordingly. Figure 5 As shown, the background of the instance mainly includes features of the sea and railings. Based on this, the empty area after removing the first instance was filled with the textures of the sea and railings, and the filling content of the empty area was basically aligned with the instance background, thereby improving the composition quality and ensuring the visual continuity and consistency of the background. On this basis, after adding the second instance to the area corresponding to the first instance, the transition between the second instance and the instance background is more natural, and the composition effect is better.
[0073] S250. Add the second instance to the area corresponding to the first instance in the template material.
[0074] S260. Adjust the color of the second instance according to the image features of the instance background.
[0075] Specifically, the co-production footage is shot by the user, while the template footage is usually shot by professionals or those familiar with video production. The shooting conditions, colors, and styles of the co-production and template footage often differ significantly, resulting in the second instance appearing abrupt and the transition unnatural compared to the original background. In this embodiment, to improve the composite effect of the second instance and the original background, the color of the second instance is adjusted based on the image characteristics of the original background, making the composite footage visually more harmonious and natural. Specifically, for the co-production footage, the second instance is compared frame by frame with the corresponding frame of the original background in the template footage, and the color of the second instance is adjusted, specifically by adjusting the color values of each pixel in the second instance.
[0076] In this embodiment, adjusting the color of the second instance based on the image features of the instance background can also be understood as transferring the color of the instance background in the template material to the second instance. During the color transfer process, the hue, filters, and / or effects of the second instance can also be adjusted based on the image features of the instance background to make the second instance blend more naturally with the instance background. Optionally, the color transfer process can be performed online.
[0077] S270. Calculate the spherical harmonic coefficient of the first instance based on the template material, and estimate the normal direction corresponding to the first instance.
[0078] In this embodiment, in order to make the synthesis result closer to the template material and more realistic, the second instance is also lit and rendered using a spherical harmonic model. This process can also be understood as transferring the ambient lighting in the template material to the second instance to enhance the realism and three-dimensionality of the second instance in the composite material.
[0079] Specifically, by sampling the ambient light around the first instance in the template material into several spherical harmonics in different directions, the ambient light around the second instance is reproduced during the lighting rendering process, thereby simplifying the calculation process for ambient light modeling. Optionally, the spherical harmonic modeling process can be performed offline.
[0080] S280. Perform lighting rendering on the second instance based on the spherical harmonic coefficient and normal direction.
[0081] In this embodiment, by modeling the ambient lighting in the template material, the spherical harmonics (SHH) coefficients used to describe the ambient lighting are obtained. Based on the image of the template material and the segmented first instance, the normal direction corresponding to the first instance is estimated. According to the SHHH coefficients and the normal direction, the light intensity distribution or depth of the first instance in the normal direction can be analyzed. Based on this, lighting rendering is performed on the second instance, providing supplementary lighting to the second instance in the composite image from different directions. Optionally, the lighting rendering process for the second instance can be performed online.
[0082] Based on the above, the final composite result can be output by combining the second instance with the instance background after color and lighting transfer.
[0083] It should be noted that this embodiment does not limit the execution order of S250-S270. For example, in some scenarios, the second instance can be color-shifted and / or lighting-shifted first, and then the second instance after color-shifting and / or lighting-shifting can be added to the template material and composited with the instance background to obtain a matching result.
[0084] In one embodiment, the content displayed on the user shooting interface also includes a second instance and a shooting background. That is, the user shooting interface displays the shooting scene when the user imitates or performs in a real shooting environment. The second instance is displayed on the real shooting background. The user can adjust the position and complete the corresponding actions according to the outline auxiliary lines. In this case, the compositing of the second instance and the instance background can be carried out after the shooting of the co-shooting material is completed, which can reduce the computing power of electronic devices and the CPU utilization rate, and the performance requirements of electronic devices are relatively low.
[0085] Alternatively, the content displayed on the user shooting interface may also include a second instance and the instance background. That is, during the process of the user shooting the combined footage, the user shooting interface displays the composite image of the second instance and the instance background in the template material in real time. In this case, the user can preview the composite effect in real time, which makes it easier for the user to flexibly adjust the shooting position and actions. In this case, the composite of the second instance and the instance background is carried out simultaneously with the shooting, which involves a high amount of computation and relatively high performance requirements for electronic devices.
[0086] In one embodiment, the user shooting interface can also display template materials, that is, in addition to displaying the shooting screen or composite screen mentioned above, template materials can also be displayed simultaneously for the user to compare. Figure 6 This is a schematic diagram of the user shooting interface in Embodiment 2 of this disclosure. Figure 6 As shown, in the user shooting interface, the upper part displays the template material, and the lower part displays the outline guide lines, as well as the composite image of the second instance and the instance background displayed in real time. Users can adjust its position according to the outline guide lines to place it within the outline guide lines. Figure 6 (Fill the white diagonal line area in the text) and complete the corresponding action.
[0087] In one embodiment, the material library contains multiple templates available for matching, and the template materials can be selected by the user through the template selection interface. Figure 7 This is a schematic diagram illustrating the determination of template materials in Embodiment 2 of this disclosure. For example... Figure 7 As shown, the material library provides multiple template materials, which are different movie clips. Users can select a template material through the template selection interface and enter the user shooting interface to complete the shooting of the collaborative material.
[0088] In one embodiment, before extracting the outline information of the first instance in the template material, the method further includes: identifying instances in the template material that support matching; and determining at least one first instance from the instances that support matching based on user selection information.
[0089] Specifically, the template material can contain multiple instances that support co-production, and the first instance can be one or more of them. Users can select the first instance through the instance selection interface. For example, if the template material is a movie clip featuring two characters, the user can choose only one character to portray. The instance selection interface can display an icon for each character, such as highlighting each character with a blinking frame. If the user clicks on a character, the blinking frames for the other characters disappear, and the selected character's blinking frame becomes solid. This selected character is the first instance. Users can also select two characters as the first instance. In this case, two users need to co-produce, with each user portraying one character.
[0090] The following is a brief explanation of the process of electronic devices achieving synchronization, using a first example:
[0091] 1) The template selection interface allows users to choose the template material they want to portray, such as selecting a movie they want to act out;
[0092] 2) Extract the outline information of the movie characters (i.e., the first instance) using the instance segmentation algorithm, and remove the movie characters from the template material;
[0093] 3) Using image restoration algorithms, based on the image features of the instance background in the template material, the missing areas after removing the person are filled in;
[0094] 4) Enter the user shooting interface. The user shooting interface displays the outline auxiliary lines generated based on the outline information, which are used to guide the user to position themselves and complete the corresponding actions, and the shooting screen is used as the material for the duet.
[0095] 5) Extract the second instance from the collaborative footage using an instance segmentation algorithm;
[0096] 6) Perform color and / or lighting migration on the second instance based on the instance background in the template material;
[0097] 6) Composite the second instance, which has undergone color migration and / or lighting migration, with the instance background in the template material to output a complete performance clip, i.e., the composite result.
[0098] This embodiment provides a composite shooting method that achieves high-quality compositing of the second instance and its background by removing a first instance, guiding the user to position themselves and complete corresponding actions using outline auxiliary lines, and performing color and lighting migration on the second instance. Specifically, the user captures composite footage guided by the outline auxiliary lines, allowing for flexible adjustments to the shooting angle and actions to ensure high consistency between the composite footage and the template footage, thereby improving the accuracy and efficiency of the compositing. Background completion of the empty areas after removing the first instance is performed based on the image features of the instance background, ensuring visual continuity and consistency of the background and improving the quality of the composite. Adjusting the color of the second instance based on the image features of the instance background makes the transition between the second instance and the instance background more natural. Utilizing a spherical harmonic model for lighting rendering of the second instance enhances the realism and three-dimensionality of the second instance in the composite footage. By flexibly displaying the shooting or composite image on the user's shooting interface, the user can preview the composite effect in real time and flexibly adjust the shooting position and actions, while also meeting the performance requirements of electronic devices.
[0099] Example 3
[0100] Figure 8 This is a flowchart of the synchronization method in Embodiment 3 of this disclosure. Embodiment 3 specifies the case where there are multiple first instances and multiple second instances, based on the above embodiments.
[0101] In this embodiment, the number of first instances is the same as the number of second instances, and there are at least two. The segmentation result of each first instance and its instance identifier can be associated, and the segmentation result of each second instance and its instance identifier can also be associated. First instances and second instances with the same instance identifier are mutually associated. Based on the instance identifier, the relative positional relationship between instances, and / or the outline information of the instances, the first instance associated with each second instance in the template material can be determined, thereby enabling multi-user collaboration.
[0102] like Figure 8 As shown in Embodiment 3 of this disclosure, a method for simultaneous shooting includes the following steps:
[0103] S310. Identify instances in template materials that support duet shooting.
[0104] S320. Based on the user's selection information, determine at least two first instances from the instances that support collaboration.
[0105] S330. Extract the outline information of the first instance in the template material. The template material includes the first instance and the instance background.
[0106] It should be noted that what is extracted here is the outline information of each first instance in the template material, and the part outside each first instance in the template material is the instance background.
[0107] S340. Obtain the collaborative shooting material imported by the user based on the outline information. The collaborative shooting material includes the second instance corresponding to each first instance.
[0108] Optionally, after obtaining the duet footage imported by the user based on the contour information, each second instance can be obtained by segmenting the duet footage into instances.
[0109] It should be noted that the collaborative footage was performed by multiple people, and there are multiple second instances, with each second instance corresponding to one of the first instances.
[0110] S350. Based on the outline information of each first instance in the template material and the outline information of each second instance in the collaborative material, determine the association between each first instance and each second instance.
[0111] Specifically, the instance segmentation algorithm can obtain the outline information of each first instance in the template material, and also the outline information of each second instance in the duet material. Each second instance corresponds one-to-one with a first instance. By comparing the outline information of each first instance and the second instance, it can be determined which first instance each second instance represents, thus establishing the relationship between the first and second instances. For example, if there are two characters in the template material, character A is standing and character B is sitting, then the relationship is: the standing user in the duet material represents character A, and the sitting user represents character B.
[0112] In one embodiment, the relationship between the first and second instances can be determined by identifying which first instance each second instance represents based on their positional relationship. For example, if there are two characters in the template material, with character A on the left and character B on the right, the relationship is that the user on the left in the combined video is representing character A, and the user on the right is representing character B.
[0113] S360. Add each second instance to the area corresponding to the first instance in the template material.
[0114] Optionally, before adding each second instance to the area corresponding to the associated first instance, the method further includes: removing each first instance from the template material; and filling the background in the empty areas after removing each first instance.
[0115] Optionally, after adding each second instance to the area corresponding to the associated first instance, the method further includes: performing color migration and / or lighting migration on each second instance in the composite image result.
[0116] In one embodiment, the collaborative footage can be filmed simultaneously by multiple people or filmed in stages by one or more users. For example, if two people simultaneously perform a template clip, one playing character A and the other playing character B, only one collaborative clip can be filmed, and the two second instances can be added to the template clip respectively. Alternatively, the first person can play character A to film the first collaborative clip, and after the first collaborative clip is filmed, the second person can play character B to film the second collaborative clip. In this case, each collaborative clip includes one second instance, and this second instance corresponds to a first instance in the template clip. It is understood that in the case of filming in stages, a single user can also film in stages and play multiple roles to enhance the flexibility and fun of collaborative filming.
[0117] This embodiment provides a collaborative shooting method that, based on the association between a second instance and a first instance, allows multiple second instances to be added to template materials, enabling multi-user collaborative shooting. This improves the flexibility and fun of collaborative shooting and can meet diverse collaborative shooting needs. Furthermore, users can experience a realistic movie atmosphere, act alongside other subjects, and engage in dialogues across time and space, increasing the diversity and playability of collaborative shooting applications.
[0118] Example 4
[0119] Figure 9 This is a schematic diagram of the timing device in Embodiment 4 of this disclosure. For details not covered in this embodiment, please refer to the above embodiments.
[0120] like Figure 9 As shown, the device includes:
[0121] The outline extraction module 310 is used to extract the outline information of the first instance in the template material, wherein the template material includes the first instance and the instance background;
[0122] The material acquisition module 320 is used to acquire the duet material imported by the user based on the outline information, wherein the duet material includes a second instance corresponding to the first instance;
[0123] The matching module 330 is used to add the second instance to the area corresponding to the first instance in the template material to obtain a matching result, wherein the matching result includes the second instance and the instance background.
[0124] The co-shooting device in this embodiment uses the outline information of the first instance to guide the user to import co-shooting material, thereby improving the consistency between the outline of the instance in the co-shooting material and the template material, and thus realizing the synthesis of the second instance and the background of the template material instance, improving the accuracy of the co-shooting.
[0125] Based on the above, the material acquisition module 320 includes:
[0126] An auxiliary line generation unit is used to generate an auxiliary line for the first instance based on the contour information.
[0127] The auxiliary line display unit is used to display the outline auxiliary line in the user shooting interface to guide the user to shoot the matching material according to the outline auxiliary line.
[0128] Based on the above, the material acquisition module 320 also includes:
[0129] The material determination unit is used to determine the captured image as the combined shooting material if the error between the outline of the second instance and the outline auxiliary line in the user shooting interface is within a set range.
[0130] Based on the above, the co-production module 330 is specifically used for:
[0131] Remove the first instance from the template material and add the second instance to the area corresponding to the first instance in the template material.
[0132] Based on the above, the device also includes:
[0133] The background completion module is used to complete the background of the missing area after removing the first instance based on the image features of the instance background.
[0134] Based on the above, the device also includes:
[0135] The segmentation module is used to perform semantic segmentation or instance segmentation on the collaborative footage imported by the user based on contour information to obtain a second instance.
[0136] Based on the above, the device also includes:
[0137] A color adjustment unit is used to adjust the color of the second instance according to the image features of the instance background.
[0138] Based on the above, the device also includes: a lighting rendering module, used for:
[0139] The spherical harmonic coefficient of the first instance is calculated based on the template material, and the normal direction corresponding to the first instance is estimated.
[0140] The second instance is then rendered with lighting based on the spherical harmonic coefficients and the normal direction.
[0141] Based on the above, the content displayed on the user shooting interface also includes the second instance and the shooting background; or,
[0142] The content displayed on the user shooting interface also includes the second instance and the background of the instance.
[0143] Based on the above, the device also includes:
[0144] The instance recognition module is used to identify instances in the template material that support matching before extracting the outline information of the first instance in the template material;
[0145] The instance determination module is used to determine at least one first instance from the instances that support matching based on the user's selection information.
[0146] Based on the above, the number of the first instance is the same as the number of the second instance, and there are at least two instances;
[0147] The co-production module 330 includes:
[0148] The relationship determination unit is used to determine the association relationship between each first instance and each second instance based on the outline information of each first instance in the template material and the outline information of each second instance in the combined shooting material.
[0149] The instance adding unit is used to add each of the second instances to the area corresponding to the first instance associated with the template material.
[0150] The aforementioned co-shooting device can execute the co-shooting method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the method.
[0151] Example 5
[0152] Figure 10 This is a schematic diagram of the hardware structure of the electronic device in Embodiment 5 of this disclosure. Figure 10 A schematic diagram of the structure of an electronic device 500 suitable for implementing embodiments of the present disclosure is shown. The electronic device 500 in the embodiments of the present disclosure includes, but is not limited to, devices with image processing capabilities such as computers, laptops, servers, tablets, or smartphones. Figure 10 The illustrated electronic device 500 is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0153] like Figure 10 As shown, the electronic device 500 may include one or more processing devices (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The one or more processing devices 501 implement the traffic packet forwarding method provided in this disclosure. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing devices 501, ROM 502, and RAM 503 are interconnected via a bus 505. An input / output (I / O) interface 504 is also connected to the bus 505.
[0154] Typically, the following devices can be connected to I / O interface 504: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc., for storing one or more programs; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 10An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0155] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.
[0156] It should be noted that the computer-readable medium described in this disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0157] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0158] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0159] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: extract the outline information of a first instance in template material, the template material including the first instance and an instance background; obtain a matching material imported by the user based on the outline information, the matching material including a second instance corresponding to the first instance; add the second instance to the area corresponding to the first instance in the template material to obtain a matching result, wherein the matching result includes the second instance and the instance background.
[0160] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0162] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".
[0163] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0164] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0165] According to one or more embodiments of this disclosure, Example 1 provides a method for synchronized shooting, including:
[0166] Extract the outline information of the first instance from the template material, wherein the template material includes the first instance and the instance background;
[0167] Obtain the duet footage imported by the user based on the outline information, wherein the duet footage includes a second instance corresponding to the first instance;
[0168] The second instance is added to the area corresponding to the first instance in the template material to obtain a matching result, wherein the matching result includes the second instance and the instance background.
[0169] Example 2, according to the method described in Example 1, the step of obtaining the matching footage imported by the user based on the contour information includes:
[0170] Generate the contour auxiliary lines of the first instance based on the contour information;
[0171] The outline guide line is displayed in the user shooting interface to guide the user to take the matching shot by following the outline guide line.
[0172] Example 3, based on the method described in Example 1, further includes:
[0173] If the error between the outline of the second instance and the outline auxiliary line is within a set range in the user shooting interface, the captured image will be used as the collaborative shooting material.
[0174] Example 4, according to the method described in Example 1, involves adding the second instance to the area corresponding to the first instance in the template material, including:
[0175] Remove the first instance from the template material and add the second instance to the area corresponding to the first instance in the template material.
[0176] Example 5, based on the method described in Example 1, further includes:
[0177] Background completion is performed on the missing area after removing the first instance based on the image features of the instance background.
[0178] Example 6, based on the method described in Example 1, further includes:
[0179] After obtaining the collaborative footage imported by the user based on the outline information, the process also includes: performing semantic segmentation or instance segmentation on the collaborative footage to obtain a second instance.
[0180] Example 7, based on the method described in Example 1, further includes:
[0181] The colors of the second instance are adjusted based on the image features of the instance background.
[0182] Example 8, according to the method described in Example 1, further includes:
[0183] The spherical harmonic coefficient of the first instance is calculated based on the template material, and the normal direction corresponding to the first instance is estimated.
[0184] The second instance is then rendered with lighting based on the spherical harmonic coefficients and the normal direction.
[0185] Example 9: According to the method described in Example 1, the content displayed on the user shooting interface further includes the second instance and the shooting background; or,
[0186] The content displayed on the user shooting interface also includes the second instance and the background of the instance.
[0187] Example 10, according to the method described in Example 1, further includes, before extracting the outline information of the first instance in the template material:
[0188] Identify instances in the template materials that support duet shooting;
[0189] Based on the user's selection information, at least one first instance is determined from the instances that support collaboration.
[0190] Example 11: According to the method described in Example 1, the number of the first instance is the same as the number of the second instance, and is at least two;
[0191] Adding the second instance to the area corresponding to the first instance in the template material includes:
[0192] Based on the outline information of each first instance in the template material and the outline information of each second instance in the duet material, the association relationship between each first instance and each second instance is determined;
[0193] Add each of the second instances to the area corresponding to the first instance in the template material.
[0194] According to one or more embodiments of this disclosure, Example 12 provides a timing device, including:
[0195] The outline extraction module is used to extract the outline information of the first instance in the template material, wherein the template material includes the first instance and the instance background;
[0196] The material acquisition module is used to acquire the duet material imported by the user based on the outline information, wherein the duet material includes a second instance corresponding to the first instance;
[0197] The "co-op" module is used to add the second instance to the area corresponding to the first instance in the template material to obtain a co-op result, wherein the co-op result includes the second instance and the instance background.
[0198] According to one or more embodiments of this disclosure, Example 13 provides an electronic device comprising:
[0199] One or more processors;
[0200] Storage device for storing one or more programs;
[0201] When the one or more programs are executed by the one or more processors, the one or more processors implement the synchronization method as described in any of Examples 1-11.
[0202] According to one or more embodiments of the present disclosure, Example 14 provides a computer-readable medium having a computer program stored thereon that, when executed by a processor, implements the synchronization method as described in any of Examples 1-11.
[0203] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.
[0204] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0205] Although this topic has been described using language specific to structural features and / or methodological logic, it should be understood that the topic defined in the accompanying example book is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the example book.
Claims
1. A method for synchronized shooting, characterized in that, include: Extract the outline information of the first instance in the template material, the template material including the first instance and the instance background; the outline information is used to describe the position and shape of the first instance, and the outline information includes one or more of the following: text, lines, symbols, simple drawings and auxiliary lines; Obtain the duet footage imported by the user based on the outline information, wherein the duet footage includes a second instance corresponding to the first instance; The second instance is added to the area corresponding to the first instance in the template material to obtain a matching result, wherein the matching result includes the second instance and the instance background; The first instance is an actual object existing in the template material, and the collaborative material is the original image or video obtained by shooting, including the shooting background and the second instance; The number of the first instance is the same as the number of the second instance; Adding the second instance to the area corresponding to the first instance in the template material includes: Based on the outline information of each first instance in the template material and the outline information of each second instance in the duet material, the association relationship between each first instance and each second instance is determined; Add each of the second instances to the area corresponding to the first instance associated with the template material; The step of adding the second instance to the area corresponding to the first instance in the template material includes: Remove the first instance from the template material and add the second instance to the area corresponding to the first instance in the template material.
2. The method according to claim 1, characterized in that, The process of acquiring the collaborative footage imported by the user based on the outline information includes: Generate the contour auxiliary lines of the first instance based on the contour information; The outline guide line is displayed in the user shooting interface to guide the user to take the matching shot by following the outline guide line.
3. The method according to claim 2, characterized in that, Also includes: If the error between the outline of the second instance and the outline auxiliary line is within a set range in the user shooting interface, the captured image will be used as the collaborative shooting material.
4. The method according to claim 1, characterized in that, Also includes: Background completion is performed on the missing area after removing the first instance based on the image features of the instance background.
5. The method according to claim 1, characterized in that, After acquiring the duet footage imported by the user based on the outline information, the process also includes: The collaborative footage is semantically segmented or instance-segmented to obtain the second instance.
6. The method according to claim 1, characterized in that, Also includes: The colors of the second instance are adjusted based on the image features of the instance background.
7. The method according to claim 1, characterized in that, Also includes: The spherical harmonic coefficient of the first instance is calculated based on the template material, and the normal direction corresponding to the first instance is estimated. The second instance is then rendered with lighting based on the spherical harmonic coefficients and the normal direction.
8. The method according to claim 2, characterized in that, The content displayed on the user shooting interface also includes the second instance and the shooting background; or... The content displayed on the user shooting interface also includes the second instance and the background of the instance.
9. The method according to claim 1, characterized in that, Before extracting the outline information of the first instance in the template material, the method further includes: Identify instances in the template materials that support duet shooting; Based on the user's selection information, at least one first instance is determined from the instances that support collaboration.
10. A timing device, characterized in that, include: The outline extraction module is used to extract the outline information of the first instance in the template material, the template material including the first instance and the instance background; the outline information is used to describe the position and shape of the first instance, and the outline information includes one or more of the following: text, lines, symbols, simple drawings and auxiliary lines; The material acquisition module is used to acquire the duet material imported by the user based on the outline information, wherein the duet material includes a second instance corresponding to the first instance; The "co-op" module is used to add the second instance to the area corresponding to the first instance in the template material to obtain a co-op result, wherein the co-op result includes the second instance and the instance background; The first instance is an actual object existing in the template material, and the collaborative material is the original image or video obtained by shooting, including the shooting background and the second instance; The number of the first instance is the same as the number of the second instance; The synchronization module includes: The relationship determination unit is used to determine the association relationship between each first instance and each second instance based on the outline information of each first instance in the template material and the outline information of each second instance in the combined shooting material. An instance adding unit is used to add each of the second instances to the area corresponding to the first instance associated with the template material; Specifically, the synchronization module is used for: Remove the first instance from the template material and add the second instance to the area corresponding to the first instance in the template material.
11. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the synchronization method as described in any one of claims 1-9.
12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the synchronization method as described in any one of claims 1-9.
Citation Information
Patent Citations
Intelligent group photo taking method and device, electronic equipment and storage medium
CN110602396A
