Methods for mounting virtual objects, and devices, equipment, media, and computer programs.

Abstract

This application discloses a method, apparatus, device, and medium for mounting virtual objects. To achieve virtual object mounting, the application acquires the original video and, at the user's selection, acquires the virtual object to be mounted. The virtual object selected by the user is mounted to the target portion of the target object in the original video. Simultaneously, the occluding region of the virtual object is removed, and rendering is performed on the video from which the occluding region has been removed to obtain the target video. That is, a removal process is performed on the portion of the target object that occludes the virtual object, so that the virtual object can fit to the target portion. In other words, with the technology proposed in this application, when mounting a virtual object to the target portion of the target object in the original video, the occluding region of the virtual object can be removed in a timely manner, thereby allowing for a better fit to the target portion and improving the mounting effect.

Description

Technical Field

[0001] This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on February 24, 2023, with an application number of 202310179314.1 and an invention title of "Method and Apparatus, Device, and Medium for Mounting Virtual Objects", the entire content of which is incorporated herein by reference. [Technical Field]

[0002] This application relates to the field of computer technology, specifically to a method and apparatus, device, and 、 medium , and computer programs for mounting virtual objects.

Background Art

[0003] High-quality rendering engines can provide a real and stable visual experience for image processing. However, the effects of current existing mounting tools are not ideal. For example, in the case of a neck mounting tool, the mounting effect is relatively poor (not good), the mounted virtual object does not fit well on the neck, and problems such as the inability to achieve a better adaptive effect according to the movement of the user's head occur.

Summary of the Invention

[0004] In view of this, embodiments of this application provide a method and apparatus, device, and medium for mounting virtual objects to achieve that the virtual object can fit well on the mounting site and improve the reality.

[0005] To achieve the above object, the technical solution according to this application is as follows.

[0006] In a first aspect of this application, a method for mounting a virtual object is provided, and the method includes acquiring an original video, acquiring a virtual object according to the user's selection, Mounting the virtual object to the target portion of the target object in the original video, To remove the shielding region of the aforementioned virtual object, This includes rendering the video with the occluded areas removed to obtain the target video.

[0007] In a second aspect of this application, a virtual object mounting device is provided, the device is An acquisition unit to acquire the original video and, furthermore, to acquire a virtual object based on the user's selection, A processing unit for mounting the virtual object to the target portion of the target object in the original video, and further for removing the occluded area of ​​the virtual object, Includes a rendering unit for rendering the video with the occluded areas removed to obtain the target video.

[0008] In a third aspect of this application, an electronic device is provided, the device including a processor and memory, The memory is used to store instructions or computer programs. The processor is used to execute the instructions or computer program in the memory to cause the electronic device to perform the method according to the first embodiment.

[0009] A fourth aspect of this application provides a computer-readable storage medium in which instructions are stored, and when the instructions are executed on a device, the device is caused to perform the method according to the first aspect.

[0010] A fifth embodiment of this application provides a computer program product which includes a computer program / instruction, and when the computer program / instruction is executed by a processor, it realizes the method described in the first embodiment.

[0011] As can be seen from this, this application has the following beneficial effects.

[0012] In this application, in order to mount a virtual object, the original video is acquired, and a virtual object to be mounted is acquired based on the user's selection. The virtual object selected by the user is mounted to the target portion of the target object in the original video. At the same time, the occluding region of the virtual object is removed, and rendering is performed on the video from which the occluding region has been removed to obtain the target video. That is, a removal process is performed on the portion of the target object that occludes the virtual object so that the virtual object can fit to the target portion. In other words, with the technology proposed in this application, when mounting a virtual object to the target portion of the target object in the original video, the occluding region of the virtual object can be removed in a timely manner, thereby allowing for a better fit to the target portion and improving the mounting effect. [Brief explanation of the drawing]

[0013] Hereinafter, in order to more clearly explain the embodiments of this disclosure or the technical concepts in the prior art, the drawings necessary for describing the embodiments or the prior art will be briefly described. The drawings in the following description are only a few embodiments of those described in this application, and it will be apparent to those skilled in the art that other drawings can be obtained from these drawings without any creative effort.

[0014] [Figure 1] This is a flowchart of the method for mounting a virtual object according to an embodiment of this application. [Figure 2] This is a schematic diagram of the structure of a virtual object according to an embodiment of this application. [Figure 3a] This is a schematic diagram of a shielding plane according to an embodiment of this application. [Figure 3b] This is a schematic diagram of another shielding plane according to an embodiment of the present application. [Figure 4a]Schematic diagram of collecting depth information at different positions according to an embodiment of the present application. [Figure 4b] Schematic diagram of shadow rendering according to an embodiment of the present application. [Figure 5] Schematic diagram of the structure of a makeup rendering device according to an embodiment of the present application. [Figure 6] Schematic diagram of the structure of an electronic device according to an embodiment of the present application.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, for those skilled in the art to better understand the solution of the present application, while referring to the drawings in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. It is obvious that the described embodiments are only a part of the embodiments of the present application and not all of them. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of the present application.

[0016] In current existing mounting tools, the movement of the mounting tool is mainly driven by target parts, such as skeleton points on the neck, wrist, ankle, etc. However, when there are few acquirable skeleton points, the mounting tool cannot fit the target part, and the mounting effect becomes poor.

[0017] In view of this, the present application provides a method for mounting virtual objects. When attempting to mount a virtual object to be mounted on the target part of the target object in the original video, the virtual object to be mounted is selected, and the virtual object is mounted on the target part. At the same time, the shielding area of the current virtual object is determined, the shielding area is removed, and rendering is performed on the video with the shielding area removed to obtain the target video. That is, in the process of mounting the virtual object on the target part, the shielding area of the virtual object can be determined and removed in real time, thereby making the virtual object fit better on the target part and improving the mounting effect.

[0018] Before using the technical solutions of each embodiment in this disclosure, it can be understood that it is necessary to inform the user about the types, scope of use, usage scenarios, etc. of relevant personal information in an appropriate manner in accordance with relevant laws and regulations, and obtain the approval of the user.

[0019] For example, when responding to a subjective request from a user, prompt information is sent to the user to explicitly indicate that the operation requested for execution needs to obtain and utilize the user's personal information. Thereby, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application, a server, or a storage medium that executes the operation of the technical solution of this disclosure according to the prompt information.

[0020] As an optional but not limited implementation form, the form of sending prompt information to the user in response to a subjective request from the user may, for example, be in the form of a pop-up window, and the prompt information may be displayed in text form in the pop-up window. Also, the pop-up window may be equipped with a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0021] It can be understood that the above-mentioned notification and user approval acquisition procedures are only exemplary and do not limit the implementation forms of this disclosure, and other forms that meet the relevant laws and regulations can also be applied to the implementation forms of this disclosure.

[0022] To facilitate understanding of the technical solutions provided by the embodiments of this application, the following will be described with reference to the drawings.

[0023] Referring to Figure 1, the figure is a flowchart of a method for mounting a virtual object according to an embodiment of the present application, the method which may be performed by a client, which may be installed on an electronic device. Here, the electronic device may include devices with communication capabilities, such as mobile phones, tablet computers, laptop computers, desktop computers, in-vehicle terminals, wearable electronic devices, multifunction printers, and smart home devices, or it may be a device simulated by a virtual machine or simulator. As shown in Figure 1, the method may include the following steps.

[0024] S101: Get the original video.

[0025] In this embodiment, when it is necessary to generate a video with a mounting effect, first, the original video is obtained, and the original video contains a target object, which contains a mounting part, i.e., a target part, that corresponds to the virtual object to be mounted. Here, if the mounting part corresponding to the virtual object to be mounted is the neck, the target part is the neck, and if the mounting part corresponding to the virtual object to be mounted is the head, wrist, ankle, or limb, the target parts are the head, wrist, ankle, or limb. For ease of understanding, the following explanation will use the case where the target part is the neck as an example.

[0026] Here, obtaining the original video may include the user recording the video in real time via the client's operating interface, or selecting a pre-recorded video.

[0027] S102: Retrieve a virtual object based on user selection.

[0028] In this embodiment, when a user opens the client interface, multiple mountable virtual objects can be displayed to the user, and the user can select the virtual object they wish to mount according to their needs. For example, the virtual objects may be virtual scarves, silk scarves, wristbands, etc.

[0029] In this embodiment, the execution order of S101 and S102 is not restricted. The user can first upload or record the original video via the client and then select a virtual object, or the user can first select a virtual object via the client and then upload or record the original video.

[0030] S103: Mount the virtual object to the target location of the target object in the original video.

[0031] After obtaining the virtual object and the original video, the virtual object is mounted to the target location of the target object in the original video.

[0032] In some embodiments, a virtual object can be mounted to the target location of a target object in the original video by obtaining the target location and reference location of the target object in the original video, controlling the movement of a virtual object based on the reference location and target location, and mounting the virtual object to the target location based on the movement information of the virtual object. That is, when mounting a virtual object to a target location, first, the movement information of the target location and reference location of the target object are obtained, and then the movement of the virtual object is controlled based on the movement information of the target location and reference location. Finally, the virtual object is controlled to be mounted to the target location based on the movement information of the virtual object.

[0033] In other words, to enable the virtual object to more accurately track and fit the target area when the target area moves, this embodiment combines the motion information of the target area with the motion information of the reference area to control the virtual object to perform matching movements, and further mounts the virtual object to the target area based on the motion information of the virtual object, thereby achieving a fitted mount of the virtual object to the target area.

[0034] For example, if the virtual object is a scarf, the target area is the neck, the reference area is the head, and the neck moves, the movement information of the neck and the movement information of the head are extracted, the movement information of the scarf is determined based on the movement information of both, and then, based on the movement information of the scarf, the system controls the mounting of the scarf to the neck, thereby allowing the scarf to fit better around the neck even while the neck is moving.

[0035] In some embodiments, controlling the movement of a virtual object based on a reference part and a target part includes obtaining first rotation information corresponding to the reference part and second rotation information corresponding to the target part, and controlling the movement of the virtual object by controlling the movement of multiple components of the virtual object based on the first and second rotation information. Here, the first rotation information may include the rotation angle and rotation direction of the reference part, and the second rotation information may include the rotation angle and rotation direction of the target part. The first rotation information may indicate the rotation information of the entire reference part, or the rotation information of a key point or position on the reference part. For example, if the reference part is the head, the first rotation information may be the rotation information corresponding to the position of the nose. The second rotation information may indicate the rotation information of the entire target part, or the rotation information corresponding to a key point or position on the target part. For example, if the target part is the neck, the second rotation information may be the rotation information corresponding to the position of the throat.

[0036] Here, the first rotation information and the second rotation information may both be quaternions, and may both include a rotation axis and a rotation angle. For example, the rotation information can be represented as (x, y, z, w), where (x, y, z) represents the rotation axis and w represents the rotation angle.

[0037] To facilitate control of a virtual object so that it follows the movement of a target part, the virtual object may be divided into multiple components, and the movement of each different component may be controlled. Specifically, the movement of the first component may be controlled based on first rotation information, and the movement of the second component may be controlled based on second rotation information. For example, in the muffler model diagram shown in Figure 2, the configuration of the muffler model is similar to the configuration of the neck skeleton, and the skeleton of the muffler model includes a first component and a second component, with the first and second components representing the upper and lower regions of the neck, respectively. The movement of the first component is controlled by a first rotation parameter corresponding to the head, and the movement of the second component is controlled by a second rotation parameter corresponding to the neck.

[0038] In this embodiment, when the second component is mounted, the mounting position is the target area, so its rotation can be controlled by directly using the second rotation parameter corresponding to the target area. If the first component and the reference area do not perfectly match, for example, if the mounting area corresponding to the first component in Figure 2 is located below the head, and there is a rotational difference between the two, and it should be in an intermediate state between the reference area and the target area, then in order to control the rotation of the first component more accurately, the first rotation information can be corrected first, and then the first component can be controlled to rotate using the corrected rotation information.

[0039] Here, the correction process may specifically include reducing or expanding the first rotation angle in the first rotation information to obtain a third rotation angle, and controlling the first component to rotate based on the second and third rotation angles. Here, the second rotation angle is the rotation angle included in the second rotation information. Whether to reduce or expand the first rotation angle generally needs to be determined based on the position between the mounting part and the reference part corresponding to the first component. If the mounting part corresponding to the first component is located below the reference part, the first rotation angle is reduced; if the mounting part corresponding to the first component is located above the reference part, the first rotation angle is expanded.

[0040] Here, controlling the rotation of the first component using the second and third rotation angles includes multiplying the second and third rotation angles to obtain a fourth rotation angle, and then controlling the rotation of the first component based on the rotation axis and the fourth rotation angle. In other words, first, the first component is controlled to rotate by the second rotation angle, and then the first component is controlled to rotate by the fourth rotation angle, thereby causing the first component to rotate tilted toward the reference part based on the second component.

[0041] In some embodiments, to more precisely control the movement of a virtual object, the virtual object may be divided into more components, and the movement of each different component may be controlled. Specifically, a third component can be obtained based on a first and a second component, and the movement of the third component can be controlled based on first and second rotation information. For example, a third component can be obtained by dividing it between the first and second components, weight information corresponding to the third component can be determined based on the distance between the center point of the third component and the center point of the first component, and the distance between the center point of the second component, and further, third rotation information corresponding to controlling the movement of the third component can be determined using the first rotation information, second rotation information, and weight information.

[0042] For example, if the weight of the third component relative to the first component is a, then the weight of the second component is (1-a), and the rotation information of the third component = a * the rotation information of the first component + (1-a) * the rotation information of the second component. The movement of the third component is controlled based on this rotation information.

[0043] S104: Removes the occluding area of ​​a virtual object.

[0044] In this embodiment, after mounting a virtual object to a target area, the target area or reference area may obscure a portion of the virtual object. To provide a better mounting effect, it is necessary to remove the obscure area on the virtual object.

[0045] Specifically, the process involves acquiring depth information for a virtual object, setting an occlusion plane based on that information, obtaining the shape of the occlusion plane by detecting a target object, and removing the occlusion region of the virtual object based on the occlusion plane. In other words, the depth information of the occlusion plane is determined by acquiring the depth information of the virtual object, and then the occlusion region of the virtual object is removed using this occlusion plane. Here, the specific shape of the occlusion plane is determined by the target object. For example, if the virtual object is a scarf, clipping can be performed on the occlusion plane based on the head and neck of the target object, thereby allowing the occlusion plane to be more accurately occluded. For example, as shown in Figure 3a, the occlusion region of the virtual object is removed using the occlusion plane, and the shape of the occlusion plane is obtained by clipping based on the user's head and neck in the original video.

[0046] In some application scenarios, the target area moves, the virtual object moves accordingly, and the occlusion region corresponding to the virtual object may also change. To ensure that the occlusion region of the virtual object is accurately removed, the occlusion plane is reset after the virtual object moves, based on the depth information of the virtual object after the movement. For example, as shown in Figure 3b, if the user's neck rotates to the left, the occlusion plane is reset, and the occlusion region of the virtual object is removed.

[0047] Specifically, setting a occlusion plane based on the depth information of a virtual object includes obtaining first and second depth information based on the depth information of the virtual object, and determining the depth information of the occlusion plane based on the first and second depth information. Here, the first and second depth information represent depth information at different positions in the virtual object, and the depth information of the occlusion plane is determined based on the depth information at two different positions. For example, as shown in Figure 4a, the first depth information is the depth information at position 1 in the virtual object, and the second depth information is the depth information at position 2 in the virtual object. Specifically, the average value of the first and second depth information is used as the depth information of the occlusion plane, or the weighted value of the first and second depth information based on pre-assigned weights is used as the depth information of the occlusion plane.

[0048] S105: Render the video with the occluded areas removed to obtain the target video.

[0049] After removing the occluded area of ​​the virtual object, rendering is performed on the video to obtain a target video, in which the virtual object in the target video can be precisely fitted to the target area.

[0050] Here, rendering to the video includes rendering the shadows that occur in the video after the virtual object is mounted on the target area. Specifically, after the virtual object is mounted on the target area, corresponding ambient light occlusion information is obtained, and using this ambient light occlusion information, rendering is performed on the video from which the occluded area has been removed to obtain the target video. Here, the ambient light occlusion information includes ambient light occlusion information at the contact point between the virtual object and the target area, and ambient light occlusion information at the contact point between the virtual object and other areas (e.g., reference areas). For example, as shown in Figure 4b, by rendering the shadow at the occluded position between the scarf and the human body, a better transition of light and shadow exists at the contact point between the human body and the scarf.

[0051] As can be seen below, in order to mount a virtual object, the original video is acquired, and the virtual object to be mounted is acquired based on the user's selection. The virtual object selected by the user is mounted to the target portion of the target object in the original video. At the same time, the occluding region of the virtual object is removed, and rendering is performed on the video from which the occluding region has been removed to obtain the target video. In other words, the portion of the target object that occludes the virtual object is removed so that the virtual object can fit to the target portion. That is, with the technology proposed in this application, when mounting a virtual object to the target portion of the target object in the original video, the occluding region of the virtual object can be removed in a timely manner, thereby allowing for a better fit to the target portion and improving the mounting effect.

[0052] Based on the above-described example, the embodiment of this application provides a virtual object mounting device and electronic equipment, which will be described below with reference to the drawings.

[0053] Referring to Figure 5, the figure is a structural diagram of a virtual object mounting device according to an embodiment of the present application, and as shown in Figure 5, the device includes an acquisition unit 501, a processing unit 502, and a rendering unit 503.

[0054] Here, the acquisition unit 501 is used to acquire the original video. The acquisition unit 501 is further used to acquire virtual objects at the user's selection. The processing unit 502 is used to mount the virtual object to the target portion of the target object in the original video. The processing unit 502 is further used to remove the shielding region of the virtual object, The rendering unit 503 is used to render the video from which the occluded area has been removed in order to obtain the target video.

[0055] In some embodiments, the processing unit 502 is specifically used to obtain the target and reference parts of the target object in the original video, to control the movement of the virtual object based on the reference and target parts, and to mount the virtual object to the target parts based on the movement information of the virtual object.

[0056] In some embodiments, the processing unit 502 is used to control the movement of the virtual object by specifically acquiring first rotation information corresponding to the reference portion, acquiring second rotation information corresponding to the target portion, and controlling the movement of multiple components of the virtual object based on the first and second rotation information.

[0057] In some embodiments, the processing unit 502 is specifically used to control the movement of a first component based on the first rotation information, to control the movement of a second component based on the second rotation information, to acquire a third component based on the first and second components, and to control the movement of the third component based on the first and second rotation information.

[0058] In some embodiments, the processing unit 502 is specifically used to acquire depth information of the virtual object, set a shielding plane based on the depth information of the virtual object, and remove the shielding region of the virtual object based on the shielding plane, the shape of the shielding plane being obtained by detecting the target object.

[0059] In some embodiments, the processing unit 502 is specifically used to acquire first depth information and second depth information based on the depth information of the virtual object, and to determine the depth information of the shielding plane based on the first depth information and the second depth information.

[0060] In some embodiments, the rendering unit 503 is specifically used to obtain ambient light occlusion information after the virtual object has been mounted on the target area, and to render the video from which the occlusion area has been removed using the ambient light occlusion information to obtain the target video.

[0061] In some embodiments, the ambient light shielding information includes ambient light shielding information for the contact location between the virtual object and the target area, and ambient light shielding information for the contact location between the virtual object and other areas.

[0062] For specific implementations of each unit in this embodiment, refer to the relevant descriptions in the embodiments of the above method. In the embodiments of this application, the unit classifications are illustrative and merely logical functional classifications; other classification methods may be used in actual implementation. Each functional unit in the embodiments of this application may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. For example, in the above embodiment, the processing unit and the transmission unit may be the same unit or different units. The integrated unit may be implemented in hardware form or in the form of a software functional unit.

[0063] Referring to Figure 6, a schematic diagram of the configuration of an electronic device 600 suitable for realizing an embodiment of the present disclosure is shown. The terminal devices in the embodiment of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., car navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The electronic device shown in Figure 6 is merely an example and should not limit the functions or scope of use of the embodiment of the present disclosure.

[0064] As shown in Figure 6, the electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601 capable of performing various appropriate operations and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data necessary for the operation of the electronic device 600. The processing unit 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0065] Typically, input devices 606, including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, and gyroscope; output devices 607, including, for example, a liquid crystal display (LCD), speaker, and vibrator; storage devices 608, including, for example, magnetic tape and hard disk; and communication devices 609 may be connected to the I / O interface 605. The communication devices 609 may allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data. Figure 6 shows an electronic device 600 with various devices, but it should be understood that it is not necessary to implement or include all of the devices shown. More or fewer devices may be implemented or included instead.

[0066] According to embodiments of the present disclosure, the processes described above with reference to the flowchart may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product which includes a computer program mounted on a non-temporary computer-readable medium, which includes program code for performing the method shown in the flowchart. In such embodiments, the computer program may be downloaded and installed from a network by a communication device 609, installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by a processing device 601, the above-described functions, limited to the methods of embodiments of the present disclosure, are performed.

[0067] The electronic devices provided in the embodiments of this disclosure belong to the same inventive concept as the methods provided in the above embodiments, and for technical details not described in detail in these embodiments, refer to the above embodiments, and these embodiments have the same beneficial effects as the above embodiments.

[0068] Embodiments of this disclosure provide a computer storage medium containing a computer program that, when executed by a processor, implements the method provided by the embodiments described above.

[0069] The computer-readable media described above in this disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of both. A computer-readable storage medium may, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, ROM, Erasable Programmable Read Only Memory (EPROM), or flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical memory device, a magnetic memory device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. On the other hand, in this disclosure, a computer-readable signal medium may include a data signal propagated in the baseband or as part of a carrier wave, on which computer-readable program code is carried. Such propagated data signals may take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. Furthermore, the computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium on which a program for use by or in combination with an instruction execution system, apparatus, or device can be transmitted, propagated, or transmitted.Program code contained in a computer-readable medium may be transmitted on any suitable medium, including, but not limited to, wires, optical cables, radio frequencies (RF), or any suitable combination thereof.

[0070] In some embodiments, clients and servers may communicate using any network protocol currently known or to be developed in the future, such as the Hypertext Transfer Protocol (HTTP), and may connect with each other to digital data communications (e.g., communication networks) in any form or medium. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the World Wide Web (e.g., the Internet), end-to-end networks (e.g., ad hoc end-to-end networks), and any network currently known or to be developed in the future.

[0071] The computer-readable medium described above may be included in the electronic device described above, or it may be a separate entity not incorporated into the electronic device.

[0072] The computer-readable medium contains one or more programs, and when these one or more programs are executed by the electronic device, the electronic device is able to perform the method described above.

[0073] Computer program code for performing the operations disclosed herein may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code may be fully executed on a first user computer, partially executed on a first user computer, executed as a standalone software package, partially executed on a first user computer and partially executed on a remote computer, or fully executed on a remote computer or server. Where a remote computer is involved, the remote computer may be connected to the first user computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (for example, connected via the Internet through an Internet service provider).

[0074] The flowcharts and block diagrams in the drawings illustrate the architectures, functions, and operations that can be implemented according to the systems, methods, and computer program products relating to each type of embodiment of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of code containing one or more executable instructions for implementing a given logic function. Note that in some implementations as replacements, the functions represented within a block may occur in a different order than shown in the drawings. For example, two blocks shown consecutively may actually be executed substantially in parallel, or in reverse order depending on the functions involved. Note that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, may be implemented in a dedicated hardware-based system that performs a given function or operation, or in a combination of dedicated hardware and computer instructions.

[0075] The units referred to in the descriptions of the embodiments of this disclosure may be implemented in software or in hardware. Herein, the names of the units / modules are not limited in any case to the units themselves.

[0076] The functions described above in this specification may be performed by at least partially one or more hardware logic components. For example, exemplary types of hardware logic components that may be used include, but are not limited to, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific general-purpose products (ASSPs), systems on chips (SOCs), and complex programmable logic devices (CPLDs).

[0077] In the context of this disclosure, machine-readable media may be tangible media that contain or can store programs used by or in combination with instruction execution systems, devices, or equipment. Machine-readable media may be machine-readable signal media or machine-readable storage media. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any appropriate combination of the above. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any appropriate combination of the above.

[0078] In this specification, each embodiment is described sequentially, and the emphasis in each embodiment is on the differences from other embodiments. Similar or identical parts between embodiments should be referenced to one another. For systems or apparatus disclosed in the embodiments, the description is simplified to correspond to the method disclosed in the embodiments; relevant sections should be referred to in the description of the method.

[0079] It should be understood that in this disclosure, “at least one” means one or more, and “multiple” means two or more. “And / or” is used to describe the relationship between related objects and indicates that there may be three possible relationships. For example, “A and / or B” may represent three cases: A only exists, B only exists, and A and B exist simultaneously, where A and B may be singular or plural. The letter “ / ” generally indicates that the related objects before and after are in an “or” relationship. “At least one of the following” or similar expressions means any combination of these objects, including any combination of single or multiple objects. For example, at least one of a, b, or c may represent a, b, c, “a and b,” “a and c,” “b and c,” or “a and b and c,” where a, b, and c may be singular or plural.

[0080] In this specification, relational terms such as "First" and "Second" are used merely to distinguish one entity or operation from another, and do not require or imply that such an actual relationship or order exists between these entities or operations. The terms "include," "incorporate," or any other variations thereof are intentionally intended to cover the non-exclusive "include," thereby including not only those elements but also other elements not explicitly listed, or elements specific to such a process, method, article, or device. Unless otherwise specified, the phrase "includes one..." does not preclude other identical elements from being present in a process, method, article, or device that includes such element.

[0081] The steps of the methods or algorithms described with reference to the embodiments disclosed herein may be carried out directly by hardware, by a software module executed by a processor, or by a combination of both. The software module may reside in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, portable magnetic disks, CD-ROMs, or any other form of storage medium well known in the art.

[0082] The above description of the disclosed embodiments enables those skilled in the art to practice or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be realized in other embodiments without departing from the spirit or scope of the present application. Thus, the present application is not limited to these embodiments shown herein, but conforms to the broadest scope that is consistent with the principles and novel features disclosed herein.

Claims

1. To obtain the original video, The user selects to obtain a virtual object, Mounting the virtual object to the target portion of the target object in the original video, To remove the shielding region of the aforementioned virtual object, A method for mounting a virtual object, including rendering to a video with occluded areas removed to obtain a target video.

2. Mounting the virtual object to the target portion of the target object in the original video is: Obtaining the target and reference parts of the target object in the original video, Controlling the movement of the virtual object based on the aforementioned reference portion and the aforementioned target portion, The method according to claim 1, comprising mounting the virtual object to the target portion based on the movement information of the virtual object.

3. Controlling the movement of the virtual object based on the aforementioned reference portion and the aforementioned target portion is, To obtain first rotation information corresponding to the aforementioned reference part, To acquire second rotation information corresponding to the target portion, The method according to claim 2, further comprising controlling the movement of a virtual object by controlling the movement of a plurality of components of the virtual object based on the first rotation information and the second rotation information.

4. Controlling the movement of multiple components of the virtual object based on the first rotation information and the second rotation information is: Controlling the movement of the first component based on the first rotation information, Controlling the movement of the second component based on the second rotation information, Obtaining a third component based on the first component and the second component, The method according to claim 3, comprising controlling the movement of the third component based on the first rotation information and the second rotation information.

5. Removing the shielding region of the virtual object means To obtain the depth information of the aforementioned virtual object, Setting a shielding plane based on the depth information of the virtual object, This includes removing the shielding region of the virtual object based on the shielding plane, The method according to claim 1, wherein the shape of the shielding plane is obtained by detecting the target object.

6. Setting a shielding plane based on the depth information of the virtual object is, Based on the depth information of the virtual object, first depth information and second depth information are obtained, The method according to claim 5, further comprising determining the depth information of the shielding plane based on the first depth information and the second depth information.

7. Rendering the video with the occluded areas removed to obtain the target video is possible. After the virtual object is mounted on the target area, the corresponding ambient light shielding information is acquired, The method according to claim 1, comprising rendering a video from which the occluded region has been removed using the ambient light occluding information to obtain a target video.

8. The method according to claim 7, wherein the ambient light shielding information includes ambient light shielding information for the contact position between the virtual object and the target part, and ambient light shielding information for the contact position between the virtual object and other parts.

9. An acquisition unit to acquire the original video and, furthermore, to acquire a virtual object based on the user's selection, A processing unit for mounting the virtual object to the target portion of the target object in the original video, and further for removing the occluded area of ​​the virtual object, A virtual object mounting device, including a rendering unit for rendering to a video from which occluded areas have been removed to obtain a target video.

10. It is an electronic device, It includes a processor and memory, The memory is used to store instructions or computer programs. The processor is used to execute the instructions or computer program in the memory to cause the electronic device to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium wherein a command is stored in the computer-readable storage medium, and when the command is executed on a device, the device is caused to execute the method according to any one of claims 1 to 8.

Images