Method and device for real-time rendering of real person shadow through XR and electronic equipment
By obtaining the human bone information of the target character and building virtual characters, and using the rendering engine to render shadow images in real time, the problem of difficult to generate realistic character shadows in the existing technology is solved, and efficient and low-cost virtual shooting effects are achieved.
Patent Information
- Application Number
- CN202311315229.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-11
- Publication Date
- 2025-04-11
AI Technical Summary
The existing XR virtual shooting technology is difficult to generate realistic character shadows in real time, and traditional motion capture technology is cumbersome and expensive.
By obtaining the human skeleton information of the target character, building virtual characters, and using the rendering engine to render the shadow screen in real time and display it on the LED display.
Real-time rendering of realistic character shadows is realized, reducing costs, simplifying operation processes, and saving operation time.
Smart Images
Figure CN119810386A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of virtual photography, and in particular to a method, device and electronic device for real-time rendering of real-person shadows in XR. Background Art
[0002] With the continuous development of science and technology, XR virtual shooting technology has made significant breakthroughs. Among them, immersive LED display is one of the main devices to realize this technology.
[0003] At present, there is a problem with XR virtual shooting, which is how to generate realistic human shadows in real time. When a person stands on an LED floor tile screen, the floor tile screen glows, and the person in the photo lacks a human shadow. Human shadows play an important role in virtual shooting scenes, and can enhance the realism and immersion of the scene. Traditional virtual shooting technology requires real people to wear motion capture clothing before human skeleton capture and human shadow rendering can be performed. Motion capture clothing is not only expensive, but the motion capture process is also cumbersome. Summary of the invention
[0004] Based on this, it is necessary to provide a method, device and electronic device for XR real-time rendering of real-person shadows to address the above technical problems, which can not only save costs, but also simplify the operation process and save operation time.
[0005] In a first aspect, an embodiment of the present application provides a method for XR real-time rendering of real-person shadows, including:
[0006] Obtaining human skeleton information of the first target person;
[0007] Constructing a second target person based on the human skeleton information of the first target person;
[0008] Rendering a shadow image of the second target person;
[0009] The shadow image of the second target person is displayed on the LED display screen as the shadow of the first target person.
[0010] In some embodiments, obtaining the human skeleton information of the first target person includes:
[0011] The human skeleton information of the first target person is obtained by using a depth camera.
[0012] In some embodiments, constructing the second target person based on the human skeleton information of the first target person includes:
[0013] Constructing a second target person in a rendering engine based on the human skeleton information of the first target person;
[0014] The second target person is driven.
[0015] In some embodiments, the rendering of the shadow image of the second target person includes:
[0016] A rendering engine is used to render a shadow image of the second target person.
[0017] In some embodiments, after rendering the shadow image of the second target person using a rendering engine, the method further includes:
[0018] The display of the second target person in the rendering engine is turned off, and the shadow image of the second target person is retained.
[0019] In a second aspect, an embodiment of the present application further provides an XR real-time rendering device for real-person shadows, including:
[0020] An acquisition module, used to acquire human skeleton information of a first target person;
[0021] A driving module, used for constructing a second target person based on the human skeleton information of the first target person;
[0022] A rendering module, used for rendering a shadow image of the second target person;
[0023] The display module is used to display the shadow image of the second target person on the LED display screen as the shadow of the first target person.
[0024] In a third aspect, an embodiment of the present application further provides an electronic device, comprising:
[0025] at least one processor; and,
[0026] a memory communicatively connected to the at least one processor; wherein,
[0027] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect above.
[0028] In a fourth aspect, an embodiment of the present application further provides an XR real-time rendering system for real-person shadows, comprising the electronic device, a depth camera and an LED display screen according to the third aspect, wherein the electronic device is connected to the depth camera and the LED display screen respectively,
[0029] The depth camera is used to obtain human skeleton information, the electronic device is used to construct a second target person based on the human skeleton information and render a shadow image of the second target person, and the LED display screen is used to display the shadow image of the second target person.
[0030] In some embodiments, the system further comprises a camera, and the depth camera is mounted on the camera.
[0031] In a fifth aspect, an embodiment of the present application further provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the processor executes the method described in the first aspect.
[0032] Compared with the prior art, the beneficial effect of the present application is: different from the prior art, the XR real-time rendering method of real-person shadows provided in the embodiment of the present application obtains the human skeleton information of the first target person; then constructs the second target person based on the human skeleton information of the first target person; then renders the shadow picture of the second target person; finally, displays the shadow picture of the second target person on the LED display screen as the shadow of the first target person, which not only saves costs, but also simplifies the operation process and saves operation time. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0034] Figure 1 This is a schematic diagram of an application scenario of a method for real-time rendering of real-person shadows by XR in one embodiment of the present application;
[0035] Figure 2 This is a flowchart of a method for real-time rendering of real-person shadows by XR in one embodiment of the present application;
[0036] Figure 3 It is a schematic diagram of the structure of an apparatus for real-time rendering of real-person shadows by XR in one embodiment of the present application;
[0037] Figure 4 It is a schematic diagram of the hardware structure of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION
[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0039] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.
[0040] The XR real-time rendering method of real-person shadows provided in the embodiment of the present application is applicable to Figure 1 In the application scenario shown, the application scenario is an XR system for real-time rendering of real-person shadows. The XR system 100 for real-time rendering of real-person shadows includes an electronic device 10, a depth camera 20, and an LED display screen 30. The electronic device 10 is connected to the depth camera 20 and the LED display screen 30, respectively. Further, the XR system 100 for real-time rendering of real-person shadows also includes a camera 40, and the depth camera 20 is installed on the camera 40. The depth camera 20 is used to obtain human skeleton information, and the electronic device 10 is used to construct a second target person based on the human skeleton information, and render the shadow picture of the second target person, and the LED display screen 30 is used to display the shadow picture of the second target person. The camera 40 is used for virtual shooting.
[0041] It should be noted that the method provided in the embodiment of the present application can be further extended to other suitable application environments, and is not limited to Figure 1 In the actual application process, the application environment may also include more LED display screens.
[0042] like Figure 2 As shown, the embodiment of the present application also provides a method for XR real-time rendering of real-person shadows, the method comprising:
[0043] Step 210, obtaining human skeleton information of the first target person.
[0044] In the embodiment of the present application, when the first target person stands on the LED display screen, the depth camera tracks the first target person, collects the human skeleton information of the first target person, and then sends the collected human skeleton information to the electronic device. The first target person is a real person in the physical world.
[0045] Step 220: construct a second target person based on the human skeleton information of the first target person.
[0046] In an embodiment of the present application, the second target person is a virtual person. After the electronic device obtains the human skeleton information of the first target person, the second target person is constructed based on the human skeleton information of the first target person. Specifically, a rendering engine is pre-installed in the electronic device, the electronic device calls and obtains the human skeleton information of the first target person from the rendering engine, and then constructs the second target person in the rendering engine based on the human skeleton information of the first target person, and drives the second target person in the rendering engine.
[0047] Since the second target person in the rendering engine is rendered based on the human skeleton information of the first target person, when the first target person performs a corresponding action, the second target person in the rendering engine also performs a corresponding action.
[0048] Step 230: Rendering a shadow image of the second target person.
[0049] The shadow rendering option in the rendering engine is turned on, and the shadow image of the second target person is rendered by the rendering engine. After the shadow image of the second target person is rendered, the display of the second target person in the rendering engine is turned off, and the shadow image of the second target person is retained.
[0050] Step 240: display the shadow image of the second target person on the LED display screen as the shadow of the first target person.
[0051] After the rendering engine renders the shadow image of the second target person, the shadow image of the second target person is displayed on the LED display screen as the shadow of the first target person.
[0052] In an embodiment of the present application, the human skeleton information of the first target person is obtained through a depth camera, and then the second target person is constructed based on the human skeleton information of the first target person, and then the shadow image of the second target person is rendered. Finally, the shadow image of the second target person is displayed on an LED display screen as the shadow of the first target person. Using a depth camera for human body tracking can not only save costs, but also simplify the operation process and save operation time.
[0053] Accordingly, the embodiment of the present application also provides an XR real-time rendering device 300 for real-person shadows, such as Figure 3 As shown, the device 300 includes:
[0054] An acquisition module 310 is used to acquire human skeleton information of a first target person;
[0055] A driving module 320, configured to construct a second target person based on the human skeleton information of the first target person;
[0056] A rendering module 330, configured to render a shadow image of the second target person;
[0057] The display module 340 is used to display the shadow image of the second target person on the LED display screen as the shadow of the first target person.
[0058] The XR real-time rendering device for real-person shadows provided in the embodiment of the present application obtains the human skeleton information of a first target person, then constructs a second target person based on the human skeleton information of the first target person through a driving module, then renders the shadow image of the second target person through a rendering module, and finally displays the shadow image of the second target person on an LED display screen through a display module as the shadow of the first target person, which can not only save costs but also simplify the operation process and save operation time.
[0059] Optionally, in other embodiments of the device, the acquisition module 310 is specifically used for:
[0060] The human skeleton information of the first target person is obtained by using a depth camera.
[0061] Optionally, in other embodiments of the device, the driving module 320 is specifically used for:
[0062] Constructing a second target person in a rendering engine based on the human skeleton information of the first target person;
[0063] The second target person is driven.
[0064] Optionally, in other embodiments of the device, the rendering module 330 is specifically used for:
[0065] A rendering engine is used to render a shadow image of the second target person.
[0066] The display of the second target person in the rendering engine is turned off, and the shadow image of the second target person is retained.
[0067] It should be noted that the above-mentioned XR real-time rendering of real-person shadows device can execute the XR real-time rendering of real-person shadows method provided in the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not fully described in the embodiment of the XR real-time rendering of real-person shadows device, refer to the XR real-time rendering of real-person shadows method provided in the embodiment of the present application.
[0068] Figure 4 is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application, such as Figure 4 As shown, the electronic device 400 includes:
[0069] One or more processors 401 and memory 402, Figure 4 A processor is taken as an example.
[0070] The processor 401 and the memory 402 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.
[0071] The memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the method for real-time rendering of real-person shadows by XR in the embodiment of the present application. The processor 401 executes various functional applications and data processing of the electronic device by running the non-volatile software programs, instructions and modules stored in the memory 402, that is, the method for real-time rendering of real-person shadows by XR in the above method embodiment is implemented.
[0072] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the device for real-time rendering of real-person shadows by XR, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely disposed relative to the processor 401, and these remote memories may be connected to the device for real-time rendering of real-person shadows by XR via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0073] An embodiment of the present application also provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the method of XR real-time rendering of real-person shadows in any of the above method embodiments.
[0074] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0075] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for real-time rendering of real-person shadows in XR, characterized in that: include: Obtaining human skeleton information of the first target person; Constructing a second target person based on the human skeleton information of the first target person; Rendering a shadow image of the second target person; The shadow image of the second target person is displayed on the LED display screen as the shadow of the first target person.
2. The method according to claim 1, characterized in that: The step of obtaining the human skeleton information of the first target person includes: The human skeleton information of the first target person is obtained by using a depth camera.
3. The method according to claim 2, characterized in that The step of constructing a second target person based on the human skeleton information of the first target person includes: Constructing a second target person in a rendering engine based on the human skeleton information of the first target person; The second target person is driven.
4. The method according to claim 3, characterized in that The rendering of the shadow picture of the second target person includes: A rendering engine is used to render a shadow image of the second target person.
5. The method according to claim 4, characterized in that After rendering the shadow image of the second target person by using the rendering engine, the method further includes: The display of the second target person in the rendering engine is turned off, and the shadow image of the second target person is retained.
6. An XR real-time rendering device for real-person shadows, characterized in that: include: An acquisition module, used to acquire human skeleton information of a first target person; A driving module, used for constructing a second target person based on the human skeleton information of the first target person; A rendering module, used for rendering a shadow image of the second target person; The display module is used to display the shadow image of the second target person on the LED display screen as the shadow of the first target person.
7. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.
8. A system for real-time rendering of real-person shadows by XR, characterized in that: The electronic device comprises the electronic device according to claim 7, a depth camera and an LED display screen, wherein the electronic device is connected to the depth camera and the LED display screen respectively. The depth camera is used to obtain human skeleton information, the electronic device is used to construct a second target person based on the human skeleton information and render a shadow image of the second target person, and the LED display screen is used to display the shadow image of the second target person.
9. The system according to claim 8, characterized in that The system also includes a camera, and the depth camera is mounted on the camera.
10. A non-volatile computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 5.