Display apparatus and display method of virtual object

Abstract

The invention relates to a display device and a display method of a virtual object. The display apparatus includes: a display configured to display an image and / or a user interface; the shooting assembly is used for shooting images; a controller configured to: determine a display position of the virtual object on the display according to a gesture position recognized in the image; determining a skeleton unit used for representing the outline of the virtual object, and determining the attitude information of the skeleton unit in the animation frame according to the motion mode of the virtual object in the to-be-generated animation; interpolation is carried out based on the attitude information of the skeleton units in the animation frames, and the attitude information of the skeleton units in the animation frames in transition between the front animation frame and the rear animation frame is obtained; and playing an animation corresponding to the virtual object at the display position according to the attitude information of the skeleton unit in the animation frame. The gesture can be captured in real time, and the animation is correspondingly generated in real time to interact with the user, so that the immersive interaction experience of the user based on somatosensory is improved.

Description

Technical Field

[0001] This application relates to the field of computer application technologies, and particularly to a display device and a method for displaying virtual objects. Background Art

[0002] Currently, in the process of generating animations, it is usually necessary to obtain the real feature data of real objects, such as motion data and facial data, and convert the real feature data into the target feature data of virtual characters, so as to obtain the animations of virtual characters. Since when generating animations, it is necessary for real objects to produce real feature data that is synchronized with the animations of virtual characters that are desired to be generated, that is, it is necessary for real objects to be able to perform exquisitely, the difficulty of generating animations is relatively large. At the same time, the usage scenarios when generating animations are also relatively limited. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a display device and a method for displaying virtual objects that can reduce the difficulty of generating animations.

[0004] In a first aspect, this application provides a display device, including:

[0005] A display configured to display images and / or user interfaces;

[0006] A shooting component for shooting images;

[0007] A controller configured to:

[0008] For the images captured by the shooting component, determine the display position of the virtual object on the display according to the gesture positions recognized in the images;

[0009] Determine the bone units representing the outer contour of the virtual object, and determine the pose information of the bone units in the animation frames according to the motion mode of the virtual object in the animation to be generated;

[0010] Interpolate based on the pose information of the bone units in the animation frames to obtain the pose information of the bone units in the animation frames that transition between two adjacent animation frames;

[0011] According to the pose information of the bone units in the animation frames, play the corresponding animation of the virtual object at the display position.

[0012] The above technical solution has the following advantages or beneficial effects: Since it is not necessary to obtain the real feature data of a real object, such as motion data and facial data, but only to obtain the gesture position of the user, the motion animation of the virtual object can be automatically generated. Thus, the difficulty of generating the animation is lower than before. At the same time, since there is no need for the user to perform delicate performances, the usage scenarios for generating animations are more extensive. In addition, through the interpolation process, the motion process of the virtual object in the generated animation can be made smoother and more natural. Finally, in the scenario where the user interacts with the display device, real-time gesture capture can be performed, and corresponding real-time animations can be generated to interact with the user, thereby improving the user's immersive interaction experience based on body sensation.

[0013] In one embodiment, the bone unit is a rectangular frame, and the pose information includes the coordinates of two end points on the diagonal of the rectangular frame.

[0014] The above technical solution has the following advantages or beneficial effects: Since the bone unit can be represented by a rectangular frame, and the pose information of the bone unit can be represented by the coordinates of two end points on the diagonal of the rectangular frame, compared with complex geometric shapes or models, the bone representation of the virtual object can be simplified. Therefore, the computational amount during animation rendering can be reduced, and it is convenient to adjust the motion state of the virtual object.

[0015] In one embodiment, the controller executes to determine the pose information of the bone unit in the animation frame according to the motion mode of the virtual object in the animation to be generated, and is configured to:

[0016] For two consecutive animation frames, according to the motion mode of the virtual object in the animation to be generated, determine the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to the diagonal of the bone unit in the previous animation frame;

[0017] According to the coordinates of the starting end point on the diagonal of the bone unit in the previous animation frame, the length of the diagonal of the bone unit in the previous animation frame, and the rotation angle, determine the coordinates of the ending end point on the diagonal of the bone unit in the subsequent animation frame.

[0018] The above technical solution has the following advantages or beneficial effects: Since the coordinates of the ending end point of the bone unit can be located through the rotation angle, length, and coordinates of the starting end point of the bone unit, the adjustment process of the bone position during the motion of the virtual object can be simulated. Therefore, the authenticity of the virtual character during motion is improved, and the user's immersive interaction experience based on body sensation is also improved.

[0019] In one embodiment, the controller executes to perform interpolation based on the pose information of the bone unit in the animation frame to obtain the pose information of the bone unit in the intermediate animation frame between two consecutive animation frames, and is configured to:

[0020] Determine the bone stretching coefficient of the current transitional animation frame to be generated according to the total number of frames of the transitional animation frames between the previous and the next animation frames and the frame number of the current transitional animation frame to be generated;

[0021] Based on the bone stretching coefficient, adjust the diagonal length of the bone unit in the previous animation frame of the current transitional animation frame to be generated to obtain the diagonal length of the bone unit in the current transitional animation frame to be generated;

[0022] Determine the rotation angle of the current transitional animation frame to be generated relative to the diagonal of the bone unit in the previous animation frame according to the motion mode of the virtual object in the animation to be generated;

[0023] Determine the coordinates of the end point on the diagonal of the bone unit in the current transitional animation frame to be generated according to the coordinates of the starting point on the diagonal of the bone unit in the previous animation frame of the current transitional animation frame to be generated, the rotation angle of the diagonal of the bone unit in the current transitional animation frame to be generated, and the diagonal length of the bone unit in the current transitional animation frame to be generated.

[0024] The above technical solution has the following advantages or beneficial effects: Since the length of the bone unit can be adjusted by interpolation to simulate the real actions of a real object during movement, the movement process of the virtual object in the generated animation can be made smoother and more natural. In addition, the immersive interactive experience of the user based on body sensation is also improved.

[0025] In one embodiment, the controller executes to determine the bone stretching coefficient of the current transitional animation frame to be generated according to the total number of frames of the transitional animation frames between the previous and the next animation frames and the frame number of the current transitional animation frame to be generated, and is configured as:

[0026] Obtain the ratio between the frame number of the current transitional animation frame to be generated and the total number of frames of the transitional animation frames, and calculate the trigonometric function value based on the ratio;

[0027] Use the trigonometric function value as the adjustment multiple to adjust the preset coefficient to obtain the bone stretching coefficient of the current transitional animation frame to be generated.

[0028] The above technical solution has the following advantages or beneficial effects: Since the bone stretching coefficient of the transitional animation frame can be determined according to the sequence of the frame number of the transitional animation frame in the overall transitional animation frames, and the length of the bone unit is adjusted according to the bone stretching coefficient to simulate the real actions of a real object during movement, the movement process of the virtual object in the generated animation can be made smoother and more natural.

[0029] In one embodiment, before the controller executes to play the corresponding animation of the virtual object at the display position according to the pose information of the bone unit in the animation frame, it is further configured as:

[0030] For the bone unit in the animation frame, convert the coordinates of the two endpoints on the diagonal of the bone unit from two-dimensional coordinates to three-dimensional coordinates;

[0031] Calculate the stretching factor based on the current time point, and stretch the coordinate components in the three-dimensional coordinates of the two endpoints on the diagonal of the bone unit based on the stretching factor.

[0032] The above technical solution has the following advantages or beneficial effects: Since the two-dimensional animation can be converted into a three-dimensional animation, and the natural deformation of the bones during the movement of a real object can be simulated through the stretching factor, the realism of the animation can be enhanced.

[0033] In one of the embodiments, before the controller executes to play the corresponding animation of the virtual object at the display position according to the pose information of the bone unit in the animation frame, it is further configured to:

[0034] Determine the main color and brightness value of the background picture in the preset display area where the virtual object is located according to the display position of the virtual object on the display;

[0035] Adjust the color and brightness value of the virtual object according to the main color and brightness value.

[0036] The above technical solution has the following advantages or beneficial effects: Since the appearance of the virtual object can be dynamically adjusted according to the color and brightness value of the displayed picture, the virtual object can be more naturally integrated with the picture currently displayed on the display device, enhancing the visual integration and the user's sensory experience.

[0037] In a second aspect, the present application also provides a method for displaying a virtual object, which is applied to the display device provided by various possible implementation manners of the first aspect. The method includes:

[0038] For the image captured by the shooting component, determine the display position of the virtual object on the display according to the gesture position recognized in the image;

[0039] Determine the bone unit that represents the outer contour of the virtual object, and determine the pose information of the bone unit in the animation frame according to the movement mode of the virtual object in the to-be-generated animation;

[0040] Interpolate based on the pose information of the bone unit in the animation frame to obtain the pose information of the bone unit in the intermediate animation frame between two adjacent animation frames;

[0041] Play the corresponding animation of the virtual object at the display position according to the pose information of the bone unit in the animation frame.

[0042] The above technical solution has the following advantages or beneficial effects: Since it is not necessary to obtain the real feature data of the real object, such as motion data and facial data, but only to obtain the gesture position of the user, the motion animation of the virtual object can be automatically generated. Thus, the difficulty of generating the animation is lower than before. At the same time, since there is no need for the user to perform delicate performances, the usage scenarios for generating animations are more extensive. In addition, through the interpolation process, the motion process of the virtual object in the generated animation can be made smoother and more natural. Finally, in the scenario where the user interacts with the display device, real-time gesture capture can be performed, and corresponding animations can be generated in real time to interact with the user, thereby improving the user's immersive interaction experience based on body sensation.

[0043] In one embodiment, the bone unit is a rectangular frame, and the pose information includes the coordinates of two end points on the diagonal of the rectangular frame.

[0044] The above technical solution has the following advantages or beneficial effects: Since the bone unit can be represented by a rectangular frame, and the pose information of the bone unit can be represented by the coordinates of two end points on the diagonal of the rectangular frame, compared with complex geometric shapes or models, the bone representation of the virtual object can be simplified. Therefore, the computational amount during animation rendering can be reduced, and it is convenient to adjust the motion state of the virtual object.

[0045] In one embodiment, according to the motion mode of the virtual object in the animation to be generated, determining the pose information of the bone unit in the animation frame includes:

[0046] For two consecutive animation frames, according to the motion mode of the virtual object in the animation to be generated, determining the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to the diagonal of the bone unit in the previous animation frame;

[0047] According to the coordinates of the starting end point on the diagonal of the bone unit in the previous animation frame, the length of the diagonal of the bone unit in the previous animation frame, and the rotation angle, determining the coordinates of the ending end point on the diagonal of the bone unit in the subsequent animation frame.

[0048] The above technical solution has the following advantages or beneficial effects: Since the coordinates of the ending end point of the bone unit can be located through the rotation angle, length, and coordinates of the starting end point of the bone unit, the adjustment process of the bone position during the motion of the virtual object can be simulated. Therefore, the authenticity of the virtual character during motion is improved, and the user's immersive interaction experience based on body sensation is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] To more clearly illustrate the technical solutions in the embodiments of the present application or in the related art, the following will briefly introduce the drawings required for the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0050] Figure 1 Schematic diagram of the operation scenario between the display device and the control device provided by some embodiments of the present application;

[0051] Figure 2 Schematic diagram of the hardware configuration of the display device provided by some embodiments of the present application;

[0052] Figure 3 Schematic diagram of the hardware configuration of the control device provided by some embodiments of the present application;

[0053] Figure 4 Schematic diagram of the software configuration of the display device provided by some embodiments of the present application;

[0054] Figure 5 Schematic diagram of the system architecture provided by some embodiments of the present application;

[0055] Figure 6 Flowchart of the method for displaying a virtual object in one embodiment;

[0056] Figure 7 Flowchart of the method for displaying a virtual object in another embodiment;

[0057] Figure 8 Flowchart of the method for displaying a virtual object in yet another embodiment;

[0058] Figure 9 Interaction flowchart between the camera component, the controller, and the display in one embodiment;

[0059] Figure 10 Block diagram of the structure of the display device for a virtual object in one embodiment. Detailed implementation manners

[0060] The following will describe the embodiments in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following embodiments do not represent all the implementation manners consistent with the present application. They are only examples of the systems and methods consistent with some aspects of the present application described in detail in the claims.

[0061] It should be noted that the brief description of terms in this application is only for the convenience of understanding the following described embodiments, rather than intending to limit the embodiments of this application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0062] In this application, terms such as "first", "second", "third", etc. in the description, claims, and the above-mentioned drawings are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0063] The terms "comprising" and "having" and any variations thereof are intended to cover but not be limited to inclusion. For example, a product or device comprising a series of components does not have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0064] The term "module" refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code that can perform functions related to that element.

[0065] In the embodiments of this application, the display device 200 generally refers to a device with the ability to display images and process data. For example, the display device 200 includes but is not limited to smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0066] Figure 1 It is a schematic diagram of the operation scenario between the display device and the control device provided for some embodiments of this application. As Figure 1 shown, the user can operate the display device 200 through touch operations, the mobile terminal 300, and the control device 100. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0067] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 to perform data interaction. In some embodiments, the mobile terminal 300 and the display device 200 can install software applications and achieve connection communication through network communication protocols to achieve the purpose of one-to-one control operations and data communication. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve the synchronous display function.

[0068] As Figure 1It is also shown that the display device 200 also communicates with the server 400 through various communication methods. The display device 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0069] The display device 200 can provide a broadcast receiving television function, and can also additionally provide an intelligent network television function with computer support functions, including but not limited to, network television, smart television, Internet Protocol Television (IPTV), etc.

[0070] Figure 2 For some embodiments of this application Figure 1 The hardware configuration block diagram of the display device 200 in

[0071] In some embodiments, the display device 200 may include at least one of a tuner demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0072] In some embodiments, the detector 230 is used to collect signals from the external environment or interact with the outside. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0073] In some embodiments, the display 260 includes a display function component for presenting a picture and a driving component for driving image display. The display 260 is used to receive the image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, components of a menu control interface, and a user control UI interface, etc.

[0074] In some embodiments, the communication device 220 is a component for communicating with external devices or the server 400 according to various communication protocol types. The display device 200 can be provided with multiple communication devices 220 according to different supported communication methods. For example, when the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0075] The communication device 220 can enable the display device 200 to communicate with an external device or a server 400 through a wireless or wired connection. Among them, the wired connection can connect the display device 200 to the external device through components such as data lines and interfaces. The wireless connection can connect the display device 200 to the external device through wireless signals or a wireless network. The display device 200 can directly establish a connection relationship with the external device or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0076] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and first to nth interfaces for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0077] In some embodiments, the controller 250 and the tuner demodulator 210 may be located in different split devices, that is, the tuner demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0078] In some embodiments, the user can input a user command on the graphical user interface (GUI) displayed on the display 260, and then the user input interface receives the user input command through the graphical user interface (GUI).

[0079] In some embodiments, the audio output device 270 may be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. Among them, for the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device can be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0080] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0081] Figure 3 For some embodiments provided in this application Figure 1 The hardware configuration block diagram of the control device. As Figure 3 shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0082] The control device 100 is configured to control the display device 200, receive input operation instructions from the user, and convert the operation instructions into instructions recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200.

[0083] In some embodiments, the control device 100 can be an intelligent device. For example, the control device 100 can install various applications for controlling the display device 200 according to user needs.

[0084] In some embodiments, as Figure 1 shown, after installing an application for controlling the display device 200, the mobile terminal 300 or other intelligent electronic devices can perform functions similar to those of the control device 100.

[0085] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and the data processing functions between the external and internal.

[0086] Under the control of the controller 110, the communication interface 130 realizes the communication of control signals and data signals with the display device 200. The communication interface 130 can include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, and other near-field communication modules.

[0087] The user input / output interface 140, where the input interface includes at least one of a microphone 141, a touchpad 142, a sensor 143, a button 144, and other input interfaces.

[0088] In some embodiments, the control device 100 includes at least one of the communication interface 130 and the input / output interface 140. The communication interface 130 is configured in the control device 100. For example, modules such as WiFi, Bluetooth, and NFC can encode user input instructions through the WiFi protocol, or the Bluetooth protocol, or the NFC protocol and send them to the display device 200.

[0089] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0090] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0091] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may (control the display device) provide a user interface, allow the user to interact with the display device 200, and support the running of various application programs.

[0092] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0093] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications layer) (referred to as "application layer"), the application framework layer (Application Framework layer) (referred to as "framework layer"), the system library layer and the kernel layer.

[0094] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0095] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions of applications in the application layer. Applications can access system resources and obtain system services during execution through the API interface.

[0096] like Figure 4As shown in the figure, in the embodiment of the present application, the application framework layer includes a view system (View System), managers, content providers, etc. Among them, the view system can design and implement the interface and interaction of the application. The view system includes lists, grids, text boxes, buttons, etc. The managers include at least one of the following modules: The Activity Manager is used to interact with all the activities running in the system; the Location Manager is used to provide access to the system location service for system services or applications; the Package Manager is used to retrieve various information related to the application packages currently installed on the device; the Notification Manager is used to control the display and clearing of notification messages; the Window Manager is used to manage the icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0097] In some embodiments, the Activity Manager is used to manage the life cycles of various applications and the usual navigation back functions, such as controlling the exit, opening, and back of the application. The Window Manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling the change of the display window. For example, the display window can be reduced, jittered, or distorted.

[0098] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions that the framework layer is to achieve.

[0099] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, as Figure 4 shown, hardware drivers can be configured in the kernel layer. The drivers included in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power drivers, etc.

[0100] It should be noted that the above examples are only a simple division of the functions of the operating system, and do not limit the specific form of the operating system of the display device 200 in the embodiments of the present application. Depending on factors such as the functions of the display device and the type of the operating system, the number of levels and the specific level types included in the operating system may be in other forms.

[0101] Combined with the specific operating system of the display device 200 in the embodiments of the present application, the system architecture diagram of the display device 200 can be referred to Figure 5 . As Figure 5 shown, in some embodiments, the system is divided into four layers, from top to bottom are the Application layer (referred to as the "application layer" for short), the Application Framework layer (referred to as the "framework layer" for short), the core layer of the input subsystem, and the driver layer.

[0102] Among them, the application layer can provide user interfaces and user interaction functions, including camera startup, the main interface of the hand shadow game, virtual object display, control panel, display of gesture recognition results, playing virtual object animations, and processing user inputs, etc. Processing user inputs includes starting the game, adjusting sensitivity, and viewing the game status, etc. Combined Figure 5 , in some embodiments, the application layer can run the application program "AI Hand Shadow Painting". This application program can further include a hand shadow capture program, a camera program, an interaction interface program, an AI animation display program, etc.

[0103] The framework layer provides a running environment for core application logics, such as real-time analysis of image data, gesture recognition, animation control logic, etc. In addition, it can also manage the interaction between applications and system services, such as camera services, sensor data acquisition, and cloud communication modules, etc. Combined Figure 5 , in some embodiments, the framework layer can further include a camera manager, a camera device, an alarm manager, an activity management service, a package management service, and a window management service.

[0104] The core layer of the input subsystem can provide system services accessible to applications, such as call interfaces for Camera services, network communication services, and image processing libraries, etc. Combined Figure 5 , in some embodiments, the core layer can include an input core driver, such as "Driver / input / input.c".

[0105] The driver layer of the input subsystem can provide underlying hardware abstraction and driver support, including camera drivers, GPU acceleration, and image processing support, etc. The driver layer can support the hardware decoding and rendering tasks of the device, and optimize the processing speed and efficiency of image data. Combined Figure 5, in some embodiments, the driver layer may include a touch screen driver (such as S3C2410TS.C) and a USB keyboard driver (such as USBKBD.C), etc.

[0106] Combined with the foregoing content, embodiments of the present application provide a display device, including:

[0107] A display configured to display images and / or user interfaces;

[0108] A shooting component for shooting images;

[0109] A controller configured to:

[0110] For the image captured by the shooting component, determine the display position of the virtual object on the display according to the gesture position recognized in the image;

[0111] Determine the bone units for characterizing the contour of the virtual object, and determine the pose information of the bone units in the animation frame according to the movement mode of the virtual object in the to-be-generated animation;

[0112] Interpolate based on the pose information of the bone units in the animation frame to obtain the pose information of the bone units in the intermediate animation frame between two adjacent animation frames;

[0113] Play the corresponding animation of the virtual object at the display position according to the pose information of the bone units in the animation frame.

[0114] Among them, the shooting component can be built-in or externally connected to the display device, and the embodiments of the present application do not make specific limitations on this. The shooting component can be continuously working to capture the user's gestures at any time; it can also capture the user's gestures after an application for generating animations with gestures is launched on the display device (such as the aforementioned AI hand shadow painting), and the embodiments of the present application do not make specific limitations on the timing of the shooting component to capture images. Taking the example of a display device installed with an application for generating animations with gestures, after the user launches the application for generating animations with gestures, the shooting component can start working to capture images. If there is no gesture in the captured image, a prompt message can be sent externally, such as a voice prompt or a picture content prompt, etc., to prompt the user to make a gesture within the shooting range. If there is a gesture in the captured image, the display position of the virtual object on the display can be determined according to the gesture position recognized in the image.

[0115] It should be noted that in actual implementation, animations can be generated in real time based on the images captured by the shooting component in real time. Therefore, although only one image can be obtained by a single shot of the camera component, during the process of generating animations in real time, the shooting component may capture not only one image but may capture an image stream. Subsequently, the images in the image stream can be continuously recognized to determine the display position of the virtual object on the display, so that the virtual object can continuously move following the gesture position.

[0116] When recognizing a gesture in an image, the image can be first converted into the Bitmap format, and then the gesture recognition function can be called to recognize the gesture position in the image. Among them, the gesture position can be characterized by the points on the gesture contour in the image. The embodiments of the present application do not specifically limit the method for determining the display position of the virtual object on the display according to the gesture position recognized in the image, including but not limited to: calculating the average value of the coordinate values of the points on the gesture contour in the image as the display position of the virtual object on the display.

[0117] It should be noted that the virtual object is usually not a single point. For example, the virtual object may be an animal, and the display position determined by the above process is mainly the display position of a single point. Therefore, in actual implementation, the display position can be used as the center position of the virtual object, so as to further determine the display position of each point on the contour of the virtual object; other methods can also be used to determine the display position of the virtual object, and the embodiments of the present application do not specifically limit this.

[0118] Since the outer contour of the virtual object can be generally represented by bone units. For example, the legs of a human object can be represented by multiple rectangular frames connected in segments. Therefore, the bone units used to represent the outer contour of the virtual object can be first determined, and subsequently, according to the expected movement mode of the virtual object in the animation, the pose information of the bone units can be adjusted, so that the virtual object moves in real time. In actual implementation, the bone units can be represented by rectangular frames. It can be understood that as a rectangular frame serving as a bone unit, it can be represented by the coordinates of the two endpoints on its diagonal line, that is, the pose information can include the coordinates of the two endpoints on the diagonal line of the rectangular frame. When the virtual object moves, the coordinates of the two endpoints on the diagonal line will change accordingly. Therefore, the pose information of the bone units in the animation frame can be determined according to the movement mode of the virtual object in the animation to be generated.

[0119] Among them, the motion mode may include walking, running, flying, etc., and the embodiments of the present application do not make specific limitations thereto. By decomposing the motion process into each animation frame, the pose information of the bone unit in the animation frame can be determined. In order to make the motion process of the virtual object in the animation smoother and more natural, interpolation can also be performed based on the pose information of the bone unit in the previous and subsequent animation frames to obtain the pose information of the bone unit in the intermediate animation frames between the previous and subsequent animation frames. The interpolation method and the number of intermediate animation frames obtained by interpolation can be set according to requirements, and the embodiments of the present application do not make specific limitations thereto. After determining the pose information of the bone unit in each animation frame, the motion pose of the virtual object in each animation frame can be obtained, and thus the corresponding motion animation of the virtual object can be generated.

[0120] For the above display device, since it does not need to obtain the real feature data of the real object, such as action data and facial data, but only needs to obtain the gesture position of the user, the motion animation of the virtual object can be automatically generated. Therefore, the difficulty of generating the animation is lower than before. At the same time, since there is no need for the user to perform delicate performances, the usage scenarios of generating the animation are more extensive. In addition, through the interpolation process, the motion process of the virtual object in the generated animation can be made smoother and more natural. Finally, since in the scenario where the user interacts with the display device, gesture capture can be performed in real time, and the animation can be generated in real time to interact with the user, the immersive interaction experience of the user based on the body sense is improved.

[0121] In some embodiments, the bone unit is a rectangular frame, and the pose information includes the coordinates of two endpoints on the diagonal of the rectangular frame.

[0122] As can be seen from the foregoing content, the outer contour of the virtual object can be generally represented by the bone unit. For example, the legs of a human object can be represented by a plurality of rectangular frames connected in segments. Therefore, the bone unit can specifically be a rectangular frame, and for a rectangular frame in a two-dimensional plane, its size and position can be represented by the coordinates of two endpoints on the diagonal. Thus, the pose information may include the coordinates of two endpoints on the diagonal of the rectangular frame. Of course, in actual implementation, the bone unit can be characterized by other geometric shapes, and the embodiments of the present application do not make specific limitations thereto.

[0123] In this embodiment, since the bone unit can be characterized by a rectangular frame, and the pose information of the bone unit can be characterized by the coordinates of two endpoints on the diagonal of the rectangular frame, compared with complex geometric shapes or models, the bone representation of the virtual object can be simplified. Therefore, the calculation amount during animation rendering can be reduced, and it is convenient to adjust the motion state of the virtual object.

[0124] In some embodiments, the controller is configured to execute determining the pose information of the bone unit in the animation frame according to the motion mode of the virtual object in the to-be-generated animation, and is configured as:

[0125] For two consecutive animation frames, determine the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to that in the previous animation frame according to the movement mode of the virtual object in the to-be-generated animation.

[0126] According to the coordinates of the starting endpoint on the diagonal of the bone unit in the previous animation frame, the length of the diagonal of the bone unit in the previous animation frame, and the rotation angle, determine the coordinates of the ending endpoint on the diagonal of the bone unit in the subsequent animation frame.

[0127] Specifically, during the process of generating the animation, key frames in the entire animation can be determined based on the movement mode of the virtual object in the to-be-generated animation. A key frame refers to an animation frame in which the object attributes (such as pose information) of the virtual object in the entire animation must change, and this change will serve as the basis for the animation effect. For example, for the running process of a human object, the contact frame when the front foot touches the ground, the low-tension frame when the body center of gravity drops forward for buffering after the foot touches the ground, the pushing frame when pushing off the ground to move the body forward, and the mid-air frame when the human body is briefly in mid-air during running, etc.

[0128] Since during the movement of the virtual object, the rectangular frame serving as the bone unit may rotate around one of the endpoints on the diagonal of the rectangular frame. Therefore, for two consecutive animation frames serving as key frames, first determine the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to that in the previous animation frame according to the movement mode of the virtual object in the to-be-generated animation. And because it rotates around one of the endpoints on the diagonal of the rectangular frame, the coordinates of the starting endpoint will not change, and the length of the bone unit can be represented by the diagonal of the rectangular frame and will not change either. However, the rotation angle of the diagonal in the subsequent animation frame compared to that in the previous animation frame will change during the movement process. Thus, based on the above three factors, the coordinates of the ending endpoint on the diagonal of the bone unit in the subsequent animation frame can be calculated.

[0129] Specifically, the diagonal length can be determined according to the distance between the two endpoints in the previous animation frame and can be represented by length. The coordinates of the starting endpoint can be represented by (startx, starty). If the coordinates of the ending endpoint on the diagonal of the bone unit in the subsequent animation frame are represented by (endx, endy), and the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to that in the previous animation frame is represented by angle, then endx can be equal to startx + length * cos(angle), and endy can be equal to starty + length * sin(angle).

[0130] In this embodiment, since the coordinates of the end point of the bone unit can be located by the rotation angle, length, and the coordinates of the starting point of the bone unit, the adjustment process of the bone position during the movement of the virtual object can be simulated. Therefore, the authenticity of the virtual character during movement is improved, and the immersive interaction experience based on the somatosensory for the user is also improved.

[0131] In some embodiments, the controller is configured to perform interpolation based on the pose information of the bone unit in the animation frame to obtain the pose information of the bone unit in the intermediate animation frame between two adjacent animation frames, and is configured as follows:

[0132] Determine the bone stretching coefficient of the intermediate animation frame to be generated according to the total number of frames of the intermediate animation frame between two adjacent animation frames and the frame number of the intermediate animation frame to be generated currently;

[0133] Based on the bone stretching coefficient, adjust the diagonal length of the bone unit in the previous animation frame of the intermediate animation frame to be generated to obtain the diagonal length of the bone unit in the intermediate animation frame to be generated;

[0134] Determine the rotation angle of the intermediate animation frame to be generated relative to the diagonal of the bone unit in the previous animation frame according to the movement mode of the virtual object in the animation to be generated;

[0135] Determine the coordinates of the end point on the diagonal of the bone unit in the intermediate animation frame to be generated according to the coordinates of the starting point on the diagonal of the bone unit in the previous animation frame of the intermediate animation frame to be generated, the rotation angle of the diagonal of the bone unit in the intermediate animation frame to be generated, and the diagonal length of the bone unit in the intermediate animation frame to be generated.

[0136] Specifically, considering that in the movement process of a real object, the body may be dynamically stretched and compressed. Therefore, in the embodiments of the present application, the dynamic stretching and compression of the body can be simulated by interpolating between two adjacent animation frames. Among them, the bone stretching coefficient of the intermediate animation frame can be calculated first. It can be understood that the dynamic stretching and compression of the body also has a process, and this process will progress as the intermediate animation frame is played backward. Therefore, for different intermediate animation frames, the bone stretching coefficient of the intermediate animation frame can be determined according to the sequence of the frame number of the intermediate animation frame in the overall intermediate animation frames.

[0137] Let the frame number of the currently to-be-generated transitional animation frame be represented as currentFrame, and the total number of transitional animation frames between two adjacent animation frames be represented as totalFrames. Among them, the ratio of currentFrame to totalFrames can be directly used as the bone stretching coefficient, or further calculations can be performed based on this ratio to obtain the bone stretching coefficient. The embodiments of the present application do not make specific limitations on this. After obtaining the bone stretching coefficient, the diagonal length of the bone unit in the previous animation frame of the currently to-be-generated transitional animation frame can be adjusted. Specifically, let the diagonal length of the bone unit in the previous animation frame be represented as originalLength, and the bone stretching coefficient of the currently to-be-generated transitional animation frame be represented as stretchFactor. The product of the two can be used as the diagonal length of the bone unit in the currently to-be-generated transitional animation frame, which can be represented as newLength.

[0138] Similarly, let the coordinates of the starting endpoint on the diagonal of the bone unit in the previous animation frame of the currently to-be-generated transitional animation frame be represented by (startx, starty), and the coordinates of the ending endpoint on the diagonal of the bone unit in the currently to-be-generated transitional animation frame be represented by (endx, endy). The rotation angle of the currently to-be-generated transitional animation frame relative to the diagonal of the bone unit in the previous animation frame is represented by angle. Then endx can be equal to startx + newLength * cos(angle), and endy can be equal to starty + newLength * sin(angle).

[0139] In this embodiment, since the length of the bone unit can be adjusted by interpolation to simulate the real actions of a real object during movement, the movement process of the virtual object in the generated animation can be made smoother and more natural. In addition, the immersive interactive experience based on the user's body feeling is also improved.

[0140] In some embodiments, the controller is configured to determine the bone stretching coefficient of the currently to-be-generated transitional animation frame according to the total number of transitional animation frames between two adjacent animation frames and the frame number of the currently to-be-generated transitional animation frame, and is configured as follows:

[0141] Obtain the ratio between the frame number of the currently to-be-generated transitional animation frame and the total number of transitional animation frames, and calculate the trigonometric function value based on the ratio;

[0142] Use the trigonometric function value as an adjustment multiple to adjust a preset coefficient to obtain the bone stretching coefficient of the currently to-be-generated transitional animation frame.

[0143] Specifically, represent the frame number of the current transitional animation frame to be generated as currentFrame, represent the total number of transitional animation frames between two adjacent animation frames as totalFrames, and represent the bone stretching coefficient of the current transitional animation frame to be generated as stretchFactor. stretchFactor can be equal to 1 + 0.2 * sin[2π * (currentFrame / totalFrames)].

[0144] Among them, sin[2π * (currentFrame / totalFrames)] is the trigonometric function value calculated based on the ratio. This trigonometric function value, as a multiple, will be multiplied by the first preset coefficient 0.2, and this product will be added to the second preset coefficient 1. Through these two adjustment processes, the bone stretching coefficient of the current transitional animation frame to be generated can be obtained.

[0145] In this embodiment, since the bone stretching coefficient of the transitional animation frame can be determined according to the sequence of the transitional animation frame in the overall transitional animation frames, and the length of the bone unit is adjusted according to the bone stretching coefficient to simulate the real actions of a real object during movement, the movement process of the virtual object in the generated animation can be made smoother and more natural.

[0146] In some embodiments, before the controller executes to play the corresponding animation of the virtual object at the display position according to the pose information of the bone unit in the animation frame, it is further configured to:

[0147] For the bone unit in the animation frame, convert the coordinates of the two endpoints on the diagonal of the bone unit from two-dimensional coordinates to three-dimensional coordinates;

[0148] Calculate the stretching factor based on the current time point, and based on the stretching factor, stretch the coordinate components in the three-dimensional coordinates of the two endpoints on the diagonal of the bone unit.

[0149] Among them, the above content is mainly the generation process of two-dimensional animation. In actual implementation, the two-dimensional animation can also be converted into three-dimensional animation. Specifically, through a 3D rendering engine, the coordinates of the two endpoints on the diagonal of the bone unit can be converted from the original two-dimensional coordinates to three-dimensional coordinates. Take the playback time point of the current animation frame as the current time point, and the trigonometric function value of the current time point can be directly used as the stretching factor and represented as stretch. Of course, the stretching factor can also be calculated in the following way, stretch is equal to 1 + 0.1 * sin(uTime). Where uTime is the current time point. After obtaining the stretching factor, the coordinate components in the three-dimensional coordinates of the two endpoints on the diagonal of the bone unit can be stretched. The specific method can be to multiply the stretching factor by the coordinate components.

[0150] In this embodiment, since a 2D animation can be converted into a 3D animation and the natural deformation of the bones during the movement of a real object can be simulated through a stretching factor, the realism of the animation can be enhanced.

[0151] In some embodiments, before the controller plays the corresponding animation of the virtual object at the display position according to the pose information of the bone units in the animation frame, the controller is further configured to:

[0152] Determine the main color and brightness value of the background picture in the preset display area where the virtual object is located according to the display position of the virtual object on the display;

[0153] Adjust the color and brightness value of the virtual object according to the main color and brightness value.

[0154] Specifically, in order to make the virtual object blend more naturally with the picture currently displayed on the display device, the appearance of the virtual object can be dynamically adjusted according to the color and brightness value of the displayed picture. Thus, the main color and brightness value of the background picture in the preset display area where the virtual object is located can be determined according to the display position of the virtual object. Among them, the preset display area includes the display position, and the range of the preset display area can be set according to requirements, and the embodiments of the present application do not make specific limitations in this regard. The main color can be at least one color that occupies a relatively large area in the preset display area, and the brightness value can be the average value of the brightness values of different pixels in the preset display area. According to the main color and brightness value, the color and brightness value of the virtual object can be adjusted by using a color harmony algorithm.

[0155] In this embodiment, since the appearance of the virtual object can be dynamically adjusted according to the color and brightness value of the displayed picture, the virtual object can blend more naturally with the picture currently displayed on the display device, enhancing the visual fusion degree and the user's sensory experience.

[0156] The above content mainly describes the display device. In an exemplary embodiment, a method for displaying a virtual object is further provided, which is applied to the display device mentioned above. Refer to Figure 6 , the method includes:

[0157] Step 602, for the image captured by the shooting component, determine the display position of the virtual object on the display according to the gesture position recognized in the image.

[0158] Step 604, determine the bone units representing the outer contour of the virtual object, and determine the pose information of the bone units in the animation frame according to the movement mode of the virtual object in the to-be-generated animation.

[0159] Step 606: Interpolate based on the pose information of the bone units in the animation frame to obtain the pose information of the bone units in the animation frame during the transition between two consecutive animation frames.

[0160] Step 608: Play the corresponding animation of the virtual object at the display position according to the pose information of the bone units in the animation frame.

[0161] For the above method of displaying a virtual object, since it is not necessary to obtain the real feature data of a real object, such as motion data and facial data, but only to obtain the gesture position of the user, the motion animation of the virtual object can be automatically generated. Thus, the difficulty of generating the animation is lower than before. At the same time, since there is no need for the user to perform delicate performances, the usage scenarios for generating the animation are more extensive. In addition, through the interpolation process, the motion process of the virtual object in the generated animation can be made smoother and more natural. Finally, in the scenario where the user interacts with the display device, real-time gesture capture can be performed, and the animation can be generated in real time to interact with the user, thereby improving the user's immersive interaction experience based on body sensation.

[0162] In some embodiments, the bone unit is a rectangular frame, and the pose information includes the coordinates of two end points on the diagonal of the rectangular frame.

[0163] In this embodiment, since the bone unit can be represented by a rectangular frame, and the pose information of the bone unit can be represented by the coordinates of two end points on the diagonal of the rectangular frame, compared with complex geometric shapes or models, the bone representation of the virtual object can be simplified. Therefore, the computational amount during animation rendering can be reduced, and it is convenient to adjust the motion state of the virtual object.

[0164] In some embodiments, refer to Figure 7 , and determine the pose information of the bone units in the animation frame according to the motion mode of the virtual object in the animation to be generated, including:

[0165] Step 6042: For two consecutive animation frames, determine the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to the diagonal of the bone unit in the previous animation frame according to the motion mode of the virtual object in the animation to be generated;

[0166] Step 6044: Determine the coordinates of the end point on the diagonal of the bone unit in the subsequent animation frame according to the coordinates of the starting end point on the diagonal of the bone unit in the previous animation frame, the length of the diagonal of the bone unit in the previous animation frame, and the rotation angle.

[0167] In this embodiment, since the coordinates of the end point of the bone unit can be located by the rotation angle, length, and coordinates of the starting point of the bone unit, the adjustment process of the bone position during the movement of the virtual object can be simulated. Therefore, the authenticity of the virtual character during movement is improved, and the immersive interaction experience based on the user's body sensation is also improved.

[0168] In some embodiments, with reference to Figure 8 , interpolation is performed based on the pose information of the bone unit in the animation frame to obtain the pose information of the bone unit in the intermediate animation frame between two consecutive animation frames, including:

[0169] Step 6062: Determine the bone stretching coefficient of the intermediate animation frame to be generated based on the total number of frames of the intermediate animation frame between two consecutive animation frames and the frame number of the intermediate animation frame to be generated currently.

[0170] Step 6064: Adjust the diagonal length of the bone unit in the previous animation frame of the intermediate animation frame to be generated based on the bone stretching coefficient to obtain the diagonal length of the bone unit in the intermediate animation frame to be generated.

[0171] Step 6066: Determine the rotation angle of the intermediate animation frame to be generated relative to the diagonal of the bone unit in the previous animation frame according to the movement mode of the virtual object in the animation to be generated.

[0172] Step 6068: Determine the coordinates of the end point on the diagonal of the bone unit in the intermediate animation frame to be generated based on the coordinates of the starting point on the diagonal of the bone unit in the previous animation frame of the intermediate animation frame to be generated, the rotation angle of the diagonal of the bone unit in the intermediate animation frame to be generated, and the diagonal length of the bone unit in the intermediate animation frame to be generated.

[0173] In this embodiment, since the length of the bone unit can be adjusted by interpolation to simulate the real actions of a real object during movement, the movement process of the virtual object in the generated animation can be made smoother and more natural. In addition, the immersive interaction experience based on the user's body sensation is also improved.

[0174] In some embodiments, referring to the foregoing content, the interaction process among the camera component, the controller, and the display can be referred to Figure 9 . In Figure 9Among them, the imaging component can capture images, and the controller can identify the images. When a gesture is recognized, the display position of the virtual object on the display is determined according to the gesture position. By determining the bone units of the virtual object, the movement process of the virtual object in the animation is converted into the pose information of the bone units in the animation frames. Interpolation is performed based on the pose information of the bone units in the animation frames to obtain the pose information of the bone units in the transitional animation frames between two adjacent animation frames. After obtaining the pose information of the bone units in each animation frame, an animation of the virtual object can be generated and the animation screen is transmitted to the display for playback.

[0175] It should be understood that although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0176] Based on the same inventive concept, an embodiment of the present application further provides a display device for a virtual object for implementing the virtual object display method described above. The implementation solutions provided by this device to solve problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the display device for a virtual object provided below can refer to the limitations on the virtual object display method in the above text, and will not be repeated here.

[0177] In an exemplary embodiment, as Figure 10 shown, a display device for a virtual object is provided, including: a determination module 1002, an interpolation module 1004, and a playback module 1006, where:

[0178] The determination module 1002 is configured to determine the display position of the virtual object on the display according to the gesture position recognized in the image for the image captured by the imaging component;

[0179] The determination module 1002 is further configured to determine the bone units representing the external contour of the virtual object, and determine the pose information of the bone units in the animation frames according to the movement mode of the virtual object in the to-be-generated animation;

[0180] The interpolation module 1004 is configured to perform interpolation based on the pose information of the bone units in the animation frames to obtain the pose information of the bone units in the animation frames during the transition between two consecutive animation frames.

[0181] The playback module 1006 is configured to play the corresponding animation of the virtual object at the display position according to the pose information of the bone units in the animation frames.

[0182] In an exemplary embodiment, the bone unit is a rectangular frame, and the pose information includes the coordinates of two end points on the diagonal of the rectangular frame.

[0183] In an exemplary embodiment, the determination module 1002 is configured to, for two consecutive animation frames, determine the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to that in the previous animation frame according to the movement mode of the virtual object in the to-be-generated animation; and determine the coordinates of the end point on the diagonal of the bone unit in the subsequent animation frame according to the coordinates of the starting end point on the diagonal of the bone unit in the previous animation frame, the length of the diagonal of the bone unit in the previous animation frame, and the rotation angle.

[0184] In an exemplary embodiment, the interpolation module 1004 is specifically configured to determine the bone stretching coefficient of the currently to-be-generated transition animation frame according to the total number of frames of the transition animation frames between two consecutive animation frames and the frame number of the currently to-be-generated transition animation frame; adjust the length of the diagonal of the bone unit in the previous animation frame of the currently to-be-generated transition animation frame based on the bone stretching coefficient to obtain the length of the diagonal of the bone unit in the currently to-be-generated transition animation frame; determine the rotation angle of the diagonal of the bone unit in the currently to-be-generated transition animation frame relative to that in the previous animation frame according to the movement mode of the virtual object in the to-be-generated animation; and determine the coordinates of the end point on the diagonal of the bone unit in the currently to-be-generated transition animation frame according to the coordinates of the starting end point on the diagonal of the bone unit in the previous animation frame of the currently to-be-generated transition animation frame, the rotation angle of the diagonal of the bone unit in the currently to-be-generated transition animation frame, and the length of the diagonal of the bone unit in the currently to-be-generated transition animation frame.

[0185] In an exemplary embodiment, the determination module 1004 is specifically configured to obtain the ratio between the frame number of the currently to-be-generated transition animation frame and the total number of frames of the transition animation frames, calculate the trigonometric function value based on the ratio; and adjust the preset coefficient with the trigonometric function value as the adjustment multiple to obtain the bone stretching coefficient of the currently to-be-generated transition animation frame.

[0186] In an exemplary embodiment, the determining module 1004 is further configured to convert the coordinates of two end points on the diagonal of the bone unit in the animation frame from two-dimensional coordinates to three-dimensional coordinates for the bone unit in the animation frame; calculate a stretching factor based on the current time point, and stretch the coordinate components in the three-dimensional coordinates of the two end points on the diagonal of the bone unit based on the stretching factor.

[0187] In an exemplary embodiment, the determining module 1004 is further configured to determine the main color and brightness value of the background picture in the preset display area where the virtual object is located according to the display position of the virtual object on the display; adjust the color and brightness value of the virtual object according to the main color and brightness value.

[0188] Each module in the above display device of the virtual object can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in the form of hardware or be independent of the processor, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0189] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the possible implementation manners provided by the above various methods are realized.

[0190] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the possible implementation manners provided by the above various methods are realized.

[0191] In an embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the possible implementation manners provided by the above various methods are realized.

[0192] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0193] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0194] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in the present application.

[0195] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A display device, characterized in that, Comprising: A display configured to display an image and / or a user interface; A shooting component for shooting an image; A controller configured to: For the image captured by the shooting component, determine the display position of the virtual object on the display according to the gesture position recognized in the image; Determine the bone unit for characterizing the outer contour of the virtual object, and determine the pose information of the bone unit in the animation frame according to the movement mode of the virtual object in the to-be-generated animation; Interpolate based on the pose information of the bone unit in the animation frame to obtain the pose information of the bone unit in the intermediate animation frame between two adjacent animation frames; Play the corresponding animation of the virtual object at the display position according to the pose information of the bone unit in the animation frame.

2. The display device according to claim 1, wherein The bone unit is a rectangular frame, and the pose information includes the coordinates of two endpoints on the diagonal of the rectangular frame.

3. The display device according to claim 2, wherein, When the controller executes to determine the pose information of the bone unit in the animation frame according to the movement mode of the virtual object in the to-be-generated animation, it is configured to: For two adjacent animation frames, determine the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to the diagonal of the bone unit in the previous animation frame according to the movement mode of the virtual object in the to-be-generated animation; Determine the coordinates of the ending endpoint on the diagonal of the bone unit in the subsequent animation frame according to the coordinates of the starting endpoint on the diagonal of the bone unit in the previous animation frame, the length of the diagonal of the bone unit in the previous animation frame, and the rotation angle.

4. The display device according to claim 2, wherein When the controller executes to interpolate based on the pose information of the bone unit in the animation frame to obtain the pose information of the bone unit in the intermediate animation frame between two adjacent animation frames, it is configured to: Determine the bone stretching coefficient of the current to-be-generated intermediate animation frame according to the total number of frames of the intermediate animation frame between two adjacent animation frames and the frame number of the current to-be-generated intermediate animation frame; Adjust the length of the diagonal of the bone unit in the previous animation frame of the current to-be-generated intermediate animation frame based on the bone stretching coefficient to obtain the length of the diagonal of the bone unit in the current to-be-generated intermediate animation frame; Determine the rotation angle of the diagonal of the bone unit in the current to-be-generated intermediate animation frame relative to the diagonal of the bone unit in the previous animation frame according to the movement mode of the virtual object in the to-be-generated animation; Determine the coordinates of the ending endpoint on the diagonal of the bone unit in the current to-be-generated intermediate animation frame according to the coordinates of the starting endpoint on the diagonal of the bone unit in the previous animation frame of the current to-be-generated intermediate animation frame, the rotation angle of the diagonal of the bone unit in the current to-be-generated intermediate animation frame, and the length of the diagonal of the bone unit in the current to-be-generated intermediate animation frame.

5. The display device according to claim 4, characterized in that, When the controller executes to determine the bone stretching coefficient of the current to-be-generated intermediate animation frame according to the total number of frames of the intermediate animation frame between two adjacent animation frames and the frame number of the current to-be-generated intermediate animation frame, it is configured to: Obtain the ratio between the frame number of the current to-be-generated intermediate animation frame and the total number of frames of the intermediate animation frame, and calculate the trigonometric function value based on the ratio; Use the trigonometric function value as an adjustment multiple to adjust the preset coefficient, and obtain the bone stretching coefficient of the current transitional animation frame to be generated.

6. The display device according to claim 2, wherein Before the controller executes to play the corresponding animation of the virtual object at the display position according to the pose information of the bone unit in the animation frame, it is further configured to: For the bone unit in the animation frame, convert the coordinates of the two endpoints on the diagonal of the bone unit from two-dimensional coordinates to three-dimensional coordinates; Calculate a stretching factor based on the current time point, and based on the stretching factor, stretch the coordinate components in the three-dimensional coordinates of the two endpoints on the diagonal of the bone unit.

7. The display device according to claim 1, wherein Before the controller executes to play the corresponding animation of the virtual object at the display position according to the pose information of the bone unit in the animation frame, it is further configured to: According to the display position of the virtual object on the display, determine the main color and brightness value of the background picture in the preset display area where the virtual object is located; Adjust the color and brightness value of the virtual object according to the main color and the brightness value.

8. A method for displaying a virtual object, characterized in that Applied to the display device according to any one of claims 1-7; The method includes: For the image captured by the photographing component, determine the display position of the virtual object on the display according to the gesture position recognized in the image; Determine the bone unit for characterizing the outer contour of the virtual object, and determine the pose information of the bone unit in the animation frame according to the movement mode of the virtual object in the animation to be generated; Interpolate based on the pose information of the bone unit in the animation frame to obtain the pose information of the bone unit in the transitional animation frame between two adjacent animation frames; Play the corresponding animation of the virtual object at the display position according to the pose information of the bone unit in the animation frame.

9. The method according to claim 8, wherein The bone unit is a rectangular frame, and the pose information includes the coordinates of the two endpoints on the diagonal of the rectangular frame.

10. The method according to claim 9, wherein The determining the pose information of the bone unit in the animation frame according to the movement mode of the virtual object in the animation to be generated includes: For two adjacent animation frames, determine the rotation angle of the diagonal of the bone unit in the subsequent animation frame relative to the diagonal of the bone unit in the previous animation frame according to the movement mode of the virtual object in the animation to be generated; According to the coordinates of the starting endpoint on the diagonal of the bone unit in the previous animation frame, the length of the diagonal of the bone unit in the previous animation frame, and the rotation angle, determine the coordinates of the ending endpoint on the diagonal of the bone unit in the subsequent animation frame.

Images