A special effect display method and device based on a mobile device

Abstract

This application relates to the field of VR / AR technologies, and provides a special effect display method and device based on a mobile device. The mobile device is equipped with a camera and can move according to a preset rule. Based on the motion pose of the camera, the current position of the mirror image of the camera sub-object is determined, and the current distance between the virtual object and the mirror image of the camera sub-object is determined. According to the comparison result between the current distance and the first distance threshold, the display type of the virtual object is determined. According to the motion speed of the camera and the display type, the virtual object is moved to the current position of the mirror image of the camera sub-object and faces the camera. Since the camera sub-object moves around the camera with the camera as the center and a preset length as the radius, in this way, the mirror image of the camera sub-object always faces the camera. Therefore, the orientation of the virtual object after moving with the camera will not shift and always faces the camera, thereby improving the authenticity of the special effect display of the virtual object, realizing the enhancement of the real world, and improving the interactivity.

Description

Technical Field

[0001] This application relates to the technical fields of virtual reality (VR) and augmented reality (AR), and particularly to a special effect display method and device based on a mobile device. Background Art

[0002] VR / AR display technology has a wide range of applications in various industries. For example, in industries such as education and training, fire drills, virtual driving, real estate, and marketing, it can bring users an immersive visual feast as if they were on the scene.

[0003] The AR motion enhancement special effect superimposes virtual objects (such as virtual cute pets: puppies, kittens, chicks, etc.) on the real world and tracks and controls their movement to better integrate them into the real world, fully demonstrating the unique visual impact experience brought by intelligence. The mobile tracking of the AR motion enhancement special effect will produce interesting dynamic effects of users' limb responses and show the advantage of high stability in extreme environments.

[0004] Currently, the traditional method for tracking and displaying the AR motion enhancement special effect mainly calculates the distance between the virtual object and the camera position of the mobile device based on the pose information of the mobile device (such as smartphones, AR glasses, VR head-mounted display devices, etc.), and realizes the movement and rotation of the virtual object according to the movement distance and rotation angle of the camera (i.e., the mobile device).

[0005] In theory, the orientation of the virtual object should always face the direction of the camera. However, when making the virtual object "little dinosaur" move with the movement of the camera according to the above method, the virtual object can only be rotated by 180°, and since the orientation of the virtual object remains unchanged, as the distance between the virtual object and the camera increases, the posture of the virtual object becomes more and more inclined relative to the orientation of the camera, as Figure 1 shown. This inclination will cause jitter visually, making the virtual object unable to simulate the natural movement and rotation of an object with the pose of the camera in the real world, resulting in poor interactivity and authenticity. Summary of the Invention

[0006] The embodiments of this application provide a special effect display method and device based on a mobile device to improve the display effect of the AR motion enhancement special effect.

[0007] In a first aspect, the embodiments of this application provide a special effect display method based on a mobile device, where the mobile device includes a camera, and the method includes:

[0008] Set the initial position of the virtual object to the initial position of the mirror image of the camera sub - object, where the camera sub - object moves around the camera with a preset length as the radius centered on the camera and faces the camera;

[0009] Determine the current position of the mirror image of the camera sub - object according to the motion pose of the camera. The orientation of the mirror image of the camera sub - object is the same as that of the camera sub - object;

[0010] Determine the current distance between the virtual object and the mirror image of the camera sub - object according to the initial position of the virtual object and the current position of the mirror image of the camera sub - object;

[0011] Determine the display type of the virtual object according to the comparison result of the current distance and the first distance threshold;

[0012] Move the virtual object to the current position of the mirror image of the camera sub - object at the motion speed of the camera, and adjust the orientation of the virtual object according to the orientation of the mirror image of the camera sub - object so that the virtual object faces the camera;

[0013] Display the special effects corresponding to the virtual object according to the determined display type.

[0014] In a second aspect, an embodiment of the present application provides a mobile device, including a processor, a memory, a display, a camera, and a communication interface. The communication interface, the camera, the display, and the memory are connected to the processor through a bus:

[0015] The camera is configured to move according to a preset rule;

[0016] A computer program is stored in the memory, and the processor realizes the following operations by executing the computer program:

[0017] Set the initial position of the virtual object to the initial position of the mirror image of the camera sub - object, where the camera sub - object moves around the camera with a preset length as the radius centered on the camera and faces the camera;

[0018] Obtain the motion pose of the camera through the communication interface, and determine the current position of the mirror image of the camera sub - object according to the motion pose. The orientation of the mirror image of the camera sub - object is the same as that of the camera sub - object;

[0019] Determine the current distance between the virtual object and the mirror image of the camera sub - object according to the initial position of the virtual object and the current position of the mirror image of the camera sub - object;

[0020] Determine the display type of the virtual object according to the comparison result of the current distance and the first distance threshold;

[0021] Move the virtual object to the current position of the mirror image of the camera sub - object according to the moving speed of the camera, and adjust the orientation of the virtual object according to the orientation of the mirror image of the camera sub - object so that the virtual object faces the camera;

[0022] According to the determined display type, the display device displays the special effects corresponding to the virtual object.

[0023] In a third aspect, an embodiment of the present application provides a computer - readable storage medium storing computer - executable instructions for causing a computer to execute the special - effect display method based on a mobile device provided by the embodiment of the present application.

[0024] In the above - mentioned embodiments of the present application, the position of the mirror image of the camera sub - object is initialized according to the position of the virtual object. The camera sub - object moves with the movement of the camera to obtain the current position of the mirror image of the camera sub - object. Further, according to the initial position of the virtual object and the current position of the mirror image of the camera sub - object, the current distance between the two is determined, and the current distance is compared with the first distance threshold. According to the comparison result, the display type of the virtual object is determined. The virtual object is moved to the current position of the mirror image of the camera sub - object according to the moving speed of the camera and faces the camera. According to the determined display type, the special effects corresponding to the virtual object are displayed. Since the camera sub - object moves around the camera with the camera as the center and a preset length as the radius, and the camera sub - object always faces the camera, the mirror image of the camera sub - object also always faces the camera. Therefore, when the virtual object is placed at the current position of the mirror image of the camera sub - object, the effect that the virtual object moves with the camera can be achieved. And after adjusting the orientation of the virtual object according to the orientation of the mirror image of the camera sub - object, the virtual object always faces the camera, so there will be no orientation deviation during the movement process, thereby improving the authenticity of the special - effect display of the virtual object, realizing the enhancement of the real world, and improving the interactivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 Exemplarily shows the effect diagram of the traditional AR motion - enhanced special - effect display provided by the embodiment of the present application;

[0027] Figure 2Schematic diagram of a camera sub-object provided by an embodiment of the present application is exemplarily shown;

[0028] Figure 3 Schematic diagram of a mirror image of a camera sub-object provided by an embodiment of the present application is exemplarily shown;

[0029] Figure 4 Flowchart of a special effect display method based on a mobile device provided by an embodiment of the present application is exemplarily shown;

[0030] Figure 5 Schematic diagram of the distance between a virtual object and a mirror image of a camera sub-object provided by an embodiment of the present application is exemplarily shown;

[0031] Figure 6 Schematic diagram of the movement process of a virtual object provided by an embodiment of the present application is exemplarily shown;

[0032] Figure 7 AR motion enhancement special effect display effect diagram provided by an embodiment of the present application is exemplarily shown;

[0033] Figure 8 Functional structure diagram of a mobile device provided by an embodiment of the present application is exemplarily shown;

[0034] Figure 9 Hardware structure diagram of a mobile device provided by an embodiment of the present application is exemplarily shown. Detailed implementation manners

[0035] To make the objectives, implementation manners, and advantages of the present application clearer, the following will clearly and completely describe the exemplary implementation manners of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0036] Based on the exemplary embodiments described in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by the claims of the present application. In addition, although the disclosed content in the present application is introduced according to one or several exemplary instances, it should be understood that each aspect of these disclosed contents can also constitute a complete implementation manner alone.

[0037] The terms "first", "second", "third", etc. in the description, claims, and above-mentioned drawings of the present application are used to distinguish similar or homogeneous objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be interchanged under appropriate circumstances, for example, it is possible to implement according to an order other than those given in the illustration or description of the embodiments of the present application.

[0038] In addition, the terms "comprising", "having" and any variations thereof are intended to cover inclusion without exclusivity. For example, a product or device comprising a series of components need not be limited to those components clearly listed, but may include other components not clearly listed or inherent to such products or devices.

[0039] To clearly describe the embodiments of the present application, the following explanations are given for the terms in the present application.

[0040] VR technology: It is a computer simulation technology that can create and experience virtual worlds. By using a computer to generate an interactive all-digital three-dimensional visual field, it can create a virtual environment.

[0041] AR technology: It is a technology that calculates the position and angle (i.e., pose) of a camera in real time and superimposes text, images, videos, and 3D models on the real world. The purpose is to seamlessly superimpose virtual information on the real world on the screen to enhance reality.

[0042] Camera sub-object: It is bound to the camera, centered on the camera, with a fixed length as the radius, and moves (including translation and rotation) as the camera moves. During the movement, the orientation of the camera sub-object always faces the camera.

[0043] As Figure 2 shown, create an empty object, represented by point A. Assume the camera position is point O, and point A is on a circle with point O as the center and radius R. The empty object point A is bound to the camera and moves as the camera moves. Then the empty object point A is the camera sub-object.

[0044] It should be noted that Figure 2 the circular motion trajectory shown is only an example. For example, the motion trajectory can also be a curve.

[0045] Mirror image of the camera sub-object: Fix the coordinates of the camera sub-object on any one coordinate in the three-dimensional space coordinate system, and let the values of the other two coordinate axes change with the change of the camera pose. The changed coordinate points are used as the mirror image of the camera sub-object. The orientation of the mirror image of the camera sub-object is the same as that of the camera sub-object and always faces the camera.

[0046] For example, taking the Y-axis coordinate of the fixed camera sub-object as an example, that is, the Y-axis of the camera sub-object point A is always the initial value, and only the coordinate values of the X and Z axes change with the movement and rotation of the camera, so as to obtain the mirror image of the camera sub-object point A.

[0047] As Figure 3As shown, assume that the initial position of the camera sub-object point A is Pg(xp, y0, zp), and the initial position of the camera is point Pc(xp, yp, zp). When the camera moves from point Pc to point Qc(xq, yq, zq), the mirror image of the camera sub-object is the position point Qg(xq, y0, zq) after the movement of point Pg.

[0048] After introducing the nouns used in this application, the embodiments of this application will be described in detail below.

[0049] The embodiment of this application provides a special effect display method and device based on a mobile device. Among them, the mobile device includes a camera that can move freely, including translation and rotation. In a VR / AR environment, when the camera moves, the virtual object moves with the movement of the camera and always faces the camera, so as to realize the tracking display of the special effects of the virtual object, make the virtual object truly and naturally integrate into the real world, and improve the substitution and authenticity of the AR motion enhancement special effects; and, through the natural movement and rotation tracking of the virtual object, bring a more real and interesting dynamic effect to the user.

[0050] It should be noted that the above-mentioned mobile devices include, but are not limited to, display devices with playback and interaction functions such as smart phones, AR glasses, VR head-mounted display devices, laptop computers, tablets, smart TVs, etc.

[0051] Figure 4 Exemplarily shows the flowchart of the special effect display method based on a mobile device provided by the embodiment of this application, as Figure 4 shown, this process is executed by a mobile device that includes a camera and can move freely, and mainly includes the following steps:

[0052] S401: Set the initial position of the virtual object to the initial position of the mirror image of the camera sub-object.

[0053] Based on the description of the above embodiment, it can be known that the camera sub-object moves around the camera in the mobile device as the center of the circle with a preset length as the radius, and always faces the camera.

[0054] In an alternative embodiment, when executing S401, initialize the position of the virtual object (such as: kitten, puppy, small dinosaur, etc.) in the real world, denoted as P0(x0, y0, z0), create a camera sub-object at this position, denoted as Pv(x0, y0, z0), the distance between the camera sub-object Pv and the camera is the preset length, denoted as R, and the camera sub-object Pv can move with the movement of the camera. During the movement, the camera sub-object always faces the camera. At the same time, initialize the mirror image position of the camera sub-object, denoted as Pg(x0, y0, z0). After the initialization is completed, the virtual object and the mirror image of the camera sub-object are in the same position.

[0055] S402: Determine the current position of the mirror image of the camera sub-object based on the motion pose of the camera. The orientation of the mirror image of the camera sub-object is the same as that of the camera sub-object.

[0056] Assume that for each frame of the image, the pose (including translation and rotation) of the camera after movement is denoted as P(x, y, z). According to the definition of the mirror image of the camera sub-object in the above embodiment, based on the motion pose of the camera, the current position of the mirror image of the camera sub-object can be determined, and the orientation of the mirror image of the camera sub-object is the same as that of the camera sub-object, always facing the camera.

[0057] During specific implementation, set a preset coordinate axis as any one of the coordinate axes in the three-dimensional space coordinate system where the camera sub-object is located. During the movement of the camera, keep the value of the mirror image of the camera sub-object on the preset coordinate axis as the initial value, that is, the Y value is always y0; according to the motion pose of the camera, determine the coordinate values of the mirror image of the camera sub-object on the other two coordinates, that is, only based on the motion pose of the camera, change the values of the mirror image of the camera sub-object on the other two coordinate axes to obtain the current position of the mirror image of the camera sub-object, denoted as Pg(x, y0, z).

[0058] For example, still taking Figure 3 as an example, the preset coordinate axis is the Y axis, and only change the values of the mirror image of the camera sub-object on the X and Z coordinate axes.

[0059] S403: Determine the current distance between the virtual object and the mirror image of the camera sub-object based on the initial position of the virtual object and the current position of the mirror image of the camera sub-object.

[0060] It can be seen from the initialization process in S401 that the initial distance between the virtual object and the mirror image of the camera sub-object is 0. After the position of the mirror image of the camera sub-object changes with the movement of the camera, the current distance between the initial position of the virtual object and the current position of the mirror image of the camera sub-object can be determined.

[0061] As Figure 5 shown, taking the virtual object as a small dinosaur as an example, B is the current position of the mirror image of the camera sub-object, and the current distance between the virtual object and the mirror image of the camera sub-object is d. That is to say, the moving distance of the camera is d.

[0062] S404: Determine the display type of the virtual object based on the comparison result between the current distance and the first distance threshold.

[0063] In some scenarios, the virtual object has different animation states, such as stationary, walking, running, jumping, etc. According to the movement speed of the camera, different animation states can be determined to better conform to the real interaction scenario. Among them, the movement speed of the camera can be reflected by the moving distance.

[0064] For example, if the virtual object is a puppy, when the camera moves fast, the puppy moves and rotates with the movement and rotation of the camera in a running animation state. When the camera moves slowly, the puppy moves and rotates with the movement and rotation of the camera in a walking animation state.

[0065] When executing S404, a preset first distance threshold D0 is used as the judgment threshold for the animation state when the virtual object is displayed. Specifically, the current distance d between the virtual object and the mirror image of the camera sub-object is compared with the first distance threshold D0. If d < D0, it is determined that the display type of the virtual object is the static state. Otherwise, it is determined that the display type of the virtual object is the motion state, such as animation states like walking, running, and jumping.

[0066] In some embodiments, after determining that the display type of the virtual object is the motion state, a second distance threshold D1 can also be set as the judgment condition for switching the motion state, that is, the current distance d between the virtual object and the mirror image of the camera sub-object is compared with the second distance threshold D1, and according to the comparison result, the motion state of the virtual object display is switched. Among them, D1 is greater than D0.

[0067] Still taking the virtual object puppy as an example, for example, when d > D1 > D0, the puppy is switched from the walking animation state to the running animation state for display.

[0068] S405: Move the virtual object to the current position of the mirror image of the camera sub-object according to the movement speed of the camera, and adjust the orientation of the virtual object according to the orientation of the mirror image of the camera sub-object so that the virtual object faces the camera.

[0069] Since the mirror image of the camera sub-object always faces the camera, when executing S405, when moving the virtual object to the current position of the mirror image of the camera sub-object, the effect that the virtual object moves with the camera can be achieved, and after adjusting the orientation of the virtual object according to the orientation of the mirror image of the camera sub-object, the virtual object always faces the camera, so there will be no orientation deviation during the movement process.

[0070] In specific implementation, when moving the virtual object to the current position of the mirror image of the camera sub-object according to the movement speed of the camera, the movement speed of the virtual object is proportional to the movement speed (i.e., distance) of the camera, that is, when the camera moves fast, it is equivalent to the mirror image of the camera sub-object moving fast, then the movement speed of the virtual object is fast. When the camera moves slowly, it is equivalent to the mirror image of the camera sub-object moving slowly, then the movement speed of the virtual object is slow.

[0071] Assume that the movement speed of the virtual object is denoted as Mspeed, then the calculation formula for Mspeed is:

[0072] Mspeed = k1 * v1 = k1 * d / t Formula 1

[0073] Among them, k1 is a preset coefficient that can be set according to experimental results, v1 is the moving speed of the camera, d is the current distance between the camera sub-object and the mirror image of the camera sub-object, that is, the moving distance of the camera, and t is the acquisition time of each frame of data of the camera.

[0074] The moving process of the virtual object is as Figure 6 shown. The virtual object is about to move to the current position B of the mirror image of the camera sub-object. During the moving process, the distance between the mirror image of the camera sub-object and the camera remains unchanged.

[0075] In S405, after moving the virtual object to the current position of the mirror image of the camera sub-object, according to the orientation of the mirror image of the camera sub-object, adjust the orientation of the virtual object so that it always faces the camera.

[0076] S406: Display the special effects corresponding to the virtual object according to the determined display type.

[0077] According to the description of the foregoing embodiments, the virtual object can be displayed in an animated state. Therefore, during the process of moving the virtual object, the display speed of the special effects should also be considered. When executing S406, display the special effects corresponding to the virtual object according to the determined display type.

[0078] Assume that the display speed of the virtual object is denoted as Aspeed, then the calculation formula of Aspeed is:

[0079] Aspeed = k2 * v1 = k2 * d / t Formula 2

[0080] Among them, k2 is a preset coefficient that can be set according to experimental results, v1 is the moving speed of the camera, d is the current distance between the camera sub-object and the mirror image of the camera sub-object, that is, the moving distance of the camera, and t is the acquisition time of each frame of data of the camera.

[0081] In some embodiments, in S405 and S406, t in Formulas 1 and 2, the acquisition time of each frame of data of the camera, is a fixed value, which can be ignored or considered as 1 ms. Then the moving speed and display speed of the virtual object can also be expressed as:

[0082] Mspeed = k11 * d Formula 3

[0083] Aspeed = k21 * d Formula 4

[0084] Among them, the coefficient values of k11 and k21 can be set according to experimental results. That is to say, it can be considered that the moving speed and display speed of the virtual object are determined according to the magnitude of the distance value between the mirror image of the camera sub-object and the virtual object in each frame.

[0085] Based on Figure 4The method flow shown Figure 7 Exemplarily shown is a schematic diagram of the result of tracking and displaying special effects for a virtual object provided by an embodiment of the present application, such as Figure 7 shown, based on the distance between the virtual object and the mirror image of the camera sub-object, the real-time tracking of the special effect display of the virtual object is performed, so that the virtual object moves and rotates naturally as the camera moves and rotates, and the rotation angle of the virtual object is 0-360°, and the virtual object always looks at the orientation of the camera.

[0086] In the above embodiments of the present application, a method for displaying an AR motion enhancement special effect with natural interaction, real-time and high efficiency is implemented, which solves the problem that the pose of the virtual object cannot be naturally tracked in the real world, such as severe pose jitter, orientation deviation and limited angle during the movement of the virtual object. In the AR / VR environment, based on the distance between the virtual object and the mirror image of the camera sub-object, the virtual object moves and rotates naturally as the camera moves and rotates, so that the virtual object is truly and naturally integrated into the real world, improving the sense of substitution and authenticity of the AR motion enhancement special effect. Moreover, through the natural movement and rotation tracking of the virtual object, a more real and interesting dynamic effect is brought to the user.

[0087] Based on the same technical concept, an embodiment of the present application provides a mobile device with a camera. This mobile device can execute the special effect display method flow provided by the embodiment of the present application and can achieve the same technical effect, which will not be repeated here.

[0088] See Figure 8 This mobile device includes an initialization module 801, a distance determination module 802, a display type determination module 803, a movement module 804, and a display module 805:

[0089] The initialization module 801 is used to set the initial position of the virtual object as the initial position of the mirror image of the camera sub-object. The camera sub-object moves around the camera with the camera as the center and a preset length as the radius and faces the camera;

[0090] The distance determination module 802 is used to determine the current position of the mirror image of the camera sub-object according to the motion pose of the camera, and determine the current distance between the virtual object and the mirror image of the camera sub-object according to the initial position of the virtual object and the current position of the mirror image of the camera sub-object. The orientation of the mirror image of the camera sub-object is the orientation of the camera sub-object;

[0091] The display type determination module 803 is used to determine the display type of the virtual object according to the comparison result between the current distance and the first distance threshold;

[0092] The movement module 804 is used to move the virtual object to the current position of the mirror image of the camera's child object according to the movement speed of the camera, and adjust the orientation of the virtual object according to the orientation of the mirror image of the camera's child object so that the virtual object faces the camera;

[0093] The display module 805 is used to display the special effects corresponding to the virtual object according to the determined display type.

[0094] Optionally, the distance determination module 802 is specifically used for:

[0095] Keep the value of the mirror image of the camera's child object on the preset coordinate axis as the initial value, and the preset coordinate axis is any one of the three-dimensional space coordinate systems where the camera's child object is located;

[0096] Determine the coordinate values of the mirror image of the camera's child object on the other two coordinates according to the movement pose of the camera, and obtain the current position of the mirror image of the camera's child object.

[0097] Optionally, the display type determination module 803 is specifically used for:

[0098] If the current distance is less than the first distance threshold, determine that the display type of the virtual object is the static state;

[0099] If the current distance is not less than the first distance threshold, determine that the display type of the virtual object is the moving state.

[0100] Optionally, the display type determination module 803 is also used for:

[0101] Compare the current distance with the second distance threshold, and switch the moving state of the virtual object display according to the comparison result. Optionally, the virtual object movement speed formula is:

[0102] Mspeed = k1 * v1 = k1 * d / t

[0103] Where k1 is a preset coefficient, v1 is the movement speed of the camera, d is the current distance between the camera's child object and the mirror image of the camera's child object, and t is the acquisition time of each frame of data of the camera.

[0104] Optionally, the display speed of the special effects corresponding to the virtual object is:

[0105] Aspeed = k2 * v1 = k2 * d / t

[0106] Where k2 is a preset coefficient, v1 is the movement speed of the camera, d is the current distance between the camera's child object and the mirror image of the camera's child object, and t is the acquisition time of each frame of data of the camera.

[0107] Optionally, the rotation angle of the virtual object is 0 - 360°.

[0108] Based on the same inventive concept, an embodiment of the present application provides a mobile device with a camera. This mobile device can execute the special effect display method flow provided by the embodiment of the present application and achieve the same technical effects, which will not be repeated here.

[0109] See Figure 9 , this mobile device includes a processor 901, a memory 902, a display 903, a camera 904, and a communication interface 905. The communication interface 905, the camera 904, the display 903, and the memory 902 are connected to the processor 901 through a bus 906;

[0110] The camera 904 is configured to move according to a preset rule;

[0111] The memory 902 stores a computer program, and the processor 901 realizes the following operations by executing the computer program:

[0112] Set the initial position of the virtual object to the initial position of the mirror image of the camera sub-object. The camera sub-object moves around the camera with a preset length as the radius centered on the camera and faces the camera;

[0113] Through the communication interface 905, obtain the motion pose of the camera 904, and determine the current position of the mirror image of the camera sub-object according to the motion pose. The orientation of the mirror image of the camera sub-object is the same as that of the camera sub-object;

[0114] Determine the current distance between the virtual object and the mirror image of the camera sub-object according to the initial position of the virtual object and the current position of the mirror image of the camera sub-object;

[0115] Determine the display type of the virtual object according to the comparison result between the current distance and the first distance threshold;

[0116] Move the virtual object to the current position of the mirror image of the camera sub-object according to the motion speed of the camera, and adjust the orientation of the virtual object according to the orientation of the mirror image of the camera sub-object so that the virtual object faces the camera;

[0117] Display the special effects corresponding to the virtual object on the display 903 according to the determined display type.

[0118] Optionally, the processor 901 determines the current position of the mirror image of the camera sub-object according to the motion pose of the camera, and is specifically configured to:

[0119] Keep the value of the mirror image of the camera sub-object on the preset coordinate axis as the initial value. The preset coordinate axis is any one of the three-dimensional space coordinate systems where the camera sub-object is located;

[0120] Determine the coordinate values of the mirror image of the camera sub-object on the other two coordinates according to the motion pose of the camera, and obtain the current position of the mirror image of the camera sub-object.

[0121] Optionally, the processor 901 determines the display type of the virtual object according to the comparison result between the current distance and the first distance threshold, and is specifically configured to:

[0122] If the current distance is less than the first distance threshold, determine that the display type of the virtual object is the stationary state;

[0123] If the current distance is not less than the first distance threshold, determine that the display type of the virtual object is the motion state.

[0124] Optionally, after the processor 901 determines that the display type of the virtual object is the motion state, it further includes:

[0125] Compare the current distance with the second distance threshold, and switch the motion state of the virtual object display according to the comparison result.

[0126] Optionally, the moving speed formula of the virtual object is:

[0127] Mspeed = k1 * v1 = k1 * d / t

[0128] Wherein, the k1 is a preset coefficient, v1 is the motion speed of the camera, d is the current distance between the camera sub-object and the mirror image of the camera sub-object, and t is the acquisition time of each frame of data of the camera.

[0129] Optionally, the display speed of the special effect corresponding to the virtual object is:

[0130] Aspeed = k2 * v1 = k2 * d / t

[0131] Wherein, the k2 is a preset coefficient, v1 is the motion speed of the camera, d is the current distance between the camera sub-object and the mirror image of the camera sub-object, and t is the acquisition time of each frame of data of the camera.

[0132] Optionally, the rotation angle of the virtual object is 0 - 360°.

[0133] It should be noted that Figure 9 The hardware of the mobile device shown is only the key part for implementing the embodiments of the present application, and the display device may further include conventional hardware such as speakers.

[0134] The embodiments of the present application further provide a computer-readable storage medium for storing some instructions, which can complete the methods of the foregoing embodiments when executed.

[0135] The embodiments of the present application further provide a computer program product for storing a computer program, and the computer program is used to execute the method in the foregoing embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0137] For ease of explanation, the above description has been presented in connection with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed. Many modifications and variations are possible in light of the above teachings. The selection and description of the embodiments were chosen to best explain the principles and practical applications, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments suited to particular uses contemplated.

Claims

1. A special effect display method based on a mobile device, characterized in that, The mobile device includes a camera, and the method includes: Setting an initial position of a virtual object as an initial position of a mirror image of a camera sub - object; wherein, the camera sub - object is bound to the camera, centered on the camera, with a fixed length as the radius, and moves along with the movement of the camera, and always faces the camera during the movement. The mirror image of the camera sub - object is the coordinate of the camera sub - object fixed on any one coordinate in the three - dimensional space coordinate system, and the values of the other two coordinate axes change with the pose of the camera. The orientation of the mirror image of the camera sub - object is the same as the orientation of the camera sub - object and always faces the camera; Determining a current position of the mirror image of the camera sub - object according to the movement pose of the camera; Determining a current distance between the virtual object and the mirror image of the camera sub - object according to the initial position of the virtual object and the current position of the mirror image of the camera sub - object; Determining a display type of the virtual object according to a comparison result between the current distance and a first distance threshold; Moving the virtual object to the current position of the mirror image of the camera sub - object according to the movement speed of the camera, and adjusting the orientation of the virtual object according to the orientation of the mirror image of the camera sub - object so that the virtual object faces the camera; Displaying a special effect corresponding to the virtual object according to the determined display type.

2. The method according to claim 1, wherein The determining a current position of the mirror image of the camera sub - object according to the movement pose of the camera includes: Keeping the value of the mirror image of the camera sub - object on a preset coordinate axis as an initial value, where the preset coordinate axis is any one coordinate axis in the three - dimensional space coordinate system where the camera sub - object is located; Determining coordinate values of the mirror image of the camera sub - object on the other two coordinates according to the movement pose of the camera to obtain the current position of the mirror image of the camera sub - object.

3. The method according to claim 1, wherein The determining a display type of the virtual object according to a comparison result between the current distance and a first distance threshold includes: If the current distance is less than the first distance threshold, determining that the display type of the virtual object is a stationary state; If the current distance is not less than the first distance threshold, determining that the display type of the virtual object is a moving state.

4. The method according to claim 3, characterized in that, After determining that the display type of the virtual object is a moving state, it further includes: Comparing the current distance with a second distance threshold, and switching the moving state of the virtual object display according to the comparison result.

5. The method according to any one of claims 1 to 4, characterized in that The moving speed formula of the virtual object is: ； Wherein, k1 is a preset coefficient, v1 is the movement speed of the camera, d is the current distance between the camera sub - object and the mirror image of the camera sub - object, and t is the acquisition time of each frame of data of the camera.

6. The method according to any one of claims 1 to 4, characterized in that The display speed of the special effect corresponding to the virtual object is: ； Wherein, k2 is a preset coefficient, v1 is the movement speed of the camera, d is the current distance between the camera sub - object and the mirror image of the camera sub - object, and t is the acquisition time of each frame of data of the camera.

7. The method according to any one of claims 1-4, characterized in that The rotation angle of the virtual object is 0 - 360°.

8. A mobile device, characterized in that, It includes a processor, a memory, a display, a camera, and a communication interface. The communication interface, the camera, the display, and the memory are connected to the processor via a bus: The camera is configured to move according to a preset rule; A computer program is stored in the memory, and the processor implements the following operations by executing the computer program: Set the initial position of the virtual object to the initial position of the mirror image of the camera sub-object; wherein, the camera sub-object is bound to the camera, centered on the camera, with a fixed length as the radius, and moves with the movement of the camera, and always faces the camera during the movement. The mirror image of the camera sub-object is the coordinate obtained by fixing any one coordinate of the camera sub-object in the three-dimensional space coordinate system and changing the values of the other two coordinate axes according to the change of the camera pose. The orientation of the mirror image of the camera sub-object is the same as that of the camera sub-object and always faces the camera; Obtain the motion pose of the camera through the communication interface, and determine the current position of the mirror image of the camera sub-object according to the motion pose; Determine the current distance between the virtual object and the mirror image of the camera sub-object according to the initial position of the virtual object and the current position of the mirror image of the camera sub-object; Determine the display type of the virtual object according to the comparison result between the current distance and the first distance threshold; Move the virtual object to the current position of the mirror image of the camera sub-object according to the movement speed of the camera, and adjust the orientation of the virtual object according to the orientation of the mirror image of the camera sub-object so that the virtual object faces the camera; Display the special effects corresponding to the virtual object on the display according to the determined display type.

9. The mobile device according to claim 8, wherein, The processor determines the current position of the mirror image of the camera sub-object according to the motion pose of the camera, and is specifically configured to: Keep the value of the mirror image of the camera sub-object on the preset coordinate axis as the initial value, and the preset coordinate axis is any one coordinate axis in the three-dimensional space coordinate system where the camera sub-object is located; Determine the coordinate values of the mirror image of the camera sub-object on the other two coordinates according to the motion pose of the camera, and obtain the current position of the mirror image of the camera sub-object.

10. The mobile device according to claim 8, wherein The processor determines the display type of the virtual object according to the comparison result between the current distance and the first distance threshold, and is specifically configured to: If the current distance is less than the first distance threshold, determine that the display type of the virtual object is the stationary state; If the current distance is not less than the first distance threshold, determine that the display type of the virtual object is the motion state.

Images