Precise follow-up interaction method, device and storage medium based on Unreal Engine platform

Abstract

The present application relates to a precise following interaction method, device and storage medium based on Unreal Engine platform, belonging to the field of three-dimensional interaction technology. The method comprises: displaying the current virtual scene corresponding to the camera position coordinates of the camera of Unreal Engine platform in the spatial coordinate system through a display screen; obtaining the screen coordinates before and after the mouse movement when a mouse movement operation is received; converting the screen coordinates to the spatial coordinate system to obtain the spatial coordinates before and after the mouse movement; determining the position update parameters of the camera based on the spatial coordinates before and after the mouse movement; updating the camera position based on the position update parameters; the problem of low mouse following accuracy of the traditional translation operation method can be solved, and compared with the native matrix transformation method of Unreal Engine, the error can be significantly reduced, and the accuracy and stability can be improved.

Description

Technical Field

[0001] The present application relates to a precise following interaction method, device and storage medium based on the Unreal Engine platform, and belongs to the field of three-dimensional interaction technology. Background Art

[0002] When displaying 3D images in games, architecture, education and other fields, it is often necessary to pan the view to view the image content at different angles in the 3D image. Generally, the pan operation following the mouse on the Unreal Engine platform allows users to quickly and accurately pull objects in virtual reality into the field of view, providing users with an efficient and comfortable operating experience.

[0003] The traditional translation operation method based on the Unreal Engine platform is implemented by a pixel difference translation method, which includes: obtaining the pixel difference between two screen positions operated by the mouse, and then multiplying the pixel difference by a preset coefficient to obtain the displacement value generated by each frame of the image.

[0004] However, the traditional translation operation method is highly sensitive to frame rate fluctuations. When the frame rate in the scene fluctuates greatly, the lens will frequently teleport, which will seriously affect the user experience. Secondly, when the camera is far away from the world coordinate origin, the error of the translation operation method will also increase. This is mainly because the method used to obtain the mouse position in the Unreal Engine platform has its inherent limitations. It is calculated based on matrix transformation. Although the calculation process is relatively simple, it is easy to produce large errors. Finally, as the viewing angle changes, this error will increase further, resulting in the algorithm being unable to accurately achieve the mouse translation follow-up effect. These problems limit the effect and usability of the traditional translation operation method in practical applications. Summary of the invention

[0005] The present application provides a precise following interaction method, device and storage medium based on the Unreal Engine platform. The method can enable the user to select a point by clicking the mouse in a plane coordinate system, and then the point can always accurately follow the position of the mouse no matter how the mouse is dragged. Secondly, this method is not only applicable to planes, but also can achieve precise mouse translation following effects on any terrain. Finally, it can even be applied to spherical coordinate systems. For example, the high-precision 3D spherical system built for Cesium for Unreal can also realize the mouse following function. These features give this technology significant advantages in improving the user interaction experience. The present application provides the following technical solutions:

[0006] In a first aspect, a precise follow-up interaction method based on an Unreal Engine platform is provided, the method comprising:

[0007] Displaying the current virtual scene corresponding to the camera position coordinates of the camera of the Unreal Engine platform in the space coordinate system through the display screen;

[0008] When a mouse movement operation is received, the screen coordinates before and after the mouse movement are acquired, where the screen coordinates are the pixel coordinates on the display screen indicated by the mouse;

[0009] Convert the screen coordinates to the spatial coordinate system to obtain the spatial coordinates before and after the mouse moves;

[0010] Determining the position update parameters of the camera based on the spatial coordinates before and after the mouse moves;

[0011] The camera position is updated based on the position update parameters to trigger the step of displaying the current virtual scene corresponding to the camera position coordinates in the space coordinate system through the display screen.

[0012] Optionally, the current virtual scene is obtained by projecting virtual objects within the viewing frustum of the camera onto a near plane according to a perspective projection relationship based on the camera position, and mapping the image on the near plane onto the display screen according to a certain proportional relationship; accordingly,

[0013] The step of converting the screen coordinates into the space coordinate system where the camera is located to obtain the space coordinates before and after the mouse moves includes:

[0014] determining a mouse position of the screen coordinates in the near plane;

[0015] Determine the yaw angle and the pitch angle of the camera based on the structural diagram of the camera and the near plane in the top view;

[0016] The spatial coordinates of the mouse before and after movement are determined based on the camera position coordinates, the mouse position, and the yaw angle and pitch angle of the camera.

[0017] Optionally, the spatial coordinates of the mouse before and after movement are determined based on the camera position coordinates, the mouse position, the yaw angle and the pitch angle of the camera, and are expressed by the following coordinate conversion formula:

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] ;

[0023] Among them, (Cx, Cy, Cz) represents the camera position coordinates, (Ax, Ay, Az) represents the calculated spatial coordinates of the mouse; CB represents the distance from the camera position coordinates of the camera to the midpoint of the near plane; DA represents the distance between the screen position and the midpoint of the near plane; BD represents the distance between the projection point of the mouse position on the vertical midline of the near plane and the midpoint of the near plane; α represents the yaw angle of the camera, and β represents the pitch angle of the camera.

[0024] Optionally, when the mouse is located at the left half of the horizontal midline of the screen, BD=0; in the coordinate conversion formula The value is 90°; accordingly, the coordinate conversion formula is transformed into:

[0025] ;

[0026] ;

[0027] .

[0028] Optionally, when the mouse is located at the right half of the horizontal midline of the screen, BD=0; in the coordinate conversion formula The value is -90°; accordingly, the coordinate conversion formula is transformed into:

[0029] ;

[0030] ;

[0031] .

[0032] Optionally, when the pitch angle of the camera is 0° and the camera is in a horizontal state, sinβ=0; accordingly, the coordinate transformation formula is modified as follows:

[0033] ;

[0034] ;

[0035] .

[0036] Optionally, the current virtual scene is generated based on a plane system; accordingly, determining the position update parameters of the camera based on the spatial coordinates before and after the mouse moves includes:

[0037] Determine the projection vector on the virtual ground from the spatial coordinates before the mouse moves to the spatial coordinates after the mouse moves;

[0038] The projection vector is subtracted from the camera position coordinates to obtain updated camera position coordinates.

[0039] Optionally, the current virtual scene is generated based on a spherical system; accordingly, determining the position update parameters of the camera based on the spatial coordinates before and after the mouse moves includes:

[0040] Determine that the spatial coordinates before the mouse moves are vertically slid on the display screen to the spatial coordinates after the mouse moves, and project them onto two first projection points on the spherical surface;

[0041] Determine the camera pitch angle increment based on the distance between the two first projection points, the radius of the spherical system, the distance between the spatial coordinates before the mouse moves and the camera position coordinates, and the angle between the line between the center of the spherical system and the camera position coordinates and the line between the spatial coordinates before the mouse moves and the camera position coordinates;

[0042] Subtract the camera pitch angle increment from the camera pitch angle to obtain an updated pitch angle;

[0043] An updated camera position coordinate is determined based on the distance between the camera position coordinate and the center of the spherical system, the yaw angle and the pitch angle of the camera, and the camera pitch angle increment.

[0044] In a second aspect, a precise following interaction device based on the Unreal Engine platform is provided, the device comprising a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement the precise following interaction method based on the Unreal Engine platform described in the first aspect.

[0045] In a third aspect, a computer-readable storage medium is provided, wherein a program is stored in the storage medium, and the program is loaded and executed by the processor to implement the precise following interaction method based on the Unreal Engine platform as described in the first aspect.

[0046] The beneficial effects of the present application are: displaying the current virtual scene corresponding to the camera position coordinates of the camera of the Unreal Engine platform in the spatial coordinate system through a display screen; obtaining the screen coordinates before and after the mouse movement when a mouse movement operation is received, the screen coordinates being the pixel coordinates on the display screen indicated by the mouse; converting the screen coordinates to the spatial coordinate system to obtain the spatial coordinates before and after the mouse movement; determining the position update parameters of the camera based on the spatial coordinates before and after the mouse movement; updating the camera position based on the position update parameters to trigger the execution of the step of displaying the current virtual scene corresponding to the camera position coordinates of the camera in the spatial coordinate system through the display screen; the problem of low mouse following accuracy of the traditional translation operation method can be solved, and compared with the native matrix transformation method of UnrealEngine, it can significantly reduce errors and improve accuracy and stability.

[0047] In addition, by calculating the projection length of the mouse on the plane or arc surface and updating the camera position, the mouse can be accurately followed in the plane coordinate system or spherical coordinate system. In the spherical coordinate system, after the mouse is dragged, the camera will move in the opposite direction similar to the orbit of a satellite.

[0048] In addition, the mouse precise following interaction method provided in this embodiment is not limited by distance and terrain height, and can accurately follow the mouse for translation, whether it is translation under a large-scale perspective, or on a flat surface, rugged terrain, or in a city building complex. In addition, this technology is also applicable to a variety of terminals, including mobile terminals, VR terminals, and web terminals.

[0049] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a structural diagram of a precise following interaction system based on the Unreal Engine platform provided by an embodiment of the present application;

[0051] Figure 2 is a schematic diagram of a camera mathematical model provided by an embodiment of the present application;

[0052] Figure 3 is a schematic diagram of a mouse position in a near plane provided by an embodiment of the present application;

[0053] Figure 4 is a schematic diagram of a mouse position in a display screen provided by an embodiment of the present application;

[0054] Figure 5Ais a structural diagram of a camera near plane in a top view provided by an embodiment of the present application, Figure 5A (a) in the figure indicates that the mouse is located in the lower right area of ​​the screen. Figure 5A (b) in the figure indicates that the mouse is located in the upper right area of ​​the screen. Figure 5A (c) in the figure indicates that the mouse is located in the upper left area of ​​the screen. Figure 5A (d) in the figure indicates that the mouse is located in the lower left area of ​​the screen;

[0055] Figure 5B is a schematic diagram of a camera pitch angle in a top view provided by an embodiment of the present application;

[0056] Figure 6 is a structural diagram of a camera near plane when a mouse is located at the left half of a horizontal midline of a screen, provided by an embodiment of the present application;

[0057] Figure 7 It is a structural diagram of the camera near plane when the mouse is located at the right half of the horizontal midline of the screen provided by an embodiment of the present application;

[0058] Figure 8 This is a structural diagram of a camera near a plane when the pitch angle increment of a camera provided by an embodiment of the present application is 0° and the camera is in a horizontal state;

[0059] Fig. 9 is a schematic diagram of obtaining a projection vector provided by an embodiment of the present application;

[0060] Fig.10 is a schematic diagram of determining updated camera position coordinates and updated pitch angle provided by an embodiment of the present application;

[0061] Fig.11 is a block diagram of a precise following interaction device based on the Unreal Engine platform provided by one embodiment of the present application;

[0062] Fig.12 It is a block diagram of a precise following interaction device based on the Unreal Engine platform provided by another embodiment of the present application. DETAILED DESCRIPTION

[0063] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but are not intended to limit the scope of the present application.

[0064] This application proposes a precise follow-up interaction method based on the Unreal Engine platform, including a novel method for calculating the precise spatial position of the mouse and a projection algorithm that moves with the mouse. It is particularly suitable for translating plane and spherical maps in Unreal Engine, especially when the camera is far away from the origin of the world coordinate system (or space coordinate system), the accuracy of the operation can still be guaranteed.

[0065] Figure 1 This is a flowchart of a precise follow-up interaction method based on the Unreal Engine platform provided by an embodiment of the present application. This embodiment takes the method applied to an electronic device with image processing capabilities as an example for explanation. The electronic device can be a computer or a virtual reality (VR) device, etc. This embodiment does not limit the implementation of the electronic device. The method includes at least the following steps:

[0066] Step 101: Displaying a current virtual scene corresponding to the camera position coordinates of a camera on the Unreal Engine platform in a spatial coordinate system through a display screen.

[0067] The camera represents the user's perspective, such as how the user views the virtual world. Therefore, the camera is only related to the character controlled by the user in the virtual world.

[0068] Initialize the Unreal Engine 3D scene and camera, and select the spherical or planar coordinate system corresponding to the 3D scene.

[0069] In Unreal Engine, the Cartesian coordinate system is usually used as the spatial coordinate system. This is a three-dimensional rectangular coordinate system that contains three mutually perpendicular axes: X (horizontal), Y (vertical), and Z (depth). Unreal Engine uses a left-handed coordinate system by default, in which the positive direction of the Z axis points outside the display screen, the positive direction of the Y axis points upward, and the positive direction of the X axis points to the right. If you need to use a spherical coordinate system or a planar coordinate system to determine the position of an object in a virtual scene, you can pre-set the conversion relationship between the spherical coordinate system or the planar coordinate system and the spatial coordinate system to achieve mutual conversion between the coordinate position in the spherical coordinate system or the planar coordinate system and the coordinate position in the spatial coordinate system.

[0070] After the camera position coordinates of the camera in the current space coordinate system are determined by initialization, the current virtual scene (ie, the initialized virtual scene) corresponding to the current camera position coordinates can be displayed through the display screen.

[0071] Optionally, the camera position coordinates can be obtained through the Unreal Engine engine interface and expressed as C (Cx, Cy, Cz).

[0072] Step 102: upon receiving a mouse movement operation, obtaining the screen coordinates before and after the mouse movement, the screen coordinates being the pixel coordinates on the display screen indicated by the mouse.

[0073] In one example, the camera mathematical model of a camera in a 3D engine can be expressed by a viewing frustum. Figure 2 The camera mathematical model shown in the figure, the 3D engine rendering process is to project the objects within the range of the visual cone onto the near plane according to the perspective projection relationship, and the picture on the near plane is mapped to the display screen according to a certain proportional relationship. Among them, the visual cone usually refers to a concept in computer graphics, which describes the part of the scene that can be seen from the observer's point of view. In 3D computer graphics, the visual cone (also called the cone or field of view) defines the scene area that the rendering engine needs to render. Objects outside this area do not need to be rendered because they are not in the observer's field of view.

[0074] The distance from the camera to the near plane is the focal length, which is 10cm in the Unreal Engine. The screen coordinates are the pixels on the screen, and there are corresponding mapping points on the near plane.

[0075] In one example, when the electronic device captures a mouse movement event, the screen coordinates before and after the mouse movement are obtained through the Unreal Engine engine interface. For example, the screen coordinates before the mouse movement are represented by S1, and the screen coordinates after the mouse movement are represented by S2.

[0076] Step 103, converting the screen coordinates into a spatial coordinate system to obtain the spatial coordinates before and after the mouse moves.

[0077] In one example, converting the screen coordinates to the space coordinate system where the camera is located to obtain the space coordinates before and after the mouse moves includes the following steps:

[0078] Step 1031, determine the mouse position in the near plane of the screen coordinates.

[0079] refer to Figure 3 A schematic diagram of the mouse position in the near plane shown, and Figure 4 Schematic diagram of the mouse position in the display screen shown. Figure 3In the figure, A represents the position of the mouse in the near plane, D represents the midpoint of the near plane, and B represents the projection point of A on the vertical midline of the near plane. Figure 4 , based on the Unreal Engine engine setting of the ratio of the screen resolution width value / the near plane width (for example, 30; in other embodiments, the ratio may also be other values, and this embodiment does not limit the value of the ratio), and the screen coordinates of the mouse, determine the length of BD and the length of AD.

[0080] BD= - ;

[0081] AB= - ;

[0082] AD= .

[0083] Step 1032, based on the structural diagram of the camera and the near plane in the top view, determine the yaw angle and pitch angle of the camera.

[0084] The top view refers to a structural diagram of the camera and the near plane obtained along the negative direction of the Y axis in the spatial coordinate system.

[0085] refer to Figure 5A The structure diagram of the camera near plane in the top view is shown in FIG. Figure 5A (a) in the figure indicates that the mouse is located in the lower right area of ​​the screen. Figure 5A (b) in the figure indicates that the mouse is located in the upper right area of ​​the screen. Figure 5A (c) in the figure indicates that the mouse is located in the upper left area of ​​the screen. Figure 5A (d) in the figure indicates that the mouse is located in the lower left area of ​​the screen. Figure 5A A” is the projection point of the mouse cursor in the top view; B” is the projection of the midpoint of the near plane in the top view. C” is the projection of the camera in the top view. D” is the projection of A” on the vertical midline of the screen. E is the intersection of the x=B”x line and the y=A”y line. α is the yaw angle of the camera, which is the angle between the line connecting C”B” and the line connecting A”E. β is the pitch angle of the camera, refer to Figure 5B , the pitch angle is the angle between the line connecting the projection point of the camera on the xz plane and the projection point of the center point of the display screen on the xz plane and the x-axis.

[0086] Step 1033, based on the camera position coordinates, the mouse position, the yaw angle and the pitch angle of the camera, determine the spatial coordinates of the mouse before and after movement.

[0087] Based on the camera position coordinates, mouse position, camera yaw angle and pitch angle, the spatial coordinates of the mouse before and after movement are determined, which are expressed by the following coordinate conversion formula:

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] Among them, (Cx, Cy, Cz) represents the camera position coordinates, (Ax, Ay, Az) represents the calculated spatial coordinates of the mouse; CB represents the distance from the camera position coordinates to the midpoint of the near plane; DA (also expressed as AD) represents the distance between the screen position and the midpoint of the near plane; BD represents the distance between the projection point of the mouse position on the vertical midline of the near plane and the midpoint of the near plane; α represents the yaw angle of the camera, and β represents the pitch angle of the camera.

[0094] According to the above coordinate conversion formula, sinβ and BD are located in the denominator. =0, but in mathematics , if we follow the calculation in Unreal Engine, we get , so we need to handle special cases. Including the following two cases:

[0095] 1. The mouse is located on the horizontal center line of the screen, BD=0, which is divided into the following two cases:

[0096] 1.1. When the mouse is located on the left half of the horizontal midline of the screen, BD=0; in the coordinate conversion formula The value is 90°; reference Figure 6 , accordingly, the coordinate transformation formula is transformed into:

[0097] ;

[0098] ;

[0099] .

[0100] 1.2. When the mouse is located on the right half of the horizontal midline of the screen, BD=0; in the coordinate conversion formula The value is -90°; reference Figure 7 , accordingly, the coordinate transformation formula is transformed into:

[0101] ;

[0102] ;

[0103] .

[0104] 2. When the camera's pitch angle is 0° and the camera is in a horizontal state, sinβ=0; Figure 8 , accordingly, the coordinate transformation formula is transformed into:

[0105] ;

[0106] ;

[0107] .

[0108] According to the above formula, the spatial coordinates before the mouse moves and the spatial coordinates after the mouse moves can be calculated.

[0109] Step 104: Determine the position update parameters of the camera based on the spatial coordinates before and after the mouse moves.

[0110] In one example, the current virtual scene is generated based on a plane system; accordingly, based on the spatial coordinates before and after the mouse movement, the camera position update parameters are determined, including: determining the projection vector from the spatial coordinates before the mouse movement to the spatial coordinates after the mouse movement on the virtual ground; subtracting the projection vector from the camera position coordinates to obtain the updated camera position coordinates.

[0111] refer to Fig. 9 As shown in the schematic diagram of obtaining the projection vector, the projection vector tt' of the mouse sliding from the spatial coordinate A1 before the mouse moves to the spatial coordinate A2 after the mouse moves is first calculated. Based on the triangle similarity principle, the projection vector tt' of the mouse moving from position A1 to position A2 on the ground is calculated. Fig. 9 In the figure, C represents the camera position coordinates; D represents the intersection of the horizontal line through A1 and the ray CA2; t represents the intersection of the ray CA1 and the horizontal plane (the world position of the mouse click); t' represents the intersection of the ray CA2 and the horizontal plane (the world position of the mouse click); h represents the intersection of the horizontal line through point A1 and the vertical line through point C; h2 represents the vertical point of point A2 on the line connecting point A1 and point h; h3 represents the vertical projection of point C on the ground.

[0112] Projection vector Calculated by the following formula:

[0113] .

[0114] The calculated projection vector Then, according to the camera original position-projection vector =Updated camera position coordinates, enabling the 3D scene to follow the mouse movement.

[0115] In another example, the current virtual scene is generated based on a spherical system; accordingly, based on the spatial coordinates before and after the mouse moves, the position update parameters of the camera are determined, including: determining the spatial coordinates before the mouse moves and sliding vertically on the display screen to the spatial coordinates after the mouse moves, and projecting them onto two first projection points on the spherical surface; determining the camera pitch angle increment based on the distance between the two first projection points, the radius of the spherical system, the distance between the spatial coordinates before the mouse moves and the camera position coordinates, and the line between the center of the spherical system and the camera position coordinates, and the angle between the line between the spatial coordinates before the mouse moves and the camera position coordinates; subtracting the camera pitch angle increment from the camera pitch angle to obtain an updated pitch angle; determining the updated camera position coordinates based on the distance between the camera position coordinates and the center of the spherical system, the yaw angle and pitch angle of the camera, and the camera pitch angle increment.

[0116] Assuming that the mouse slides only in the vertical direction, only the camera pitch angle changes, the above method can be used to calculate the pitch angle increment. When the mouse slides in the horizontal direction, the pitch angle calculation principle can be reused. When the mouse slides in both vertical and horizontal directions, the sliding process can be disassembled into pure vertical and pure horizontal movements, and the above calculation principle can be reused.

[0117] According to A1 and A2, the updated camera position coordinates and updated pitch angle are calculated. The specific process is as follows:

[0118] refer to Fig.10 , the mouse slides vertically from point A1 to point A2 on the screen and projects onto the sphere Dot and point, yes The projection on the yz plane is yes Projection on the yz plane, the camera pitch angle is β, the yaw angle is α, the camera position is C, the center of the sphere is G, the radius of the sphere is r, calculate the camera pitch angle increment The formula can be expressed as follows:

[0119] ;

[0120] Updated pitch angle = original pitch angle - ;

[0121] Updated camera position coordinates It is expressed by the following formula:

[0122] ;

[0123] ;

[0124] .

[0125] Step 105 , updating the camera position based on the position update parameters, so as to trigger the step of displaying the current virtual scene corresponding to the camera position coordinates in the space coordinate system through the display screen.

[0126] If the mouse continues to move, step 102 is repeated to recalculate the camera position coordinates based on the updated camera position coordinates and the screen coordinates before and after the mouse is moved, so that the target point follows the mouse movement in real time.

[0127] In summary, the precise following interaction method based on the Unreal Engine platform provided in this embodiment displays the current virtual scene corresponding to the camera position coordinates of the camera of the Unreal Engine platform in the spatial coordinate system through a display screen; when a mouse movement operation is received, the screen coordinates before and after the mouse movement are obtained, and the screen coordinates are the pixel coordinates on the display screen indicated by the mouse; the screen coordinates are converted to the spatial coordinate system to obtain the spatial coordinates before and after the mouse movement; based on the spatial coordinates before and after the mouse movement, the position update parameters of the camera are determined; based on the position update parameters, the camera position is updated to trigger the step of displaying the current virtual scene corresponding to the camera position coordinates of the camera in the spatial coordinate system through the display screen; the problem of low mouse following accuracy of the traditional translation operation method can be solved, and compared with the native matrix transformation method of UnrealEngine, it can significantly reduce errors and improve accuracy and stability.

[0128] In addition, by calculating the projection length of the mouse on the plane or arc surface and updating the camera position, the mouse can be accurately followed in the plane coordinate system or spherical coordinate system. In the spherical coordinate system, after the mouse is dragged, the camera will move in the opposite direction similar to the orbit of a satellite.

[0129] In addition, the mouse precise following interaction method provided in this embodiment is not limited by distance and terrain height, and can accurately follow the mouse for translation, whether it is translation under a large-scale perspective, or on a flat surface, rugged terrain, or in a city building complex. In addition, this technology is also applicable to a variety of terminals, including mobile terminals, VR terminals, and web terminals.

[0130] Fig.11This is a block diagram of a precise follow-up interaction device based on the Unreal Engine platform provided by an embodiment of the present application. The device includes at least the following modules: a scene display module 1110, a coordinate acquisition module 1120, a coordinate conversion module 1130, a parameter update module 1140 and a position update module 1150.

[0131] A scene display module 1110 is used to display a current virtual scene corresponding to the camera position coordinates of the camera of the Unreal Engine platform in the space coordinate system through a display screen;

[0132] The coordinate acquisition module 1120 is used to acquire the screen coordinates before and after the mouse movement when receiving the mouse movement operation, wherein the screen coordinates are the pixel coordinates on the display screen indicated by the mouse;

[0133] A coordinate conversion module 1130 is used to convert the screen coordinates into the space coordinate system to obtain the space coordinates before and after the mouse moves;

[0134] A parameter updating module 1140, configured to determine the position update parameters of the camera based on the spatial coordinates before and after the mouse moves;

[0135] The position updating module 1150 is used to update the camera position based on the position updating parameters to trigger the step of displaying the current virtual scene corresponding to the camera position coordinates in the space coordinate system through the display screen.

[0136] For relevant details, refer to the above method embodiment.

[0137] It should be noted that: the precise following interaction device based on the Unreal Engine platform provided in the above embodiment is only illustrated by the division of the above functional modules when performing precise following interaction based on the Unreal Engine platform. In actual application, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the precise following interaction device based on the Unreal Engine platform is divided into different functional modules to complete all or part of the functions described above. In addition, the precise following interaction device based on the UnrealEngine platform provided in the above embodiment and the precise following interaction method embodiment based on the Unreal Engine platform belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0138] Fig.1212 is a block diagram of an accurate follow-up interaction device based on the Unreal Engine platform provided by an embodiment of the present application. The device may be a smart phone, a tablet computer, a laptop computer, a desktop computer or a server. The accurate follow-up interaction device based on the UnrealEngine platform may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, a control terminal, etc., which is not limited in this embodiment. The device includes at least a processor 1201 and a memory 1202.

[0139] The processor 1201 may include one or more processing cores, such as a 4-core processor, a 6-core processor, etc. The processor 1201 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1201 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1201 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1201 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0140] The memory 1202 may include one or more computer-readable storage media, which may be non-transitory. The memory 1202 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1202 is used to store at least one instruction, which is used to be executed by the processor 1201 to implement the precise follow-up interaction method based on the Unreal Engine platform provided in the method embodiment of the present application.

[0141] In some embodiments, the precise follow-up interactive device based on the Unreal Engine platform may optionally include: a peripheral device interface and at least one peripheral device. The processor 1201, the memory 1202 and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Schematically, the peripheral devices include but are not limited to: a radio frequency circuit, a touch display screen, an audio circuit, and a power supply.

[0142] Of course, the precise following interaction device based on the Unreal Engine platform may also include fewer or more components, which is not limited in this embodiment.

[0143] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the precise following interaction method based on the Unreal Engine platform of the above method embodiment.

[0144] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium, wherein the computer-readable storage medium stores a program, and the program is loaded and executed by a processor to implement the precise following interaction method based on the Unreal Engine platform of the above method embodiment.

[0145] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A precise follow-up interaction method based on the Unreal Engine platform, characterized in that: The method comprises: Displaying a current virtual scene corresponding to the camera position coordinates of the camera of the Unreal Engine platform in the space coordinate system on a display screen; the current virtual scene is obtained by projecting virtual objects within the viewing frustum of the camera onto a near plane according to a perspective projection relationship based on the camera position, and mapping the picture on the near plane onto the display screen according to a certain proportional relationship; When a mouse movement operation is received, the screen coordinates before and after the mouse movement are acquired, where the screen coordinates are the pixel coordinates on the display screen indicated by the mouse; Converting the screen coordinates to the spatial coordinate system where the camera is located to obtain the spatial coordinates before and after the mouse moves; including: determining the mouse position of the screen coordinates in the near plane; determining the yaw angle and pitch angle of the camera based on the structural diagram of the camera and the near plane in the top view; determining the spatial coordinates before and after the mouse moves based on the camera position coordinates, the mouse position, the yaw angle and the pitch angle of the camera; Determining the position update parameters of the camera based on the spatial coordinates before and after the mouse moves; The camera position is updated based on the position update parameters to trigger the step of displaying the current virtual scene corresponding to the camera position coordinates in the space coordinate system through the display screen.

2. The method according to claim 1, characterized in that The spatial coordinates of the mouse before and after movement are determined based on the camera position coordinates, the mouse position, the yaw angle and the pitch angle of the camera, and are expressed by the following coordinate conversion formula: ； ； ； ； ； Among them, (Cx, Cy, Cz) represents the camera position coordinates, (Ax, Ay, Az) represents the calculated spatial coordinates of the mouse; CB represents the distance from the camera position coordinates of the camera to the midpoint of the near plane; DA represents the distance between the screen position and the midpoint of the near plane; BD represents the distance between the projection point of the mouse position on the vertical midline of the near plane and the midpoint of the near plane; α represents the yaw angle of the camera, and β represents the pitch angle of the camera.

3. The method according to claim 2, characterized in that When the mouse is located at the left half of the horizontal midline of the screen, BD=0; in the coordinate conversion formula The value is 90°; accordingly, the coordinate conversion formula is transformed into: ； ； 。 4. The method according to claim 2, characterized in that: When the mouse is located at the right half of the horizontal midline of the screen, BD=0; in the coordinate conversion formula The value is -90°; accordingly, the coordinate conversion formula is transformed into: ； ； 。 5. The method according to claim 2, characterized in that: When the pitch angle of the camera is 0° and the camera is in a horizontal state, sinβ= 0; accordingly, the coordinate conversion formula is transformed into: ； ； 。 6. The method according to claim 1, characterized in that The current virtual scene is generated based on a plane system; accordingly, determining the position update parameters of the camera based on the spatial coordinates before and after the mouse moves includes: Determine the projection vector on the virtual ground from the spatial coordinates before the mouse moves to the spatial coordinates after the mouse moves; The projection vector is subtracted from the camera position coordinates to obtain updated camera position coordinates.

7. The method according to claim 1, characterized in that The current virtual scene is generated based on a spherical system; accordingly, determining the position update parameters of the camera based on the spatial coordinates before and after the mouse moves includes: Determine that the spatial coordinates before the mouse moves are vertically slid on the display screen to the spatial coordinates after the mouse moves, and project them onto two first projection points on the spherical surface; Determine the camera pitch angle increment based on the distance between the two first projection points, the radius of the spherical system, the distance between the spatial coordinates before the mouse moves and the camera position coordinates, and the angle between the line between the center of the spherical system and the camera position coordinates and the line between the spatial coordinates before the mouse moves and the camera position coordinates; Subtract the camera pitch angle increment from the camera pitch angle to obtain an updated pitch angle; An updated camera position coordinate is determined based on the distance between the camera position coordinate and the center of the spherical system, the yaw angle and the pitch angle of the camera, and the camera pitch angle increment.

8. A precise following interactive device based on the Unreal Engine platform, characterized in that: The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement the precise following interaction method based on the Unreal Engine platform as claimed in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The storage medium stores a program, and when the program is executed by the processor, it is used to implement the precise following interaction method based on the Unreal Engine platform as claimed in any one of claims 1 to 7.

Images