User interaction method and apparatus, and device, storage medium and product
By identifying and displaying the user's virtual avatar within the visible range of a VR device, the problem of difficulty in user interaction with video images in VR technology is solved, enabling rich visual experiences and virtual social interactions.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- CHINA MOBILEHANGZHOUINFORMATION TECH CO LTD
- Filing Date
- 2025-09-10
- Publication Date
- 2026-05-15
AI Technical Summary
In existing VR technologies, users cannot interact with the video screen when using treadmills and exercise bikes, resulting in an isolated experience.
By acquiring video files corresponding to multiple travel routes, the system identifies and responds to user selections, determines other users within the field of view, and displays their virtual avatars to enable interaction.
It enhances the user experience, breaks the isolation of traditional video viewing, constructs a virtual social space, and enables virtual interaction across spaces.
Smart Images
Figure CN2025120230_15052026_PF_FP_ABST
Abstract
Description
User interaction methods, devices, equipment, storage media and products
[0001] Cross-reference to related applications
[0002] This disclosure claims priority to Chinese Patent Application No. 202411574309.1, filed in China on November 5, 2024, the entire contents of which are incorporated herein by reference. Technical Field
[0003] This disclosure belongs to the field of computer text processing technology, and in particular relates to a user interaction method, apparatus, device, storage medium and product. Background Technology
[0004] The experience mode of playing video footage at the speed of a user's treadmill or stationary bike is now widely used in various running and cycling fitness scenarios. With the rising popularity of the metaverse concept in recent years, hardware devices such as Virtual Reality (VR) glasses are considered key entry points for metaverse interaction, capable of bringing users a completely new sensory experience between the virtual and real worlds. Therefore, immersive running and cycling fitness applications based on VR glasses are emerging in large numbers. Currently, the videos used for synchronized playback on treadmills and stationary bikes mainly include two categories: live-action footage and 3D (3D) modeled and rendered footage.
[0005] Currently, the production cost of high-definition video and 360-degree panoramic video recorded by cameras is relatively low. When combined with 3D glasses, they can give users an immersive experience in a real environment. However, the playback process is only related to the user's cycling and running speed. During cycling and running, users usually only watch a video on their own and cannot interact with the screen much. Summary of the Invention
[0006] This disclosure provides a user interaction method, apparatus, device, storage medium, and product to solve the problem that users are unable to interact with the screen during the use of VR technology.
[0007] In a first aspect, embodiments of this disclosure provide a user interaction method, the method comprising:
[0008] Obtain multiple first video files that correspond one-to-one with multiple travel routes;
[0009] Upon receiving a selection instruction from a first user, a second video file is played on the display terminal. The second video file is the video file selected from the plurality of first video files by the selection instruction.
[0010] Using the currently playing video frame in the second video file, a second user is determined who is within the visual range of the first user. The second user is a user who is playing any video file in the plurality of first video files at the same time as the first user.
[0011] The virtual avatar of the second user is displayed on the display terminal so that the first user can interact with the second user through the virtual avatar.
[0012] Secondly, embodiments of this disclosure provide a user interaction device, the device comprising:
[0013] The first acquisition module is used to acquire multiple first video files that correspond one-to-one with multiple travel routes;
[0014] The playback module is used to play a second video file on a display terminal upon receiving a selection instruction sent by a first user. The second video file is a video file selected from a plurality of first video files by the selection instruction.
[0015] The second acquisition module is used to determine a second user within the visual range of the first user by using the currently playing video frame in the second video file. The second user is a user who is playing any video file in the plurality of first video files at the same time as the first user.
[0016] A display module is used to display the virtual avatar of the second user on the display terminal, so that the first user can interact with the second user through the virtual avatar.
[0017] Thirdly, embodiments of this disclosure provide a terminal device, the device including: a processor and a memory storing computer program instructions;
[0018] The processor implements user interaction methods as described in the first aspect when executing computer program instructions.
[0019] Fourthly, embodiments of this disclosure provide a computer storage medium on which computer program instructions are stored, and when executed by a processor, the computer program instructions implement the user interaction method as described in the first aspect.
[0020] Fifthly, embodiments of this disclosure provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the user interaction method as described in the first aspect.
[0021] The user interaction method provided in this disclosure offers users a rich variety of visual content choices by acquiring video files that match multiple travel routes. When a first user selects and plays a specific video file, the method can recognize and respond to their selection, instantly displaying the selected content on the display terminal, thus enhancing the user experience. It also identifies and acquires information about other users (i.e., second users) who are simultaneously watching different video content with the first user through the current video frame. This not only breaks the isolation of traditional video viewing but also constructs a virtual social space based on shared interests or activity time. Furthermore, by displaying the virtual avatar of the second user, cross-space virtual interaction is achieved, allowing the first user to intuitively experience the coexistence and connection with other users. Through the interaction between the first and second users, with the second user displayed on the playback screen, interaction between the first user and the playback screen is realized. Attached Figure Description
[0022] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments of this disclosure will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 is a flowchart illustrating the user interaction method provided in an embodiment of this disclosure;
[0024] Figure 2 is a schematic diagram of recording multiple travel routes provided in an embodiment of this disclosure;
[0025] Figure 3 is a schematic diagram of the second user in a three-dimensional coordinate system according to an embodiment of this disclosure;
[0026] Figure 4 is a schematic diagram of the structure of the user interaction device provided in an embodiment of this disclosure;
[0027] Figure 5 is a schematic diagram of the structure of the terminal device provided in an embodiment of this disclosure. Detailed Implementation
[0028] The features and exemplary embodiments of various aspects of this disclosure will now be described in detail. To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description, in conjunction with the accompanying drawings and specific embodiments, will provide a further detailed description. It should be understood that the specific embodiments described herein are intended only to explain this disclosure and not to limit it. For those skilled in the art, this disclosure can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this disclosure by illustrating examples.
[0029] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0030] To address the related technical issues, this disclosure provides a user interaction method, apparatus, device, storage medium, and product.
[0031] The user interaction method provided in the embodiments of this disclosure will be introduced first.
[0032] Figure 1 shows a flowchart of a user interaction method provided in an embodiment of this disclosure. As shown in Figure 1, the method includes:
[0033] Step 101: Obtain multiple first video files that correspond one-to-one with multiple travel routes;
[0034] In this embodiment, the disclosed solution is applied to the server side. An application scenario could be that when a first user is running or cycling while wearing a VR device, the VR device projects other users in the same area, also wearing VR devices, onto the first user's VR device, and then the VR device enables interaction between the first and second users.
[0035] As shown in Figure 2, which is a schematic diagram of multiple travel routes, these routes are located within the same area, such as a park or scenic spot. Each travel route can be filmed using a camera device.
[0036] Specifically, in one embodiment, step 101 further includes:
[0037] For any given route, the route is recorded using a camera device, and during the recording process, the longitude, latitude, altitude, and direction of travel for each frame are recorded to obtain the first video file.
[0038] In this embodiment, for any of the multiple travel routes, a camera device can be used to record ordinary high-definition video or panoramic video at a constant speed. During the recording process, the camera height and viewing angle need to remain unchanged. At the same time, the latitude y, longitude x, altitude h and travel direction information of each current frame are recorded into the video frame extension information to obtain the first video file.
[0039] In addition to recording the above data, it is also necessary to record the distance D between the video frame and the camera device in each frame, as well as the height H of the camera device, and store them together with the first video file.
[0040] Step 102: Upon receiving a selection instruction from the first user, play a second video file on the display terminal. The second video file is the video file selected from the plurality of first video files by the selection instruction.
[0041] In this embodiment, the display terminal can be a VR device, such as VR glasses. The first user is a user using the VR device. The first user selects and plays a second video file corresponding to any movement route on the display device. During playback, the display device can collect the first user's movement speed and play the second video file at the playback speed corresponding to that movement speed, thereby achieving synchronization between the first user and the video. The second video file refers to any one of multiple first video files.
[0042] Step 103: Using the currently playing video frame in the second video file, determine the second user who is within the visual range of the first user. The second user is the user who is playing any video file in the plurality of first video files at the same time as the first user.
[0043] In this embodiment, when the first user plays the second video file, the first virtual longitude and first virtual latitude corresponding to the currently played video frame are used to determine the second user within the first user's line of sight. The second user is any user who plays any of the first video files simultaneously with the first user; that is, the second user can be either a user who plays the second video file simultaneously with the first user, or a user who plays other video files besides the second video file among multiple first video files.
[0044] It should be noted that when recording the first video file, each frame includes the longitude, latitude, and altitude at the time of recording. However, when playing the video file, the playback location is not the same as the latitude and longitude of the original video file. Therefore, when playing the first video file, the latitude, longitude, and altitude in the first video file relative to the first user's location are referred to as virtual longitude, virtual latitude, and virtual altitude.
[0045] Specifically, in one embodiment, step 103 includes:
[0046] Obtain a third user who can simultaneously play any video file from the multiple first video files;
[0047] Using the second virtual longitude and second virtual latitude of the third user, and the first virtual longitude and first virtual latitude of the first user, the third user located within the visible range is identified as the second user;
[0048] In this embodiment, the third user can be any user simultaneously playing any video file from multiple first video files. As shown in the above embodiment, the second virtual longitude and second virtual latitude of the third user can be determined by the video frame being played by the third user. Therefore, by using the first virtual longitude and first virtual latitude of the first user to traverse the virtual longitude and latitude of the third user, it is determined whether the third user is within the first user's line of sight. The third user within the line of sight is then identified as the second user. Specifically, the line of sight is defined as the range formed by the straight-line distance between the first virtual longitude and the second virtual longitude being less than a preset viewing distance, and the straight-line distance between the first virtual latitude and the second virtual latitude being less than a preset viewing distance.
[0049] The visible range is represented by the first formula;
[0050] The first formula is:
[0051] A=(|x-x0|<m)∩(|y-y0|<m)
[0052] Where A is the visible range, x is the second virtual longitude, x0 is the first virtual longitude, y is the second virtual latitude, y0 is the second virtual latitude, and m is the preset viewing distance of the first user.
[0053] In addition to using the visible range to identify the second user, the hardware performance rating of the display terminal used by the first user, the first user's occupation and hobbies, and the type of the first video file can be used to further filter for the third user and obtain the second user.
[0054] As an example, the second user could be someone located within the first user's field of vision, sharing the same profession and hobbies, playing either panoramic or standard videos, and having the same hardware performance rating on their display terminal. By selecting a second user with similar interests to the first user, the interactive experience between different users can be improved.
[0055] Step 104: Display the virtual avatar of the second user on the display terminal so that the first user can interact with the second user through the virtual avatar.
[0056] In this embodiment, after the second user is determined, the virtual avatar of the second user can be displayed on the display interface of the display terminal, so that the first user can communicate and interact with the virtual avatar, realizing interaction between two users located in different locations and enhancing the user experience.
[0057] In this embodiment, by acquiring video files matching multiple travel routes, a rich variety of visual content choices are provided to the user. When the first user selects and plays a specific video file, the system can recognize and respond to their selection, instantly displaying the selected content on the display terminal, thus enhancing the user experience. Information about other users (i.e., second users) simultaneously watching different video content with the first user is also obtained through the current video frame. This not only breaks the isolation of traditional video viewing but also constructs a virtual social space based on shared interests or activity time. Based on this, by displaying the virtual avatar of the second user, cross-space virtual interaction is achieved, allowing the first user to intuitively experience the coexistence and connection with other users. Through the interaction between the first and second users, and with the second user displayed on the playback screen of the display device, interaction between the first user and the playback screen is realized.
[0058] In one embodiment of this disclosure, displaying the virtual avatar of the second user on the display terminal includes:
[0059] Based on the video frame and the user information of the second user, the first coordinates of the second user in a three-dimensional coordinate system are obtained. The first coordinates are three-dimensional pixel coordinates, and the three-dimensional coordinate system is constructed based on the second video file.
[0060] The first coordinate is mapped to a second coordinate in the playback screen of the display terminal, where the second coordinate is a two-dimensional pixel coordinate;
[0061] Based on the user information, obtain the pixel distance between the second user and the first user;
[0062] Using the pixel distance, the virtual avatar is displayed at the second coordinate, and the display terminal includes the second coordinate.
[0063] In this embodiment, as shown in Figure 3, which is a schematic diagram of the second user in a three-dimensional coordinate system, the three-dimensional coordinate system is constructed with the center point of the bottom edge of the video rectangle of the second video file as the origin O, the forward direction as the V-axis, the direction perpendicular to the ground as the Z-axis, and the direction parallel to the ground as the U-axis.
[0064] The method for determining the virtual avatar point P in the three-dimensional coordinate system can be obtained by calculating the third virtual longitude, third virtual latitude and third virtual altitude of the second user and obtaining the three-dimensional coordinates of the virtual avatar point P relative to the origin O as (u0, v0, z0), which is the first coordinate.
[0065] Since the display terminal shows a two-dimensional image, the second user's first coordinates need to be mapped to the two-dimensional image in order to display the second user's virtual avatar on the display screen. Therefore, it is also necessary to map the three-dimensional first coordinates to the two-dimensional second coordinates.
[0066] In order to display the second user in the playback screen proportionally based on the actual distance between the first user and the second user, in this embodiment, it is also necessary to obtain the pixel distance between the first user and the second user. This pixel distance is used to adjust the size of the second user's virtual avatar in the playback screen, so that different second users can reasonably display their corresponding virtual avatars in the playback screen based on the actual distance between them and the first user.
[0067] After obtaining the pixel distance, the virtual avatar can be displayed at the second coordinate in the playback screen using this pixel distance, so that the first user can see the virtual avatar in the playback screen.
[0068] In this embodiment, by converting the second user's position in the three-dimensional video space (three-dimensional pixel coordinates) into a specific position on the display terminal's playback screen (two-dimensional pixel coordinates), and combining this with the pixel distance between users, precise display of the virtual avatar on the display terminal is achieved. This not only enhances the immersive experience of video viewing but also greatly improves the interactive experience between users.
[0069] In one embodiment of this disclosure, obtaining the first coordinates of the second user in a three-dimensional coordinate system based on the user information includes:
[0070] According to the second formula, obtain the first coordinates of the second user in the three-dimensional coordinate system;
[0071] The second formula is:
[0072] Where x is the third virtual longitude, y is the third virtual latitude, h is the third virtual altitude, x0 is the first virtual longitude, y0 is the first virtual latitude, h0 is the first virtual altitude, γ is the angle between the first virtual longitude and the direction of travel, and (u0, v0, z0) are the first coordinates.
[0073] In this embodiment, the user information includes a third virtual longitude, a third virtual latitude, and a third virtual altitude; the video frame includes the first user's first virtual longitude, first virtual latitude, first virtual altitude, and direction of travel.
[0074] It should be noted that since the second user is also the user who played the first video file, the principle for obtaining the first virtual longitude, first virtual latitude, and first virtual altitude is the same as that for the first user. The user information of the second user also includes the corresponding virtual longitude, latitude, and altitude.
[0075] In this embodiment, by using the second formula to obtain the first coordinates of the second user in the three-dimensional coordinate system, the accurate display of the virtual avatar in the subsequent playback screen can be guaranteed.
[0076] In one embodiment of this disclosure, mapping the first coordinates to a second coordinate in the playback screen of the display terminal includes:
[0077] The first view rotation angle and the second view rotation angle of the first user are obtained. The first view rotation angle is the left and right rotation angle of the first user in the three-dimensional pixel coordinate system, and the second view rotation angle is the up and down rotation angle of the first user in the three-dimensional pixel coordinate system.
[0078] The first coordinate is rotated by the first viewpoint rotation angle to obtain the third coordinate;
[0079] The third coordinate is rotated by the second viewpoint rotation angle to obtain the fourth coordinate;
[0080] Using a preset focal length, the fourth coordinate is mapped to the second coordinate in the playback screen, where the focal length is the distance between the playback screen and the camera device recording the first video file.
[0081] In this embodiment, since the first user's viewpoint rotates up, down, left, and right, when a change in the first user's viewpoint is detected, the first coordinate needs to be rotated by the corresponding viewpoint rotation angle. Specifically, this viewpoint rotation angle can be monitored by the display terminal and then uploaded to the server via the display terminal to participate in the coordinate transformation calculation.
[0082] The first-view rotation angle α is the angle of rotation along the Z-axis in the three-dimensional pixel coordinate system, and the second-view rotation angle β is the angle of rotation along the U-axis in the three-dimensional pixel coordinate system.
[0083] The specific rotation process is as follows: After the first coordinate is rotated around the Z-axis by an angle α, the new third coordinates u1, v1, z1 are calculated using the following method:
[0084] The third coordinate is rotated vertically to obtain the fourth coordinate u2, v2, z2. The calculation method after rotating β around the U-axis is as follows:
[0085] Alternatively, in this embodiment, the rotation can be performed up and down first, followed by the rotation to the left and right; or only the rotation up and down or the rotation to the left and right can be performed. This embodiment does not impose any restrictions on this.
[0086] After a series of rotations around the first coordinate, the fourth coordinate is obtained. This fourth coordinate is then mapped to the second coordinate in the playback screen. The focal length mentioned above refers to the distance between the camera device and its focal point when recording the first video file. However, since there is no focal point during playback, this focal length is the distance between the playback screen and the camera device. The height mentioned above is the height between the camera device and the ground plane.
[0087] In one embodiment, the fourth coordinate is mapped to the second coordinate in the playback screen using a third formula;
[0088] The third formula is:
[0089] Where (u2, v2, z2) is the fourth coordinate, (u, v) is the second coordinate, D is the focal length, and zc is the height.
[0090] It should also be noted that if the first coordinate is rotated only by the first viewpoint rotation angle, the third coordinate can be used to replace the fourth coordinate for mapping to obtain the second coordinate; if the first coordinate is rotated only by the second viewpoint rotation angle, the coordinate after rotation by the second viewpoint rotation angle can be used for mapping to obtain the second coordinate.
[0091] The coordinate transformation and mapping process described above is based on the premise that the first video file is a panoramic video. If the first video file is a regular video, the third formula can be used directly to map the first coordinate to the second coordinate, without any coordinate transformation.
[0092] In this embodiment, the third formula is used to map the fourth coordinate to the second coordinate in the playback screen, which can convert the three-dimensional coordinate into a two-dimensional coordinate, so that it can be displayed in the playback screen of the display terminal.
[0093] In one embodiment of this disclosure, displaying the virtual avatar at the second coordinates using the pixel distance includes:
[0094] The pixel size of the virtual avatar at the second coordinate is adjusted using the pixel distance, and the display effect of the virtual avatar at the second coordinate is adjusted using the second user's walking direction and speed.
[0095] In this embodiment, the pixel distance is calculated in the following way:
[0096] Obtain the actual distance between the first user and the second user, the scaling ratio of the playback screen, and the number of pixels per inch in the playback screen;
[0097] Based on the actual distance, scaling ratio, and number of pixels, the pixel distance is calculated using the fourth formula.
[0098] The fourth formula is:
[0099] Where L is the actual distance, S is the scaling ratio, and Dpi is the number of pixels.
[0100] After determining the pixel distance, the second user's original height is displayed inversely proportional to the pixel distance. The virtual avatar is also given the same display effect as the second user's movement direction and speed, thus achieving synchronization between the virtual avatar and the second user.
[0101] In this embodiment, when the virtual avatar is displayed at the second coordinate, multiple factors such as pixel distance, direction of travel, and speed are fully considered to achieve a more dynamic and realistic display effect.
[0102] As shown in Figure 4, this embodiment of the present disclosure also provides a user interaction device 400, the device comprising:
[0103] The first acquisition module 401 is used to acquire multiple first video files that correspond one-to-one with multiple travel routes;
[0104] The playback module 402 is used to play a second video file on a display terminal when a selection instruction is received from a first user. The second video file is a video file selected from a plurality of first video files by the selection instruction.
[0105] The second acquisition module 403 is used to determine a second user within the visual range of the first user by using the currently playing video frame in the second video file. The second user is a user who plays any video file in the plurality of first video files at the same time as the first user.
[0106] Display module 404 is used to display the virtual avatar of the second user on the display terminal so that the first user can interact with the second user through the virtual avatar.
[0107] Optionally, the second acquisition module 403 includes:
[0108] The first acquisition submodule is used to acquire a third user who is simultaneously playing any video file among the multiple first video files;
[0109] The first determining submodule is used to determine the third user located within the visible range as the second user by using the second virtual longitude and second virtual latitude of the third user, and the first virtual longitude and first virtual latitude of the first user;
[0110] The visible range is represented by a first formula;
[0111] The first formula is:
[0112] A=(|x-x0|<m)∩(|y-y0|<m)
[0113] Where A is the visible range, x is the second virtual longitude, x0 is the first virtual longitude, y is the second virtual latitude, y0 is the second virtual latitude, and m is the preset viewing distance of the first user.
[0114] Optionally, the display module 404 includes:
[0115] The second acquisition submodule is used to acquire the first coordinates of the second user in a three-dimensional coordinate system based on the video frame and the user information of the second user. The first coordinates are three-dimensional pixel coordinates, and the three-dimensional coordinate system is constructed based on the second video file.
[0116] The mapping submodule is used to map the first coordinates to a second coordinate in the playback screen of the display terminal, wherein the second coordinate is a two-dimensional pixel coordinate;
[0117] The third acquisition submodule is used to acquire the pixel distance between the second user and the first user based on the user information;
[0118] The display submodule is used to display the virtual avatar at the second coordinate using the pixel distance, and the display terminal includes the second coordinate.
[0119] Optionally, the second acquisition submodule is specifically used for:
[0120] According to the second formula, obtain the first coordinates of the second user in the three-dimensional coordinate system;
[0121] The second formula is:
[0122] Where x is the third virtual longitude, y is the third virtual latitude, h is the third virtual altitude, x0 is the first virtual longitude, y0 is the first virtual latitude, h0 is the first virtual altitude, γ is the angle between the first virtual longitude and the direction of travel, and (u0, v0, z0) are the first coordinates.
[0123] Optionally, the mapping submodule includes:
[0124] The acquisition unit is used to acquire the first view rotation angle and the second view rotation angle of the first user. The first view rotation angle is the left-right rotation angle of the first user in the three-dimensional pixel coordinate system, and the second view rotation angle is the up-down rotation angle of the first user in the three-dimensional pixel coordinate system.
[0125] The first rotation unit is used to rotate the first coordinate by the first viewpoint rotation angle to obtain the third coordinate;
[0126] The second rotation unit is used to rotate the third coordinate by the second viewpoint rotation angle to obtain the fourth coordinate;
[0127] The mapping unit is used to map the fourth coordinate to the second coordinate in the playback screen using a preset focal length and height. The focal length is the distance between the playback screen and the camera device recording the first video file, and the height is the height between the camera device and the ground plane.
[0128] Optionally, the mapping unit is specifically used for:
[0129] Using the third formula, the fourth coordinate is mapped to the second coordinate in the playback screen;
[0130] The third formula is:
[0131] Where (u2, v2, z2) is the fourth coordinate, (u, v) is the second coordinate, D is the focal length, and zc is the height.
[0132] Optionally, display submodules are used specifically for:
[0133] The pixel size of the virtual avatar at the second coordinate is adjusted using the pixel distance, and the display effect of the virtual avatar at the second coordinate is adjusted using the second user's walking direction and speed.
[0134] Optionally, the first acquisition module 401 is specifically used for:
[0135] For any given route, the route is recorded using a camera device, and during the recording process, the longitude, latitude, altitude, and direction of travel for each frame are recorded to obtain the first video file.
[0136] It should be noted that the user interaction device 400 is a device corresponding to the above-described user interaction method. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.
[0137] Figure 5 shows a schematic diagram of the hardware structure of the terminal device provided in an embodiment of this disclosure.
[0138] The terminal device may include a processor 501 and a memory 502 storing computer program instructions.
[0139] Specifically, the processor 501 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this disclosure.
[0140] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.
[0141] In a particular embodiment, memory 502 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.
[0142] The processor 501 implements any of the user interaction methods described in the above embodiments by reading and executing computer program instructions stored in the memory 502.
[0143] In one example, the terminal device may also include a communication interface 505 and a bus 510. As shown in Figure 5, the processor 501, memory 502, and communication interface 505 are connected via the bus 510 and communicate with each other.
[0144] The communication interface 505 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this disclosure.
[0145] Bus 510 includes hardware, software, or both, that couples components of an online data flow metering device together. For example, and not limited to, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCI-X or PCI-Express) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 510 may include one or more buses. Although specific buses are described and illustrated in embodiments of this disclosure, this disclosure contemplates any suitable bus or interconnection.
[0146] Furthermore, in conjunction with the user interaction methods in the above embodiments, this disclosure can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the user interaction methods in the above embodiments.
[0147] It should be clarified that this disclosure is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this disclosure is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this disclosure.
[0148] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this disclosure are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0149] It should also be noted that the exemplary embodiments mentioned in this disclosure describe methods or systems based on a series of steps or apparatus. However, this disclosure is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0150] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable user interactive device to produce a machine such that these instructions, executable via the processor of the computer or other programmable user interactive device, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.
[0151] The above are merely specific embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and these modifications or substitutions should all be covered within the protection scope of this disclosure.
Claims
1. A user interaction method, the method comprising: Obtain multiple first video files that correspond one-to-one with multiple travel routes; Upon receiving a selection instruction from a first user, a second video file is played on the display terminal. The second video file is the video file selected from the plurality of first video files by the selection instruction. Using the currently playing video frame in the second video file, a second user is determined who is within the visual range of the first user. The second user is a user who is playing any video file in the plurality of first video files at the same time as the first user. The virtual avatar of the second user is displayed on the display terminal so that the first user can interact with the second user through the virtual avatar.
2. The user interaction method as described in claim 1, wherein, The video frame includes the first virtual longitude and the first virtual latitude of the first user; The step of determining the second user within the first user's field of vision using the currently playing video frame in the second video file includes: Obtain a third user who can simultaneously play any video file from the multiple first video files; Using the second virtual longitude and second virtual latitude of the third user, and the first virtual longitude and first virtual latitude of the first user, the third user located within the visible range is identified as the second user; The visible range is represented by a first formula; The first formula is: A=(|x-x0|<m)∩(|y-y0|<m) Where A is the visible range, x is the second virtual longitude, x0 is the first virtual longitude, y is the second virtual latitude, y0 is the second virtual latitude, and m is the preset viewing distance of the first user.
3. The user interaction method as described in claim 1, wherein, Displaying the virtual avatar of the second user on the display terminal includes: Based on the video frame and the user information of the second user, the first coordinates of the second user in a three-dimensional coordinate system are obtained. The first coordinates are three-dimensional pixel coordinates, and the three-dimensional coordinate system is constructed based on the second video file. The first coordinate is mapped to a second coordinate in the playback screen of the display terminal, where the second coordinate is a two-dimensional pixel coordinate; Based on the user information, obtain the pixel distance between the second user and the first user; Using the pixel distance, the virtual avatar is displayed at the second coordinate, and the display terminal includes the second coordinate.
4. The user interaction method as described in claim 3, wherein, The user information includes a third virtual longitude, a third virtual latitude, and a third virtual altitude; the video frame includes the first user's first virtual longitude, first virtual latitude, first virtual altitude, and direction of travel; The step of obtaining the first coordinates of the second user in the three-dimensional coordinate system based on the user information includes: According to the second formula, obtain the first coordinates of the second user in the three-dimensional coordinate system; The second formula is: Where x is the third virtual longitude, y is the third virtual latitude, h is the third virtual altitude, x0 is the first virtual longitude, y0 is the first virtual latitude, h0 is the first virtual altitude, γ is the angle between the first virtual longitude and the direction of travel, and (u0, v0, z0) are the first coordinates.
5. The user interaction method as described in claim 3, wherein, The step of mapping the first coordinates to the second coordinates in the playback screen of the display terminal includes: The first view rotation angle and the second view rotation angle of the first user are obtained. The first view rotation angle is the left and right rotation angle of the first user in the three-dimensional pixel coordinate system, and the second view rotation angle is the up and down rotation angle of the first user in the three-dimensional pixel coordinate system. The first coordinate is rotated by the first viewpoint rotation angle to obtain the third coordinate; The third coordinate is rotated by the second viewpoint rotation angle to obtain the fourth coordinate; Using a preset focal length and height, the fourth coordinate is mapped to the second coordinate in the playback screen. The focal length is the distance between the playback screen and the camera device recording the first video file, and the height is the height between the camera device and the ground plane.
6. The user interaction method as described in claim 5, wherein, The step of mapping the fourth coordinate to the second coordinate in the playback screen includes: Using the third formula, the fourth coordinate is mapped to the second coordinate in the playback screen; The third formula is: Where (u2, v2, z2) is the fourth coordinate, (u, v) is the second coordinate, D is the focal length, and zc is the height.
7. The user interaction method as described in claim 3, wherein, The step of displaying the virtual avatar at the second coordinate using the pixel distance includes: The pixel size of the virtual avatar at the second coordinate is adjusted using the pixel distance, and the display effect of the virtual avatar at the second coordinate is adjusted using the second user's walking direction and speed.
8. The user interaction method as described in claim 1, wherein, The step of obtaining multiple first video files corresponding one-to-one with multiple travel routes includes: For any given route, the route is recorded using a camera device, and during the recording process, the longitude, latitude, altitude, and direction of travel for each frame are recorded to obtain the first video file.
9. A user interaction device, the device comprising: The first acquisition module is used to acquire multiple first video files that correspond one-to-one with multiple travel routes; The playback module is used to play a second video file on a display terminal upon receiving a selection instruction sent by a first user. The second video file is a video file selected from a plurality of first video files by the selection instruction. The second acquisition module is used to determine, by using the currently playing video frame in the second video file, a second user who is within the visual range of the first user, and the second user is a user who is playing any video file in the plurality of first video files at the same time as the first user. A display module is used to display the virtual avatar of the second user on the display terminal, so that the first user can interact with the second user through the virtual avatar.
10. A terminal device, the device comprising: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the user interaction method as described in any one of claims 1-8.
11. A computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the user interaction method as described in any one of claims 1-8.
12. A computer program product comprising a computer program that, when executed by a processor, implements the user interaction method according to any one of claims 1-8.