Ray generation method, device, equipment and medium

By synchronizing the motion data of the data transmission module between the interactive controller and the controller model, an accurate target ray is generated, which solves the problem of insufficient accuracy of the interactive controller in selecting distant virtual objects in the virtual scene and improves the user operation experience.

CN118897618APending Publication Date: 2024-11-05BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310491513.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-04
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

The ray generated by the interactive controller has poor accuracy when selecting distant virtual objects in the virtual scene, mainly due to errors and data loss during data transmission.

Method used

By acquiring motion data from the first data transmission module of the interactive controller, the second data transmission module in the controller model is synchronized to generate an accurate target ray, reducing errors during data transmission.

Benefits of technology

It improves the accuracy of virtual object selection and enhances the user's human-computer interaction experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118897618A_ABST
    Figure CN118897618A_ABST
Patent Text Reader

Abstract

The invention provides a ray generation method and device, equipment and a medium. The method comprises the following steps: acquiring motion data of a first data transmission module in an interaction controller; according to the motion data of the first data transmission module, motion data of a second data transmission module in a controller model is subjected to synchronous processing, and the controller model corresponds to the interaction controller and is located in the virtual scene; and generating a target ray for human-computer interaction according to the current motion data of the second data transmission module. According to the method and the device, the controller model in the virtual scene can obtain the accurate data used for generating the ray, and the accurate ray is generated according to the data, so that the selection accuracy of a remote virtual object can be improved based on the accurate ray, and the man-machine interaction operation experience of a user is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of human-computer interaction technology, and in particular to a ray generation method, device, equipment and medium. Background Art

[0002] As an indispensable accessory of Extended Reality (XR) devices, the interactive controller provides strong support for interactive control. At present, when using the interactive controller for human-computer interaction, the interactive controller is mainly used for ray interaction. For example, when using the interactive controller to control the corresponding controller model for far-field interaction in a virtual scene, the user can use the interactive controller to control the controller model to send a centered ray to the distant virtual object to select the distant virtual object, and move or rotate the distant virtual object to achieve interaction with the virtual object.

[0003] However, when the corresponding controller model is controlled by the interactive controller to interact with the virtual object, the controller model has poor accuracy in selecting distant virtual objects based on the rays generated by the controller model based on the acquired data, because the data used to generate rays obtained from the interactive controller end has errors. Summary of the invention

[0004] The embodiments of the present application provide a ray generation method, apparatus, device and medium, so that a controller model located in a virtual scene can obtain accurate data for generating rays, and generate an accurate ray based on the data, so as to improve the selection accuracy of distant virtual objects based on the accurate ray, thereby improving the user's human-computer interaction operation experience.

[0005] In a first aspect, an embodiment of the present application provides a ray generation method, including:

[0006] Acquire motion data of a first data transmission module in the interactive controller;

[0007] According to the motion data of the first data transmission module, synchronously processing the motion data of the second data transmission module in the controller model, wherein the controller model corresponds to the interactive controller and the controller model is located in the virtual scene;

[0008] A target ray for human-computer interaction is generated according to the current motion data of the second data transmission module.

[0009] In a second aspect, an embodiment of the present application provides a ray generating device, including:

[0010] A data acquisition module, used to acquire motion data of the first data transmission module in the interactive controller;

[0011] a data synchronization module, configured to synchronize the motion data of the second data transmission module in the controller model according to the motion data of the first data transmission module, wherein the controller model corresponds to the interactive controller and is located in the virtual scene;

[0012] The ray generation module is used to generate a target ray for human-computer interaction according to the current motion data of the second data transmission module.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0014] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the ray generation method as described in the embodiment of the first aspect or its various implementation methods.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the ray generation method as described in the embodiment of the first aspect or its various implementation methods.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product comprising program instructions, which, when executed on an electronic device, enables the electronic device to execute the ray generation method as described in the embodiment of the first aspect or its various implementations.

[0017] The technical solution disclosed in the embodiments of the present application has at least the following beneficial effects:

[0018] By acquiring the motion data of the first data transmission module in the interactive controller, the motion data of the second data transmission module in the controller model corresponding to the interactive controller is synchronously processed according to the motion data of the first data transmission module, and then the target ray for human-computer interaction is generated according to the motion data after the synchronous processing by the second data transmission module, that is, the current motion data. In this way, the controller model located in the virtual scene can obtain accurate data for generating rays, and generate an accurate ray according to the data, so that the selection accuracy of distant virtual objects can be improved based on the accurate ray, thereby improving the user's human-computer interaction operation experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of a process flow of a ray generation method provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of creating a handle model corresponding to a handle in a virtual scene provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a flow chart of another ray generation method provided in an embodiment of the present application;

[0023] Figure 4 A schematic block diagram of a ray generating device provided in an embodiment of the present application;

[0024] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. According to the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0028] As an indispensable interactive accessory of XR devices, the interactive controller is widely used in various interactive scenarios. At present, when using the interactive controller for various human-computer interactions, the interactive controller is usually used for ray interaction. However, when the interactive controller is used to control the controller model located in the virtual scene to interact with the virtual object, because the data for generating rays sent by the interactive controller to the controller model needs to be transmitted to the controller model through some intermediate transmission components, there will be data transmission loss or errors introduced due to a series of data conversion operations, etc., which makes it impossible for the controller model to obtain accurate data for generating rays, resulting in poor accuracy of the controller model to select distant virtual objects based on the rays generated by the inaccurate data obtained. To this end, the present application provides a method for generating an accurate ray corresponding to the real scene interaction operation, so that the selection accuracy of distant virtual objects can be improved based on the ray, thereby improving the user's human-computer interaction operation experience.

[0029] In order to facilitate understanding of the embodiments of the present application, before describing each embodiment of the present application, some concepts involved in all embodiments of the present application are first appropriately explained, as follows:

[0030] 1) Virtual Reality (VR), a technology for creating and experiencing a virtual world, generates a virtual environment by computation, and is a multi-source information (the virtual reality mentioned in this article includes at least visual perception, and can also include auditory perception, tactile perception, motion perception, and even taste perception, olfactory perception, etc.), realizing the fusion of virtual environment, interactive three-dimensional dynamic vision and simulation of entity behavior, allowing users to immerse themselves in a simulated virtual reality environment, and realizing applications in various virtual environments such as maps, games, videos, education, medical care, simulation, collaborative training, sales, assisted manufacturing, maintenance and repair.

[0031] 2) Virtual reality devices (VR devices), terminals for realizing virtual reality effects, can usually be provided in the form of glasses, helmet-mounted displays (Head Mount Display, HMD), and contact lenses to realize visual perception and other forms of perception. Of course, the form of virtual reality devices is not limited to this, and can be further miniaturized or enlarged according to actual needs.

[0032] Optionally, the virtual reality devices described in the embodiments of the present application may include but are not limited to the following types:

[0033] 2.1) PC-based virtual reality (PCVR) devices use the PC to perform relevant calculations and data output for virtual reality functions. External PC-based virtual reality devices use the data output by the PC to achieve virtual reality effects.

[0034] 2.2) Mobile virtual reality devices support the configuration of mobile terminals (such as smart phones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations of virtual reality functions and outputs data to the mobile virtual reality device, such as watching virtual reality videos through the mobile terminal's APP.

[0035] 2.3) All-in-one virtual reality devices have a processor for performing relevant calculations of virtual functions, and thus have independent virtual reality input and output functions. They do not need to be connected to a PC or mobile terminal and have a high degree of freedom in use.

[0036] 3) Augmented Reality (AR): A technology that calculates the camera's posture parameters in the real world (or three-dimensional world, real world) in real time during the process of the camera capturing images, and adds virtual elements to the images captured by the camera based on the camera posture parameters. Virtual elements include but are not limited to: images, videos, and three-dimensional models. The goal of AR technology is to integrate the virtual world with the real world on the screen for interaction.

[0037] 4) Mixed Reality (MR): A simulated setting that integrates computer-created sensory input (e.g., virtual objects) with sensory input from a physical setting or its representation. In some MR settings, the computer-created sensory input can adapt to changes in sensory input from the physical setting. In addition, some electronic systems used to present MR settings can monitor the orientation and / or position relative to the physical setting so that virtual objects can interact with real objects (i.e., physical elements from the physical setting or their representations). For example, the system can monitor movement so that virtual plants appear stationary relative to physical buildings.

[0038] 5) Extended Reality (XR) refers to all real and virtual combined environments and human-computer interactions generated by computer technology and wearable devices, which include virtual reality (VR), augmented reality (AR) and mixed reality (MR) and other forms.

[0039] 6) A virtual scene (also known as a virtual space) is a virtual scene displayed (or provided) when an application is running on an electronic device. The virtual scene can be a simulation of the real world, a semi-simulated and semi-fictitious virtual scene, or a purely fictitious virtual scene. The virtual scene can be any of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiments of the present application do not limit the dimensions of the virtual scene. For example, a virtual scene can include the sky, land, ocean, etc., and the land can include environmental elements such as deserts and cities.

[0040] 7) Virtual objects (also known as virtual objects) are objects that interact in virtual scenes and are controlled by users or robot programs (for example, robot programs based on artificial intelligence) and can be stationary, move, and perform various behaviors in virtual scenes.

[0041] After introducing some concepts involved in the embodiments of the present application, a ray generation method, device, equipment and medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0042] Figure 1 A flowchart of a ray generation method provided in an embodiment of the present application. The ray generation method provided in the present application can be performed by a ray generation device. The ray generation device can be composed of hardware and / or software and can be integrated into an electronic device. In the embodiment of the present application, the electronic device can be any hardware device that can provide a virtual scene, such as an XR device, etc., which is not specifically limited here. Among them, the XR device can be but is not limited to: a VR device, an AR device, and an MR device.

[0043] like Figure 1 As shown, the method may include the following steps:

[0044] S101, obtaining motion data of a first data transmission module in an interactive controller.

[0045] In the present application, the interactive controller may be selected as but not limited to at least one of the following: a handle, a glove, a finger ring, a wristband, and a hand controller.

[0046] Furthermore, usually, a first data transmission module for acquiring the position information and attitude information of the interactive controller is provided inside the interactive controller. In the present application, the first data transmission module is preferably a first inertial measurement unit (IMU), also called a first inertial sensor.

[0047] It should be understood that an inertial measurement unit is a device that measures the angular velocity and acceleration of an object in three-dimensional space. The main components of an inertial measurement unit are gyroscopes, accelerometers, and magnetometers. In general, an inertial measurement unit can generally include three single-axis gyroscopes and three single-axis accelerometers, wherein the gyroscope can detect the angular velocity of an object, and the accelerometer can detect the acceleration of an object, so the data directly measured by the inertial measurement unit is acceleration and angular velocity, but the inertial measurement unit can achieve indirect measurement values ​​through integration, that is, the inertial measurement unit can integrate the angular velocity once to calculate the angle, that is, calculate the relative attitude, and integrate the acceleration twice to obtain the distance, that is, calculate the relative position. Therefore, the motion data of the first data transmission module in the interactive controller can represent the relative position and relative attitude of the interactive controller.

[0048] In addition, when the first data transmission module is set in the interactive controller, the positive direction of the first data transmission module can also be set, so that when the interactive controller moves, the interactive controller and / or the head-mounted display HMD (or head-mounted display device) connected to the interactive controller for communication can use the positive direction of the first data transmission module as a reference to determine the direction information (angle information, or posture information) of the interactive controller.

[0049] The positive direction of the first data transmission module may be any direction as the positive direction of the first data transmission module; or the front of the first data transmission module may be determined as the positive direction of the first data transmission module. It should be noted that the front of the first data transmission module here can be understood as the positive direction of any coordinate axis in the coordinate system of the first data transmission module, etc., and no specific limitation is imposed here.

[0050] That is to say, the motion data of the first data transmission module includes at least one of: position information, posture information and positive direction information.

[0051] It should be understood that when the motion data of the first data transmission module is position information, it means that the current operation data of the interactive controller has only changed position compared to the operation data at the previous moment, such as only translation, while the positive direction and posture remain unchanged, that is, the positive direction and posture have not changed. At this time, only sending the position information of the first data transmission module can not only reduce the amount of data transmission and improve the data transmission efficiency, but also further improve the utilization rate of the positive direction information and posture information in the first data transmission module at the previous moment.

[0052] When the motion data of the first data transmission module is positive direction information, it means that the current motion data of the interactive controller has only changed in direction, such as rotating in place, compared with the motion data at the previous moment, that is, the positive direction changes as the posture of the first data transmission module changes, but the position information remains unchanged. At this time, only the posture information of the first data transmission module can be sent to reversely determine the positive direction information of the first data transmission module based on the posture information. Because the posture information can be determined based on the positive direction information, only the positive direction information of the first data transmission module can be sent at this time. Thereby, the number of data transmissions can be reduced to improve the data transmission efficiency. In order to reduce the subsequent cumbersome calculations of determining the corresponding posture information based on the positive direction information of the first data transmission module, or determining the corresponding positive direction information based on the posture information, the positive direction information and posture information of the first data transmission module can also be sent optionally.

[0053] When the motion data of the first data transmission module is position information and positive direction information, it means that the current operation data of the interactive controller has not only changed in position but also in posture compared to the operation data at the previous moment, such as an angle change based on the translational motion. Then it is necessary to send the position information and positive direction information of the first data transmission module. Alternatively, the position information and posture information of the first data transmission module can also be sent; of course, the present application can optionally send the position information, positive direction information and posture information of the first data transmission module, thereby reducing the subsequent tedious calculations for determining the corresponding posture information based on the positive direction information of the first data transmission module.

[0054] Optionally, when the present application obtains the motion data of the first data transmission module in the interactive controller, the head mounted display HMD in the electronic device may communicate with the first data transmission module in the interactive controller to obtain the motion data of the first data transmission module.

[0055] Among them, the head-mounted display HMD can provide various virtual scenes to the user, and present a controller model corresponding to the interactive controller in the virtual scene.

[0056] S102, synchronously processing the motion data of the second data transmission module in the controller model according to the motion data of the first data transmission module, wherein the controller model corresponds to the interactive controller and the controller model is located in the virtual scene.

[0057] It should be understood that the controller model in the present application is a virtual model corresponding to the interactive controller and located in the virtual scene.

[0058] The controller model may have the same shape, size and orientation as the interaction controller.

[0059] For example, when the interactive controller is a handle, the controller model in the virtual scene is the handle model. And when the handle is a left handle, the handle model is a left handle model corresponding to the left handle, and a right handle model corresponding to the right handle, such as Figure 2 shown.

[0060] For another example, when the interactive controller is a hand controller, the controller model in the virtual scene is the hand controller model. Also, when the hand controller is a left-hand hand controller, the hand controller model is a left-hand controller model corresponding to the left-hand hand controller, and a right-hand controller model corresponding to the right-hand hand controller.

[0061] Since the controller model is designed and created based on the interactive controller, a second data transmission module identical to the first data transmission module in the interactive controller is also provided in the controller model. Thus, the motion of the controller model can be determined through the second data transmission module.

[0062] Among them, the second data transmission module is preferably a second inertial measurement unit IMU, also called a second inertial sensor.

[0063] In order to ensure that the controller model in the virtual scene can achieve fully synchronized interaction with the interactive controller in the real scene (real space). For example, when the interactive controller moves to the left, the controller model in the virtual scene also moves to the left synchronously, etc. To this end, after obtaining the motion data of the first data transmission module in the interactive controller, the present application can synchronize the motion data of the second data transmission module in the controller model based on the motion data of the first data transmission module.

[0064] Considering that the motion data of the first data transmission module includes at least one of: position information, posture information and positive direction information. Therefore, the present application performs synchronous processing on the motion data of the second data transmission module in the controller model based on the motion data of the first data transmission module, which may include at least one of the following situations:

[0065] Scenario 1

[0066] According to the position information of the first data transmission module, the position information of the second data transmission module in the controller model is adjusted to the position information of the first data transmission module.

[0067] Scenario 2

[0068] According to the posture information of the first data transmission module, the posture information of the second data transmission module in the controller model is adjusted to the posture information of the first data transmission module.

[0069] Scenario 3

[0070] According to the positive direction information of the first data transmission module, the positive direction information of the second data transmission module in the controller model is adjusted to the positive direction information of the first data transmission module.

[0071] It should be noted that the implementation principles of the above three situations correspond to the motion data of the first data transmission module obtained above, that is, they can be implemented separately or in combination with each other, and no limitation is imposed here.

[0072] In other words, if only the position information of the first data transmission module is obtained, only the position data of the second data transmission module in the controller model is synchronized, and the positive direction information and posture information in the second data transmission module continue to use the positive direction information and posture information at the previous moment; if only the positive direction information of the first data transmission module is obtained, only the positive direction information of the second data transmission module in the controller model is synchronized, and the position information and posture information in the second data transmission module continue to use the position information and posture information at the previous moment; if only the posture information of the first data transmission module is obtained, only the second data transmission module in the controller model is synchronized. The position information and positive direction information of the transmission module are synchronously processed, while the position information and positive direction information in the second data transmission module continue to use the position information and positive direction information of the previous moment; if the position information and positive direction information of the first data transmission module are obtained, the position information and positive direction information of the second data transmission module in the controller model are synchronously processed, while the posture information in the second data transmission module continues to use the posture information of the previous moment; if the position information, positive direction information and posture information of the first data transmission module are obtained, the position information, positive direction information and posture information of the second data transmission module in the controller model are synchronously processed, and so on.

[0073] S103, generating a target ray for human-computer interaction according to the current motion data of the second data transmission module.

[0074] Optionally, after adjusting the motion data of the second data transmission module in the controller model according to the motion data of the first data transmission module in the interactive controller, the present application can generate a target ray according to the adjusted motion data of the second data transmission module in the controller model.

[0075] Optionally, when generating a target ray, the current position information of the second data transmission module can be used as the starting point of the target ray, and the current positive direction information of the second data transmission module can be used as the direction of the target ray to generate the target ray for human-computer interaction.

[0076] Among them, the current position information is the position information after the position of the first data transmission module changes, or the position information of the previous moment when the first data transmission module has not changed; correspondingly, the current positive direction information is the positive direction information after the direction of the first data transmission module changes; or, it is the positive direction information of the previous moment when the first data transmission module has not changed.

[0077] It can be understood that since the data sent by the interactive controller for generating the target ray is the motion data of the first data transmission module, and the motion data of the first data transmission module is directly sent to the head-mounted display HMD providing the virtual scene through the first data transmission module, there is no need to transfer through an intermediate transmission component, and there is no need to perform a series of coordinate system conversion operations, so that data loss or errors introduced during data transmission will not occur, so that the controller model can obtain the accurate data sent by the interactive controller, and thus an accurate ray corresponding to the real scene interactive operation can be generated based on the accurate data, and then based on the accurate ray, the interactive operation of the interactive controller in the real scene can be realized, and the interactive operation of the controller model in the virtual scene can be fully synchronized, thereby improving the accuracy of using the interactive controller to control the controller model to interact with distant virtual objects for ray interactions.

[0078] The technical solution provided by the embodiment of the present application obtains the motion data of the first data transmission module in the interactive controller, and synchronously processes the motion data of the second data transmission module in the controller model corresponding to the interactive controller according to the motion data of the first data transmission module, and then generates a target ray for human-computer interaction according to the motion data after the synchronous processing by the second data transmission module, that is, the current motion data, so that the controller model located in the virtual scene can obtain accurate data for generating rays, and generate an accurate ray according to the data, so that the selection accuracy of distant virtual objects can be improved based on the accurate ray, thereby improving the user's human-computer interaction operation experience.

[0079] Based on the above embodiments, Figure 3 , further explanation is given on obtaining motion data of the first data transmission module in the interactive controller in this application.

[0080] like Figure 3 As shown, the method may include the following steps:

[0081] S201, obtaining a first application programming interface API of a first data transmission module in an interactive controller.

[0082] Among them, the application programming interface API, also known as the Application Programming Interface in English, is a set of definitions, programs and protocols that enable mutual communication between computer software through the API interface. One of the main functions of API is to provide a common set of functions. Developers can develop applications by calling API functions, which can reduce programming tasks. API is also a kind of middleware that provides data sharing for various platforms.

[0083] Generally speaking, when designers design an interactive controller, they will use the center point in the interactive controller as the parent. Therefore, when transmitting data for generating rays, the data acquisition component needs to first transmit the collected data to the component at the parent level, and then the parent component transmits the received data to the controller model in the virtual scene. However, this data transmission method may result in data loss or errors introduced by a series of data conversion operations, making it impossible for the controller model to obtain accurate data for generating rays.

[0084] Based on the above reasons, when the present application performs industrial design on the interactive controller, the first data transmission module is directly used as the parent, and the motion data of the first data transmission module is sent to the controller model located in the virtual scene as the data for generating rays, so that the controller model can generate an accurate target ray based on the motion data of the first data transmission module.

[0085] Optionally, the first data transmission module is used as a parent, and the position of the first data transmission module can be marked as the starting position of the interactive controller and the starting position of the target ray.

[0086] Among them, when marking the position of the first data transmission module, the position of the first data transmission module can be taken as the starting position of the interactive controller and the starting position of the target ray by default; or, an identification information can be set for the position of the first data transmission module to indicate through the identification information that the position information of the first data transmission module is the starting position of the interactive controller and the starting position of the target ray, etc. Here, there is no restriction on the implementation process of marking the position of the first data transmission module.

[0087] Optionally, after marking the position of the first data transmission module, the industrially designed interactive controller can be converted into a model format corresponding to the virtual engine used by the head mounted display HMD. In addition, the size, shape and direction of the interactive controller, as well as the first application program interface API of the first data transmission module in the interactive controller and other parameters are transferred to the virtual engine, so that a controller model completely corresponding to the interactive controller can be created in the virtual engine.

[0088] It should be noted that the virtual engines supported by the head-mounted display HMD may include but are not limited to: unity, unreal, CryEngine, StingRay, jMonkey Engine, Lumberyard and OpenVR.

[0089] Furthermore, each virtual engine can support different model formats. Therefore, when the present application converts the industrially designed interactive controller into the model format corresponding to the virtual engine used by the head-mounted display HMD, it is necessary to perform the conversion operation according to the model format supported by the virtual engine in the head-mounted display HMD, so as to ensure that the virtual engine in the head-mounted display HMD can create a controller model that completely corresponds to the interactive controller based on the converted model format.

[0090] In other words, the controller model format created by the virtual engine in the head-mounted display HMD in the present application corresponds to the type of the virtual engine.

[0091] For example, when the virtual engine used by the head-mounted display HMD is unity, the industrial designed interactive controller needs to be converted into a supported model format, such as .fbx, .dae (Collada), .dxf, and .obj.

[0092] For another example, when the virtual engine used by the head-mounted display HMD is Unreal, the industrial designed interactive controller needs to be converted into a model format supported by Unreal, such as .fbx, .obj, etc.

[0093] Furthermore, when the head mounted display HMD obtains the interactive controller conversion file in the virtual engine, the size, shape and direction of the interactive controller, as well as the first application program interface API and other parameters of the first data transmission module in the interactive controller can be obtained through the conversion file. Thus, according to the size, shape and direction of the interactive controller, as well as the first application program interface API and other parameters of the first data transmission module in the interactive controller, a controller model that is completely the same as the interactive controller can be created.

[0094] S202, adjusting the second application program interface API of the second data transmission module in the controller model to the first application program interface API of the first data transmission module, so as to establish a communication connection between the first data transmission module and the second data transmission module.

[0095] Optionally, based on the interactive controller conversion file, while creating a controller model in the virtual scene, the head-mounted display HMD of the present application can also adjust the second application program interface API of the second data transmission module in the controller model to the first application program interface API of the first data transmission module, so as to establish a binding relationship between the communication data between the interactive controller and the controller model. That is, the communication link between the interactive controller and the controller model is opened, so that the motion data of the first data transmission module in the interactive controller can be transmitted to the second data transmission module in the controller model through the communication link, and then the controller model can obtain the motion data of the first data transmission module based on the second data transmission module. Thus, an accurate target ray can be generated based on the motion data of the first data transmission module.

[0096] S203: Acquire motion data of a first data transmission module in the interactive controller through a first application programming interface (API).

[0097] S204, synchronously processing the motion data of the second data transmission module in the controller model according to the motion data of the first data transmission module, wherein the controller model corresponds to the interactive controller and the controller model is located in the virtual scene.

[0098] S205 , generating a target ray for human-computer interaction according to the current motion data of the second data transmission module.

[0099] The technical solution provided by the embodiment of the present application obtains the motion data of the first data transmission module in the interactive controller, and synchronously processes the motion data of the second data transmission module in the controller model corresponding to the interactive controller according to the motion data of the first data transmission module, and then generates a target ray for human-computer interaction according to the motion data after the synchronous processing by the second data transmission module, that is, the current motion data, so that the controller model located in the virtual scene can obtain accurate data for generating rays, and generate an accurate ray according to the data, so that the selection accuracy of distant virtual objects can be improved based on the accurate ray, thereby improving the user's human-computer interaction operation experience.

[0100] Please refer to the attached Figure 4 , a ray generating device proposed in an embodiment of the present application is described. Figure 4 A schematic block diagram of a ray generating device provided in an embodiment of the present application.

[0101] like Figure 4 As shown, the ray generating device 300 includes: a data acquisition module 310 , a data synchronization module 320 and a ray generating module 330 .

[0102] The data acquisition module 310 is used to acquire the motion data of the first data transmission module in the interactive controller;

[0103] A data synchronization module 320, configured to synchronize the motion data of the second data transmission module in the controller model according to the motion data of the first data transmission module, wherein the controller model corresponds to the interactive controller and is located in the virtual scene;

[0104] The ray generation module 330 is used to generate a target ray for human-computer interaction according to the current motion data of the second data transmission module.

[0105] In one or more optional implementations of the embodiment of the present application, the motion data of the first data transmission module includes at least one of: position information and positive direction information.

[0106] In one or more optional implementations of the embodiment of the present application, the data synchronization module 320 performs at least one of the following:

[0107] The position information of the second data transmission module in the controller model is adjusted to the position information of the first data transmission module; the positive direction information of the second data transmission module in the controller model is adjusted to the positive direction information of the first data transmission module.

[0108] In one or more optional implementations of the embodiment of the present application, the ray generation module 330 is specifically configured to:

[0109] The target ray for human-computer interaction is generated by taking the current position information of the second data transmission module as the starting point of the target ray and the current positive direction information of the second data transmission module as the direction of the target ray.

[0110] In one or more optional implementations of the embodiment of the present application, the apparatus 300 further includes:

[0111] An interface acquisition module, used to acquire a first application program interface API of a first data transmission module in the interactive controller;

[0112] The interface adjustment module is used to adjust the second application program interface API of the second data transmission module in the controller model to the first application program interface API of the first data transmission module to establish a communication connection between the first data transmission module and the second data transmission module.

[0113] In one or more optional implementations of the embodiment of the present application, the data acquisition module 310 is specifically used to:

[0114] The motion data of the first data transmission module in the interactive controller is obtained through the first application programming interface API.

[0115] One or more optional implementations of the embodiment of the present application, the interface acquisition module is specifically used to:

[0116] A first application programming interface API of a first data transmission module in the interactive controller is obtained from a virtual engine, wherein the virtual engine is used to construct a controller model corresponding to the interactive controller.

[0117] One or more optional implementations of the embodiments of the present application,

[0118] The first data transmission module is a first inertial measurement unit IMU;

[0119] The second data transmission module is a second inertial measurement unit IMU.

[0120] It should be understood that the device embodiment and the aforementioned method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 4 The device 300 shown can perform Figure 1 The corresponding method embodiments, and the aforementioned and other operations and / or functions of each module in the device 300 are respectively to implement Figure 1 For the sake of brevity, the corresponding processes in each method are not repeated here.

[0121] In the above, the device 300 of the embodiment of the present application is described from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the first aspect method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the first aspect method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above-mentioned first aspect method embodiment in conjunction with its hardware.

[0122] Figure 5 A schematic block diagram of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 400 may include:

[0123] The memory 410 and the processor 420, the memory 410 is used to store the computer program and transmit the program code to the processor 420. In other words, the processor 420 can call and run the computer program from the memory 410 to implement the ray generation method in the embodiment of the present application.

[0124] For example, the processor 420 may be configured to execute the above-mentioned ray generation method embodiment according to the instructions in the computer program.

[0125] Among them, the ray generation method includes: obtaining motion data of a first data transmission module in an interactive controller; synchronously processing motion data of a second data transmission module in a controller model according to the motion data of the first data transmission module, wherein the controller model corresponds to the interactive controller and the controller model is located in a virtual scene; and generating target rays for human-computer interaction according to the current motion data of the second data transmission module.

[0126] Optionally, in one or more optional implementations of the embodiments of the present application, the motion data of the first data transmission module includes at least one of: position information and positive direction information.

[0127] Optionally, one or more optional implementations of the embodiment of the present application, wherein the synchronous processing of the motion data of the second data transmission module in the controller model according to the motion data of the first data transmission module includes at least one of the following:

[0128] Adjusting the position information of the second data transmission module in the controller model to the position information of the first data transmission module;

[0129] The positive direction information of the second data transmission module in the controller model is adjusted to the positive direction information of the first data transmission module.

[0130] Optionally, in one or more optional implementations of the embodiment of the present application, generating a target ray for human-computer interaction according to the current motion data of the second data transmission module includes:

[0131] The target ray for human-computer interaction is generated by taking the current position information of the second data transmission module as the starting point of the target ray and the current positive direction information of the second data transmission module as the direction of the target ray.

[0132] Optionally, in one or more optional implementations of the embodiment of the present application, before acquiring the motion data of the first data transmission module in the interactive controller, the method further includes:

[0133] Acquire a first application programming interface API of a first data transmission module in the interactive controller;

[0134] The second application program interface API of the second data transmission module in the controller model is adjusted to the first application program interface API of the first data transmission module to establish a communication connection between the first data transmission module and the second data transmission module.

[0135] Optionally, in one or more optional implementations of the embodiment of the present application, the acquiring the motion data of the first data transmission module in the interactive controller includes:

[0136] The motion data of the first data transmission module in the interactive controller is obtained through the first application programming interface API.

[0137] Optionally, in one or more optional implementations of the embodiment of the present application, the obtaining of a first application programming interface API of a first data transmission module in the interactive controller includes:

[0138] A first application programming interface API of a first data transmission module in the interactive controller is obtained from a virtual engine, wherein the virtual engine is used to construct a controller model corresponding to the interactive controller.

[0139] Optionally, one or more optional implementations of the embodiments of the present application,

[0140] The first data transmission module is a first inertial measurement unit IMU;

[0141] The second data transmission module is a second inertial measurement unit IMU.

[0142] In some embodiments of the present application, the processor 420 may include but is not limited to:

[0143] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0144] In some embodiments of the present application, the memory 410 includes but is not limited to:

[0145] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0146] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 410 and executed by the processor 420 to complete the ray generation method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0147] like Figure 5 As shown, the electronic device 400 may further include:

[0148] The transceiver 430 may be connected to the processor 420 or the memory 410 .

[0149] The processor 420 may control the transceiver 430 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 430 may include a transmitter and a receiver. The transceiver 430 may further include an antenna, and the number of antennas may be one or more.

[0150] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0151] The present application also provides a computer storage medium on which a computer program is stored. When the computer program is executed by a computer, the computer is enabled to execute the ray generation method of the above method embodiment.

[0152] An embodiment of the present application also provides a computer program product including program instructions. When the program instructions are executed on an electronic device, the electronic device executes the ray generation method of the above method embodiment.

[0153] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0154] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0156] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0157] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A ray generation method, characterized in that: include: Acquire motion data of a first data transmission module in the interactive controller; According to the motion data of the first data transmission module, synchronously processing the motion data of the second data transmission module in the controller model, wherein the controller model corresponds to the interactive controller and the controller model is located in the virtual scene; A target ray for human-computer interaction is generated according to the current motion data of the second data transmission module.

2. The method according to claim 1, characterized in that The motion data of the first data transmission module includes at least one of position information and positive direction information.

3. The method according to claim 2, characterized in that The synchronous processing of the motion data of the second data transmission module in the controller model according to the motion data of the first data transmission module includes at least one of the following: Adjusting the position information of the second data transmission module in the controller model to the position information of the first data transmission module; The positive direction information of the second data transmission module in the controller model is adjusted to the positive direction information of the first data transmission module.

4. The method according to claim 2, characterized in that: The generating a target ray for human-computer interaction according to the current motion data of the second data transmission module includes: The target ray for human-computer interaction is generated by taking the current position information of the second data transmission module as the starting point of the target ray and the current positive direction information of the second data transmission module as the direction of the target ray.

5. The method according to any one of claims 1 to 4, characterized in that: Before acquiring the motion data of the first data transmission module in the interactive controller, the method further includes: Acquire a first application programming interface API of a first data transmission module in the interactive controller; The second application program interface API of the second data transmission module in the controller model is adjusted to the first application program interface API of the first data transmission module to establish a communication connection between the first data transmission module and the second data transmission module.

6. The method according to claim 5, characterized in that The acquiring of motion data of the first data transmission module in the interactive controller includes: The motion data of the first data transmission module in the interactive controller is obtained through the first application programming interface API.

7. The method according to claim 5, characterized in that The step of obtaining a first application programming interface API of a first data transmission module in the interactive controller includes: A first application programming interface API of a first data transmission module in the interactive controller is obtained from a virtual engine, wherein the virtual engine is used to construct a controller model corresponding to the interactive controller.

8. The method according to claim 1, characterized in that The first data transmission module is a first inertial measurement unit IMU; The second data transmission module is a second inertial measurement unit IMU.

9. A ray generating device, characterized in that: include: A data acquisition module, used to acquire motion data of the first data transmission module in the interactive controller; a data synchronization module, configured to synchronize the motion data of the second data transmission module in the controller model according to the motion data of the first data transmission module, wherein the controller model corresponds to the interactive controller and is located in the virtual scene; The ray generation module is used to generate a target ray for human-computer interaction according to the current motion data of the second data transmission module.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the ray generation method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the ray generation method according to any one of claims 1 to 8.

12. A computer program product comprising program instructions, characterized in that When the program instructions are executed on an electronic device, the electronic device is enabled to execute the ray generation method according to any one of claims 1 to 8.