Human-computer interaction method and device for virtual light field three-dimensional space

Abstract

The application provides a kind of virtual light field three-dimensional space human-computer interaction method and device, it is related to human-computer interaction technical field, the method includes: the user hand image that camera gathers is acquired, and hand three-dimensional model is constructed based on user hand image;Collision detection is carried out to hand three-dimensional model and other three-dimensional objects displayed in three-dimensional light field display based on hand movement trajectory, and the display position of hand three-dimensional model is updated based on collision detection result;In the case where the collision detection result indicates that hand three-dimensional model and other three-dimensional objects generate gesture interaction, the content shown in three-dimensional light field display is updated based on gesture interaction result.The virtual light field three-dimensional space human-computer interaction method and device provided by the application, by camera, user hand is modeled, and it is displayed in the mode of three-dimensional light field display, realizes that user hand and three-dimensional object in three-dimensional space carry out three-dimensional space interaction, greatly improves the three-dimensional interaction experience of user.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a human-computer interaction method and device in virtual light field three-dimensional space. Background Technology

[0002] Light field display technology is an advanced display technology that reconstructs the light field distribution of three-dimensional objects in space by simulating the physical laws of light propagation in the real world. It is not only an upgrade to traditional 2D / 3D displays, but also a groundbreaking solution that enables naked-eye viewing of stereoscopic levitation images and supports natural interaction.

[0003] In related technologies, to achieve interactive content display using light field displays, it is usually necessary to suspend the planar content in space in a three-dimensional form and then interact with the hand in a spatial manner.

[0004] However, this interaction method makes it very easy for hands to obscure objects in space or to pass through objects, reducing the immersive experience of 3D interaction. Summary of the Invention

[0005] The purpose of this application is to provide a human-computer interaction method and device in a virtual light field three-dimensional space. By modeling the user's hand with a camera and displaying it on a three-dimensional light field display, the user's hand can interact with three-dimensional objects in three-dimensional space, which greatly improves the user's three-dimensional interaction experience.

[0006] This application provides a human-computer interaction method in a virtual light field three-dimensional space, including:

[0007] The system acquires user hand images captured by the camera and constructs a 3D hand model based on the user hand images; it performs collision detection between the 3D hand model and other 3D objects displayed on the 3D light field display based on the hand movement trajectory, and updates the display position of the 3D hand model based on the collision detection results; if the collision detection results indicate that the 3D hand model interacts with other 3D objects using gestures, it updates the content displayed on the 3D light field display based on the gesture interaction results; wherein the hand movement trajectory is determined based on at least two adjacent user hand images.

[0008] Optionally, the step of performing collision detection between the 3D hand model and other 3D objects displayed in the 3D light field display based on the hand movement trajectory, and updating the display position of the 3D hand model based on the collision detection results, includes: determining the user's hand position information based on the relative position of the user's hand in the user's hand image, and obtaining a hand-eye connection based on the user's hand position information and the user's eye position information; determining the target position information of the 3D hand model based on the intersection of the user's hand position information and the target point, and performing collision detection between the 3D hand model and the other 3D objects based on the target position information; the target intersection point is the intersection of the hand-eye connection and the 3D light field display; updating the target position information according to the collision detection results, and displaying the 3D hand model in the 3D light field display based on the updated target position information.

[0009] Optionally, the front of the three-dimensional light field display is provided with a sensor for collecting user eye position information; the step of obtaining the hand-eye connection based on the user's hand position information and the user's eye position information includes: determining the user's eye position information based on the sensor, and obtaining the hand-eye connection based on the user's hand position information and the user's eye position information.

[0010] Optionally, updating the target position information based on the collision detection result includes: when the collision detection result indicates that the hand 3D model comes into contact with any of the target 3D objects among other 3D objects, updating the target position information based on the position constraint of the hand 3D model on the outer contour of the target 3D object; wherein the position constraint is used to restrict the movement of the hand 3D model in the virtual light field 3D space.

[0011] Optionally, updating the target position information based on the position constraint of the hand 3D model on the external contour of the target 3D object includes: splitting the movement direction of the hand 3D model to determine each movement direction component of the hand movement direction; determining the movable direction component that is not constrained by the position constraint on each movement direction component, and updating the target position information based on the movable direction component.

[0012] Optionally, updating the target position information based on the collision detection result includes: keeping the target position information unchanged when the collision detection result indicates that the three-dimensional hand model has not come into contact with other three-dimensional objects.

[0013] Optionally, when the collision detection result indicates that the three-dimensional hand model interacts with other three-dimensional objects, updating the content displayed in the three-dimensional light field display based on the gesture interaction result includes: when the collision detection result indicates that the three-dimensional hand model comes into contact with any target three-dimensional object among other three-dimensional objects, performing gesture interaction based on the interaction logic pre-set for the target three-dimensional object, and updating the content displayed in the three-dimensional light field display based on the gesture interaction result.

[0014] This application also provides a human-computer interaction device in a virtual light field three-dimensional space, comprising:

[0015] The system includes an image acquisition module for acquiring user hand images captured by the camera; a model building module for constructing a 3D model of the hand based on the user hand images; a model display module for performing collision detection between the 3D hand model and other 3D objects displayed on the 3D light field display based on the hand movement trajectory, and updating the display position of the 3D hand model based on the collision detection results; the model display module is also used to update the content displayed on the 3D light field display based on the gesture interaction results when the collision detection results indicate that the 3D hand model interacts with other 3D objects; wherein the hand movement trajectory is determined based on at least two adjacent user hand images.

[0016] Optionally, the model display module is specifically used to determine the user's hand position information based on the relative position of the user's hand in the user's hand image, and to obtain a hand-eye connection based on the user's hand position information and the user's eye position information; the model display module is further used to determine the target position information of the hand 3D model based on the intersection of the user's hand position information and the target point, and to perform collision detection between the hand 3D model and other 3D objects based on the target position information; the target intersection point is the intersection of the hand-eye connection and the 3D light field display; the model display module is further used to update the target position information according to the collision detection result, and to display the hand 3D model on the 3D light field display based on the updated target position information.

[0017] Optionally, the front of the three-dimensional light field display is provided with a sensor for collecting user eye position information; the model display module is specifically used to determine the user's eye position information based on the sensor, and to obtain a hand-eye connection based on the user's hand position information and the user's eye position information.

[0018] Optionally, the model display module is specifically used to update the target position information based on the position constraint of the hand 3D model on the target 3D model according to the position constraint of the target 3D object on the target 3D object when the collision detection result indicates that the hand 3D model has come into contact with any target 3D object among other 3D objects; wherein, the position constraint is used to restrict the movement of the hand 3D model in the virtual light field 3D space.

[0019] Optionally, the model display module is specifically used to decompose the movement direction of the three-dimensional hand model to determine each movement direction component of the hand movement direction; the model display module is also specifically used to determine the movable direction component that is not constrained by the position constraint in each movement direction component, and update the target position information based on the movable direction component.

[0020] Optionally, the model display module is specifically used to keep the target position information unchanged when the collision detection result indicates that the three-dimensional model of the hand has not come into contact with other three-dimensional objects.

[0021] Optionally, the model display module is specifically used to perform gesture interaction based on the interaction logic pre-set for the target three-dimensional object when the collision detection result indicates that the hand three-dimensional model comes into contact with any target three-dimensional object among other three-dimensional objects, and to update the content displayed in the three-dimensional light field display based on the gesture interaction result.

[0022] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the human-computer interaction method in the virtual light field three-dimensional space as described above.

[0023] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the human-computer interaction method for virtual light field three-dimensional space as described above.

[0024] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the human-computer interaction method in the virtual light field three-dimensional space as described above.

[0025] The human-computer interaction method and apparatus in virtual light field 3D space provided in this application first acquires a user's hand image captured by a camera and constructs a 3D model of the hand based on the user's hand image; then, based on the hand movement trajectory, collision detection is performed between the 3D model of the hand and other 3D objects displayed on the 3D light field display, and the display position of the 3D model of the hand is updated based on the collision detection results; finally, if the collision detection results indicate that the 3D model of the hand interacts with other 3D objects using gestures, the content displayed on the 3D light field display is updated based on the gesture interaction results; wherein, the hand movement trajectory is determined based on at least two adjacent user hand images. In this way, by modeling the user's hand using a camera and displaying it on a 3D light field display, 3D spatial interaction between the user's hand and 3D objects in 3D space is realized, greatly improving the user's 3D interactive experience. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is one of the human-computer interaction diagrams in the virtual light field three-dimensional space provided in this application;

[0028] Figure 2 This is the second of the human-computer interaction diagrams provided in this application for a virtual light field three-dimensional space;

[0029] Figure 3 This is a flowchart illustrating the human-computer interaction method in a virtual light field three-dimensional space provided in this application;

[0030] Figure 4 This is a schematic diagram illustrating the determination of the position of the three-dimensional hand model provided in this application;

[0031] Figure 5 This is a schematic diagram of the structure of the human-computer interaction device in the virtual light field three-dimensional space provided in this application;

[0032] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0035] like Figure 1 As shown, the 3D display method of virtual light field 3D space in related technologies relies on binocular parallax or integrated imaging to display 3D objects, so the light source of the 3D object is on the screen. In 3D interaction, hands will attempt to touch the 3D object. However, in reality, hands often pass through or obscure the object, thus affecting both the viewing experience and the interaction efficiency.

[0036] To address the aforementioned technical problems in related technologies, embodiments of this application provide a human-computer interaction method in a virtual light field three-dimensional space, such as... Figure 2 As shown, this method allows users to place their hands behind the screen, and the camera models and displays the hands on a 3D display. This method uses hand models to interact with 3D objects, avoiding clipping and occlusion issues, greatly enhancing the interactive experience.

[0037] The following description, in conjunction with the accompanying drawings, details the human-computer interaction method for virtual light field three-dimensional space provided in this application through specific embodiments and application scenarios.

[0038] like Figure 3 As shown in the embodiment of this application, a human-computer interaction method in a virtual light field three-dimensional space is provided. This method may include the following steps 301 to 303:

[0039] Step 301: Obtain the user's hand image captured by the camera, and construct a three-dimensional model of the hand based on the user's hand image.

[0040] For example, to avoid hand obstruction during user interaction, a camera is installed on the back of the 3D light field display. The 3D light field display can acquire images of the user's hand captured by the camera and perform 3D modeling of the hand. The user can control the movement of the 3D hand model by moving the position of their hand.

[0041] Step 302: Based on the hand movement trajectory, perform collision detection between the 3D hand model and other 3D objects displayed in the 3D light field display, and update the display position of the 3D hand model based on the collision detection results.

[0042] The hand movement trajectory is determined based on at least two adjacent user hand images.

[0043] For example, to avoid clipping, the generated 3D model of the user's hand needs to undergo collision detection with other 3D objects in the virtual light field 3D space. If the 3D model of the hand comes into contact with other 3D models, the two 3D models should be prevented from overlapping to avoid clipping and make the interaction more realistic.

[0044] Specifically, step 302 above, which involves performing collision detection between the 3D hand model and other 3D objects displayed in the 3D light field display based on the hand movement trajectory, may further include steps 302a, 302b, and 302c:

[0045] Step 302a: Based on the relative position of the user's hand in the user's hand image, determine the user's hand position information, and based on the user's hand position information and the user's eye position information, obtain the hand-eye connection.

[0046] For example, such as Figure 4 As shown, the camera behind the 3D light field display can only determine the horizontal position of the user's hand, but not its spatial position. Therefore, in order to accurately locate the user's hand and improve the rationality of the interaction, the spatial position information of the user's hand can be determined based on the connection between the user's hand and eye.

[0047] Specifically, the front of the three-dimensional light field display is provided with a sensor for collecting user eye position information, and step 302a above may further include the following step 302a1:

[0048] Step 302a1: Determine the user's eye position information based on the sensor, and obtain the hand-eye connection based on the user's hand position information and the user's eye position information.

[0049] For example, the sensor described above can be a camera or a time-of-flight (TOF) sensor.

[0050] Step 302b: Based on the intersection of the user's hand position information and the target, determine the target position information of the three-dimensional hand model, and perform collision detection between the three-dimensional hand model and other three-dimensional objects based on the target position information.

[0051] The target intersection point is the intersection point of the hand-eye connection line and the three-dimensional light field display.

[0052] Step 302c: Update the target position information according to the collision detection results, and display the three-dimensional model of the hand on the three-dimensional light field display based on the updated target position information.

[0053] For example, based on the user's hand-eye connection, the spatial position information of the user's hand can be determined more accurately, which not only improves the rationality of the interaction but also makes the user's interactive experience more realistic. It can be understood that, from the user's perspective, the spatial position of the 3D hand model generated based on the user's hand-eye connection is closer to the spatial position of the real hand, resulting in a better user experience.

[0054] Specifically, step 302c above, the step of updating the target position information based on the collision detection result, may include the following step 302c1:

[0055] Step 302c1: If the collision detection result indicates that the hand 3D model is in contact with any of the target 3D objects, update the target position information based on the position constraint of the hand 3D model on the outer contour of the target 3D object.

[0056] The positional constraint is used to limit the movement of the three-dimensional hand model in the virtual light field three-dimensional space.

[0057] For example, when a 3D hand model comes into contact with a target 3D object, the target 3D object can constrain the movement of the 3D hand model. At this time, the position information of the 3D hand model can be updated according to the constraint.

[0058] Specifically, step 302c above, the step of updating the target position information based on the collision detection result, may include steps 302c11 and 302c12:

[0059] Step 302c11: Decompose the movement direction of the three-dimensional hand model and determine the various movement direction components of the hand movement direction.

[0060] Step 302c12: Determine the movable direction component that is not constrained by the position constraint in each of the movable direction components, and update the target position information based on the movable direction component.

[0061] For example, if the target 3D object is to the left of the hand 3D model, and the user's hand moves to the lower left, the hand 3D model cannot continue to move to the left, but can move downwards. In this case, the hand 3D model can be controlled to move only downwards.

[0062] Specifically, in step 302c above, the step of updating the target position information based on the collision detection result may include the following step 302c2:

[0063] Step 302c2: If the collision detection result indicates that the three-dimensional model of the hand has not come into contact with other three-dimensional objects, keep the target position information unchanged.

[0064] For example, when the 3D hand model does not come into contact or collide with any 3D object, the 3D hand model is not constrained and can move as the user's hand moves.

[0065] Step 303: If the collision detection result indicates that the 3D hand model interacts with other 3D objects, update the content displayed in the 3D light field display based on the gesture interaction result.

[0066] For example, users can interact with other 3D objects in the virtual light field 3D space, such as controlling the rotation and movement of other 3D objects, or clicking buttons to trigger specific functions.

[0067] Specifically, step 303 above may include the following step 303a:

[0068] Step 303a: If the collision detection result indicates that the hand 3D model comes into contact with any target 3D object among other 3D objects, perform gesture interaction based on the interaction logic pre-set for the target 3D object, and update the content displayed in the 3D light field display based on the gesture interaction result.

[0069] For example, after the 3D hand model interacts with other 3D objects, pre-set interaction logic can be executed, and the interaction results can be displayed in real time.

[0070] The human-computer interaction method in virtual light field 3D space provided in this application embodiment first acquires a user's hand image captured by a camera and constructs a 3D model of the hand based on the user's hand image; then, based on the hand movement trajectory, collision detection is performed between the 3D model of the hand and other 3D objects displayed on the 3D light field display, and the display position of the 3D model of the hand is updated based on the collision detection results; finally, if the collision detection results indicate that the 3D model of the hand interacts with other 3D objects using gestures, the content displayed on the 3D light field display is updated based on the gesture interaction results; wherein, the hand movement trajectory is determined based on at least two adjacent user hand images. In this way, by modeling the user's hand using a camera and displaying it on a 3D light field display, 3D spatial interaction between the user's hand and 3D objects in 3D space is realized, greatly improving the user's 3D interactive experience.

[0071] It should be noted that the human-computer interaction method in the virtual light field three-dimensional space provided in this application embodiment can be executed by a human-computer interaction device in the virtual light field three-dimensional space, or by a control module in the human-computer interaction device in the virtual light field three-dimensional space for executing the human-computer interaction method in the virtual light field three-dimensional space. This application embodiment uses the execution of the human-computer interaction method in the virtual light field three-dimensional space by the human-computer interaction device in the virtual light field three-dimensional space as an example to illustrate the human-computer interaction device in the virtual light field three-dimensional space provided in this application embodiment.

[0072] It should be noted that, in the embodiments of this application, the human-computer interaction methods in the virtual light field three-dimensional space shown in the accompanying drawings are all illustrated by way of example with reference to one of the accompanying drawings in the embodiments of this application. In specific implementation, the human-computer interaction methods in the virtual light field three-dimensional space shown in the accompanying drawings of the above methods can also be implemented in conjunction with any other accompanying drawings that can be combined with the above embodiments, which will not be elaborated here.

[0073] The following describes the human-computer interaction device in the virtual light field three-dimensional space provided in this application. The human-computer interaction method in the virtual light field three-dimensional space described below can be referred to in correspondence with the one described above.

[0074] Figure 5 This is a schematic diagram of the structure of the human-computer interaction device in the virtual light field three-dimensional space provided in the embodiments of this application, as shown below. Figure 5 As shown, it specifically includes:

[0075] The image acquisition module 501 is used to acquire images of the user's hand captured by the camera; the model construction module 502 is used to construct a three-dimensional model of the hand based on the user's hand image; the model display module 503 is used to perform collision detection between the three-dimensional model of the hand and other three-dimensional objects displayed in the three-dimensional light field display based on the hand movement trajectory, and update the display position of the three-dimensional model of the hand based on the collision detection result; the model display module 503 is also used to update the content displayed in the three-dimensional light field display based on the gesture interaction result when the collision detection result indicates that the three-dimensional model of the hand interacts with other three-dimensional objects; wherein, the hand movement trajectory is determined based on at least two adjacent user hand images.

[0076] Optionally, the model display module 503 is specifically used to determine the user's hand position information based on the relative position of the user's hand in the user's hand image, and to obtain a hand-eye connection based on the user's hand position information and the user's eye position information; the model display module 503 is also specifically used to determine the target position information of the hand 3D model based on the intersection of the user's hand position information and the target point, and to perform collision detection between the hand 3D model and other 3D objects based on the target position information; the target intersection point is the intersection of the hand-eye connection and the 3D light field display; the model display module 503 is also specifically used to update the target position information according to the collision detection result, and to display the hand 3D model on the 3D light field display based on the updated target position information.

[0077] Optionally, the front of the three-dimensional light field display is provided with a sensor for collecting user eye position information; the model display module 503 is specifically used to determine the user's eye position information based on the sensor, and to obtain the hand-eye connection based on the user's hand position information and the user's eye position information.

[0078] Optionally, the model display module 503 is specifically used to update the target position information based on the position constraint of the hand 3D model on the target 3D model according to the position constraint of the target 3D object on the target 3D object when the collision detection result indicates that the hand 3D model has come into contact with any target 3D object among other 3D objects; wherein, the position constraint is used to restrict the movement of the hand 3D model in the virtual light field 3D space.

[0079] Optionally, the model display module 503 is specifically used to decompose the movement direction of the hand 3D model and determine each movement direction component of the hand movement direction; the model display module 503 is also specifically used to determine the movable direction component that is not constrained by the position constraint in each movement direction component, and update the target position information based on the movable direction component.

[0080] Optionally, the model display module 503 is specifically used to keep the target position information unchanged when the collision detection result indicates that the three-dimensional hand model has not come into contact with other three-dimensional objects.

[0081] Optionally, the model display module 503 is specifically used to perform gesture interaction based on the interaction logic pre-set for the target three-dimensional object when the collision detection result indicates that the hand three-dimensional model comes into contact with any target three-dimensional object among other three-dimensional objects, and to update the content displayed in the three-dimensional light field display based on the gesture interaction result.

[0082] The human-computer interaction device in a virtual light field 3D space provided in this application first acquires an image of a user's hand captured by a camera, and constructs a 3D model of the hand based on the image. Then, based on the hand's movement trajectory, collision detection is performed between the 3D hand model and other 3D objects displayed on the 3D light field display, and the display position of the 3D hand model is updated based on the collision detection results. Finally, if the collision detection results indicate that the 3D hand model interacts with other 3D objects using gestures, the content displayed on the 3D light field display is updated based on the gesture interaction results. The hand movement trajectory is determined based on at least two adjacent images of the user's hand. Thus, by modeling the user's hand using a camera and displaying it on a 3D light field display, 3D spatial interaction between the user's hand and 3D objects in 3D space is achieved, greatly improving the user's 3D interactive experience.

[0083] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a human-computer interaction method in a virtual light field three-dimensional space. This method includes: first, acquiring a user's hand image captured by the camera and constructing a three-dimensional hand model based on the user's hand image; then, performing collision detection between the three-dimensional hand model and other three-dimensional objects displayed in the three-dimensional light field display based on the hand movement trajectory, and updating the display position of the three-dimensional hand model based on the collision detection result; finally, if the collision detection result indicates that the three-dimensional hand model interacts with other three-dimensional objects using gestures, updating the content displayed in the three-dimensional light field display based on the gesture interaction result; wherein the hand movement trajectory is determined based on at least two adjacent user hand images. In this way, by modeling the user's hand with a camera and displaying it on a 3D light field display, the user's hand can interact with 3D objects in 3D space, greatly improving the user's 3D interactive experience.

[0084] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0085] On the other hand, this application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the human-computer interaction method in virtual light field three-dimensional space provided by the above methods. The method includes: first, acquiring a user's hand image captured by the camera, and constructing a three-dimensional hand model based on the user's hand image; then, performing collision detection between the three-dimensional hand model and other three-dimensional objects displayed in the three-dimensional light field display based on the hand movement trajectory, and updating the display position of the three-dimensional hand model based on the collision detection result; finally, if the collision detection result indicates that the three-dimensional hand model interacts with other three-dimensional objects by gesture, updating the content displayed in the three-dimensional light field display based on the gesture interaction result; wherein, the hand movement trajectory is determined based on at least two adjacent user hand images. Thus, by modeling the user's hand using a camera and displaying it in a three-dimensional light field display, three-dimensional spatial interaction between the user's hand and three-dimensional objects in three-dimensional space is realized, greatly improving the user's three-dimensional interaction experience.

[0086] Furthermore, this application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the aforementioned human-computer interaction methods in a virtual light field three-dimensional space. The method includes: first, acquiring an image of a user's hand captured by a camera, and constructing a three-dimensional model of the hand based on the image; then, performing collision detection between the three-dimensional hand model and other three-dimensional objects displayed in the three-dimensional light field display based on the hand's movement trajectory, and updating the display position of the three-dimensional hand model based on the collision detection result; finally, if the collision detection result indicates that the three-dimensional hand model interacts with other three-dimensional objects using gestures, updating the content displayed in the three-dimensional light field display based on the gesture interaction result; wherein the hand movement trajectory is determined based on at least two adjacent user hand images. Thus, by modeling a user's hand using a camera and displaying it in a three-dimensional light field display, three-dimensional spatial interaction between the user's hand and three-dimensional objects in three-dimensional space is achieved, greatly improving the user's three-dimensional interaction experience.

[0087] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A human-computer interaction method in a virtual light field three-dimensional space, characterized in that, The device is used in a three-dimensional light field display, wherein a camera is mounted on the back of the three-dimensional light field display. The method includes: The system acquires a user's hand image captured by the camera and constructs a three-dimensional hand model based on the user's hand image. The target position information of the three-dimensional hand model is determined based on the intersection point between the user's hand position information and the target. The target intersection point is the intersection point between the line connecting the hand and the eye and the three-dimensional light field display. Collision detection is performed between the 3D hand model and other 3D objects displayed in the 3D light field display based on the hand movement trajectory, and the display position of the 3D hand model is updated based on the collision detection results. If the collision detection result indicates that the 3D hand model interacts with other 3D objects, the content displayed in the 3D light field display is updated based on the gesture interaction result. The hand movement trajectory is determined based on at least two adjacent user hand images.

2. The method according to claim 1, characterized in that, The step of performing collision detection between the 3D hand model and other 3D objects displayed in the 3D light field display based on the hand movement trajectory, and updating the display position of the 3D hand model based on the collision detection results, includes: Based on the relative position of the user's hand in the user's hand image, the position information of the user's hand is determined, and based on the position information of the user's hand and the position information of the user's eye, a hand-eye connection is obtained; Based on the intersection point between the user's hand position information and the target, the target position information of the three-dimensional hand model is determined, and collision detection between the three-dimensional hand model and other three-dimensional objects is performed based on the target position information. The target position information is updated based on the collision detection results, and the three-dimensional model of the hand is displayed on the three-dimensional light field display based on the updated target position information.

3. The method according to claim 2, characterized in that, The front of the three-dimensional light field display is equipped with a sensor for collecting information about the user's eye position. The step of obtaining the hand-eye connection based on the user's hand position information and eye position information includes: The sensor determines the user's eye position information, and the hand-eye connection is obtained based on the user's hand position information and the user's eye position information.

4. The method according to claim 2 or 3, characterized in that, The step of updating the target position information based on the collision detection results includes: If the collision detection result indicates that the hand 3D model comes into contact with any of the target 3D objects, the target position information is updated based on the position constraint of the hand 3D model on the outer contour of the target 3D object. The positional constraint is used to limit the movement of the three-dimensional hand model in the virtual light field three-dimensional space.

5. The method according to claim 4, characterized in that, The step of updating the target position information based on the positional constraints of the hand 3D model on the external contour of the target 3D object includes: The movement direction of the three-dimensional hand model is decomposed to determine the individual movement direction components of the hand movement direction; The position constraint is determined to be the movable direction component that does not generate a constraint on each of the movable direction components, and the target position information is updated based on the movable direction component.

6. The method according to claim 2 or 3, characterized in that, The step of updating the target position information based on the collision detection results includes: If the collision detection result indicates that the 3D model of the hand has not come into contact with other 3D objects, the target position information remains unchanged.

7. The method according to claim 1, characterized in that, When the collision detection result indicates that the 3D hand model interacts with other 3D objects, updating the content displayed in the 3D light field display based on the gesture interaction result includes: If the collision detection result indicates that the hand 3D model comes into contact with any of the target 3D objects, a gesture interaction is performed based on the interaction logic pre-set for the target 3D object, and the content displayed in the 3D light field display is updated based on the gesture interaction result.

8. A human-computer interaction device for virtual light field three-dimensional space, characterized in that, The device is used in a three-dimensional light field display, wherein a camera is mounted on the back of the three-dimensional light field display. The device includes: The image acquisition module is used to acquire images of the user's hand captured by the camera; The model building module is used to build a three-dimensional model of the hand based on the user's hand image; the target position information of the three-dimensional hand model is determined based on the intersection of the user's hand position information and the target; the target intersection is the intersection of the line connecting the hand and the eye and the three-dimensional light field display. The model display module is used to perform collision detection between the three-dimensional hand model and other three-dimensional objects displayed in the three-dimensional light field display based on the hand movement trajectory, and update the display position of the three-dimensional hand model based on the collision detection results; The model display module is also used to update the content displayed in the three-dimensional light field display based on the gesture interaction result when the collision detection result indicates that the three-dimensional hand model interacts with other three-dimensional objects. The hand movement trajectory is determined based on at least two adjacent user hand images.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the human-computer interaction method for virtual light field three-dimensional space as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the steps of the human-computer interaction method in the virtual light field three-dimensional space as described in any one of claims 1 to 7.

Images