Method and system for processing surface interactions of virtual 3D objects - Patents.com

Abstract

The present invention provides a means for aligning a first virtual object with a second virtual object within a scene volume in a 3D virtual space, where the second virtual object is different from the first virtual object. The method includes performing a 3D scan of a user's head, where the first virtual object comprises the 3D scan of the user's head, and the second virtual object comprises a 3D model of a mask. The method further includes determining positions of first surface points on a surface of the first virtual object based on first ray casting of the first virtual object. The method further includes determining positions of second surface points on a surface of the second virtual object based on second ray casting of the second virtual object, where the second surface points have a desired positional relationship with respect to the first surface points, where in the desired positional relationship, the inner surface of the 3D model of the mask is coplanar with the surface of the 3D model of the user's head. The first ray casting occurs from outside the scene volume, and the second ray casting occurs from inside the scene volume. A relative position between the position of the first surface point and the position of the second surface point is then determined, and the position of the second surface point is adjusted based on the relative position.

Description

[Technical field]

[0001] The present invention relates to the field of 3D digital imaging, and more particularly to the field of surface interactions between 3D digital objects. [Background technology]

[0002] It is often useful to visualize the interaction between two objects in a digital space before implementing the changes in the real world.

[0003] For example, a system for visualizing the fit of a medical device to a subject's body allows a user to easily and objectively determine the optimal medical device for the subject.

[0004] FIG. 1 illustrates such an example, where a model of a user's head 100 is shown in combination with a respiratory mask 110 .

[0005] In this case, the subject sees a 3D model of their head wearing the breathing mask of their choice, the shape of which must be adjusted to fit the 3D face scan to accommodate the individual face shape of any given subject. Summary of the Invention [Problem to be solved by the invention]

[0006] Current algorithms typically position the mask correctly over the face scan plane, but often gaps appear between the face and the mask, or the mask disappears partially below the face surface (as shown by region 120), which is unattractive to the subject and misleads the clinician into selecting a mask that is an inadequate fit.

[0007] Francois Lehericey et al., "Extended version: GPU Ray-Traced Collision Detection for Cloth Simulation", 2015-11-02, pages 1-9, XP055638417, the open archive Hal, presents a method for performing collision detection with cloth using ray tracing at interactive frame rates. [Means for solving the problem]

[0008] The invention is defined by the claims.

[0009] According to an example according to an aspect of the present invention, there is provided a method of aligning a first virtual object with a second virtual object within a scene volume in a 3D virtual space, where the second virtual object is different from the first virtual object, the method comprising: performing a 3D scan of the user's head, the first virtual object comprising the 3D scan of the user's head and the second virtual object comprising a 3D model of a mask; determining a position of a first surface point on a surface of the first virtual object based on a first ray casting of the first virtual object, the first ray casting being in a direction from outside a scene volume towards inside the scene volume; determining a position of a second surface point on a surface of the second virtual object based on a second ray casting of the second virtual object, the second ray casting being in a direction from inside the scene volume to outside the scene volume, the second surface point having a desired relationship to the first surface point, wherein in the desired relationship an inner surface of the 3D model of the mask is flush with a surface of a 3D model of the user's head; determining a relative position between the position of the first surface point and the position of the second surface point; and adjusting the position of the second surface point based on the relative position. has.

[0010] The method provides means for aligning a second virtual object with the first virtual object and for adjusting a surface of the second virtual object according to a position of the second virtual object relative to the first virtual object.

[0011] The method provides a means for preventing a surface of a second virtual object from penetrating a surface of a first virtual object in instances where a collision might otherwise occur.

[0012] By raycasting a first object from outside the scene and a second object from inside the scene, it is possible to obtain a detailed image of the surfaces of these objects which interact with each other and therefore require adjustments.

[0013] By using a 3D scan of the user's head and a 3D model of the mask, a personalized representation of a given user's head is obtained and the relationship between the mask and the user's face is virtually determined, where the method provides a means to visualize how the mask will fit an individual user without accidentally colliding and clipping parts of the mask with the user's 3D scan.

[0014] In one embodiment, adjusting the position of the second surface point comprises adjusting a position of a second virtual object.

[0015] In this manner, the entire second virtual object can be moved to obtain a desired positional relationship between the first surface points and the second surface points.

[0016] In one embodiment, the steps of the method are repeated for a plurality of surface points on a first virtual object and a corresponding plurality of surface points on a second virtual object.

[0017] In this way, the entire surface of the second virtual object can be adjusted based on its spatial relationship to the surface of the first virtual object.

[0018] In another embodiment, adjusting the position of the second surface point comprises deforming the second virtual object based on the adjusted position of the second surface point.

[0019] In this manner, the shape of the second virtual object may change to accommodate the desired interaction with the first virtual object.

[0020] In another embodiment, deforming the second virtual object comprises adjusting positions of additional second surface points based on adjusted second surface point positions of the plurality of second surface points.

[0021] In this way, when a given surface point of the second virtual object moves, adjacent surface points also move to provide a smooth transition along the surface of the second virtual object during deformation.

[0022] In another embodiment, the 3D model of the mask comprises: CPAP mask, Medical sensory devices, hearing aid, Neck brace, Dental equipment, Dental equipment, e.g. orthodontic appliances; Eyewear, e.g. glasses, headphone, Virtual reality headsets, Visualization of postoperative scars, and Implants has one or more of the following.

[0023] In one embodiment, the method includes performing a preliminary alignment of a first virtual object and a second virtual object.

[0024] In this way, the first and second virtual objects can be approximately in the correct position before fine adjustments are made to the surface of the second object.

[0025] In one embodiment, the method further comprises: deforming a first virtual object; and Transforming a second virtual object based on the transformation of the first virtual object. It further has:

[0026] In this way, the effect that a deformation of a first virtual object has on a second virtual object is modeled.

[0027] According to an example according to an aspect of the invention there is provided a computer program comprising computer program code means adapted to perform the method described above when said computer program is run on a computer.

[0028] According to an example according to an aspect of the present invention, there is provided a system for aligning a first virtual object to a second virtual object within a scene volume in a 3D virtual space, where the second virtual object is different from the first virtual object, the system comprising: a scanner adapted to generate a 3D model of a user's head based on a scan of the user's head; and Processor The processor includes: receiving a 3D model of the user's head from the scanner as the first virtual object, where the second virtual object comprises a 3D model of a mask; determining a position of a first surface point on a surface of the first virtual object based on a first ray casting of the first virtual object, where the first ray casting is in a direction from outside the scene volume to inside the scene volume; determining a position of a second surface point on a surface of the second virtual object based on a second ray casting of the second virtual object, where the second ray casting is in a direction from inside the scene volume to outside the scene volume, and where the second surface point has a desired relationship to the first surface point, where in the desired relationship an inner surface of the 3D model of the mask is flush with a surface of the 3D model of the user's head. determining a relative position between the position of the first surface point and the position of the second surface point; and Adjusting the position of the second surface point based on the relative position. Adapt as follows.

[0029] In one embodiment, the system further comprises a display adapted to display the scene volume to a user.

[0030] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. [Brief description of the drawings]

[0031] For a better understanding of the present invention, and to show more clearly how the same may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which: [Figure 1] FIG. 1 shows a graphical representation of the prior art. [Diagram 2] FIG. 2 illustrates the method of the present invention. [Figure 3a] FIG. 3a shows a schematic diagram of ray casting of the method of FIG. [Figure 3b] FIG. 3b shows a schematic diagram of ray casting of the method of FIG. [Figure 4] FIG. 4 shows a graphical representation of implementing the method of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] The present invention will now be described with reference to the drawings.

[0033] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the devices, systems and methods, are for illustrative purposes only and are not intended to limit the scope of the invention. These and other features, aspects and advantages of the devices, systems and methods of the present invention will become better understood from the following description, appended claims and accompanying drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0034] The present invention provides a means for aligning a first virtual object with a second virtual object within a scene volume in a 3D virtual space, where the second virtual object is different from the first virtual object. The method includes the step of performing a 3D scan of a user's head, where the first virtual object comprises the 3D scan of the user's head, and the second virtual object comprises a 3D model of a mask. The method further includes the step of determining a position of a first surface point on a surface of the first virtual object based on a first ray casting of the first virtual object. The method further includes the step of determining a position of a second surface point on a surface of the second virtual object based on a second ray casting of the second virtual object, where the second surface point has a desired positional relationship to the first surface point, where in the desired positional relationship, an inner surface of the 3D model of the mask lies flush with a surface of the 3D model of the user's head. The first ray casting occurs from outside the scene volume, and the second ray casting occurs from inside the scene volume. A relative position between the position of the first surface point and the position of the second surface point is then determined, and the position of the second surface point is adjusted based on the relative position.

[0035] FIG. 2 illustrates a method 200 for aligning a first virtual object with a second virtual object within a scene volume in a 3D virtual space.

[0036] The method begins at step 210 by determining a position of a first surface point on a surface of a first virtual object based on a first ray casting of the first virtual object, the first ray casting originating from outside the scene volume.

[0037] In step 220, a position of a second surface point on a surface of a second virtual object is determined based on a second ray casting of the second virtual object, where the second surface point has a desired positional relationship to the first surface point, the second ray casting originating from inside the scene volume.

[0038] In other words, the locations of two surface points that are on the surfaces of separate virtual objects and have a desired spatial relationship to each other are determined using ray casting, a ray casting technique that is further described below with reference to FIG.

[0039] In step 230, the relative position between the position of the first surface point and the position of the second surface point is determined.

[0040] In step 240, the position of the second surface point is adjusted based on the relative position.

[0041] As an example, the first virtual object may be a 3D scan of the user's head, in which case the method may also include the step of performing a 3D scan of the user's head, and the second virtual object may be a 3D model of a mask, including a CPAP mask or any other type of mask adapted to be worn by a user.

[0042] In this example, the method is used to adjust the inner surface of the mask to fit the surface of the user's face, thus preventing visual clipping as shown in Figure 1, where the mask passes through the user's face.

[0043] For example, the position of the second surface point is adjusted by adjusting the position of the second virtual object. In other words, the entire second virtual object is translated to adjust the position of the second surface point. In the example outlined above, the position of the entire 3D model of the mask may be adjusted around the 3D model of the user's head.

[0044] Alternatively, the steps of method 200 above may be repeated for a number of surface points on a first virtual object and a corresponding number of surface points on a second virtual object.

[0045] In other words, for complex objects having multiple interacting surface points, the method may be applied across any number of surface points to properly align the first and second virtual objects.

[0046] Returning to the example of the 3D model of the user's head and the 3D model of the mask, a number of surface points along the mask are adjusted according to their relationship to corresponding surface points on the user's face, in this way the entire mask can be fine-tuned to show the exact fit of the mask to the user's face.

[0047] Additionally, adjusting the position of the second surface point may include deforming the second virtual object based on the adjusted position of the second surface point or the adjusted positions of the surface points.

[0048] For example, if the second virtual object has a wireframe connecting each of the second surface points, additional second surface points may be generated along the predetermined wires to enable a smooth transition along the surface of the second virtual object. Further, when deforming the second virtual object, positions of such additional second surface points may be adjusted based on positions of the adjusted second surface points of the plurality of second surface points.

[0049] In other words, as a first second surface point moves, additional adjacent second surfaces may move an appropriate amount such that the integrity of the surface of the second virtual object is maintained in 3D space.

[0050] In the above example, this may be visualized as a mask that stretches to fit the individual user's face. In this way, each user can be provided with a personalized representation of the fit, rather than providing a standardized view that does not accurately represent how the mask actually fits.

[0051] Additionally, the method allows for simulating an interaction between a first virtual object and a second virtual object, for example, the first virtual object may deform and the second virtual object may deform based on the deformation of the first virtual object.

[0052] For example, a 3D model of the user's head may be animated to simulate speaking or yawning, and in response, the 3D model of the mask may deform accordingly to visualize how the mask will function during everyday use.

[0053] Furthermore, the second virtual object may be programmed to follow certain predefined constraints, e.g. relating to material properties. For example, if the second virtual object is a mask, a flexibility measure may be assigned to the mask according to the material it is made of. In this case, if the adjustment of the second surface points to adapt to a 3D model of the user's head exceeds a flexibility threshold, a warning may be generated informing the user that the selected mask is not suitable for the given user.

[0054] It should be noted that a preliminary alignment of the first and second virtual objects may be performed before starting the method. During this preliminary alignment, larger adjustments may be made, whereas the method described above can be reserved for finer adjustments of the second surface points.

[0055] In another example, the first virtual object and the second virtual object may be any 3D models of two objects whose interaction requires investigation. For example, the first virtual object may be a 3D model of a user's head and the second virtual object may be one or more 3D models of hearing aids, eyewear such as glasses, cervical braces, dental equipment such as orthodontic appliances, post-operative scar visualization, and implants, etc.

[0056] Furthermore, the first virtual object may be a 3D model of any body part of the user and the second virtual object may be a 3D model of any device or visualization adapted to interact with said body part.

[0057] 3a and 3b show a graphical representation of the ray casting method described above.

[0058] Ray casting refers to the use of ray-surface intersection tests in computer graphics and computational mechanics. For example, ray casting is used to address the common problem of determining the first object that a ray intersects in a scene, or in direct volume rendering methods where a ray penetrates a virtual object and the 3D scalar field of interest is a sample along the ray within this virtual object.

[0059] Ray casting is a computer graphics rendering algorithm based on the geometric algorithm of ray tracing. To represent (render) a 3D scene into a 2D image, a ray-tracing-based rendering algorithm operates on the image.

[0060] The speed and simplicity of ray casting comes from computing the color of light without recursively tracing additional rays that sample the incident radiance to the point where the ray hits.

[0061] In reality, a light source emits a ray of light, which travels until it reaches a surface that eventually blocks its progress. At this point, any combination of three things can happen to this ray: absorption, reflection and refraction. The surface can reflect all or part of the ray in one or more directions. The surface can also absorb part of the ray, resulting in a loss of intensity of the reflected and / or refracted light. If the surface has some transparent or translucent properties, it will refract parts of the light beam into itself in different directions, absorbing part (or all) of the spectrum (and possibly changing color). All of the incident light must be considered between absorption, reflection and refraction, and nothing more. A surface cannot, for example, reflect 66% of the incident ray and refract 50% (because the two add up to 116%). From here, the reflected and / or refracted ray can hit other surfaces, where their absorption, refraction and reflection properties are calculated again based on the incident ray. Some of these rays are directed such that they strike the eye 310 and contribute to the final rendered image.

[0062] In virtual space, it is not necessary to simulate the lighting of a scene in this way: rather, geometric rays may be traced directly from the eye so that only rays that contribute to the rendered image are simulated.

[0063] In operation, and looking at Figure 3a, which illustrates a first ray casting operation of the method as an example, a geometric ray 300 is traced from outside the scene volume 330, from the eye of an observer 310 to a first virtual object 320. This geometric ray is used to sample the virtual light traveling towards the observer from the direction of this ray. In Figure 3a, the geometric ray hits the first virtual object at a first surface point 350.

[0064] 3b shows a graphical representation of a second ray casting operation taking place within the scene volume 330. In this example, a geometric ray 300 is traced from the eye 310 to a second virtual object 360. The geometric ray strikes the second virtual object at a second surface point 370.

[0065] The scene volume is defined as a 3D virtual space in which the first 3D virtual object and the second 3D virtual object reside. This space defines a coordinate system that indicates the positions of the virtual objects. In addition, the scene volume also includes other components that are common in 3D computer graphics, such as a camera, a light source, and background decorations. The scene volume can have any shape suitable for a given implementation. The scene volume is defined by the smallest 3D volume required to contain the first and second 3D virtual objects, or by a 3D volume of any other shape or size that contains these 3D virtual objects.

[0066] A ray, such as geometric ray 300, is a straight line through the 3D scene volume. Collision checks are applied to this line (the raycast) in one of two directions along this line.

[0067] In the above-mentioned method, ray casting is performed in both directions along the ray when colliding with a first 3D virtual object (first ray casting) and another time when colliding with a second 3D virtual object (second ray casting), which means that the first ray cast goes along the ray from outside the scene volume towards the center of the scene where the first and second virtual objects are placed, whereas the second ray cast goes in the opposite direction, i.e. from within the scene volume, away from said center.

[0068] When the positioning of a first 3D model relative to a second 3D model is constrained by anatomical aspects, e.g., a 3D model of a respiratory mask that must be above the nose and not above the chin, conventional positioning methods can work well when dealing with known, predefined 3D model shapes.

[0069] However, when one or more of the shapes of the 3D model are not known in advance because the 3D model is created on-site by a user using a scanner, for example a 3D model of a user's head, the multi-stage ray casting technique described above provides a system that takes into account the anatomical aspects and is flexible enough to accommodate changing shapes over the life of the system.

[0070] From the known positions of the eye in the first and second ray casting operations, and the geometric ray characteristics in each case, the positions of the first and second surface points within the scene volume may be determined.

[0071] The relative positions of these surface points are compared to a desired spatial relationship between these surface points. For example, in the example shown in Figure 1, one or more surface points of the 3D model of the mask are shown to encroach on the 3D model of the user's head, meaning that the desired spatial relationship is not met. The desired spatial relationship in this example is for the inner surface of the 3D model of the mask to lie flush with the surface of the 3D model of the user's head.

[0072] Figure 4 shows an updated version of the scenario of figure 1 after the above-mentioned method has been applied. As can be seen in figure 4, the second virtual object, mask 110, has been adjusted according to the first virtual object, user's head 100, such that there is no longer an erroneously intruded area between the first virtual object and the second virtual object.

[0073] Variations to the disclosed embodiments can be understood and implemented by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the term "comprising" does not exclude other elements or steps, nor does a recitation of a plurality of elements exclude a plurality of the elements. A single processor or other unit may perform the functions of several items recited in the claims. The mere fact that certain methods are recited in mutually different dependent claims does not indicate that a combination of these methods cannot be advantageously used. Where a computer program is described above, the computer program may be stored or distributed on a suitable medium, for example an optical storage medium or a solid-state medium provided together with or as part of other hardware, but may also be distributed in other forms, for example via the Internet or other wired or wireless telecommunication systems. When the term "adapted for" is used in the claims or the description, this term "adapted for" is meant to be equivalent to the term "configured for". Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. 1. A method of aligning a first virtual object with a second virtual object within a scene volume defined as a three-dimensional (3D) virtual space in which the first virtual object and the second virtual object exist, the second virtual object being different from the first virtual object, the method comprising: performing a three-dimensional (3D) scan of a user's head, the first virtual object comprising the three-dimensional (3D) scan of the user's head and the second virtual object comprising a three-dimensional (3D) model of a mask; determining a position of a first surface point on a surface of the first virtual object based on a first ray casting of the first virtual object, the first ray casting originating from outside the scene volume and in a direction from outside the scene volume towards an inside of the scene volume; determining a position of a second surface point on a surface of the second virtual object based on a second ray casting of the second virtual object, the second ray casting being in a direction from inside the scene volume to outside the scene volume in a direction away from a center of the scene volume, the second surface point having a desired relationship to the first surface point, wherein in the desired relationship an inner surface of a three-dimensional (3D) model of the mask is flush with a surface of a three-dimensional (3D) model of the user's head; determining a relative position between the position of the first surface point and the position of the second surface point; and adjusting the position of the second surface point based on the relative position. The method according to claim 1,

2. The method of claim 1 , wherein adjusting a position of the second surface point comprises adjusting a position of the second virtual object.

3. The method of claim 1 or 2, wherein the method steps are repeated for a plurality of surface points on the first virtual object and a corresponding plurality of surface points on the second virtual object.

4. The method of claim 3 , wherein adjusting a position of the second surface point comprises deforming the second virtual object based on the adjusted position of the second surface point.

5. 5. The method of claim 4, wherein transforming the second virtual object comprises adjusting positions of additional second surface points based on positions of the adjusted second surface points of the plurality of second surface points.

6. The three-dimensional (3D) model of the mask comprises: Continuous positive airway pressure (CPAP) masks, Medical sensory devices, hearing aid, Cervical orthosis, Dental equipment, Eyewear, headphone, Virtual reality headsets, Visualization of postoperative scars, and Implants The method according to claim 1 , further comprising one or more of the following steps:

7. The method according to claim 1 , further comprising a step of performing a preliminary alignment of the first virtual object and the second virtual object.

8. The method comprises: deforming the first virtual object; and Transforming the second virtual object based on the transformation of the first virtual object.

8. The method of claim 1, further comprising:

9. A computer program comprising computer program code means adapted to perform a method according to any one of claims 1 to 8, when said computer program is run on a computer.

10. 1. A system for aligning a first virtual object with a second virtual object within a scene volume defined as a three-dimensional (3D) virtual space in which the first virtual object and the second virtual object exist, the second virtual object being different from the first virtual object, the system comprising: a scanner adapted to generate a three-dimensional (3D) model of a user's head based on a scan of the user's head; and Processor The processor includes: receiving a three-dimensional (3D) model of the user's head from the scanner as the first virtual object, the second virtual object having a three-dimensional (3D) model of a mask; determining a position of a first surface point on a surface of the first virtual object based on a first ray casting of the first virtual object, the first ray casting originating from outside the scene volume and in a direction from outside the scene volume towards an inside of the scene volume; determining a location of a second surface point on a surface of the second virtual object based on a second ray casting of the second virtual object, the second ray casting being in a direction from inside the scene volume to outside the scene volume in a direction away from a center of the scene volume, the second surface point having a desired relationship to the first surface point, wherein in the desired relationship an inner surface of a three-dimensional (3D) model of the mask is flush with a surface of the three-dimensional (3D) model of the user's head. determining a relative position between the position of the first surface point and the position of the second surface point; and Adjusting the position of the second surface point based on the relative position. So adapt the system.

11. The system of claim 10 , further comprising a display adapted to display the scene volume to a user.

Images