Method for realizing virtual home product layout scene and virtual reality system

Abstract

The present invention provides a method for realizing a virtual home product layout scene and a virtual reality system. The method for realizing a virtual home product layout scene comprises: creating a background image; superimposing at least one element model to be laid out on the background image, wherein the at least one element model at least comprises a three-dimensional model of a home product drawn in proportion to the actual size; obtaining user actions captured by a somatosensory device, so that the element model is arranged in the background image in a posture responsive to the user action. The present invention arranges the three-dimensional model of home products in a virtual background image by capturing user actions, which can simply and conveniently realize the user's free arrangement of home products, give full play to the user's imagination, get rid of the limitations of external conditions such as manpower and material resources, and improve the user experience.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method for realizing a virtual home product layout scene and a virtual reality system. Background Art

[0002] At present, home furnishing layout mainly relies on users to arrange home furnishing products at home after purchase or rely on design drawings issued by designers. The first method not only consumes too much manpower, but also the purchased home furnishing products are likely to be difficult to match reasonably, resulting in low user satisfaction, while the second method costs more and has low user participation. Taking all factors into consideration, a method and virtual reality system that can realize virtual home furnishing product layout scenes are needed in design. Summary of the invention

[0003] An object of the first aspect of the present invention is to overcome at least one technical defect in the prior art and provide a method for realizing a virtual home product layout scene.

[0004] A further object of the first aspect of the present invention is to simplify operation.

[0005] Another further object of the first aspect of the present invention is to more accurately identify user intention.

[0006] An object of the second aspect of the present invention is to provide a virtual reality system.

[0007] According to a first aspect of the present invention, a method for implementing a virtual home product layout scene is provided, comprising:

[0008] Create a background image;

[0009] Overlaying at least one element model to be laid out on the background image, wherein the at least one element model at least includes a three-dimensional model of a household product drawn in proportion to an actual size;

[0010] The user action captured by the somatosensory device is acquired, and the element model is arranged in the background image in a posture responsive to the user action.

[0011] Optionally, the step of acquiring a user action captured by a somatosensory device and arranging the element model in the background image in a posture responsive to the user action comprises:

[0012] A target element model is determined in the at least one element model, and the user action is associated with the target element model.

[0013] Optionally, the step of determining a target element model in the at least one element model and associating the user action with the target element model includes:

[0014] Acquire a virtual distance between the somatosensory device and an element model located at a center of a field of view of a head mounted display;

[0015] If the virtual distance is less than or equal to a preset distance threshold, the element model is determined as the target element model.

[0016] Optionally, after the step of associating the user action with the target element model, the method further includes:

[0017] The display state of the target element model is changed to an activated state, and the visual effect of the activated state is different from that of other element models.

[0018] Optionally, the somatosensory device is configured to capture a user's hand motion, wherein after the step of associating the user motion with the target element model, the method further includes:

[0019] The target element model is configured to reduce in size and be attached to the user's hand position in the background image in response to the user's first hand motion; and / or

[0020] Configuring the target element model to restore the size and place the target element model at the user's hand position in the background image in a current posture in response to a second hand motion of the user; and / or

[0021] The target element model is configured to rotate in response to a third hand motion of the user.

[0022] Optionally, the somatosensory device is configured to capture a hand motion of a user, wherein the step of modifying the user motion according to a motion feature of the user motion comprises:

[0023] If the movement speed of the user's hand is less than or equal to a preset speed threshold or the movement acceleration of the user's hand is greater than or equal to a preset acceleration threshold, the target element model responds to the hand movement at a speed less than the movement speed of the user's hand.

[0024] Optionally, the somatosensory device is configured to capture a hand motion of a user, wherein the step of modifying the user motion according to a motion feature of the user motion comprises:

[0025] If the user's hand stays at a certain position or a certain posture for a period of time greater than or equal to a preset time threshold, the target element model is directly moved to the position or changed to the posture at a first speed.

[0026] Optionally, the user action is modified according to an action feature of the user action, where the action feature is at least one of speed, acceleration, holding time at the same position, and holding time at the same posture.

[0027] Optionally, the step of creating a background image includes:

[0028] Acquire room information input by a user, where the room information includes at least one of room size and room background;

[0029] A background image is created according to the room information.

[0030] According to a second aspect of the present invention, there is provided a virtual reality system, comprising:

[0031] A head mounted display for outputting virtual images;

[0032] A somatosensory device for capturing the user's movements;

[0033] Processor; and

[0034] A memory stores a computer program, which, when executed by the processor, is used to implement the method for realizing a virtual home product layout scene according to any of the above methods.

[0035] The present invention arranges three-dimensional models of household products in a virtual background image by capturing user actions, which can simply and conveniently enable users to freely arrange household products, give full play to the user's imagination, get rid of the limitations of external conditions such as manpower and material resources, and improve user experience.

[0036] Furthermore, the present invention determines the target element model through the virtual distance between the somatosensory device and the element model located at the center of the field of view of the head-mounted display. Without the need for the user to issue instructions, the target element model can be automatically determined and then the model can be arranged, thereby improving the intelligence of the virtual reality system.

[0037] Furthermore, the present invention modifies user actions according to the speed, acceleration, and holding time of the same position or the same posture of the user action, which can more accurately identify the user's intention and avoid unexpected repeated movements of the target element model, further improving the intelligence of the virtual reality system and improving the user experience.

[0038] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0040] Figure 1is a schematic structural diagram of a virtual reality system according to an embodiment of the present invention;

[0041] Figure 2 is a schematic flow chart of a method for implementing a virtual home product layout scene according to an embodiment of the present invention;

[0042] Figure 3 It is a schematic and detailed flow chart of a method for realizing a virtual home product layout scene according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Figure 1 is a schematic structural diagram of a virtual reality system 100 according to an embodiment of the present invention. Figure 1 The virtual reality system 100 of the present invention may include a head mounted display 110, a somatosensory device 120, a processor 130 and a memory 140.

[0044] The head mounted display 110 can be worn on the head of a user and output a virtual image of a real object or a virtual object to the user. The head mounted display 110 can be a display device of a virtual display helmet or smart glasses.

[0045] The somatosensory device 120 may be configured to capture user movements, thereby allowing the user to interact with the virtual image output by the head mounted display 110. The somatosensory device 120 may be a sensing device that captures user movements by means of inertial sensing, optical sensing, tactile sensing, and combinations thereof, such as a data glove.

[0046] In some embodiments, the somatosensory device 120 may be configured to capture the user's hand movements, so as to more flexibly preset command movements, so that small user movements can achieve interaction with the virtual image. In other embodiments, the somatosensory device 120 may also be configured to capture the user's arm movements.

[0047] The memory 140 may store a computer program 141. The computer program 141 may be executed by the processor 130 to implement the method for implementing a virtual home product layout scene according to an embodiment of the present invention.

[0048] In particular, the processor 130 may be configured to create a background image, and superimpose at least one element model to be laid out on the background image, obtain the user action captured by the somatosensory device 120, and arrange the element model in the background image in a posture responsive to the user action. Among them, at least one element model includes at least a three-dimensional model of a household product drawn in proportion to the actual size. For example, household appliances, furniture, etc. In the present invention, at least one is one, two, or more than two.

[0049] The virtual reality system 100 of the present invention captures user movements to arrange three-dimensional models of home products in a virtual background image, which can simply and conveniently enable users to freely arrange home products, give full play to the user's imagination, get rid of the limitations of external conditions such as manpower and material resources, and improve user experience.

[0050] In some embodiments, the processor 130 may be configured to obtain room information input by the user and create a background image according to the room information. The room information includes at least one of room size and room background to improve the practicality of the virtual reality system 100.

[0051] In some embodiments, before acquiring a specific user action captured by the somatosensory device 120 , the processor 130 may be configured to determine a target element model in at least one element model and associate the user action with the target element model to arrange the element models one by one.

[0052] Specifically, the processor 130 can be configured to obtain the virtual distance between the somatosensory device 120 and the element model located at the center of the field of view of the head-mounted display 110, and determine the element model as the target element model when the virtual distance is less than or equal to a preset distance threshold, and further associate the user action with the element model. The association process does not require the user to issue instructions, and can automatically determine the target element model and then arrange the model, thereby improving the intelligence of the virtual reality system 100.

[0053] After associating the user action with the target element model, the processor 130 may also be configured to change the display state of the target element model to an activated state to prompt the user to perform a layout operation on the element model. The visual effect of the activated state may be different from other element models, for example, by changing the display color or transparency of the target element model.

[0054] In an embodiment where the somatosensory device 120 is configured to capture hand movements of the user, the preset instruction movements may include at least a first hand movement, a second hand movement, and a third hand movement.

[0055] After associating the user action with the target element model, the processor 130 may be configured to configure the target element model to reduce the size and adsorb to the user's hand position in the background image in response to the user's first hand action, so that the target element model moves with the user's hand in the background image. In some exemplary embodiments, the first hand action may be a grabbing action.

[0056] The processor 130 may be configured to configure the target element model to restore the size and place the target element model at the user's hand position in the background image in response to the user's second hand motion, so as to place the target element model at a desired virtual position. In some exemplary embodiments, the second hand motion may be an open palm or a throwing motion.

[0057] The processor 130 may be configured to configure the target element model to rotate in response to a third hand motion of the user to change the posture of the target element model. In some exemplary embodiments, the third hand motion may be a wrist rotation motion.

[0058] In some embodiments, the processor 130 may be configured to modify the user action according to the action feature of the user action, so that the corresponding movement of the target element model is more consistent with the user's real intention. The action feature may be at least one of speed, acceleration, holding time at the same position, and holding time at the same posture.

[0059] Specifically, if the speed of the user's hand movement is less than or equal to a preset speed threshold or the acceleration of the user's hand movement is greater than or equal to a preset acceleration threshold, the target element model responds to the hand movement at a speed less than the speed of the user's hand movement to avoid undesired movement of the target element model. In the present invention, both the movement speed and the movement acceleration are scalars.

[0060] If the user's hand is held at the same position or the same posture for a time greater than or equal to a preset time threshold, the target element model is directly moved to the position or changed to the posture at a first speed to improve the response speed.

[0061] If the user's hand movement speed is greater than the preset speed threshold and the acceleration is less than the preset acceleration threshold, or the time the user's hand is in the same position and in the same posture is less than the preset time threshold, the target element model moves to the user's hand position in the background image at a second speed, and then responds to the hand movement at the same speed as the user's hand movement speed to avoid excessive movement delay.

[0062] In other words, when the user's hand movement speed is greater than the preset speed threshold and the acceleration is less than the preset acceleration threshold, or the user's hand is in the same position and the same posture and maintains a time less than the preset time threshold, if the movement of the target element model at this moment lags behind the user's hand movement, the target element model is first synchronized with the user's hand movement at a second speed, and then responds at the same speed; if the movement of the target element model and the user's hand movement at this moment are already in real-time response, the target element model continues to respond at the same speed.

[0063] In this embodiment, the first speed and the second speed may both be greater than the movement speed of the user's hand at that moment, and when the movement speeds of the user's hand at that moment are the same, the first speed may be greater than the second speed.

[0064] In some embodiments, the processor 130 may be configured to display the user as fully visible or partially visible in the background image to improve the user experience and provide a reference for the user to correct his or her movements. In an embodiment where the somatosensory device 120 is configured to capture the user's hand movements, the processor 130 may be configured to display only the user's hands as visible in the background image.

[0065] Figure 2 is a schematic flow chart of a method for implementing a virtual home product layout scene according to an embodiment of the present invention. Figure 2 The method for realizing a virtual home product layout scene of this embodiment can be realized by the virtual reality system 100 of any of the above embodiments, and the method may include the following steps:

[0066] Step S202: Create a background image. The background image can be used to present the scene to be laid out.

[0067] Step S204: Overlay at least one element model to be laid out on the background image. The at least one element model includes at least a three-dimensional model of a household product drawn in proportion to the actual size. The household product may include but is not limited to household appliances such as televisions, air conditioners, refrigerators, washing machines, kitchen appliances, and water heaters.

[0068] Step S206: acquiring the user action captured by the somatosensory device 120, so that the element model is arranged in the background image in a posture in response to the user action.

[0069] The method of the present invention arranges three-dimensional models of household products in a virtual background image by capturing user actions, which can simply and conveniently enable users to freely arrange household products, give full play to the user's imagination, get rid of the limitations of external conditions such as manpower and material resources, and improve user experience.

[0070] In some embodiments, step S204 may include the following steps:

[0071] Downloading corresponding product drawings from a server according to at least one product number input by a user or at least one product picture selected by a user;

[0072] Convert product drawings into element models, and add properties such as collision bodies, rigid bodies, friction, etc. to the element models.

[0073] In other embodiments, at least one element model may be pre-stored in the memory 140 for user selection.

[0074] In some embodiments, in step S206 , a target element model may be determined in at least one element model, and the user action may be associated with the target element model to arrange the element models one by one.

[0075] In some embodiments, in step S206, the user action may be specifically a hand action of the user, so as to facilitate more flexible presetting of command actions, so that small user actions can achieve interaction with the virtual image.

[0076] Step S206 may further include the following steps:

[0077] If the first hand movement captured by the somatosensory device 120 is obtained, the target element model is configured to be reduced in size and adsorbed to the user's hand position in the background image in response to the user's first hand movement, so that the target element model moves with the user's hand in the background image.

[0078] If the second hand motion captured by the somatosensory device 120 is obtained, the target element model is configured to restore the size in response to the user's second hand motion and be placed at the user's hand position in the background image with the current posture, so as to place the target element model at the desired virtual position.

[0079] If the third hand motion captured by the somatosensory device 120 is acquired, the target element model is configured to rotate in response to the third hand motion of the user, so as to change the posture of the target element model.

[0080] The above-mentioned first hand movement, second hand movement, and third hand movement can be configured according to the user interaction habits of virtual reality. For example, the first hand movement corresponds to the grasping movement of the hand, the second hand movement corresponds to the releasing movement (opening the palm or throwing, etc.), and the third hand movement corresponds to the flipping movement of the hand (turning the wrist, etc.).

[0081] In some embodiments, in step S206, the user action may be modified according to the action feature of the user action, so that the corresponding movement of the target element model is more consistent with the user's real intention. The action feature may be at least one of speed, acceleration, holding time at the same position, and holding time at the same posture.

[0082] Specifically, if the speed of the user's hand movement is less than or equal to the preset speed threshold or the acceleration of the user's hand movement is greater than or equal to the preset acceleration threshold, the target element model responds to the hand movement at a speed less than the speed of the user's hand movement to avoid the target element model from moving unexpectedly. In the present invention, both the speed of movement and the acceleration of movement are scalars. That is to say, when the speed of the user's hand movement is greater than the preset speed threshold and the acceleration is less than the preset acceleration threshold, or the user's hand is in the same position and the same posture is kept for less than the preset time threshold, if the movement of the target element model at this moment lags behind the user's hand movement, the target element model is first synchronized with the user's hand movement at the second speed, and then responds at the same speed; if the movement of the target element model at this moment and the user's hand movement are already in real-time response, the target element model continues to respond at the same speed. In this embodiment, the first speed and the second speed may both be greater than the speed of the user's hand movement at this moment, and when the speed of the user's hand movement at this moment is the same, the first speed may be greater than the second speed.

[0083] Figure 3 is a schematic and detailed flow chart of a method for implementing a virtual home product layout scene according to an embodiment of the present invention. Figure 3 The method for realizing a virtual home product layout scene of the present invention may include the following detailed steps:

[0084] Step S302: Acquire room information input by the user and at least one element model selected by the user. In this step, the room information includes at least one of room size and room background, so as to improve the practicality of the virtual reality system 100.

[0085] Step S304: Create a background image according to the room information.

[0086] Step S306: superimposing at least one element model to be laid out on the background image.

[0087] Step S308 : obtaining a virtual distance between the somatosensory device 120 and the element model located at the center of the field of view of the head mounted display 110 .

[0088] Step S310: Determine whether the virtual distance is less than or equal to a preset distance threshold. If so, execute step S312; if not, return to step S308.

[0089] Step S312: determining the element model to be associated as the target element model, associating the user action with the target element model and changing the display state of the target element model to an activated state to prompt the user to perform a layout operation on the element model.

[0090] Step S314: obtaining the user's hand movements captured by the somatosensory device 120 .

[0091] Step S316: Determine whether the speed of the user's hand motion is less than or equal to a preset speed threshold or whether the acceleration of the hand motion is greater than or equal to a preset acceleration threshold. If yes, execute step S318; if not, execute step S320.

[0092] Step S318: Make the target element model respond to the hand movement at a speed slower than the user's hand movement speed to avoid undesired movement of the target element model. Execute step S326.

[0093] Step S320: Determine whether the time for which the user's hand is held in the same position or in the same posture is greater than or equal to a preset time threshold. If so, execute step S322; if not, execute step S324.

[0094] Step S322: Make the target element model move directly to the position or change to the posture at the first speed to improve the response speed. Execute step S326.

[0095] Step S324: Move the target element model to the position of the user's hand in the background image at a second speed, and then respond to the hand movement at a speed that is the same as the movement speed of the user's hand to avoid excessive movement delay. Execute step S326.

[0096] Step S326: Determine whether a save or exit instruction is received. If so, end the output of the virtual image; if not, return to step S314 to continue acquiring the user's hand movements captured by the somatosensory device 120.

[0097] In the above virtual layout process, the background image may be displayed as fully visible or partially visible to the user, so as to improve the user experience and provide a reference for the user to correct his or her actions.

[0098] In a specific embodiment, the virtual reality system of this embodiment can be constructed by using the HTC vive VR helmet and its matching locator and leap motion gesture recognition device. The construction process of the virtual reality system of this embodiment may include:

[0099] Get three-dimensional drawings of household appliances. Each household appliance can have three types of drawings: large, medium and small. UG (Unigraphics NX) can be used in the drawing process.

[0100] The drawn drawings are imported into 3D animation rendering software (such as 3D animation rendering 3DsMax) and converted into 3D element models of home appliances.

[0101] For other 3D element models, you can directly use 3D animation rendering software for design. For example, beds, coffee tables, sofas, tiles and other furniture or decoration materials can be designed through 3DsMax.

[0102] Import home appliances and other 3D element models into virtual development platforms (such as Unity3D)

[0103] Connect the virtual development platform (Unity3D) in the SDK (software development kit) of the HTC vive virtual reality helmet, adjust the positioning device of the helmet, and build a virtual scene.

[0104] The leap motion gesture recognition device is connected to the HTC vive virtual reality helmet. Create hand attributes in Unity3D, add collision and rigid body attributes after placing it in the virtual scene. Leap motion recognizes hand information and reads hand status.

[0105] Write scripts to associate the hands with various parts, add properties such as collision and friction, etc. Use data collection algorithms to collect and calculate hand movement information.

[0106] When using the virtual reality system, the virtual distance between the hand and the 3D element model to be arranged is obtained through the leap motion gesture recognition device and the HTC vive virtual reality helmet. When the virtual distance meets the arrangement conditions, the target element model is activated (for example, the target element model is changed to green). Then, by detecting the user's hand movements, the target element model is made to move accordingly. For example, by defining a gesture bend grab, the target element model can be adsorbed in the hand, and the properties of the target element model can be changed to complete the layout operation.

[0107] In addition, the method of realizing the layout scene of virtual home products of this embodiment can also set layout conditions for the element models to be laid out, such as setting the priority of the layout, instructing the user to lay out the elements in order according to the priority of the element models, thereby instructing the user to gradually complete the layout. In addition, the method of this embodiment can also automatically generate restriction conditions for subsequent element models according to the layout completed by the user, for example, the element models that can be selected later can be restricted from the aspects of space size, overall cost, space function, and visual unity.

[0108] For example, after the user arranges tiles and wallpaper on the background image, the user can be recommended to layout larger furniture such as sofas and beds first, and then, based on the size of the space after layout, the furniture that does not meet the size requirements can be configured into an inoperable state.

[0109] For another example, after a coffee table and a sofa are arranged in a virtual space, it can be considered that the space may be arranged as a living room environment, and the element models that meet the functions of a living room can be displayed first.

[0110] For another example, before layout, the total layout price set by the user can be obtained. During the layout process, the element models that can be used subsequently are limited according to the prices of the products that have been laid out.

[0111] For another example, during the layout process, home appliances with suitable appearance can be recommended based on the visual effects of the layout.

[0112] The method of realizing a virtual home product layout scene of the present invention can more accurately identify user intentions, improve the intelligence of the virtual reality system 100, avoid unexpected repeated movements of the target element model, and does not require the user to perform complex actions, thereby shortening the user's learning process and providing an excellent user experience.

[0113] At this point, those skilled in the art should recognize that, although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived based on the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and recognized as covering all these other variations or modifications.

Claims

1. A method for realizing a virtual home product layout scene, comprising: Step A: Create background image; Step B: superimposing at least one element model to be laid out on the background image, wherein the at least one element model at least includes a three-dimensional model of a household product drawn in proportion to the actual size; Step C: obtaining a user action captured by a somatosensory device, so that the element model is arranged in the background image in a posture responsive to the user action; wherein, step C includes: Step C1: determining a target element model in the at least one element model, and associating the user action with the target element model; Step C2: modifying the user action according to the action features of the user action, wherein the user action includes a hand action; wherein step C2 includes: If the speed of the user's hand movement is less than or equal to a preset speed threshold or the acceleration of the user's hand movement is greater than or equal to a preset acceleration threshold, causing the target element model to respond to the hand movement at a speed less than the speed of the user's hand movement; and When the movement speed of the user's hand is greater than the preset speed threshold and the acceleration is less than the preset acceleration threshold, or the time the user's hand maintains the same position and the same posture is less than the preset time threshold, if the movement of the target element model lags behind the user's hand movement, the target element model is first synchronized with the user's hand movement at a second speed, and then responds at the same speed as the user's hand movement speed, and the second speed is greater than the user's hand movement speed at that moment.

2. The method according to claim 1, wherein: The step C1 comprises: Acquire a virtual distance between the somatosensory device and an element model located at a center of a field of view of a head mounted display; If the virtual distance is less than or equal to a preset distance threshold, the element model is determined as the target element model.

3. The method according to claim 1, wherein: After step C1, the method further includes: The display state of the target element model is changed to an activated state, and the visual effect of the activated state is different from that of other element models.

4. The method according to claim 1, wherein: After step C1, the method further includes: The target element model is configured to reduce in size and be attached to the user's hand position in the background image in response to the user's first hand motion; and / or Configuring the target element model to restore the size and place the target element model at the user's hand position in the background image in a current posture in response to a second hand motion of the user; and / or The target element model is configured to rotate in response to a third hand motion of the user.

5. The method according to claim 1, wherein step C2 further comprises: If the user's hand pauses at a certain position or a certain posture for a time period greater than or equal to the preset time threshold, the target element model is directly moved to the position or changed to the posture at the first speed.

6. The method according to claim 1, wherein: The step A comprises: Acquire room information input by a user, where the room information includes at least one of room size and room background; A background image is created according to the room information.

7. A virtual reality system, comprising: A head mounted display for outputting virtual images; A somatosensory device for capturing the user's movements; processor; as well as A memory storing a computer program, wherein the computer program, when executed by the processor, is used to implement the method for realizing a virtual home product layout scene according to any one of claims 1-6.

Images