VR (virtual reality) deep interaction method and interaction equipment for creative and cultural products

Abstract

The invention discloses a VR (virtual reality) deep interaction method and interaction equipment for cultural and creative products. The method comprises the following steps: establishing a three-dimensional model of an object through an electronic scanner; projecting a non-contact virtual image of the three-dimensional model of the object in the intelligent wearable component; the resonance unit is triggered by touching the non-contact virtual image, and the resonance unit triggers resonance reaction to start a sensing mode; and completing the deep interaction of the three-dimensional model of the touch object in the starting perception mode. According to the invention, the non-contact virtual image of the three-dimensional model of the object established by the electronic scanner is projected in the head-mounted intelligent wearing assembly, which is convenient for the user to view and emulate, and when the user touches the non-contact virtual image to trigger the resonance unit, the resonance unit triggers the resonance reaction to start the sensing mode, so that the deep interaction of the three-dimensional model of the touched object is completed, and the user experience is improved. The user is in zero-distance contact with the object three-dimensional model, and the real experience of the user is improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual interaction technology, and in particular to a cultural and creative product VR deep interaction method and interaction device. Background Art

[0002] Cultural and creative products are a product that combines cultural elements with creative design. In today's digital age, with the continuous advancement of VR technology, the combination of cultural and creative products and VR has become an innovative trend. In recent years, with the continuous development of the tourism industry, the dataization and virtualization of cultural and creative products have been gradually improved, and the virtual reality technology of cultural and creative products has been strengthened, which is convenient for users to improve their understanding of the historical background and close contact experience of cultural and creative products during the journey. For example, in the prior art, the publication number is CN115617222B, which discloses a VR virtual display system for cultural and creative products. By wearing VR glasses to watch cultural and creative products and animations in a specific area, it is convenient for the corresponding users to integrate into the viewing, improve the user's viewing experience, and facilitate the promotion and sale of related cultural and creative products. Wearing VR glasses to watch cultural and creative products can obtain a three-dimensional model of the cultural and creative products in a virtual environment, and combine the external explanation of the sound and music to improve the user experience. However, in the virtual environment, users cannot have a sense of touch on the cultural and creative products, and there is still a certain distance in the virtual matrix, which reduces the real experience of users. Summary of the invention

[0003] The purpose of the present invention is to provide a cultural and creative product VR deep interaction method and interaction device to solve the problems described in the background technology.

[0004] The technical solution of the present invention is achieved in this way:

[0005] On the one hand, a method for VR deep interaction of cultural and creative products is provided, the method comprising:

[0006] Create a three-dimensional model of the object using an electronic scanner;

[0007] Projecting a non-contact virtual image of the three-dimensional model of the object in the smart wearable component;

[0008] The resonance unit is triggered by touching the non-contact virtual image, and the resonance unit triggers a resonance reaction to start a perception mode;

[0009] When the perception mode is activated, depth interaction of touching the three-dimensional model of the object is completed.

[0010] A further technical solution is that the step of establishing a three-dimensional model of the object by means of an electronic scanner comprises:

[0011] Using a three-dimensional scanner to collect a three-dimensional coordinate image of the object, wherein the three-dimensional coordinate image includes the height, width, outer surface contour line and outer surface thickness of the object;

[0012] Purifying the three-dimensional coordinate image;

[0013] The processed three-dimensional coordinate image is used to create the three-dimensional model of the object.

[0014] A further technical solution is that the purification process of the three-dimensional coordinate image includes:

[0015] The three-dimensional coordinate image is subjected to noise reduction and filling processing, and the object texture is added during the noise reduction and filling processing.

[0016] A further technical solution is that projecting a non-contact virtual image of the three-dimensional model of the object in the smart wearable component includes:

[0017] The smart wearable component plays the three-dimensional model of the object to generate the non-contact virtual image;

[0018] In the smart wearable component, the non-contact virtual image is observed using an adjustable visual mirror, and the smart wearable component is adapted to broadcast the sound of touching the non-contact virtual image.

[0019] A further technical solution is that the non-contact virtual image observed by using an adjustable eyepiece includes:

[0020] According to the visual distance of the visual observer, the distance between the adjustable visual observer and the eyeball of the visual observer is adjusted, and the inclination angle of the visual observer is adjusted.

[0021] A further technical solution is that the non-contact virtual image is touched to trigger a resonance unit, and the resonance unit triggers a resonance reaction to start a perception mode, including:

[0022] The touch feedback component is mounted on the user's hand, and a tactile feedback element is installed in the touch feedback component, and the virtual image of the user's hand contacts the non-contact virtual image, so as to trigger the tactile feedback element to generate the resonance reaction with a particle effect;

[0023] In the tactile feedback element, a signal is outputted to a vibrator through a hand skeleton marker point, and the vibrator feeds back the tactile sensation to the user's hand, thereby completing a perception mode.

[0024] On the other hand, the present invention provides a cultural and creative product VR deep interaction device, including a smart wear component module, a scanning module, a projection module, a resonance module and an observation module, wherein the smart wear component module is provided with the scanning module, the projection module and the observation module, and the resonance module is electrically connected to the smart wear component module;

[0025] The scanning module is used to establish a three-dimensional model of the object, the projection module is used to project the three-dimensional model of the object into a non-contact virtual image in the smart wearable component, the resonance module is used to touch the non-contact virtual image to trigger the resonance unit, the resonance unit triggers the resonance reaction to start the perception mode, and the observation module is used to watch the non-contact virtual image.

[0026] A further technical solution is that the smart wearable component includes a wearing box, in which the scanning module, the projection module and the observation module are arranged, wherein the scanning module includes a three-dimensional scanner, the input end of the three-dimensional scanner is used to scan an object, and the input end is input into the smart wearable component through the output end, and the output end of the smart wearable component outputs the three-dimensional model of the object, which is projected through the projection module.

[0027] A further technical solution is that the resonance module includes a feedback glove, the outer surface of the feedback glove is provided with a plurality of marking points and a linear motor, the marking points and the three-dimensional model of the object are reacted through light particles to determine touch, and the linear motor is used to simulate gravity.

[0028] A further technical solution is that the observation module includes two pupils and a shell, each pupil includes a first eyepiece and a second eyepiece, the shell has the first eyepiece and the second eyepiece built in, and the first eyepiece and the second eyepiece are used to observe the three-dimensional model of the object, a micro-electric push rod is provided in the shell, and the side edge of the first eyepiece is slidably connected to the inner wall of the shell, and one side is connected to the micro-electric push rod;

[0029] A micro motor is connected to the side of the second eyepiece, and the micro motor is connected to the control center of the smart wearable component. At the same time, the input end of the control center is electrically connected to an external light marking point.

[0030] The beneficial effects of the present invention are:

[0031] In the head-mounted smart wearable component, an electronic scanner is projected to establish a non-contact virtual image of the three-dimensional model of the object, which is convenient for users to observe. When the user touches the non-contact virtual image to trigger the resonance unit, the resonance unit triggers the resonance reaction to start the perception mode, completing the deep interaction of the three-dimensional model of the touched object, allowing the user to have zero-distance contact with the three-dimensional model of the object, thereby improving the user's real experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of the method of the present invention;

[0033] Figure 2 Processing flow chart for 3D models;

[0034] Figure 3 A flowchart for the occurrence of perception;

[0035] Figure 4 A flow chart for touch generation;

[0036] Figure 5 is a schematic diagram of an interactive device of the present invention;

[0037] In the figure, 1. wearing box; 2. outer shell; 3. micro-electric push rod; 4. micro-motor; 5. first eyepiece; 6. second eyepiece. DETAILED DESCRIPTION

[0038] In order to better understand the technical content of the present invention, specific embodiments are provided below, and the present invention is further described in conjunction with the accompanying drawings.

[0039] See also Figure 1 On the one hand, the present invention provides a method for deep interaction of cultural and creative products VR. In the head-mounted smart wearable component, a non-contact virtual image of a three-dimensional model of an object is projected by an electronic scanner, which is convenient for users to observe. When the user touches the non-contact virtual image to trigger the resonance unit, the resonance unit triggers the resonance reaction to start the perception mode, and completes the deep interaction of the three-dimensional model of the touched object, so that the user has zero-distance contact with the three-dimensional model of the object, thereby improving the user's real experience.

[0040] In this embodiment, the method includes the following steps:

[0041] Step S1: Create a three-dimensional model of the object through an electronic scanner, such as scanning a model of a historical relic or a hand-made physical model, which is stored in the system after scanning for subsequent screening.

[0042] Specifically, it includes:

[0043] Step S101: using a 3D scanner to collect a 3D coordinate image of the object, wherein the 3D coordinate image includes the height, width, outer surface contour and outer thickness of the object;

[0044] Step S102: performing purification processing on the three-dimensional coordinate image, including performing noise reduction and filling processing on the three-dimensional coordinate image, and adding the object texture during the noise reduction and filling processing;

[0045] Step S103: creating a three-dimensional model of the object using the processed three-dimensional coordinate image.

[0046] Exemplarily, the 3D scanner uses the EinScan Pro EP model to scan the cultural and creative products that need to be scanned, including but not limited to a comprehensive scan of the height, width, outer surface contour line and outer thickness of the cultural and creative products, to facilitate subsequent processing. Then, the images of the cultural and creative products that have been scanned are processed by noise reduction and filling processing to improve the cleanliness of the images of the cultural and creative products. At the same time, the actual texture of the cultural and creative products is added to the images of the cultural and creative products to improve the realism of the images. Finally, the 3D model of the cultural and creative products is established by completing the images after noise reduction and filling processing.

[0047] Specifically, the specific method of noise reduction is: determine the area size in the neighborhood map of each cultural and creative product image, set the voxel of each cultural and creative product image as a 3×3×3 cubic neighborhood, and for each voxel, calculate the sum of all voxel values ​​in its neighborhood, and then divide it by the number of voxels in the neighborhood to get a new voxel value, thereby effectively removing a certain degree of noise. The filling processing method is: select a suitable structural element of the cultural and creative product image, usually the structural element is a sphere. Then, move the structural element on the cultural and creative product image. When the structural element contacts the empty area, set the voxel value covered by the structural element to the voxel value of the non-empty area, so as to quickly fill the missing part of the cultural and creative product image.

[0048] In this embodiment, the method includes the following steps:

[0049] Step S2: Projecting a non-contact virtual image of the three-dimensional model of the object in the smart wearable component. The smart wearable component can project a non-contact virtual image of the corresponding cultural and creative product in a virtual environment for viewing.

[0050] Specifically, it includes:

[0051] Step S201: the smart wearable component plays the three-dimensional model of the object to generate the non-contact virtual image.

[0052] Step S202: In the smart wearable component, the non-contact virtual image is observed by using an adjustable visual mirror, and the broadcast of the sound played by touching the non-contact virtual image is adapted in the smart wearable component. According to the visual distance of the visual person, the distance between the adjustable visual mirror and the visual person's eyeball is adjusted, and the inclination angle of the visual mirror is adjusted.

[0053] For example, the non-contact virtual image is a projected image of a three-dimensional model of a cultural and creative product, which is convenient for customers to watch. When the user watches, the adjustable eyepiece is adjusted according to the user's visual distance, so that the non-contact virtual image can be clearly seen, thereby improving the user experience. At the same time, the angle of the eyepiece can also be adjusted to the best lighting rate to improve the best image quality of the projection.

[0054] In this embodiment, the method includes the following steps:

[0055] Step S3: The resonance unit is triggered by touching the non-contact virtual image, and the resonance unit triggers a resonance reaction to start the perception mode. The resonance unit simulates the hand touching the non-contact virtual image, and the non-contact virtual image can be flipped and cut open, and a sense of simulation is generated while touching, and the sense of reality of touching the real object can be felt through the resonance unit. At the same time, the sound generated by touching the non-contact virtual image is played by onomatopoeia broadcasting, and when a specific component is touched, the onomatopoeia broadcasting plays the origin and function of the component, thereby improving user cognition and enhancing the user's sense of immersion when using it.

[0056] In this embodiment, the method includes the following steps:

[0057] Step S4: When the perception mode is started, depth interaction of touching the three-dimensional model of the object is completed.

[0058] See also Figure 2 and Figure 3 On the other hand, the present invention provides an interactive device using a cultural and creative product VR deep interaction method, including a smart wearable component module, a scanning module, a projection module, a resonance module and an observation module, wherein the smart wearable component module is provided with the scanning module, the projection module and the observation module, and the resonance module is electrically connected to the smart wearable component module; the scanning module is used to establish a three-dimensional model of an object by a scanner, the projection module is used to project a non-contact virtual image of the three-dimensional model of the object in the smart wearable component, the resonance module is used to touch the non-contact virtual image to trigger a resonance unit, the resonance unit triggers a resonance reaction to start a perception mode, and the observation module is used to watch the non-contact virtual image.

[0059] For example, first, a scanning module is used to scan the cultural and creative products to obtain a three-dimensional model of the cultural and creative products, and the model is stored in the smart wearable component. The user wears the smart wearable component on his head and starts the projection module to project the three-dimensional model of the cultural and creative product from the smart wearable component into a non-contact virtual image, which is convenient for the user to watch through the observation module. Then, when the user wants to touch the non-contact virtual image, the resonance module is activated, and when the user's hand touches the non-contact virtual image, a sense of touch, friction, and gravity is generated, thereby improving the user's real interaction reaction with the non-contact virtual image.

[0060] Preferably, the smart wearable component includes a wearing box 1, in which the scanning module, the projection module and the observation module are arranged, wherein the scanning module includes a three-dimensional scanner, the input end of the three-dimensional scanner is used to scan an object, and the input end is input into the smart wearable component through the output end, and the output end of the smart wearable component outputs the three-dimensional model of the object, which is projected through the projection module.

[0061] Furthermore, the resonance module includes a feedback glove, the outer surface of the feedback glove is provided with a plurality of marking points and a linear motor, the marking points and the three-dimensional model of the object are reacted with light particles to determine touch, and the linear motor is used to simulate gravity.

[0062] Exemplarily, the wearing box 1 is used to be worn on the head, and it is equipped with an elastic rope to be fastened to the head, and the image of the cultural and creative product is observed through the visual mirror in the wearing box 1. Before use, the cultural and creative product is scanned by a three-dimensional scanner, so that the cultural and creative product is stored in the storage of the wearing box 1. During scanning, the three-dimensional scanner obtains the structure and outline of the cultural and creative product through the visual mirror. After the scanning is completed, the image of the cultural and creative product stored in the storage is played through a micro projector. A plurality of LED light sources are provided on the outer surface of the feedback glove corresponding to the fingers and joints. The LED light sources are captured by a computer, and the image is obtained by a three-dimensional scanner. A hand animation is formed in a non-contact virtual image. The hand animation is consistent with the feedback glove action. When the user wears the feedback glove, when the user's fingers move, the hand animation moves in real time. When the hand animation contacts the cultural and creative product, a light particle reaction is generated to identify the contact between the hand animation and the three-dimensional model of the cultural and creative product. Then, the controller conducts and starts the linear motor to simulate gravity, press the user's hand, and improve the sense of reality. At the same time, the physical engine is used to simulate the friction of the feedback gloves, allowing users to truly experience the structural friction of cultural and creative products through the feedback gloves.

[0063] Preferably, the observation module includes two pupils and a shell 2, each of the pupils includes a first eyepiece 5 and a second eyepiece 6, the shell 2 has the first eyepiece 5 and the second eyepiece 6 built in, and the first eyepiece 5 and the second eyepiece 6 are used to observe the three-dimensional model of the object, a micro-electric push rod 3 is provided in the shell 2, the side of the first eyepiece 5 is slidably connected to the inner wall of the shell 2, one side of which is connected to the micro-electric push rod 3; the side of the second eyepiece 6 is connected to a micro motor, and the micro motor is connected to the control center of the smart wearable component. At the same time, the input end of the control center is electrically connected to an external light marking point.

[0064] It should be noted that the adjustment range between the two pupil lenses is 58mm-74mm, and the existing hinge axis is used for rotation, which ensures that users get a better visual experience, reduce dizziness, and improve comfort.

[0065] For example, the controller starts the micro-electric push rod 3 to slide the first eyepiece 5 in the housing 2, and can move toward the second eyepiece 6, or move in the opposite direction of the second eyepiece 6, to a position suitable for the user to clearly observe, thereby improving the user experience. At the same time, the controller starts the micro-motor 4 to rotate the second eyepiece 6, so that the second eyepiece 6 can reach a comfortable slope for the user to observe, improve the non-contact virtual image picture sense, and make the hand animation in the non-contact virtual image have micro-light particles when it contacts the three-dimensional model of the cultural and creative product, thereby increasing the user's touch playability. Among them, the controller is an STM32 single-chip microcomputer controller.

[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for VR deep interaction of cultural and creative products, characterized in that: The method includes: Create a three-dimensional model of the object using an electronic scanner; Projecting a non-contact virtual image of the three-dimensional model of the object in the smart wearable component; The resonance unit is triggered by touching the non-contact virtual image, and the resonance unit triggers a resonance reaction to start a perception mode; When the perception mode is activated, depth interaction of touching the three-dimensional model of the object is completed.

2. A method for VR deep interaction of cultural and creative products according to claim 1, characterized in that: The method of establishing a three-dimensional model of the object by means of an electronic scanner comprises: Using a three-dimensional scanner to collect a three-dimensional coordinate image of the object, wherein the three-dimensional coordinate image includes the height, width, outer surface contour line and outer surface thickness of the object; Purifying the three-dimensional coordinate image; The processed three-dimensional coordinate image is used to create the three-dimensional model of the object.

3. A method for VR deep interaction of cultural and creative products according to claim 2, characterized in that: The purification process of the three-dimensional coordinate image comprises: The three-dimensional coordinate image is subjected to noise reduction and filling processing, and the object texture is added during the noise reduction and filling processing.

4. A method for VR deep interaction of cultural and creative products according to claim 1, characterized in that: The projecting of a non-contact virtual image of the three-dimensional model of the object in the smart wearable component comprises: The smart wearable component plays the three-dimensional model of the object to generate the non-contact virtual image; In the smart wearable component, the non-contact virtual image is observed using an adjustable visual mirror, and the smart wearable component is adapted to broadcast the sound of touching the non-contact virtual image.

5. A method for VR deep interaction of cultural and creative products according to claim 4, characterized in that: The method of observing the non-contact virtual image by using an adjustable eyepiece includes: According to the visual distance of the visual observer, the distance between the adjustable visual observer and the eyeball of the visual observer is adjusted, and the inclination angle of the visual observer is adjusted.

6. The method for VR deep interaction of cultural and creative products according to claim 1, characterized in that: The triggering of the resonance unit by touching the non-contact virtual image, wherein the resonance unit triggers a resonance reaction to start the perception mode, comprises: The touch feedback component is mounted on the user's hand, and a tactile feedback element is installed in the touch feedback component, and the virtual image of the user's hand contacts the non-contact virtual image, so as to trigger the tactile feedback element to generate the resonance reaction with a particle effect; In the tactile feedback element, a signal is outputted to a vibrator through a hand skeleton marker point, and the vibrator feeds back the tactile sensation to the user's hand, thereby completing a perception mode.

7. A device using the cultural and creative product VR deep interaction method according to any one of claims 1 to 6, characterized in that: It includes a smart wear component module, a scanning module, a projection module, a resonance module and an observation module, wherein the smart wear component module is provided with the scanning module, the projection module and the observation module, and the resonance module is electrically connected to the smart wear component module; The scanning module is used to establish a three-dimensional model of the object, the projection module is used to project the three-dimensional model of the object into a non-contact virtual image in the smart wearable component, the resonance module is used to touch the non-contact virtual image to trigger the resonance unit, the resonance unit triggers the resonance reaction to start the perception mode, and the observation module is used to watch the non-contact virtual image.

8. The VR deep interaction device for cultural and creative products according to claim 7, characterized in that: The smart wearable component includes a wearing box, in which the scanning module, the projection module and the observation module are arranged, wherein the scanning module includes a three-dimensional scanner, the input end of the three-dimensional scanner is used to scan an object, and the input end is input into the smart wearable component through the output end, and the output end of the smart wearable component outputs the three-dimensional model of the object, which is projected through the projection module.

9. The cultural and creative product VR deep interaction device according to claim 7, characterized in that: The resonance module includes a feedback glove, wherein a plurality of marking points and a linear motor are arranged on the outer surface of the feedback glove, wherein the marking points and the three-dimensional model of the object are reacted with light particles to determine touch, and the linear motor is used to simulate gravity.

10. A cultural and creative product VR deep interaction device according to claim 9, characterized in that: The observation module includes two pupils and a shell, each pupil includes a first eyepiece and a second eyepiece, the shell has the first eyepiece and the second eyepiece built in, and is used to observe the three-dimensional model of the object through the first eyepiece and the second eyepiece, the shell is provided with a micro-electric push rod, the side edge of the first eyepiece is slidably connected to the inner wall of the shell, and one side is connected to the micro-electric push rod; A micro motor is connected to the side of the second eyepiece, and the micro motor is connected to the control center of the smart wearable component. At the same time, the input end of the control center is electrically connected to an external light marking point.

Images