A virtual museum guiding method, device, equipment and storage medium

Abstract

The present invention relates to the fields of information technology and virtual reality technology, and discloses a virtual museum guidance method, apparatus, device, and storage medium. The method comprises: obtaining the current state of a VR head-mounted display device, and when the VR head-mounted display device is currently in a wearing state, obtaining virtual exhibits in a virtual museum; displaying the virtual exhibits and obtaining the amplitude of head or hand movement; determining a demonstration video of interaction between a presenter and the virtual exhibits based on the amplitude of movement; replacing the background portion of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background; playing the demonstration video with the semi-transparent background, and after completion of the playback, capturing gestures of the user interacting with the virtual exhibits; and displaying guidance content for the virtual museum when the gestures are not standard. The present invention shows users how to interact with virtual exhibits in a virtual museum through guidance content, thereby improving the efficiency of interaction between users and virtual exhibits.

Description

Technical Field

[0001] The present invention relates to the fields of information technology and virtual reality technology, and in particular to a virtual museum guiding method, device, equipment and storage medium. Background Art

[0002] A virtual museum is an online exhibition space created using digital technology. It presents the collections, exhibitions, and cultural information of a physical museum in a digital form. Breaking the limitations of physical space, virtual museums make cultural resources more accessible and accessible, offering visitors a new way to explore and learn about cultural heritage anytime, anywhere.

[0003] However, the process of acquiring existing virtual museum guide content is cumbersome, hindering efficiency. This is because existing technologies primarily rely on manual acquisition, which consumes significant human and time resources, increasing the time required to acquire virtual museum guide content. Furthermore, most virtual museum guide content is in text and voice, which is not intuitive enough. Therefore, this approach hinders the quality of virtual museum guide content. Summary of the Invention

[0004] The present invention provides a virtual museum guidance method, device, computer equipment and storage medium to solve the technical problem that the acquisition process of the existing virtual museum guidance content is cumbersome and is not conducive to improving the efficiency of obtaining the guidance content.

[0005] In a first aspect, a virtual museum guiding method is provided, comprising:

[0006] Get the current state of the VR headset. When the VR headset is currently in the wearing state, get the permission level corresponding to the device identifier. When the permission level is higher than the preset level, get the virtual exhibits in the virtual museum.

[0007] Displaying a virtual exhibit, and obtaining a head or hand movement amplitude within a preset time from the start of displaying the virtual exhibit;

[0008] Determine the demonstration video of the interaction between the presenter and the virtual exhibit based on the movement amplitude;

[0009] Replace the background part of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background;

[0010] Play a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capture the user's gestures when interacting with the virtual exhibits;

[0011] When the gesture action is not a standard action, a gesture animation is obtained based on a predefined method, the gesture animation is selected as the guide content of the virtual museum, and the guide content of the virtual museum is displayed.

[0012] Furthermore, the displaying of the virtual exhibit, starting from the start of displaying the virtual exhibit, obtaining the movement amplitude of the head or hand within a preset time, includes:

[0013] Displaying a virtual exhibit, obtaining a recording instruction, executing the recording instruction, and recording the start time of displaying the virtual exhibit;

[0014] The movement range of the head or hand is obtained within a preset time from the start of display of the virtual exhibit.

[0015] Furthermore, the method of determining a demonstration video of interaction between a presenter and a virtual exhibit based on the movement amplitude includes:

[0016] When the movement amplitude is smaller than a preset amplitude, accessing a preset file, and obtaining a plurality of videos of the presenter and the virtual exhibit interacting in the preset file;

[0017] Get the score and playback frequency corresponding to each video, multiply the score and playback frequency corresponding to each video to get the evaluation value corresponding to each video, and select the video with the highest evaluation value as the demonstration video.

[0018] Furthermore, replacing the background portion of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background includes:

[0019] Input the demonstration video into an image processing tool, process the demonstration video using the image processing tool, and obtain the background part of the demonstration video;

[0020] A replacement instruction is obtained and executed to replace a background portion in the demonstration video with a semi-transparent background, thereby obtaining a demonstration video with a semi-transparent background.

[0021] Furthermore, the playing of the demonstration video with the semi-transparent background and capturing the user's gestures for interacting with the virtual exhibit after the playing of the demonstration video with the semi-transparent background is completed include:

[0022] Playing a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capturing an action of a user grabbing a virtual exhibit, an action of a user rotating the virtual exhibit, an action of a user clicking on the virtual exhibit, and an action of a user waving the virtual exhibit;

[0023] The actions of the user grabbing the virtual exhibit, the user rotating the virtual exhibit, the user clicking the virtual exhibit, and the user waving the virtual exhibit are selected as gesture actions.

[0024] Furthermore, when the gesture action is not a standard action, obtaining a gesture animation based on a predefined method, selecting the gesture animation as the guide content of the virtual museum, and displaying the guide content of the virtual museum include:

[0025] When the gesture action is not a standard action, multiple posture data and model files corresponding to the standard action are obtained, a hand model is loaded from the model file through a loading script, the multiple posture data are mapped to the skeleton of the hand model, and a hand model carrying the multiple posture data is obtained. The hand model carrying the multiple posture data is input into a graphics rendering engine to obtain multiple frames of images simulating the standard action;

[0026] According to the chronological order, multiple frames of images simulating standard actions are encoded to generate gesture animations, and the gesture animations are selected as the guiding content of the virtual museum, and the guiding content of the virtual museum is displayed.

[0027] Furthermore, when the gesture action is not a standard action, based on a predefined method, a gesture animation is obtained, the gesture animation is selected as the guide content of the virtual museum, and after the guide content of the virtual museum is displayed, the virtual museum guidance method includes:

[0028] Create a storage area, store the guide content of the virtual museum in the storage area, and create an access interface for the storage area.

[0029] In a second aspect, a virtual museum guiding device is provided, comprising:

[0030] The first acquisition module is used to obtain the current state of the VR head display device. When the current state of the VR head display device is the wearing state, the permission level corresponding to the device identifier is obtained. When the permission level is higher than the preset level, the virtual exhibits in the virtual museum are obtained;

[0031] The second acquisition module is used to display the virtual exhibit and acquire the movement amplitude of the head or hand within a preset time from the start of display of the virtual exhibit;

[0032] a determination module for determining a demonstration video in which a presenter and a virtual exhibit interact based on the movement amplitude;

[0033] A replacement module, used to replace the background part of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background;

[0034] a playback module, configured to play a demonstration video with a semi-transparent background, and capture gestures of a user interacting with a virtual exhibit after the demonstration video with the semi-transparent background has finished playing;

[0035] The guidance module is used to obtain the gesture animation based on a predefined method when the gesture action is not a standard action, select the gesture animation as the guidance content of the virtual museum, and display the guidance content of the virtual museum.

[0036] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned virtual museum guiding method when executing the computer program.

[0037] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned virtual museum guiding method are implemented.

[0038] The present application provides a virtual museum guidance method, apparatus, computer equipment and storage medium, which obtain the current state of a VR head display device, obtain the authority level corresponding to the device identifier when the current state of the VR head display device is a wearing state, and obtain the virtual exhibits in the virtual museum when the authority level is higher than a preset level; display the virtual exhibits, obtain the movement amplitude of the head or hand within a preset time from the start time of displaying the virtual exhibits; determine a demonstration video of the interaction between the presenter and the virtual exhibits based on the movement amplitude; replace the background part of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background; play the demonstration video with a semi-transparent background, and after the demonstration video with a semi-transparent background is played, capture the gestures of the user interacting with the virtual exhibits; when the gesture is not a standard gesture, obtain a gesture animation based on a predefined method, and select a gesture animation The invention provides the guiding content of the virtual museum and displays the guiding content of the virtual museum. The beneficial effects are in two aspects. On the one hand, when the gesture action is not a standard action, the gesture animation is obtained based on a predefined method, and the gesture animation is selected as the guiding content of the virtual museum to display the guiding content of the virtual museum. Since there is no need to manually obtain the guiding content of the virtual museum, the obtaining time of the guiding content of the virtual museum is reduced, which is conducive to improving the obtaining efficiency of the guiding content of the virtual museum; on the other hand, the gesture animation is selected as the guiding content of the virtual museum, and the standard action is simulated by the gesture animation, which is conducive to guiding the user to use the standard action to interact with the virtual exhibits, without the need for the user to perform additional voice analysis or text reading. Therefore, compared with voice and text, gesture animation has a more intuitive expression advantage, so it can reduce the guiding time of the virtual museum, which is conducive to improving the guiding efficiency of the virtual museum. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 This is a schematic diagram of an application environment of a virtual museum guidance method according to an embodiment of the present invention;

[0041] Figure 2 A schematic flow chart of a virtual museum guidance method provided by one embodiment of the present invention;

[0042] Figure 3 yes Figure 2 A schematic flow chart of a specific implementation of step S23;

[0043] Figure 4 yes Figure 2 A schematic flow chart of a specific implementation of step S25;

[0044] Figure 5 yes Figure 2 A schematic flow chart of a specific implementation of step S26;

[0045] Figure 6 is a schematic structural diagram of a virtual museum guiding device according to an embodiment of the present invention;

[0046] Figure 7 It is a structural diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] See also Figure 1 , Figure 1 This is a schematic diagram of an application environment of a virtual museum guidance method according to an embodiment of the present invention. The virtual museum guidance method provided by the embodiment of the present invention can be applied in the following situations: Figure 1 In an application environment, a client communicates with a server through a network.

[0049] The server obtains the current state of the VR headset through the client. When the VR headset is currently in the wearing state, it obtains the permission level corresponding to the device identifier. When the permission level is higher than the preset level, it obtains the virtual exhibits in the virtual museum.

[0050] Displaying a virtual exhibit, and obtaining a head or hand movement amplitude within a preset time from the start of displaying the virtual exhibit;

[0051] Determine the demonstration video of the interaction between the presenter and the virtual exhibit based on the movement amplitude;

[0052] Replace the background part of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background;

[0053] Playing a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capturing the user's gestures when interacting with the virtual exhibits;

[0054] When the gesture action is not a standard action, a gesture animation is obtained based on a predefined method, the gesture animation is selected as the guide content of the virtual museum, and the guide content of the virtual museum is displayed.

[0055] In the scheme implemented by the above-mentioned virtual museum guidance method, device, equipment and medium, the beneficial effects are in two aspects. On the one hand, when the gesture action is not a standard action, the gesture animation is obtained based on a predefined method, and the gesture animation is selected as the guidance content of the virtual museum to display the guidance content of the virtual museum. Since there is no need to manually obtain the guidance content of the virtual museum, the time for obtaining the guidance content of the virtual museum is reduced, which is conducive to improving the efficiency of obtaining the guidance content of the virtual museum; on the other hand, the gesture animation is selected as the guidance content of the virtual museum, and the standard action is simulated by the gesture animation, which is conducive to guiding the user to use standard actions to interact with the virtual exhibits, without the user having to perform additional voice analysis or text reading. Therefore, compared with voice and text, gesture animation has a more intuitive expression advantage, thereby reducing the guidance time of the virtual museum and helping to improve the guidance efficiency of the virtual museum.

[0056] The device running the client is referred to as a client device.

[0057] Among them, the device running the server is referred to as: server device.

[0058] Among them, client devices include but are not limited to VR head-mounted display devices, smartphones, personal computers, Internet of Vehicles terminals, tablets and portable wearable devices.

[0059] The server device can be implemented as an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments. Figure 2 , Figure 2 A flowchart of a virtual museum guidance method provided in one embodiment of the present invention includes the following steps:

[0060] S21, obtaining the current state of the VR head display device. When the current state of the VR head display device is the wearing state, obtaining the permission level corresponding to the device identifier. When the permission level is higher than the preset level, obtaining the virtual exhibits in the virtual museum;

[0061] VR (Virtual Reality) is the core technology foundation of VR headsets. VR headsets use computer technology to create a three-dimensional virtual environment that is completely different from the real world. VR headsets are devices that integrate high-precision displays, advanced sensors, and complex processing systems.

[0062] VR headsets use specialized display technology to present stereoscopic images. Precise sensors track the user's head movements, updating the virtual perspective in real time, allowing the user to experience a truly immersive three-dimensional environment. VR headsets, a crucial enabler of virtual reality technology, require both a high level of technical integration and powerful computing power to support a smooth virtual experience.

[0063] Exemplarily, obtaining the current state of a VR head-mounted display device, and when the current state of the VR head-mounted display device is a wearing state, obtaining the permission level corresponding to the device identifier, and when the permission level is higher than a preset level, obtaining virtual exhibits in a virtual museum, including:

[0064] Obtaining a current state of the VR head display device. When the current state of the VR head display device is a wearing state, displaying multiple virtual scene identifiers, and obtaining a virtual scene identifier selected by the user from the multiple virtual scene identifiers;

[0065] When the virtual scene is identified as a virtual museum, the device identification of the VR head display device is obtained, and the permission level corresponding to the device identification is obtained. When the permission level is higher than the preset level, the 3D file corresponding to the virtual exhibit is obtained, and the 3D file is input into the graphics processor in the VR head display device. The 3D file is rendered by the graphics processor to obtain the virtual exhibits in the virtual museum.

[0066] The virtual scene identifier is an identifier corresponding to the virtual scene.

[0067] Among them, different virtual scene identifiers correspond to different virtual scene identifiers.

[0068] Among them, virtual exhibits are exhibits that exist in virtual form. Virtual exhibits are not physical objects, but digital display content created using digital technology.

[0069] Among them, virtual exhibits include but are not limited to bronzes that exist in virtual form, ceramics that exist in virtual form, calligraphy and painting works that exist in virtual form, rubbings of inscriptions on monuments that exist in virtual form, animal and plant specimens that exist in virtual form, mineral fossils that exist in virtual form, stone tools that exist in virtual form, and jade artifacts that exist in virtual form.

[0070] S22, displaying a virtual exhibit, and obtaining a head or hand movement amplitude within a preset time from the start of displaying the virtual exhibit;

[0071] Among them, virtual exhibits can be displayed to users in a more vivid and intuitive way, which not only enhances the user's immersion and participation, enabling users to have a deeper understanding of the historical background and cultural connotations of the exhibits, but also improves the display efficiency and protection level of the exhibits.

[0072] The display of the virtual exhibit, starting from the start of display of the virtual exhibit, obtaining the movement amplitude of the head or hand within a preset time, includes:

[0073] Displaying a virtual exhibit, obtaining a recording instruction, executing the recording instruction, and recording the start time of displaying the virtual exhibit;

[0074] The movement range of the head or hand is obtained within a preset time from the start of display of the virtual exhibit.

[0075] For ease of explanation, the following examples are given:

[0076] For example, if the virtual exhibit starts displaying at 2:30 and the preset time is 1 minute, the head or hand movement amplitude is acquired within 1 minute starting from 2:30, that is, the head or hand movement amplitude is acquired between 2:30 and 2:31.

[0077] For example, the virtual exhibit starts displaying at 2:40 and the preset time is 2 minutes.

[0078] From 2:40 onwards, obtain the movement range of the head or hand within 2 minutes, that is, obtain the movement range of the head or hand between 2:40 and 2:42.

[0079] Among them, the user's behavior pattern can be understood by obtaining the movement amplitude of the head or hand within a preset time from the start of the display of the virtual exhibit.

[0080] S23, based on the movement amplitude, determining a demonstration video of the interaction between the presenter and the virtual exhibit;

[0081] Among them, a small movement amplitude indicates that the user is confused or unfamiliar with the interactive methods of the virtual exhibit. This is because when users interact with virtual exhibits, they perform corresponding actions based on their own understanding and experience. If users are confused about the interactive methods of virtual exhibits, they tend to show hesitation and uncertainty in their behavior, which may be reflected in their physical movements as small movements. This is because users may be unsure of how to proceed or fear making mistakes, so they adopt a cautious and tentative approach. Therefore, a small movement amplitude can serve as an indicator of user confusion or unfamiliarity.

[0082] If the movement amplitude is too small, a demonstration video of the presenter interacting with the virtual exhibit is obtained. The demonstration video can intuitively show the correct operation methods and steps, helping users quickly understand and master the operation skills.

[0083] S24, replacing the background portion of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background;

[0084] The step of replacing the background portion of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background includes:

[0085] Input the demonstration video into an image processing tool, process the demonstration video using the image processing tool, and obtain the background part of the demonstration video;

[0086] A replacement instruction is obtained and executed to replace a background portion in the demonstration video with a semi-transparent background, thereby obtaining a demonstration video with a semi-transparent background.

[0087] The background of the demo video is partially replaced with a semi-transparent background, resulting in a demo video with a semi-transparent background. This background preserves some of the visual information of the demo video while allowing for overlaying other elements or backgrounds, enhancing the video's flexibility and editability. Furthermore, the semi-transparent background can highlight the video content, reduce distractions, and help users focus on the key information in the video.

[0088] S25, playing a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capturing gestures of the user interacting with the virtual exhibit;

[0089] Wherein, a demonstration video with a semi-transparent background is played, and the user watches the demonstration video, and the user can learn standard movements in the demonstration video with a semi-transparent background.

[0090] VR headsets, through integrated sensors and tracking systems, can capture the user's gestures as they interact with virtual exhibits in real time. By capturing these gestures, the system can understand the user's confusion or needs during the interaction process, thereby providing targeted guidance and assistance.

[0091] S26 , when the gesture action is not a standard action, obtaining a gesture animation based on a predefined method, selecting the gesture animation as the guide content of the virtual museum, and displaying the guide content of the virtual museum.

[0092] Among them, gesture animation is obtained, gesture animation is selected as the guidance content of the virtual museum, the guidance content of the virtual museum is displayed, and the interaction method of virtual exhibits in the virtual museum is shown to the user through the guidance content, which is conducive to improving the interaction efficiency between the user and the virtual exhibits.

[0093] Wherein, when the gesture action is not a standard action, based on a predefined method, a gesture animation is obtained, the gesture animation is selected as the guide content of the virtual museum, and after the guide content of the virtual museum is displayed, the virtual museum guidance method includes:

[0094] Create a storage area, store the guide content of the virtual museum in the storage area, and create an access interface for the storage area.

[0095] Accessing the virtual museum's guide content through an interface allows for a quick response to user needs. Users can immediately access the virtual museum's guide content when they need help, improving the user experience. Furthermore, the virtual museum's guide content can be reused by different applications, significantly increasing its utilization.

[0096] Exemplarily, after S25, the virtual museum guiding method includes:

[0097] In step A, when the gesture action is a standard action, a single vibration instruction is sent to the tactile feedback module of the VR head display device, so that the tactile feedback module completes the vibration according to the single vibration instruction.

[0098] After S25, S26 or step A is executed.

[0099] The tactile feedback module is a system component that integrates hardware and software technologies, and is designed to convey tactile information to users through vibration. For ease of explanation, the following example is provided:

[0100] When the haptic feedback module receives a single vibration command, it interprets the command parameters, drives the vibration motor, executes the vibration, and stops after completion. The entire process is highly controllable, accurately simulating various tactile effects and providing users with an immersive interactive experience.

[0101] In the embodiment of the present invention, the beneficial effects are in two aspects. On the one hand, when the gesture action is not a standard action, the gesture animation is obtained based on a predefined method, and the gesture animation is selected as the guidance content of the virtual museum to display the guidance content of the virtual museum. Since there is no need to manually obtain the guidance content of the virtual museum, the acquisition time of the guidance content of the virtual museum is reduced, which is conducive to improving the acquisition efficiency of the guidance content of the virtual museum; on the other hand, the gesture animation is selected as the guidance content of the virtual museum, and the standard action is simulated by the gesture animation, which is conducive to guiding the user to use standard actions to interact with the virtual exhibits, without the user performing additional voice analysis or text reading. Therefore, compared with voice and text, gesture animation has a more intuitive expression advantage, so it can reduce the guidance time of the virtual museum, which is conducive to improving the guidance efficiency of the virtual museum.

[0102] See also Figure 3 , Figure 3 yes Figure 2 A specific implementation flow diagram of step S23 is described in detail as follows:

[0103] S31, when the movement amplitude is less than a preset amplitude, accessing a preset file, and obtaining multiple videos of the presenter and the virtual exhibit interacting in the preset file;

[0104] S32, obtaining the score and playback frequency corresponding to each video, multiplying the score and playback frequency corresponding to each video to obtain the evaluation value corresponding to each video, and selecting the video with the highest evaluation value as the demonstration video.

[0105] For ease of explanation, the following examples are given:

[0106] For example, multiple videos are video 1, video 2, and video 3;

[0107] Get the score corresponding to video 1 and the playback frequency of video 1, multiply the score corresponding to video 1 and the playback frequency of video 1 to obtain the evaluation value corresponding to video 1;

[0108] Get the score corresponding to video 2 and the playback frequency of video 2, multiply the score corresponding to video 2 and the playback frequency of video 2 to obtain the evaluation value corresponding to video 2;

[0109] The score corresponding to video 3 and the playback frequency of video 3 are obtained, and the score corresponding to video 3 and the playback frequency of video 3 are multiplied to obtain the evaluation value corresponding to video 3.

[0110] Among Video 1, Video 2, and Video 3, when Video 1 has the highest corresponding evaluation value, Video 1 is selected as the demonstration video; among Video 1, Video 2, and Video 3, when Video 2 has the highest corresponding evaluation value, Video 2 is selected as the demonstration video; among Video 1, Video 2, and Video 3, when Video 3 has the highest corresponding evaluation value, Video 3 is selected as the demonstration video.

[0111] In the embodiment of the present invention, the video with the highest evaluation value is selected as the demonstration video, which can ensure the reliability of the demonstration video.

[0112] See also Figure 4 , Figure 4 yes Figure 2 A specific implementation flow diagram of step S25 is described in detail as follows:

[0113] S41, playing a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is played, capturing an action of a user grabbing a virtual exhibit, an action of a user rotating the virtual exhibit, an action of a user clicking on the virtual exhibit, and an action of a user waving the virtual exhibit;

[0114] S42 , selecting an action of a user grabbing a virtual exhibit, an action of a user rotating a virtual exhibit, an action of a user clicking on a virtual exhibit, and an action of a user waving a virtual exhibit as gesture actions.

[0115] Among them, the actions of users grabbing virtual exhibits, rotating virtual exhibits, clicking virtual exhibits, and waving virtual exhibits are selected as gesture actions. This design can simplify development and maintenance. It processes the actions of users grabbing virtual exhibits, rotating virtual exhibits, clicking virtual exhibits, and waving virtual exhibits through a unified framework, reducing duplicate code and resource overhead and improving development efficiency.

[0116] In an embodiment of the present invention, by capturing the user's actions of grabbing the virtual exhibit, rotating the virtual exhibit, clicking the virtual exhibit, and waving the virtual exhibit, the user's needs and intentions can be understood more accurately, thereby providing more personalized and considerate services.

[0117] See also Figure 5 , Figure 5 yes Figure 2 A specific implementation flow diagram of step S26 is described in detail as follows:

[0118] S51, when the gesture action is not a standard action, obtaining multiple posture data and a model file corresponding to the standard action, loading a hand model from the model file using a loading script, mapping the multiple posture data to the skeleton of the hand model to obtain a hand model carrying the multiple posture data, and inputting the hand model carrying the multiple posture data into a graphics rendering engine to obtain multiple frames of images simulating the standard action;

[0119] Among them, the hand model is a digital reproduction of the structure and form of the human hand. Usually created through 3D modeling technology, the hand model can simulate the hand's bone structure, muscle distribution, skin texture, and joints.

[0120] Among them, hand models carrying different posture data correspond to different pictures simulating standard movements.

[0121] For the sake of explanation, let's take the standard action of rotating a virtual exhibit with your hands as an example, as follows:

[0122] The plurality of posture data are posture data 1, posture data 2, posture data 3, posture data 4, posture data 5, posture data 5, posture data 6, and posture data 7;

[0123] The hand model carrying the posture data 1 is input into the graphics rendering engine to generate a first frame simulating a standard action. The first frame simulating a standard action is used to simulate the hand holding the virtual exhibit in the standard action, with the front of the virtual exhibit facing forward.

[0124] The hand model carrying the posture data 2 is input into the graphics rendering engine to generate the second frame simulating the standard action. The second frame simulating the standard action is used to simulate the hand in the standard action starting to rotate, and the virtual exhibit rotates 30 degrees.

[0125] The hand model carrying posture data 3 is input into the graphics rendering engine to generate the third frame simulating the standard action. The third frame simulating the standard action is used to simulate the continued rotation of the hand in the standard action, and the product rotates 60 degrees.

[0126] The hand model carrying posture data 4 is input into the graphics rendering engine to generate the 4th frame simulating the standard action. The 4th frame simulating the standard action is used to simulate the continued rotation of the hand in the standard action, and the product rotates 90 degrees.

[0127] The hand model carrying posture data 5 is input into the graphics rendering engine to generate the 5th frame simulating the standard action. The 5th frame simulating the standard action is used to simulate the continued rotation of the hand in the standard action, and the product rotates 120 degrees.

[0128] The hand model carrying posture data 6 is input into the graphics rendering engine to generate the 6th frame simulating the standard action. The 6th frame simulating the standard action is used to simulate the continued rotation of the hand in the standard action, and the product rotates 160 degrees.

[0129] The hand model carrying posture data 7 is input into the graphics rendering engine to generate the 7th frame simulating the standard action. The 7th frame simulating the standard action is used to simulate the continued rotation of the hand in the standard action. The product rotates 180 degrees. At this time, the back of the virtual exhibit faces forward.

[0130] Among them, posture data 1, posture data 2, posture data 3, posture data 4, posture data 5, posture data 5, posture data 6, and posture data 7 are different posture data.

[0131] For ease of explanation, let's take the standard action of grabbing a virtual exhibit as an example, as follows:

[0132] The plurality of posture data are posture data A, posture data B, posture data C, and posture data D;

[0133] The hand model carrying the posture data A is input into the graphics rendering engine to generate the A-th frame simulating the standard action. The A-th frame simulating the standard action is used to simulate the opening of the hand in the standard action, and the virtual exhibit is still.

[0134] The hand model carrying the posture data B is input into the graphics rendering engine to generate the B-th frame simulating the standard action. The B-th frame simulating the standard action is used to simulate the hand in the standard action starting to close and approach the virtual exhibit.

[0135] The hand model carrying the posture data C is input into the graphics rendering engine to generate the Cth frame simulating the standard action. The Cth frame simulating the standard action is used to simulate the partial closure of the hand in the standard action and contact with the virtual exhibits.

[0136] The hand model carrying posture data D is input into the graphics rendering engine to generate the D-th frame simulating the standard action. The D-th frame simulating the standard action is used to simulate the complete closure of the hand part in the standard action to grab the virtual exhibit.

[0137] Among them, posture data A, posture data B, posture data C, and posture data D are different posture data.

[0138] S52 , encoding multiple frames of images simulating standard actions in chronological order to generate gesture animation, selecting the gesture animation as the guide content of the virtual museum, and displaying the guide content of the virtual museum.

[0139] In an embodiment of the present invention, multiple frames of images simulating standard actions are encoded in chronological order to generate gesture animations, which are selected as the guide content of the virtual museum and displayed. Since there is no need to manually obtain the guide content of the virtual museum, the time for obtaining the guide content of the virtual museum is reduced, which is conducive to improving the efficiency of obtaining the guide content of the virtual museum.

[0140] See also Figure 6 , Figure 6 FIG. 1 is a schematic diagram of the structure of a virtual museum guide device according to an embodiment of the present invention. Figure 6 As shown, the virtual museum guidance device includes a first acquisition module 101, a second acquisition module 102, a determination module 103, a replacement module 104, a playback module 105, and a guidance module 106. The functional modules are described in detail as follows:

[0141] The first acquisition module 101 is used to obtain the current state of the VR head display device. When the current state of the VR head display device is the wearing state, the permission level corresponding to the device identifier is obtained. When the permission level is higher than the preset level, the virtual exhibits in the virtual museum are obtained;

[0142] The second acquisition module 102 is used to display the virtual exhibit and acquire the movement amplitude of the head or hand within a preset time from the start of display of the virtual exhibit;

[0143] A determination module 103 is configured to determine a demonstration video of interaction between a presenter and a virtual exhibit based on the movement amplitude;

[0144] A replacement module 104 is used to replace the background portion of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background;

[0145] The playback module 105 is configured to play a demonstration video with a semi-transparent background, and capture gestures of the user interacting with the virtual exhibit after the demonstration video with the semi-transparent background is played;

[0146] The guidance module 106 is configured to obtain a gesture animation based on a predefined method when the gesture action is not a standard action, select the gesture animation as the guidance content of the virtual museum, and display the guidance content of the virtual museum.

[0147] In one embodiment, the second acquisition module 102 includes:

[0148] The first acquisition subunit is used to display the virtual exhibit, acquire a recording instruction, execute the recording instruction, and record the start time of displaying the virtual exhibit;

[0149] The second acquisition subunit is used to acquire the movement amplitude of the head or hand within a preset time from the start of display of the virtual exhibit.

[0150] In one embodiment, the determining module 103 includes:

[0151] An access subunit, configured to access a preset file when the movement amplitude is smaller than a preset amplitude, and obtain multiple videos of the presenter and the virtual exhibit interacting in the preset file;

[0152] The third acquisition subunit is used to obtain the score and playback frequency corresponding to each video, multiply the score and playback frequency corresponding to each video to obtain the evaluation value corresponding to each video, and select the video with the highest evaluation value as the demonstration video.

[0153] In one embodiment, the replacement module 104 includes:

[0154] An input subunit, configured to input the demonstration video into an image processing tool, process the demonstration video through the image processing tool, and obtain a background portion of the demonstration video;

[0155] The replacement subunit is used to obtain a replacement instruction, execute the replacement instruction, replace the background part in the demonstration video with a semi-transparent background, and obtain a demonstration video with a semi-transparent background.

[0156] In one embodiment, the playback module 105 includes:

[0157] a playback subunit, configured to play a demonstration video with a semi-transparent background, and after the demonstration video with a semi-transparent background is played, capture the user's action of grabbing the virtual exhibit, the user's action of rotating the virtual exhibit, the user's action of clicking the virtual exhibit, and the user's action of waving the virtual exhibit;

[0158] The selection subunit is used to select the user's action of grabbing the virtual exhibit, the user's action of rotating the virtual exhibit, the user's action of clicking the virtual exhibit, and the user's action of waving the virtual exhibit as gesture actions.

[0159] In one embodiment, the guiding module 106 includes:

[0160] The fourth acquisition subunit is configured to, when the gesture action is not a standard action, obtain multiple posture data and model files corresponding to the standard action, load a hand model from the model file via a loading script, map the multiple posture data onto the skeleton of the hand model, obtain a hand model carrying the multiple posture data, and input the hand model carrying the multiple posture data into a graphics rendering engine to obtain multiple frames simulating the standard action;

[0161] The display subunit is used to encode multiple frames of simulated standard actions in chronological order to generate gesture animation, select the gesture animation as the guide content of the virtual museum, and display the guide content of the virtual museum.

[0162] In one embodiment, the virtual museum guiding device further includes:

[0163] The creation module is used to create a storage area, store the guide content of the virtual museum in the storage area, and create an access interface for the storage area.

[0164] In the embodiment of the present invention, the beneficial effects are in two aspects. On the one hand, when the gesture action is not a standard action, the gesture animation is obtained based on a predefined method, and the gesture animation is selected as the guidance content of the virtual museum to display the guidance content of the virtual museum. Since there is no need to manually obtain the guidance content of the virtual museum, the acquisition time of the guidance content of the virtual museum is reduced, which is conducive to improving the acquisition efficiency of the guidance content of the virtual museum; on the other hand, the gesture animation is selected as the guidance content of the virtual museum, and the standard action is simulated by the gesture animation, which is conducive to guiding the user to use standard actions to interact with the virtual exhibits, without the user performing additional voice analysis or text reading. Therefore, compared with voice and text, gesture animation has a more intuitive expression advantage, so it can reduce the guidance time of the virtual museum, which is conducive to improving the guidance efficiency of the virtual museum.

[0165] The specific definition of the virtual museum guiding device can be found in the definition of the virtual museum guiding method above, which will not be repeated here.

[0166] Each module in the aforementioned virtual museum guidance device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor within a computer device in the form of hardware, or can be stored in a computer device's memory in the form of software, allowing the processor to call and execute the corresponding operations of each module.

[0167] See also Figure 7 , Figure 7 Schematic diagram of the structure of a computer device in one embodiment of the present invention. In one embodiment, a computer device is provided. The computer device is a server device or a client device. The internal structure diagram thereof can be as follows: Figure 7As shown. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with external devices.

[0168] When the computer program is executed by a processor, the following steps may be implemented:

[0169] Get the current state of the VR headset. When the VR headset is currently in the wearing state, get the permission level corresponding to the device identifier. When the permission level is higher than the preset level, get the virtual exhibits in the virtual museum.

[0170] Displaying a virtual exhibit, and obtaining a head or hand movement amplitude within a preset time from the start of displaying the virtual exhibit;

[0171] Determine the demonstration video of the interaction between the presenter and the virtual exhibit based on the movement amplitude;

[0172] Replace the background part of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background;

[0173] Play a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capture the user's gestures when interacting with the virtual exhibits;

[0174] When the gesture action is not a standard action, a gesture animation is obtained based on a predefined method, the gesture animation is selected as the guide content of the virtual museum, and the guide content of the virtual museum is displayed.

[0175] In some embodiments, the processor is configured to implement:

[0176] Displaying a virtual exhibit, obtaining a recording instruction, executing the recording instruction, and recording the start time of displaying the virtual exhibit;

[0177] The movement range of the head or hand is obtained within a preset time from the start of display of the virtual exhibit.

[0178] In some embodiments, the processor is configured to implement:

[0179] When the movement amplitude is smaller than a preset amplitude, accessing a preset file, and obtaining a plurality of videos of the presenter and the virtual exhibit interacting in the preset file;

[0180] Get the score and playback frequency corresponding to each video, multiply the score and playback frequency corresponding to each video to get the evaluation value corresponding to each video, and select the video with the highest evaluation value as the demonstration video.

[0181] In some embodiments, the processor is configured to implement:

[0182] Input the demonstration video into an image processing tool, process the demonstration video using the image processing tool, and obtain the background part of the demonstration video;

[0183] A replacement instruction is obtained and executed to replace a background portion in the demonstration video with a semi-transparent background, thereby obtaining a demonstration video with a semi-transparent background.

[0184] In some embodiments, the processor is configured to implement:

[0185] Playing a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capturing an action of a user grabbing a virtual exhibit, an action of a user rotating the virtual exhibit, an action of a user clicking on the virtual exhibit, and an action of a user waving the virtual exhibit;

[0186] The actions of the user grabbing the virtual exhibit, the user rotating the virtual exhibit, the user clicking the virtual exhibit, and the user waving the virtual exhibit are selected as gesture actions.

[0187] In some embodiments, the processor is configured to implement:

[0188] When the gesture action is not a standard action, multiple posture data and model files corresponding to the standard action are obtained, a hand model is loaded from the model file through a loading script, the multiple posture data are mapped to the skeleton of the hand model, and a hand model carrying the multiple posture data is obtained. The hand model carrying the multiple posture data is input into a graphics rendering engine to obtain multiple frames of images simulating the standard action;

[0189] According to the chronological order, multiple frames of images simulating standard actions are encoded to generate gesture animations, and the gesture animations are selected as the guiding content of the virtual museum, and the guiding content of the virtual museum is displayed.

[0190] In some embodiments, the processor is configured to implement:

[0191] Create a storage area, store the guide content of the virtual museum in the storage area, and create an access interface for the storage area.

[0192] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0193] Get the current state of the VR headset. When the VR headset is currently in the wearing state, get the permission level corresponding to the device identifier. When the permission level is higher than the preset level, get the virtual exhibits in the virtual museum.

[0194] Displaying a virtual exhibit, and obtaining a head or hand movement amplitude within a preset time from the start of displaying the virtual exhibit;

[0195] Determine the demonstration video of the interaction between the presenter and the virtual exhibit based on the movement amplitude;

[0196] Replace the background part of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background;

[0197] Play a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capture the user's gestures when interacting with the virtual exhibits;

[0198] When the gesture action is not a standard action, a gesture animation is obtained based on a predefined method, the gesture animation is selected as the guide content of the virtual museum, and the guide content of the virtual museum is displayed.

[0199] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant description of the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0200] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), and a network processor (NP); it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0201] The above description and accompanying drawings sufficiently illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Each embodiment herein may focus on the differences from other embodiments, and similar portions between the embodiments may be referenced to each other. For methods disclosed in the embodiments, if they correspond to the method portion disclosed in the embodiments, then the relevant portions may be referenced to the description of the method portion.

[0202] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0203] In the embodiments disclosed herein, the device embodiments described above are merely illustrative. For example, the division of units may be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components may be combined or integrated into another system, or some subsamples may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments disclosed herein may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0204] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the system, method and computer program according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A virtual museum guiding method, characterized in that: include: Get the current state of the VR headset. When the VR headset is currently in the wearing state, get the permission level corresponding to the device identifier. When the permission level is higher than the preset level, get the virtual exhibits in the virtual museum. Displaying a virtual exhibit, and obtaining a head or hand movement amplitude within a preset time from the start of displaying the virtual exhibit; When the movement amplitude is less than a preset amplitude, access a preset file, obtain multiple videos of the presenter and the virtual exhibit interacting in the preset file, obtain the score and playback frequency corresponding to each video, multiply the score and playback frequency corresponding to each video to obtain the evaluation value corresponding to each video, and select the video with the highest evaluation value as the demonstration video; Replace the background part of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background; Play a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capture the user's gestures when interacting with the virtual exhibits; When the gesture action is not a standard action, multiple gesture data and model files corresponding to the standard action are obtained, the hand model is loaded from the model file through a loading script, the multiple gesture data are mapped to the skeleton of the hand model, and a hand model carrying multiple gesture data is obtained. The hand model carrying multiple gesture data is input into a graphics rendering engine to obtain multiple frames of images simulating standard actions. The hand model simulates the bone structure, muscle distribution, skin texture and joints of the hand. The multiple frames of images simulating standard actions are encoded in chronological order to generate a gesture animation. The gesture animation is selected as the guide content of the virtual museum, and the guide content of the virtual museum is displayed.

2. The virtual museum guiding method according to claim 1, characterized in that: The display of the virtual exhibit, starting from the start of display of the virtual exhibit, obtains the movement amplitude of the head or hand within a preset time, including: Displaying a virtual exhibit, obtaining a recording instruction, executing the recording instruction, and recording the start time of displaying the virtual exhibit; The movement range of the head or hand is obtained within a preset time from the start of display of the virtual exhibit.

3. The virtual museum guiding method according to claim 1, characterized in that: The method of replacing the background portion of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background includes: Input the demonstration video into an image processing tool, process the demonstration video using the image processing tool, and obtain the background part of the demonstration video; A replacement instruction is obtained and executed to replace a background portion in the demonstration video with a semi-transparent background, thereby obtaining a demonstration video with a semi-transparent background.

4. The virtual museum guiding method according to claim 1, characterized in that: The method of playing a demonstration video with a semi-transparent background and capturing a user's gesture action of interacting with a virtual exhibit after the demonstration video with a semi-transparent background is finished playing includes: Playing a demonstration video with a semi-transparent background, and after the demonstration video with the semi-transparent background is finished playing, capturing an action of a user grabbing a virtual exhibit, an action of a user rotating the virtual exhibit, an action of a user clicking on the virtual exhibit, and an action of a user waving the virtual exhibit; The actions of the user grabbing the virtual exhibit, the user rotating the virtual exhibit, the user clicking the virtual exhibit, and the user waving the virtual exhibit are selected as gesture actions.

5. The virtual museum guiding method according to claim 1, characterized in that: After selecting the gesture animation as the virtual museum's guide content and displaying the virtual museum's guide content, the virtual museum's guide method includes: Create a storage area, store the guide content of the virtual museum in the storage area, and create an access interface for the storage area.

6. A virtual museum guiding device, characterized in that: include: The first acquisition module is used to obtain the current state of the VR head display device. When the current state of the VR head display device is the wearing state, the permission level corresponding to the device identifier is obtained. When the permission level is higher than the preset level, the virtual exhibits in the virtual museum are obtained; The second acquisition module is used to display the virtual exhibit and acquire the movement amplitude of the head or hand within a preset time from the start of display of the virtual exhibit; a determination module configured to, when the movement amplitude is less than a preset amplitude, access a preset file, obtain from the preset file multiple videos of the presenter and the virtual exhibit interacting, obtain a rating and a playback frequency corresponding to each video, multiply the rating and the playback frequency corresponding to each video to obtain an evaluation value corresponding to each video, and select the video with the highest evaluation value as the demonstration video; A replacement module, used to replace the background part of the demonstration video with a semi-transparent background to obtain a demonstration video with a semi-transparent background; a playback module, configured to play a demonstration video with a semi-transparent background, and capture gestures of a user interacting with a virtual exhibit after the demonstration video with the semi-transparent background has finished playing; The guidance module is used to obtain multiple posture data and model files corresponding to standard actions when the gesture action is not a standard action, load the hand model from the model file through the loading script, map the multiple posture data to the bones of the hand model, obtain the hand model carrying the multiple posture data, input the hand model carrying the multiple posture data into the graphics rendering engine, obtain multiple frames of images simulating standard actions, the hand model simulates the bone structure, muscle distribution, skin texture and joints of the hand, encode the multiple frames of images simulating standard actions in chronological order, generate gesture animation, select the gesture animation as the guidance content of the virtual museum, and display the guidance content of the virtual museum.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the virtual museum guiding method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the virtual museum guiding method according to any one of claims 1 to 5 are implemented.

Images