VR scene establishment method and device

Abstract

The present application relates to the field of VR technology, and provides a method and device for establishing a VR scene. The method comprises: receiving a plurality of captured images of a first scene acquired by an image acquisition terminal; determining the displacement of a target object according to the position of a target object in the first scene in the first target image, and the position of the target object in the second target image; and establishing a VR scene according to the displacement of the target object, the captured image, and the pre-stored image. The VR scene establishment method and device provided in the embodiment of the present application, by determining the motion and stillness of the image acquisition terminal that shoots the first scene according to the plurality of captured images of the first scene acquired by the image acquisition terminal, and then establishing different VR game scenes according to different states by combining the captured images and the pre-stored images, can improve the fun of the combination of robots and VR technology, effectively expand the practicality of the combination of robots and VR technology, and has a very broad application prospect.

Description

Technical Field

[0001] The present application relates to the field of VR technology, and in particular to a method and device for establishing a VR scene. Background Art

[0002] At present, robots are equipped with a variety of software and hardware functions, providing a lot of convenience in life and work. On the other hand, VR (Virtual Reality) technology is increasingly being used in life, with typical applications such as VR games.

[0003] However, there are few existing technologies that combine robotics and VR technologies. Even if there are, they are mostly used in specific work scenarios, such as combining robotics and VR technologies for dangerous and complex work scenarios such as deep-well mining, earthquake rescue, and seabed exploration.

[0004] Therefore, how to expand the application of the combination of robots and VR technology is of great significance to improving the practicality of robots and VR technology. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a VR scene establishment method, which can improve the fun of combining robots with VR technology and effectively expand the practicality of combining robots with VR technology.

[0006] This application also proposes a VR scene establishment system.

[0007] This application also proposes a VR scene creation device.

[0008] The VR scene establishment method according to the first aspect of the present application includes:

[0009] Receiving a plurality of captured images of a first scene acquired by an image capture terminal; the plurality of captured images include a first target image and a second target image;

[0010] Determine the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene;

[0011] A VR scene is established according to the displacement of the target object, the acquired image and the pre-stored image.

[0012] The VR scene establishment method provided in the embodiment of the present application determines the motion and stillness states of the image acquisition terminal that shoots the first scene based on multiple acquired images of the first scene acquired by the image acquisition terminal, and then establishes different VR game scenes based on different states by combining the acquired images and pre-stored images. This can improve the fun of the combination of robots and VR technology, effectively expand the practicality of the combination of robots and VR technology, and has a very broad application prospect.

[0013] According to an embodiment of the present application, establishing a VR scene according to the displacement, the acquired image, and the pre-stored image includes:

[0014] Determine that the magnitude of the displacement of the target object is greater than 0, and determine, according to the direction of the displacement of the target object, a first matching image that matches the direction of the displacement of the target object from a plurality of pre-stored images;

[0015] The captured image and the first matching image are superimposed to establish the VR scene.

[0016] The VR scene establishment method provided in the embodiment of the present application establishes a VR game scene by determining the first matching image specifically according to the displacement direction of the target object in the first scene when the displacement of the target object is greater than 0. This can ensure that the established VR game scene is compatible with the motion state of the image acquisition terminal, effectively improve the realism of the VR game scene, and ensure user experience.

[0017] According to an embodiment of the present application, superimposing the acquired image and the first matching image includes:

[0018] The first matching image is superimposed on the acquired image.

[0019] According to an embodiment of the present application, the first matching image is an animated image, and the method further includes:

[0020] The display of the animation image is controlled according to at least one of a magnitude of the displacement of the target object or a direction of the displacement of the target object.

[0021] According to an embodiment of the present application, establishing a VR scene according to the displacement of the target object, the acquired image, and the pre-stored image includes:

[0022] Determine that the displacement of the target object is 0, and determine a second matching image that matches the displacement of the target object being 0 from a plurality of pre-stored images;

[0023] The captured image and the second matching image are superimposed to establish the VR scene.

[0024] The VR scene establishment method provided in the embodiment of the present application can ensure that the established VR game scene is compatible with the motion state of the image acquisition terminal, effectively improve the realism of the VR game scene, and ensure the user experience by determining the second matching image from multiple pre-stored images when the displacement of the target object in the first scene is 0, and establishing the VR game scene according to the captured image and the second matching image.

[0025] According to one embodiment of the present application, the second matching image includes a target object, and the method further includes:

[0026] The position of the target object in the superimposed image is determined according to the content of the collected image; the superimposed image is an image formed by superimposing the second matching image on the collected image.

[0027] According to an embodiment of the present application, after superimposing the acquired image and the second matching image to establish the VR scene, the method further includes:

[0028] Receive user input;

[0029] In response to the input, a control instruction is generated and sent to the image acquisition terminal; the control instruction is used to control the movement of the image acquisition terminal.

[0030] The VR scene establishment method provided in the embodiment of the present application can realize the interaction between the user and the established VR game scene by generating control instructions in response to the user's input and sending the control instructions to the image acquisition terminal, thereby effectively improving the user's participation in the VR game scene and further improving the user experience.

[0031] According to an embodiment of the present application, the image acquisition terminal includes a mechanical arm and a shooting device; the shooting device is arranged at one end of the mechanical arm;

[0032] The collected image is obtained by any of the following methods:

[0033] When the robotic arm moves, the photographing device moves along with the robotic arm and acquires the captured image during the movement;

[0034] When the robot arm is stationary, the shooting device acquires the collected image from the stationary position of the shooting device.

[0035] According to an embodiment of the present application, the displacement of the target object corresponds to the displacement generated by the mechanical arm driving the camera to move within the target time interval;

[0036] The target time interval is the time interval between acquiring the first target image and acquiring the second target image.

[0037] A VR scene establishment system according to an embodiment of the second aspect of the present application includes:

[0038] An image acquisition terminal, used to acquire a plurality of acquired images of a first scene;

[0039] A VR display terminal is communicatively connected with the image acquisition terminal; the VR display terminal is used for:

[0040] Receiving the multiple acquired images acquired by the image acquisition terminal; the multiple acquired images include a first target image and a second target image;

[0041] Determine the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene;

[0042] A VR scene is established according to the displacement of the target object, the acquired image and the pre-stored image.

[0043] The VR scene establishment system provided in the embodiment of the present application determines the motion and stillness states of the image acquisition terminal shooting the first scene according to multiple acquired images of the first scene acquired by the image acquisition terminal, and then establishes different VR game scenes according to different states by combining the acquired images and pre-stored images. It can improve the fun of the combination of robots and VR technology, effectively expand the practicality of the combination of robots and VR technology, and has a very broad application prospect.

[0044] According to an embodiment of the present application, the image acquisition terminal includes a mechanical arm and a shooting device; the shooting device is arranged at one end of the mechanical arm;

[0045] The image acquisition terminal is specifically used for:

[0046] When the robotic arm moves, the photographing device moves along with the robotic arm and acquires the captured image during the movement;

[0047] When the robot arm is stationary, the shooting device acquires the collected image from the stationary position of the shooting device.

[0048] According to an embodiment of the present application, the displacement of the target object corresponds to the displacement generated by the mechanical arm driving the camera to move within the target time interval;

[0049] The target time interval is the time interval between acquiring the first target image and acquiring the second target image.

[0050] According to an embodiment of the present application, when the displacement magnitude generated by the mechanical arm driving the shooting device to move within the target time interval is greater than 0, the VR display terminal is specifically used to determine a first matching image that matches the displacement direction of the target object from a plurality of pre-stored images according to the displacement direction of the target object, and superimpose the captured image and the first matching image to establish the VR scene;

[0051] When the displacement size generated by the movement of the shooting device by the robotic arm within the target time interval is 0, the VR display terminal is specifically used to determine a second matching image that matches the displacement size of 0 of the target object from multiple pre-stored images, and superimpose the acquired image and the second matching image to establish the VR scene.

[0052] According to one embodiment of the present application, the image acquisition terminal is further used for:

[0053] Receiving a control instruction sent by the VR display terminal;

[0054] The movement of the robot arm is controlled based on the control instruction.

[0055] A VR scene establishment device according to an embodiment of the third aspect of the present application includes:

[0056] A receiving module, used for receiving a target image of a first scene acquired by an image acquisition terminal;

[0057] a determination module, configured to determine the displacement of the target object in the first scene according to the position of the target object in the first target image and the position of the target object in the second target image; wherein the first target image is a target image acquired before the second target image; and the target object is a stationary object in the first scene;

[0058] A building module is used to build a VR scene according to the displacement, the target image and the pre-stored image.

[0059] The VR scene establishment device provided in the embodiment of the present application determines the motion and stillness states of the image acquisition terminal that shoots the first scene based on multiple captured images of the first scene acquired by the image acquisition terminal, and then establishes different VR game scenes based on different states in combination with the target image and pre-stored images. This can improve the fun of the combination of robots and VR technology, effectively expand the practicality of the combination of robots and VR technology, and has a very broad application prospect.

[0060] According to an electronic device of an embodiment of the fourth aspect of the present application, the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the VR scene establishment method as described in the first aspect is implemented.

[0061] According to the non-transitory computer-readable storage medium of the fifth aspect embodiment of the present application, a computer program is stored thereon, and when the computer program is executed by a processor, the VR scene establishment method as described in the first aspect is implemented.

[0062] According to the computer program product of the sixth aspect of the present application, the computer program includes a computer program, which, when executed by a processor, implements the VR scene establishment method as described in the first aspect.

[0063] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0064] By establishing a VR scene through the displacement of the target object in the first scene, the captured images obtained by the image capture terminal and the pre-stored images, the established VR scene can be realistic and vivid, thereby improving the user's gaming experience.

[0065] Since the images captured by the image acquisition terminal can be of any scene, such as home scenes, etc., it also effectively improves the fun of the VR scene.

[0066] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0068] Figure 1 It is a flowchart of a VR scene establishment method provided in an embodiment of the present application;

[0069] Figure 2 It is a structural diagram of a VR scene establishment system provided in an embodiment of the present application;

[0070] Figure 3 It is a structural schematic diagram of a VR scene establishment device provided in an embodiment of the present application;

[0071] Figure 4 It is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0073] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0074] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0075] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0076] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0077] Figure 1 Schematic diagram of the VR scene creation method provided in the embodiment of the present application. Figure 1 The VR scene establishment method provided in the embodiment of the present application may include:

[0078] Step 110: receiving a plurality of captured images of a first scene acquired by an image acquisition terminal; the plurality of captured images include a first target image and a second target image;

[0079] Step 120: determining the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene;

[0080] Step 130: Create a VR scene based on the displacement of the target object, the captured image, and the pre-stored image.

[0081] It should be noted that the execution subject of the VR scene establishment method provided in the embodiment of the present application may be a VR display terminal, such as VR glasses. The image acquisition terminal may be a combination of a robot arm and a shooting device. Optionally, the shooting device may be disposed at one end of the robot arm, for example, fixedly or detachably disposed at one end of the robot arm, and the image captured by the shooting device may change with the movement of the robot arm.

[0082] In step 110, the image acquisition terminal can acquire the captured image of the first scene (that is, the current scene that can be photographed by the image acquisition terminal) in real time, and send the captured image to the VR display terminal.

[0083] Among them, the image acquisition terminal and the VR display terminal can be connected in communication. Furthermore, the data interaction between the image acquisition terminal and the VR display terminal can be implemented using a high-speed communication protocol to reduce data transmission delay and ensure the real-time data interaction between the image acquisition terminal and the VR display terminal. For example, the image acquisition terminal can send the collected image to the VR display terminal through UWB (Ultra WideBand) technology.

[0084] It should be noted that when the image acquisition terminal includes a mechanical arm and a shooting device, and the shooting device is arranged at one end of the mechanical arm, the captured image can be obtained by any of the following methods:

[0085] Method 1: When the robot arm moves, the camera moves with the robot arm and acquires images during the movement;

[0086] It is understandable that when the robotic arm moves, the robotic arm will drive the shooting device to move along with the movement of the robotic arm. At this time, the shooting device can obtain the captured images of the shooting device during the movement in real time or at certain time intervals.

[0087] For example, when the robotic arm dives from top to bottom, the shooting device can send the captured image of the first scene captured during the top-down movement to the VR display terminal in real time. When the robotic arm rotates from left to right, the shooting device can send the captured image of the first scene captured during the left-to-right movement to the VR display terminal in real time.

[0088] Method 2: When the robot arm is stationary, the camera device acquires the captured image from the stationary position of the camera device.

[0089] It is understandable that when the robotic arm is stationary, the shooting device will send the captured image of the first scene to the VR display terminal in real time.

[0090] For example, when the robotic arm makes the shooting device stationary to shoot the food in the refrigerator (the first scene), the shooting device can send the captured image including the food in the refrigerator to the VR display terminal in real time; when the robotic arm makes the shooting device stationary to shoot the obstacles on the desktop (the first scene), the shooting device can send the captured image including the obstacles on the desktop to the VR display terminal in real time.

[0091] By combining the robotic arm and the shooting device to acquire images in a variety of ways, the diversity of acquired images can be achieved, thereby providing a basis for the subsequent establishment of different VR game scenes, thereby ensuring the fun of the combination of robots and VR technology.

[0092] In step 120, the VR display terminal may determine the first target image and the second target image based on the multiple captured images of the first scene transmitted in real time by the image acquisition terminal, and then determine the displacement of the target object based on the position of the target object in the first scene in the first target image and the position of the target object in the second target image.

[0093] The target object in the first scene may be any stationary object in the first scene. For example, when the first scene is a room, the target object may be furniture such as tables, chairs, and cabinets in the room, or may be decorations such as murals and wall clocks on the walls of the room. The displacement of the target object refers to the displacement of the target object in the second target image relative to the displacement of the target object in the first target image.

[0094] The VR display terminal can determine the displacement of the target object based on two consecutive captured images. For example, the VR display terminal can use the first captured image of the two consecutive captured images as the first target image, and the later captured image of the two consecutive captured images as the second target image, to determine that the displacement of the table in a unit time is 0.01m upward; the VR display terminal can also randomly sample two of the multiple consecutive captured images obtained by the image acquisition terminal as the first target image and the second target image, respectively, to determine that the displacement of the table in a unit time is 0.02m downward, etc.

[0095] It should be noted that the displacement of the target object corresponds to the displacement caused by the robot arm driving the camera device to move within the target time interval;

[0096] The target time interval is the time interval between acquiring the first target image and acquiring the second target image.

[0097] It is understandable that, when the robot arm moves, the camera will move with the robot arm and acquire the target image during the movement. Therefore, the displacement of the camera driven by the robot arm in the target time interval will be reflected in the content included in the first target image and the second target image. That is, when both the first target image and the second target image include a stationary target object, the displacement of the target object can correspond to the displacement caused by the robot arm driving the camera to move in the target time interval.

[0098] By combining the robotic arm and the camera device, the displacement of the target object can be made to correspond to the displacement caused by the robotic arm driving the camera device to move within the target time interval. This can make the captured image accurately reflect the displacement of the robotic arm, so that the established VR game scene has a realistic effect, thereby ensuring the fun of the combination of robot and VR technology.

[0099] In step 130, the VR display terminal may establish a VR scene according to the displacement of the target object, the captured image, and the pre-stored image.

[0100] For example, when the VR display terminal determines that the displacement of the target object is 0, it can be determined that the mechanical arm is in a stationary state at this time, that is, the mechanical arm does not drive the shooting device to move, and the VR display terminal can establish a game scene corresponding to the mechanical arm being in a stationary state in the first scene based on the collected image and the pre-stored image. The pre-stored image can be an image pre-stored by the VR display terminal.

[0101] When the VR display terminal determines that the displacement of the target object is not 0, it can be determined that the robotic arm is in motion at this time, that is, the robotic arm is driving the shooting device to move. The VR display terminal can then establish a game scene corresponding to the robotic arm being in motion in the first scene based on the displacement of the target object, the captured image and the pre-stored image.

[0102] The VR scene establishment method provided in the embodiment of the present application determines the motion and stillness states of the image acquisition terminal that shoots the first scene based on multiple acquired images of the first scene acquired by the image acquisition terminal, and then establishes different VR game scenes based on different states by combining the acquired images and pre-stored images. This can improve the fun of the combination of robots and VR technology, effectively expand the practicality of the combination of robots and VR technology, and has a very broad application prospect.

[0103] In one embodiment, step 130 may include:

[0104] Determine that the magnitude of the displacement of the target object is greater than 0, and determine a first matching image that matches the direction of the displacement of the target object from a plurality of pre-stored images according to the direction of the displacement of the target object;

[0105] The captured image and the first matching image are superimposed to establish a VR scene.

[0106] The VR display terminal can first determine whether the displacement of the target object in a unit time is greater than 0. If it is greater than 0, it indicates that the robotic arm is in motion, that is, the robotic arm is driving the shooting device to move. At this time, the VR display terminal can further determine the first matching image that matches the direction of the displacement of the target object from the pre-stored multiple images according to the direction of the displacement.

[0107] For example, when the robotic arm of the image acquisition terminal dives from top to bottom, the shooting device of the image acquisition terminal will send the captured image of the first scene captured from top to bottom to the VR display terminal in real time. The VR display terminal first determines whether the displacement of the target object is greater than 0 based on the position of the target object in the second target image and the position of the target object in the first target image. If so, the VR display terminal will further determine the direction of the displacement. If the displacement direction is from top to bottom, then at this time, the VR display terminal will determine the image including the roller coaster element that matches the displacement direction from top to bottom from the pre-stored multiple images as the first matching image, and superimpose the captured image and the first matching image to establish a VR game scene for playing a roller coaster game in the first scene.

[0108] The image including roller coaster elements may include roller coaster tracks and other objects reflecting roller coaster elements, such as roller coaster seats, the front of the roller coaster, and the front seats of the roller coaster.

[0109] For example, when the mechanical arm of the image acquisition terminal rotates from left to right, the shooting device of the image acquisition terminal will send the captured image of the first scene captured from left to right to the VR display terminal in real time. The VR display terminal first determines whether the displacement of the target object is greater than 0 based on the position of the target object in the second target image and the position of the target object in the first target image. If so, the VR display terminal will further determine the direction of the displacement of the target object. If the displacement direction of the target object is from left to right, then at this time, the VR display terminal will determine the image including the racing element that matches the displacement direction from left to right from the pre-stored multiple images as the first matching image, and superimpose the captured image and the first matching image to establish a VR game scene for playing a racing game in the first scene.

[0110] The image including the racing elements may include the racing track and other objects reflecting the racing elements, such as the cockpit of the racing car, the rearview mirror of the racing car, other racing cars in front, and the like.

[0111] The VR scene establishment method provided in the embodiment of the present application establishes a VR game scene by determining the first matching image specifically according to the displacement direction of the target object in the first scene when the displacement of the target object is greater than 0. This can ensure that the established VR game scene is compatible with the motion state of the image acquisition terminal, effectively improve the realism of the VR game scene, and ensure user experience.

[0112] In one embodiment, superimposing the target image and the first matching image may include:

[0113] The first matching image is superimposed on the acquired image.

[0114] After determining the target image and the first matching image, the VR display terminal may superimpose the captured image and the first matching image to obtain a superimposed image, and establish a VR scene based on the superimposed image.

[0115] In this embodiment, the VR display terminal will preferentially superimpose the first matching image on the captured image, that is, the VR display terminal uses the captured image as the bottom layer image and the first matching image as the top layer image, and then superimposes the captured image and the first matching image.

[0116] It should be noted that by superimposing the first matching image on the captured image, it can be ensured that the first matching image in the superimposed image can be displayed more clearly and obviously, so that the fun of the game can be more prominent in the final VR scene.

[0117] In one embodiment, the first matching image may be an animated image, and the VR scene establishment method provided in the embodiment of the present application may further include:

[0118] The display of the animation image is controlled according to at least one of the magnitude of the displacement of the target object or the direction of the displacement of the target object.

[0119] For example, when the VR display terminal establishes a VR game scene of a roller coaster game in the first scene based on the captured image and the first matching image, and the first matching image is an animated image of a roller coaster track, the VR display terminal can control the display of the roller coaster track according to the displacement direction of the target object. For example, when the displacement of the target object is upward, the VR display terminal can control the roller coaster track to extend in the vertical direction to simulate the scene of climbing up on the train.

[0120] The VR display terminal can also control the display of the roller coaster track according to the displacement of the target object. For example, when the displacement of the target object per unit time is large, the VR display terminal can control the roller coaster track to move at a faster speed to simulate the scene when the roller coaster accelerates.

[0121] The VR display terminal can also control the display of the roller coaster track according to the size and direction of the displacement of the target object. For example, when the displacement of the target object is upward and the displacement per unit time is large, the VR display terminal can control the roller coaster track to extend in the vertical direction and move at a faster speed to simulate the scene when the roller coaster climbs quickly. When the displacement of the target object is upward and the displacement per unit time is small, the VR display terminal can control the roller coaster track to extend in the vertical direction and move at a slower speed to simulate the scene when the roller coaster climbs at a low speed.

[0122] Of course, when the VR display terminal establishes a VR game scene of a roller coaster based on the captured image and the first matching image, the VR display terminal can also control the display of the roller coaster seats, the front of the roller coaster, the front seats of the roller coaster, etc. according to the size and / or direction of the displacement of the target object. The embodiments of the present application are not listed one by one.

[0123] When the VR display terminal establishes a VR game scene for a racing game in the first scene based on the captured image and the first matching image, and the first matching image is an animated image of a track, the VR display terminal can control the display of the track according to the displacement direction of the target object. For example, when the displacement of the target object is to the left, the VR display terminal can control the track to extend to the left to simulate the scene when the car turns left.

[0124] The VR display terminal can control the display of the track according to the displacement of the target object. For example, when the displacement of the target object per unit time is large, the VR display terminal can control the track to move at a faster speed to simulate the scene when the car accelerates.

[0125] The VR display terminal can also control the display of the track according to the size and direction of the displacement of the target object. For example, when the displacement of the target object is to the left and the displacement per unit time is large, the VR display terminal can control the track to extend to the left and move at a faster speed to simulate the scene when the car turns left quickly. When the displacement of the target object is to the left and the displacement per unit time is small, the VR display terminal can control the track to extend to the left and move at a slower speed to simulate the scene when the car turns left at a low speed.

[0126] Of course, when the VR display terminal establishes a VR game scene of a car based on the captured image and the first matching image, the VR display terminal can also control the display of the car's cockpit, the car's rearview mirror, other cars in front, etc. according to the size and / or direction of the displacement of the target object. The embodiments of the present application are not listed one by one.

[0127] The VR scene establishment method provided in the embodiment of the present application can further improve the realism and fun of the VR game scene by controlling the first matching image to change with at least one of the displacement size or displacement direction of the target object, thereby further improving the user experience.

[0128] In one embodiment, step 130 may include:

[0129] Determine that the size of the displacement is 0, and determine a second matching image that matches the size of the displacement of the target object 0 from a plurality of pre-stored images;

[0130] The captured image and the second matching image are superimposed to create a VR scene.

[0131] The VR display terminal can first determine whether the displacement of the target object in a unit time is 0. If so, it indicates that the robotic arm is in a stationary state, that is, the robotic arm does not drive the shooting device to move. At this time, the VR display terminal can determine a second matching image that matches the displacement size of 0 from multiple pre-stored images.

[0132] When the robotic arm of the image acquisition terminal is stationary, the shooting device of the image acquisition terminal will send the captured image of the first scene to the VR display terminal in real time. The VR display terminal first determines whether the displacement of the target object is 0 according to the position of the target object in the second target image and the position of the target object in the first target image. If so, the VR display terminal will determine an image including a cartoon image that matches the displacement size of 0 from multiple pre-stored images as the second matching image, and superimpose the captured image and the second matching image to establish a VR game scene.

[0133] For example, when the robotic arm of the image acquisition terminal makes the shooting device stationary to shoot the food in the refrigerator (the first scene), the VR display terminal can determine the image including the Charizard from the pre-stored multiple images as the second matching image, and superimpose the acquired image and the second matching image to create a VR game scene of the Charizard exploring the refrigerator.

[0134] For example, when the robotic arm of the image acquisition terminal makes the shooting device stationary to shoot obstacles on the desktop (first scene), the VR display terminal can determine an image including a cartoon character from multiple pre-stored images as a second matching image, and superimpose the acquired image and the second matching image to create a VR game scene of a cartoon character exploring the desktop.

[0135] The VR scene establishment method provided in the embodiment of the present application can ensure that the established VR game scene is compatible with the motion state of the image acquisition terminal, effectively improve the realism of the VR game scene, and ensure the user experience by determining the second matching image from multiple pre-stored images when the displacement of the target object is 0, and establishing the VR game scene according to the captured image and the second matching image.

[0136] In one embodiment, the second matching image may include a target object, and the VR scene establishment method provided in the embodiment of the present application may further include:

[0137] According to the content of the captured image, the position of the target object in the superimposed image is determined; the superimposed image is an image formed by superimposing the second matching image on the captured image. The content of the captured image may be a target content that has a matching relationship with the target object; the matching relationship may be a custom relationship, for example, the matching relationship may include: the target object may be placed on the target content, the target content is the food of the target object, the target content may hinder the target object from moving forward, etc.

[0138] For example, when the target object is a fire-breathing dragon and the captured image includes a cake with candles, the VR display terminal can place the fire-breathing dragon next to the candles in the superimposed image to simulate a VR game scene in which the fire-breathing dragon lights the candles with fire. When the target object is a cartoon character and the captured image includes a cup on the dining table, the VR display terminal can place the cartoon character on the cup in the superimposed image to simulate a VR game scene in which the cartoon character jumps off the cup to explore.

[0139] It should be noted that by superimposing the second matching image on the captured image, it can be ensured that the second matching image in the superimposed image can be displayed more clearly and obviously, so that the fun of the game can be more prominent in the final VR scene.

[0140] The VR scene establishment method provided in the embodiment of the present application can ensure that the established VR game scene is compatible with the content of the superimposed image by determining the position of the target object in the superimposed image according to the content of the captured image, thereby effectively improving the realism of the VR game scene and ensuring the user experience.

[0141] In one embodiment, after the captured image and the second matching image are superimposed to establish a VR scene, the VR scene establishment method provided in the embodiment of the present application may further include:

[0142] Receive user input;

[0143] In response to the input, a control instruction is generated and sent to the image acquisition terminal; the control instruction is used to control the movement of the image acquisition terminal.

[0144] After the VR display terminal establishes the VR scene according to the captured image and the second matching image, it can also receive user input and control the movement of the image capture terminal according to the input, thereby simulating an adventure in the VR game scene.

[0145] The following example uses a VR display terminal to create a VR game scene of a cartoon character exploring a desktop based on a target image and a second matching image: When the cartoon character is on a teacup, the VR display terminal can receive a user input indicating "jump", and the VR display terminal generates a control instruction in response to the input and sends the control instruction to the image acquisition terminal. According to the control instruction, the image acquisition terminal will first move the robotic arm diagonally upward (at this time, the captured image captured by the shooting device can simulate the perspective of the cartoon character when jumping up), and then move the robotic arm diagonally downward along a certain parabola (at this time, the captured image captured by the shooting device can simulate the perspective of the cartoon character when jumping from a high place). Through the above operations of the VR display terminal and the image acquisition terminal, the scene of a cartoon character jumping off a teacup can be completely simulated.

[0146] The VR scene establishment method provided in the embodiment of the present application can realize the interaction between the user and the established VR game scene by generating control instructions in response to the user's input and sending the control instructions to the image acquisition terminal, thereby effectively improving the user's participation in the VR game scene and further improving the user experience.

[0147] To sum up, the VR scene establishment method provided in the embodiment of the present application, on the one hand, establishes the VR scene through the displacement of the target object in the first scene, the captured image obtained by the image acquisition terminal and the pre-stored image, so that the established VR scene can be realistic and vivid, thereby improving the user's gaming experience; on the other hand, since the captured image taken by the image acquisition terminal can be any scene, such as a home scene, etc., it also effectively improves the fun of the VR scene.

[0148] The embodiment of the present application also provides a VR scene establishment system, which can be used to execute the VR scene establishment method described above.

[0149] Figure 2 Schematic diagram of the structure of the VR scene establishment system provided in the embodiment of the present application. Figure 2 The VR scene establishment system provided in the embodiment of the present application may include:

[0150] An image acquisition terminal 210 is used to acquire a plurality of acquired images of a first scene;

[0151] The VR display terminal 220 is connected to the image acquisition terminal 210 for communication; the VR display terminal 220 is used for:

[0152] Receiving a plurality of collected images acquired by the image acquisition terminal 210; the plurality of collected images include a first target image and a second target image;

[0153] Determine the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene;

[0154] A VR scene is established based on the displacement of the target object, the captured images, and the pre-stored images.

[0155] It should be noted that the image acquisition terminal 210 can be set on the robot, and the data interaction between the image acquisition terminal 210 and the VR display terminal 220 can be implemented using a high-speed communication protocol to reduce data transmission delay and ensure the real-time data interaction between the image acquisition terminal 210 and the VR display terminal 220. For example, the image acquisition terminal 210 can send the acquired image to the VR display terminal 220 through UWB (Ultra Wide Band) technology.

[0156] It should be noted that in the VR scene establishment system provided in the embodiment of the present application, the working principle of the image acquisition terminal 210 is consistent with the image acquisition terminal in the above method embodiment, and the working principle of the VR display terminal 220 is consistent with the VR display terminal in the above method embodiment, which will not be repeated here.

[0157] The VR scene establishment system provided in the embodiment of the present application determines the motion and stillness states of the image acquisition terminal that shoots the first scene based on multiple captured images of the first scene acquired by the image acquisition terminal, and then establishes different VR game scenes based on different states by combining the captured images and pre-stored images. This can improve the fun of the combination of robots and VR technology, effectively expand the practicality of the combination of robots and VR technology, and has a very broad application prospect.

[0158] In one embodiment, the image acquisition terminal 210 includes a mechanical arm 211 and a shooting device 212; the shooting device 212 is disposed at one end of the mechanical arm 211;

[0159] The image acquisition terminal 210 is specifically used for:

[0160] When the robot arm 211 moves, the camera 212 moves with the robot arm 211 and acquires images during the movement.

[0161] When the robot arm 211 is stationary, the photographing device 212 acquires a captured image from the stationary position of the photographing device 212 .

[0162] The photographing device 212 may be, for example, fixedly or detachably disposed at one end of the robotic arm 211 , so that the image captured by the photographing device may change with the movement of the robotic arm 211 .

[0163] It is understandable that when the robotic arm 211 moves, the robotic arm 211 will drive the shooting device 212 to move along with the movement of the robotic arm 211. At this time, the shooting device 212 can obtain the captured images of the shooting device 212 during the movement in real time or at a certain time interval.

[0164] For example, when the robotic arm 211 dives from top to bottom, the shooting device 212 can send the captured image of the first scene captured during the top-down movement to the VR display terminal 220 in real time. When the robotic arm 211 rotates from left to right, the shooting device 212 can send the captured image of the first scene captured during the left-to-right movement to the VR display terminal 220 in real time.

[0165] It is understandable that when the robotic arm 211 is stationary, the shooting device 212 will send the captured image of the first scene to the VR display terminal 220 in real time.

[0166] For example, when the robotic arm 211 causes the shooting device 212 to remain stationary to shoot food in the refrigerator (first scene), the shooting device 212 can send the captured image including the food in the refrigerator to the VR display terminal 220 in real time; when the robotic arm 211 causes the shooting device 212 to remain stationary to shoot obstacles on the desktop (first scene), the shooting device 212 can send the captured image including the obstacles on the desktop to the VR display terminal 220 in real time.

[0167] By combining the robotic arm 212 and the shooting device 222 to acquire captured images in a variety of ways, the diversity of captured images can be achieved, thereby providing a basis for the subsequent establishment of different VR game scenes, thereby ensuring the fun of the combination of robots and VR technology.

[0168] In one embodiment, the displacement of the target object corresponds to the displacement generated by the mechanical arm 211 driving the camera 212 to move within the target time interval;

[0169] The target time interval is the time interval between acquiring the first target image and acquiring the second target image.

[0170] It is understandable that, when the mechanical arm 211 moves, the camera 212 moves with the mechanical arm 211 and acquires the target image during the movement. Therefore, within the target time interval, the displacement of the camera 212 driven by the mechanical arm 211 will be reflected in the content included in the first target image and the second target image. That is, when both the first target image and the second target image include a stationary target object, the displacement of the target object can correspond to the displacement generated by the mechanical arm 211 driving the camera 212 to move within the target time interval.

[0171] The combination of the robotic arm 211 and the shooting device 212 enables the displacement of the target object to correspond to the displacement caused by the robotic arm 211 driving the shooting device 212 to move within the target time interval. This allows the captured image to accurately reflect the displacement of the robotic arm 211, thereby making the established VR game scene realistic, thereby ensuring the fun of the combination of robots and VR technology.

[0172] In one embodiment, when the displacement magnitude generated by the mechanical arm 211 driving the shooting device 212 to move within the target time interval is greater than 0, the VR display terminal 220 is specifically used to determine a first matching image that matches the displacement direction of the target object from a plurality of pre-stored images according to the displacement direction of the target object, and superimpose the captured image and the first matching image to establish a VR scene;

[0173] For example, when the robotic arm 211 dives from top to bottom within the target time interval, the shooting device 212 will send the captured image of the first scene captured from top to bottom to the VR display terminal 220 in real time. The VR display terminal 220 first determines whether the displacement of the target object is greater than 0 based on the position of the target object in the second target image and the position of the target object in the first target image. If so, the VR display terminal 220 will further determine the direction of the displacement. If the displacement direction is from top to bottom, then at this time, the VR display terminal 220 will determine the image including the roller coaster element that matches the displacement direction from top to bottom from the pre-stored multiple images as the first matching image, and superimpose the captured image and the first matching image to establish a VR game scene for playing a roller coaster game in the first scene.

[0174] The image including roller coaster elements may include roller coaster tracks and other objects reflecting roller coaster elements, such as roller coaster seats, the front of the roller coaster, and the front seats of the roller coaster.

[0175] For example, when the mechanical arm 211 rotates from left to right within the target time interval, the shooting device 212 will send the captured image of the first scene captured from left to right to the VR display terminal 220 in real time. The VR display terminal 220 first determines whether the displacement of the target object is greater than 0 based on the position of the target object in the second target image and the position of the target object in the first target image. If so, the VR display terminal 220 will further determine the direction of the displacement of the target object. If the displacement direction of the target object is from left to right, then at this time, the VR display terminal 220 will determine the image including the racing element that matches the displacement direction from left to right from the pre-stored multiple images as the first matching image, and superimpose the captured image and the first matching image to establish a VR game scene for playing a racing game in the first scene.

[0176] The image including the racing elements may include the racing track and other objects reflecting the racing elements, such as the cockpit of the racing car, the rearview mirror of the racing car, other racing cars in front, and the like.

[0177] The VR scene establishment method provided in the embodiment of the present application establishes a VR game scene by determining the first matching image specifically according to the displacement direction of the target object in the first scene when the displacement of the target object is greater than 0. This can ensure that the established VR game scene is compatible with the motion state of the image acquisition terminal, effectively improve the realism of the VR game scene, and ensure user experience.

[0178] In one embodiment, when the displacement size generated by the robotic arm 211 driving the shooting device 212 to move within the target time interval is 0, the VR display terminal 220 is specifically used to determine a second matching image that matches the displacement size of the target object of 0 from multiple pre-stored images, and superimpose the captured image and the second matching image to establish a VR scene.

[0179] When the robotic arm 211 is stationary within the target time interval, the shooting device 212 will send the captured image of the first scene to the VR display terminal 220 in real time. The VR display terminal 220 first determines whether the displacement of the target object is 0 based on the position of the target object in the second target image and the position of the target object in the first target image. If so, the VR display terminal 220 will determine an image including a cartoon image that matches the displacement size of 0 from the pre-stored multiple images as the second matching image, and superimpose the captured image and the second matching image to establish a VR game scene.

[0180] For example, when the robotic arm 211 makes the shooting device 212 stationary to shoot the food in the refrigerator (the first scene) within the target time interval, the VR display terminal 220 can determine the image including the Charizard as the second matching image from the pre-stored multiple images, and superimpose the captured image and the second matching image to create a VR game scene of the Charizard exploring the refrigerator.

[0181] For example, when the robotic arm 211 makes the shooting device 212 stationary to shoot obstacles on the desktop (first scene) within the target time interval, the VR display terminal 220 can determine an image including a cartoon character from multiple pre-stored images as a second matching image, and superimpose the captured image and the second matching image to establish a VR game scene of a cartoon character exploring the desktop.

[0182] The VR scene establishment method provided in the embodiment of the present application can ensure that the established VR game scene is compatible with the motion state of the image acquisition terminal, effectively improve the realism of the VR game scene, and ensure the user experience by determining the second matching image from multiple pre-stored images when the displacement of the target object is 0, and establishing the VR game scene according to the captured image and the second matching image.

[0183] In one embodiment, the image acquisition terminal 210 is further used for:

[0184] Receive control instructions sent by the VR display terminal 220;

[0185] The movement of the robot arm 211 is controlled based on the control command.

[0186] It should be noted that after the VR display terminal 220 establishes the VR scene based on the captured image and the second matching image, it can also receive user input, generate a control instruction in response to the input, and then send the control instruction to the image capture terminal 210.

[0187] After receiving the control instruction sent by the VR display terminal 220, the image acquisition terminal 210 can control the movement of the robotic arm 211 based on the control instruction, thereby simulating an adventure in a VR game scene.

[0188] The following takes the VR display terminal 220 establishing a VR game scene of a cartoon character exploring a desktop according to the target image and the second matching image as an example:

[0189] When the cartoon character is on the teacup, the VR display terminal 220 can receive the user's input indicating "jump", and the VR display terminal 220 generates a control instruction in response to the input and sends the control instruction to the image acquisition terminal 210. According to the control instruction, the image acquisition terminal 210 will first move the mechanical arm 211 obliquely upward (at this time, the captured image captured by the shooting device 212 can simulate the perspective of the cartoon character when jumping up), and then move the mechanical arm 211 obliquely downward along a certain parabola (at this time, the captured image captured by the shooting device 212 can simulate the perspective of the cartoon character when jumping from a high place). Through the above operations of the VR display terminal 220 and the image acquisition terminal 210, the scene of the cartoon character jumping off the teacup can be completely simulated.

[0190] The VR scene establishment method provided in the embodiment of the present application can realize the interaction between the user and the established VR game scene by enabling the image acquisition terminal 210 to receive the control instructions sent by the VR display terminal 220 and controlling the movement of the robotic arm 211 based on the control instructions, thereby effectively improving the user's participation in the VR game scene and further improving the user experience.

[0191] The embodiment of the present application also provides a VR scene establishment device, which can correspond to the VR scene establishment method described above.

[0192] Figure 3 Schematic diagram of the structure of the VR scene establishment device provided in the embodiment of the present application. Figure 3 , the VR scene establishment device provided in the embodiment of the present application may include:

[0193] The receiving module 310 is used to receive a plurality of captured images of a first scene acquired by an image acquisition terminal; the plurality of captured images include a first target image and a second target image;

[0194] A determination module 320, configured to determine the displacement of the target object in the first scene according to the position of the target object in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene;

[0195] The establishing module 330 is used to establish a VR scene according to the displacement, the collected image and the pre-stored image.

[0196] The VR scene establishment device provided in the embodiment of the present application determines the motion and stillness states of the image acquisition terminal that shoots the first scene based on multiple acquired images of the first scene acquired by the image acquisition terminal, and then establishes different VR game scenes based on different states by combining the acquired images and pre-stored images. This can improve the fun of the combination of robots and VR technology, effectively expand the practicality of the combination of robots and VR technology, and has a very broad application prospect.

[0197] In one embodiment, the establishing module 330 is specifically used for:

[0198] Determine that the magnitude of the displacement of the target object is greater than 0, and determine, according to the direction of the displacement of the target object, a first matching image that matches the direction of the displacement of the target object from a plurality of pre-stored images;

[0199] The captured image and the first matching image are superimposed to establish the VR scene.

[0200] In one embodiment, the establishing module 330 is specifically used for:

[0201] The first matching image is superimposed on the acquired image.

[0202] In one embodiment, the establishing module 330 is further configured to:

[0203] The display of the animation image is controlled according to at least one of a magnitude of the displacement of the target object or a direction of the displacement of the target object.

[0204] In one embodiment, the establishing module 330 is specifically used for:

[0205] Determine that the displacement of the target object is 0, and determine a second matching image that matches the displacement of the target object being 0 from a plurality of pre-stored images;

[0206] The captured image and the second matching image are superimposed to establish the VR scene.

[0207] In one embodiment, the establishing module 330 is further configured to:

[0208] The position of the target object in the superimposed image is determined according to the content of the collected image; the superimposed image is an image formed by superimposing the second matching image on the collected image.

[0209] In one embodiment, the VR scene establishment device provided in the embodiment of the present application further includes a generation module for:

[0210] Receive user input;

[0211] In response to the input, a control instruction is generated and sent to the image acquisition terminal; the control instruction is used to control the movement of the image acquisition terminal.

[0212] In one embodiment, the image acquisition terminal includes a mechanical arm and a shooting device; the shooting device is arranged at one end of the mechanical arm;

[0213] The collected image is obtained by any of the following methods:

[0214] When the robotic arm moves, the photographing device moves along with the robotic arm and acquires the captured image during the movement;

[0215] When the robot arm is stationary, the shooting device acquires the collected image from the stationary position of the shooting device.

[0216] In one embodiment, the displacement of the target object corresponds to the displacement generated by the mechanical arm driving the camera to move within the target time interval;

[0217] The target time interval is the time interval between acquiring the first target image and acquiring the second target image.

[0218] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the following method:

[0219] Receiving multiple acquisitions of a first scene acquired by an image acquisition terminal; the multiple acquired images include a first target image and a second target image;

[0220] Determine the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene;

[0221] A VR scene is established according to the displacement of the target object, the acquired image and the pre-stored image.

[0222] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0223] On the other hand, an embodiment of the present application discloses a computer program product, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer can perform the methods provided by the above-mentioned method embodiments, for example, including:

[0224] Receiving a plurality of captured images of a first scene acquired by an image capture terminal; the plurality of captured images include a first target image and a second target image;

[0225] Determine the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene;

[0226] A VR scene is established according to the displacement of the target object, the acquired image and the pre-stored image.

[0227] On the other hand, the embodiments of the present application further provide a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method provided by the above method embodiments is implemented, for example, including:

[0228] Receiving a plurality of captured images of a first scene acquired by an image capture terminal; the plurality of captured images include a first target image and a second target image;

[0229] Determine the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene;

[0230] A VR scene is established according to the displacement of the target object, the acquired image and the pre-stored image.

[0231] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0232] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can be essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0233] Finally, it should be noted that the above implementation modes are only used to illustrate the present application, rather than to limit the present application. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that various combinations, modifications or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application, and should be included in the scope of the claims of the present application.

Claims

1. A VR scene establishment method, characterized in that: include: Receiving a plurality of captured images of a first scene acquired by an image capture terminal; The multiple acquired images include a first target image and a second target image; Determine the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene; Determine a matching image from pre-stored images according to the displacement of the target object, and establish a VR scene according to the acquired image and the matching image; The image acquisition terminal includes a mechanical arm and a shooting device; the shooting device is arranged at one end of the mechanical arm; The collected image is obtained by any of the following methods: When the robotic arm moves, the photographing device moves along with the robotic arm and acquires the captured image during the movement; When the robot arm is stationary, the shooting device acquires the collected image from the stationary position of the shooting device.

2. The VR scene establishment method according to claim 1, characterized in that: The step of determining a matching image from pre-stored images according to the displacement of the target object, and establishing a VR scene according to the acquired image and the matching image includes: Determine that the magnitude of the displacement of the target object is greater than 0, and determine, according to the direction of the displacement of the target object, a first matching image that matches the direction of the displacement of the target object from a plurality of pre-stored images; The captured image and the first matching image are superimposed to establish the VR scene.

3. The VR scene establishment method according to claim 2, characterized in that: The step of superimposing the collected image and the first matching image includes: The first matching image is superimposed on the acquired image.

4. The VR scene establishment method according to claim 2, characterized in that: The first matching image is an animated image, and the method further includes: The display of the animation image is controlled according to at least one of a magnitude of the displacement of the target object or a direction of the displacement of the target object.

5. The VR scene establishment method according to claim 1, characterized in that: The step of determining a matching image from pre-stored images according to the displacement of the target object, and establishing a VR scene according to the acquired image and the matching image includes: Determine that the displacement of the target object is 0, and determine a second matching image that matches the displacement of the target object being 0 from a plurality of pre-stored images; The captured image and the second matching image are superimposed to establish the VR scene.

6. The VR scene establishment method according to claim 5, characterized in that: The second matching image includes a target object, and the method further includes: The position of the target object in the superimposed image is determined according to the content of the collected image; the superimposed image is an image formed by superimposing the second matching image on the collected image.

7. The VR scene establishment method according to claim 5, characterized in that: After superimposing the collected image and the second matching image to establish the VR scene, the method further includes: Receive user input; In response to the input, a control instruction is generated and sent to the image acquisition terminal; the control instruction is used to control the movement of the image acquisition terminal.

8. The VR scene establishment method according to claim 1, characterized in that: The displacement of the target object corresponds to the displacement generated by the mechanical arm driving the shooting device to move within the target time interval; The target time interval is the time interval between acquiring the first target image and acquiring the second target image.

9. A VR scene establishment system, characterized in that: include: An image acquisition terminal, used to acquire a plurality of acquired images of a first scene; A VR display terminal, which is communicatively connected to the image acquisition terminal; The VR display terminal is used for: Receiving the multiple acquired images acquired by the image acquisition terminal; the multiple acquired images include a first target image and a second target image; Determine the displacement of the target object according to the position of the target object in the first scene in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene; Determine a matching image from pre-stored images according to the displacement of the target object, and establish a VR scene according to the acquired image and the matching image; The image acquisition terminal includes a mechanical arm and a shooting device; the shooting device is arranged at one end of the mechanical arm; The collected image is obtained by any of the following methods: When the robotic arm moves, the photographing device moves along with the robotic arm and acquires the captured image during the movement; When the robot arm is stationary, the shooting device acquires the collected image from the stationary position of the shooting device.

10. The VR scene establishment system according to claim 9, characterized in that: The displacement of the target object corresponds to the displacement generated by the mechanical arm driving the shooting device to move within the target time interval; The target time interval is the time interval between acquiring the first target image and acquiring the second target image.

11. The VR scene establishment system according to claim 10, characterized in that: When the displacement magnitude generated by the mechanical arm driving the shooting device to move within the target time interval is greater than 0, the VR display terminal is specifically used to determine a first matching image matching the displacement direction of the target object from a plurality of pre-stored images according to the displacement direction of the target object, and superimpose the captured image and the first matching image to establish the VR scene; When the displacement size generated by the movement of the shooting device by the robotic arm within the target time interval is 0, the VR display terminal is specifically used to determine a second matching image that matches the displacement size of 0 of the target object from multiple pre-stored images, and superimpose the acquired image and the second matching image to establish the VR scene.

12. The VR scene establishment system according to claim 10 or 11, characterized in that: The image acquisition terminal is also used for: Receiving a control instruction sent by the VR display terminal; The movement of the robot arm is controlled based on the control instruction.

13. A VR scene creation device, characterized in that: include: A receiving module, used for receiving a plurality of captured images of a first scene acquired by an image acquisition terminal; The multiple acquired images include a first target image and a second target image; a determination module, configured to determine the displacement of the target object in the first scene according to the position of the target object in the first target image and the position of the target object in the second target image; wherein the first target image is a captured image acquired before the second target image; and the target object is a stationary object in the first scene; An establishing module, used to determine a matching image from pre-stored images according to the displacement of the target object, and establish a VR scene according to the acquired image and the matching image; The image acquisition terminal includes a mechanical arm and a shooting device; the shooting device is arranged at one end of the mechanical arm; The collected image is obtained by any of the following methods: When the robotic arm moves, the photographing device moves along with the robotic arm and acquires the captured image during the movement; When the robot arm is stationary, the shooting device acquires the collected image from the stationary position of the shooting device.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the VR scene establishment method according to any one of claims 1 to 8 is implemented.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the VR scene establishment method according to any one of claims 1 to 8 is implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the VR scene establishment method according to any one of claims 1 to 8 is implemented.

Images