High-risk indoor environment VR navigation method of mobile robot and related device

Abstract

The invention discloses a high-risk indoor environment VR (virtual reality) navigation method of a mobile robot and a related device, and the method comprises the following steps: the mobile robot performs mapping processing on a to-be-navigated indoor area based on an SLAM (simultaneous localization and mapping) technology to obtain an SLAM map; carrying out image acquisition processing on a to-be-navigated indoor area based on a panoramic camera carried by the mobile robot to obtain an environment image set; performing splicing processing based on the environment image data in the environment image set to generate a panoramic environment image; carrying out fusion processing on the panoramic environment image and the SLAM map to generate a fused SLAM map with the panoramic environment image; and when navigation operation is executed, real-time VR navigation information is generated by utilizing the fused SLAM map based on sensor data acquired by a sensor carried by the mobile robot. According to the embodiment of the invention, the autonomous navigation precision of the mobile robot in a high-risk indoor environment is improved, and a high-definition, three-dimensional and immersive environment map is presented to a user.

Description

Technical Field

[0001] The present invention relates to the field of VR technology, and in particular to a high-risk indoor environment VR navigation method for a mobile robot and related devices. Background Art

[0002] Simultaneous Localization And Mapping (SLAM) technology refers to a mobile robot moving from an unknown position in an unknown environment, positioning itself based on position estimates and sensor observations, and creating an environmental map at the same time; however, due to the high degree of professionalism of SLAM maps, non-professional users cannot experience the immersive feeling of SLAM maps when navigating, and cannot provide users with a good user experience.

[0003] Virtual reality technology (VR for short) uses VR software to stitch multiple images from different angles to generate a 720-degree 3D panoramic view, which can be displayed on a two-dimensional screen or played through VR glasses, providing users with an immersive feeling.

[0004] This technical solution is to solve how to integrate SLAM technology and VR technology into mobile robot navigation, and to achieve the presentation of high-definition, three-dimensional, immersive environmental maps. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a high-risk indoor environment VR navigation method and related devices for a mobile robot, which improves the autonomous navigation accuracy of the mobile robot in a high-risk indoor environment and realizes the presentation of a high-definition, three-dimensional, immersive environmental map to the user.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a high-risk indoor environment VR navigation method for a mobile robot, the method comprising:

[0007] An indoor area to be navigated is obtained, and a mobile robot performs mapping processing on the indoor area to be navigated based on a simultaneous localization and mapping (SLAM) technology to obtain a SLAM map of the mobile robot in the indoor area to be navigated;

[0008] During the mapping process of the mobile robot, image acquisition and processing are performed on the indoor area to be navigated based on the panoramic camera carried by the mobile robot to obtain an environmental image set corresponding to the indoor area to be navigated;

[0009] Generate a panoramic environment image corresponding to the indoor area to be navigated by performing stitching processing based on the environment image data in the environment image set corresponding to the indoor area to be navigated;

[0010] The panoramic environment image is fused with the SLAM map to generate a fused SLAM map with the panoramic environment image;

[0011] When the mobile robot performs navigation operations in the indoor area to be navigated, the fused SLAM map is used to generate real-time VR navigation information based on sensor data collected by sensors carried by the mobile robot, and the sensor data includes the current positioning information, current odometer position information and motion trend information of the mobile robot.

[0012] Optionally, the mobile robot performs mapping processing on the indoor area to be navigated based on the simultaneous localization and mapping SLAM technology to obtain a SLAM map of the mobile robot in the indoor area to be navigated, including:

[0013] Constructing a motion model of the mobile robot in a plane coordinate system when the mobile robot moves, wherein the motion model represents the position of the mobile robot in terms of a horizontal coordinate x, a vertical coordinate y, and an angle θ between the orientation of the mobile robot and the square of the x-axis of the plane coordinate system;

[0014] Obtaining predicted position information of the mobile robot at the current moment by calculating the probability value at the position using the conditional probability density function based on the motion model;

[0015] Based on the predicted position information of the mobile robot at the current moment, the indoor area to be navigated is mapped using the SLAM technology to obtain a SLAM map of the mobile robot in the indoor area to be navigated.

[0016] Optionally, the performing image acquisition and processing on the indoor area to be navigated based on the panoramic camera carried by the mobile robot to obtain an environmental image set corresponding to the indoor area to be navigated includes:

[0017] When the mobile robot moves a preset interval distance in the indoor area to be navigated, the panoramic camera carried by the mobile robot is started to perform image acquisition and processing on the indoor area to be navigated to obtain a certain amount of environmental image data;

[0018] After marking a number of environmental image data with the acquisition location and the acquisition time, the data are sequentially placed into a pre-constructed folder to form an environmental image set corresponding to the indoor area to be navigated.

[0019] Optionally, performing stitching processing based on environmental image data in an environmental image set corresponding to the indoor area to be navigated to generate a panoramic environmental image corresponding to the indoor area to be navigated includes:

[0020] Importing each piece of environmental image data in the environmental image set corresponding to the indoor area to be navigated into the multifunctional panoramic production software in sequence;

[0021] After the multifunctional panoramic production software receives the imported environmental image data, it performs splicing processing on the received environmental image data based on preset adjustment parameters to generate a panoramic environmental image corresponding to the indoor area to be navigated.

[0022] Optionally, fusing the panoramic environment image with the SLAM map to generate a fused SLAM map with the panoramic environment image includes:

[0023] Extracting and processing the obstacle target in the panoramic environment image to obtain first extracted obstacle target data corresponding to the panoramic environment image;

[0024] Extracting and processing the obstacle target in the SLAM map to obtain second extracted obstacle target data corresponding to the SLAM map;

[0025] Based on the first extracted obstacle target data and the second extracted obstacle target data, the panoramic environment image and the SLAM map are superimposed and fused using an orthogonalization mechanism to form a fused SLAM map with a panoramic environment image.

[0026] Optionally, the generating real-time VR navigation information based on the sensor data collected by the sensor carried by the mobile robot using the fused SLAM map includes:

[0027] Performing posture estimation processing of the mobile robot based on sensor data collected by the sensor carried by the mobile robot to obtain current posture data of the mobile robot;

[0028] Controlling the panoramic camera carried by the mobile robot to perform image acquisition and processing on the current environment in the forward direction to obtain image data of the current environment in the forward direction;

[0029] Based on the current posture data, a panoramic environment image index matching process is performed using the current environment image data in the fused SLAM map to obtain a panoramic environment image area that matches the current environment image data;

[0030] Based on the current posture data, the motion trend information and the panoramic environment image area matching the current environment image data, real-time VR navigation generation processing is performed on the fused SLAM map to form real-time VR navigation information.

[0031] Optionally, performing posture estimation processing of the mobile robot based on sensor data collected by a sensor carried by the mobile robot to obtain current posture data of the mobile robot includes:

[0032] Performing data collection and processing based on the sensors carried by the mobile robot to obtain current positioning information and current odometer position data of the mobile robot;

[0033] The current position and posture of the mobile robot is estimated based on the current positioning information and the current odometer position data to obtain the current position and posture data of the mobile robot.

[0034] In addition, an embodiment of the present invention further provides a high-risk indoor environment VR navigation device for a mobile robot, the device comprising:

[0035] Mapping module: used to obtain the indoor area to be navigated, and the mobile robot performs mapping processing on the indoor area to be navigated based on the simultaneous localization and map creation SLAM technology to obtain the SLAM map of the mobile robot in the indoor area to be navigated;

[0036] Image acquisition module: used for performing image acquisition and processing on the indoor area to be navigated based on the panoramic camera carried by the mobile robot during the mapping process of the mobile robot, so as to obtain an environmental image set corresponding to the indoor area to be navigated;

[0037] An image stitching module: used for performing stitching processing based on the environmental image data in the environmental image set corresponding to the indoor area to be navigated, so as to generate a panoramic environmental image corresponding to the indoor area to be navigated;

[0038] Fusion module: used for fusing the panoramic environment image with the SLAM map to generate a fused SLAM map with the panoramic environment image;

[0039] VR navigation generation module: used to generate real-time VR navigation information using the fused SLAM map based on sensor data collected by sensors carried by the mobile robot when the mobile robot performs navigation operations in the indoor area to be navigated, wherein the sensor data includes the current positioning information, current odometer position information and motion trend information of the mobile robot.

[0040] In addition, an embodiment of the present invention further provides a controller of a mobile robot, including a processor and a memory, wherein the processor runs a computer program or code stored in the memory to implement a high-risk indoor environment VR navigation method as described in any one of the above.

[0041] In addition, an embodiment of the present invention further provides a computer-readable storage medium for storing a computer program or code. When the computer program or code is executed by a processor, the high-risk indoor environment VR navigation method as described in any one of the above is implemented.

[0042] In an embodiment of the present invention, a mobile robot performs mapping processing on an indoor area to be navigated based on SLAM technology to obtain a SLAM map; performs image acquisition processing on the indoor area to be navigated based on a panoramic camera carried by the mobile robot to obtain an environmental image set; performs splicing processing on environmental image data in the environmental image set to generate a panoramic environmental image; fuses the panoramic environmental image with the SLAM map to generate a fused SLAM map with the panoramic environmental image; when performing navigation operations, real-time VR navigation information is generated using the fused SLAM map based on sensor data collected by sensors carried by the mobile robot; thereby improving the autonomous navigation accuracy of the mobile robot in high-risk indoor environments, and realizing the presentation of high-definition, three-dimensional, immersive environmental maps to users; and providing users with a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 is a flow chart of a high-risk indoor environment VR navigation method for a mobile robot in an embodiment of the present invention;

[0045] Figure 2 is a flow chart of a high-risk indoor environment VR navigation method for a mobile robot in another embodiment of the present invention;

[0046] Figure 3 2 is a schematic diagram of the structure of a high-risk indoor environment VR navigation device for a mobile robot in an embodiment of the present invention;

[0047] Figure 4 It is a schematic diagram of the structural composition of the controller of the mobile robot in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] For example, see Figure 1 , Figure 1 It is a flowchart of a high-risk indoor environment VR navigation method for a mobile robot in an embodiment of the present invention.

[0050] like Figure 1 As shown, a high-risk indoor environment VR navigation method for a mobile robot, the method comprising:

[0051] S101: obtaining an indoor area to be navigated, and the mobile robot performing mapping processing on the indoor area to be navigated based on the simultaneous localization and mapping SLAM technology to obtain a SLAM map of the mobile robot in the indoor area to be navigated;

[0052] In the specific implementation process of the present invention, the mobile robot maps the indoor area to be navigated based on the simultaneous positioning and map creation SLAM technology to obtain the SLAM map of the mobile robot in the indoor area to be navigated, including: constructing a motion model of the mobile robot in a plane coordinate system when the mobile robot moves, the motion model using a horizontal coordinate x, a vertical coordinate y, and an angle θ between the orientation of the mobile robot and the x-axis square of the plane coordinate system to represent the position of the mobile robot; obtaining the predicted posture information of the mobile robot at the current moment based on the probability value calculated by the conditional probability density function based on the motion model; mapping the indoor area to be navigated based on the predicted posture information of the mobile robot at the current moment using the SLAM technology to obtain the SLAM map of the mobile robot in the indoor area to be navigated.

[0053] Specifically, the simultaneous localization and mapping (SLAM) technology of mobile robots can be described as follows: the robot starts to move from an unknown posture in an unknown environment, and during the movement, it locates itself according to the posture estimation value and sensor observation value, and simultaneously creates an environment map. According to different sensors, mainstream SLAM is divided into laser SLAM and visual SLAM: laser SLAM uses laser radar sensors to observe the environment and uses the principle of laser ranging to perceive the position coordinates, reflectivity and other information of the surface points of objects in the environment; visual SLAM uses visual sensors and can be further divided into monocular visual SLAM, binocular visual SLAM and RGB-D SLAM. In addition to perceiving the position information of objects in the environment, visual sensors can also perceive richer information, such as color and texture. Mobile robots can also be equipped with ultrasonic sensors, inertial measurement units (IMUs), and wireless communication modules such as global positioning systems (GPS), wireless Internet access (WIFI), Bluetooth and ultra-wideband (UWB) [4] to assist in completing positioning and map creation.

[0054] When a mobile robot is mapping, the first step is to build a motion model of the mobile robot using the plane coordinate system when the mobile robot is moving. The motion model represents the position of the mobile robot using the horizontal coordinate x, the vertical coordinate y, and the angle θ between the orientation of the mobile robot and the x-axis square of the plane coordinate system. These three variables are then expressed in vector form, and the vector is called the robot's posture x. t (x,y,θ).

[0055] Using the conditional density function P(x t |u t ,x t-1 ), the probability value given represents the robot’s current position, x t-1 is the position of the mobile robot at the previous moment, u t is the control variable of the mobile robot, including the translation control variable v t , and the rotation control variable ω t , thereby obtaining the predicted posture information of the mobile robot at the current moment by calculating the probability value; and using the predicted posture information of the mobile robot at the current moment to map the indoor area to be navigated using SLAM technology to obtain the SLAM map of the mobile robot in the indoor area to be navigated.

[0056] In this embodiment, a SLAM algorithm based on the extended Kalman filter (EKF) algorithm is selected. The EKF algorithm is one of the most widely used data filtering methods. Data filtering can update and process system data in real time, and is easy to implement through computer programming. The EKF algorithm uses a linearization method based on Taylor series expansion to approximate the nonlinear system, and then recursively estimates and corrects the system state based on the measurement value of the sensor.

[0057] Under the probability-based velocity motion model, assuming that the position x of the mobile robot at time t is t The uncertainty can be expressed as μ t , the variance is Σ t Gaussian distribution bel(x t ). The state equation and measurement equation of the mobile robot are:

[0058] x t =g(u t ,x t-1 )+ε t

[0059] z t =h(x t )+δ t ;

[0060] In the formula, the function g(u t ,x t-1 )、h(x t ) are nonlinear equations, which are applicable to the motion models of most mobile robots in real environments; ε t , δ t They are used to describe the uncertainty of the state model and the measurement model respectively. Assume that ε t The mean is 0 and the variance is R t Gaussian distribution, δ t The mean is 0 and the variance is Q t Gaussian distribution.

[0061] The EKF algorithm is based on the posture x at the previous moment. t-1 and the control quantity u at the current moment t , get the random pose x at the current moment t Estimated value of Then according to the current measurement z t Estimated value Correction is performed, and finally a relatively accurate random pose x at the current moment is obtained t ; In the algorithm, K t It is called Kalman gain, which is a key parameter for state prediction and update. t and H t They are the function g(ut ,x t-1 )、h(x t ) is expanded by Taylor series and the first-order term is taken to approximate the linearization.

[0062] Compared with other algorithms, the EKF algorithm has good timeliness, and the time required for each update is O(k 2.4 +n 2 ), k is the dimension of the measured variable zt, n is the state variable x t The dimension of the particle filter, for example, the update time of the particle filter is exponential in n; the EKF algorithm also has certain limitations. The Taylor series expansion takes the first-order term to obtain an approximate linear equation, which is still different from the nonlinear equation of the actual system. In some cases, the approximate error will cause the EKF algorithm to be unable to accurately estimate the posture of the mobile robot.

[0063] In the algorithm, it is assumed that the known environment map m is feature-based and the measurement value z obtained by the robot's measurement sensor is t At least two-dimensional, including a distance variable r t and an angle variable φ t .

[0064] Based on the EKF SLAM algorithm, a new variable is first introduced, the joint variable y t , which contains the robot's pose variables and map information of the working environment:

[0065]

[0066] x t is the position variable of the mobile robot, and m represents the map of the working environment, where (m N,x ,m N,y ) represents the plane coordinates of the Nth feature point in the map; the SLAM problem is to study how to obtain the joint state variable y t process.

[0067] S102: During the mapping process of the mobile robot, image acquisition and processing are performed on the indoor area to be navigated based on the panoramic camera carried by the mobile robot to obtain an environment image set corresponding to the indoor area to be navigated;

[0068] In the specific implementation process of the present invention, the panoramic camera carried by the mobile robot is used to perform image acquisition and processing on the indoor area to be navigated to obtain an environmental image set corresponding to the indoor area to be navigated, including: when the mobile robot moves a preset interval distance in the indoor area to be navigated, the panoramic camera carried by the mobile robot is started to perform image acquisition and processing on the indoor area to be navigated to obtain a certain number of environmental image data; after marking the certain number of environmental image data with the acquisition position and the acquisition time, they are placed in a pre-constructed folder in order to form an environmental image set corresponding to the indoor area to be navigated.

[0069] Specifically, since the mobile robot is equipped with a panoramic camera, when the mobile robot is controlled to move a preset interval distance in the indoor area to be navigated, the panoramic camera carried by the mobile robot is started to perform image acquisition and processing on the indoor area to be navigated, and a certain number of environmental image data are obtained; at the same time, the certain number of environmental image data are marked with the acquisition position and the acquisition time, and are placed in a pre-constructed folder in sequence, so as to form an environmental image set corresponding to the indoor area to be navigated.

[0070] S103: performing stitching processing based on environmental image data in an environmental image set corresponding to the indoor area to be navigated, to generate a panoramic environmental image corresponding to the indoor area to be navigated;

[0071] In the specific implementation process of the present invention, stitching processing is performed based on the environmental image data in the environmental image set corresponding to the indoor area to be navigated to generate a panoramic environmental image corresponding to the indoor area to be navigated, including: importing each environmental image data in the environmental image set corresponding to the indoor area to be navigated into a multifunctional panoramic production software in sequence; after the multifunctional panoramic production software receives the imported environmental image data, stitching processing is performed on the received environmental image data based on preset adjustment parameters to generate a panoramic environmental image corresponding to the indoor area to be navigated.

[0072] Specifically, each environmental image data in the environmental image set corresponding to the indoor area to be navigated is imported into the multi-functional panoramic production software in turn, wherein the multi-functional panorama is Panorama Tools software, and PTgui is a graphical user interface of Panorama Tools (multi-functional panoramic production tool); the software is free and widely used but complicated to operate; PTgui provides a graphical user interface for Panorama Tools (multi-functional panoramic production tool) for graphic stitching, so that the panorama can be presented to the user in a high-quality form.

[0073] That is, after the multifunctional panoramic production software receives the imported environmental image data, it splices the received environmental image data through preset adjustment parameters to generate a panoramic environmental image corresponding to the indoor area to be navigated.

[0074] Using PTgui, you can easily create more dazzling panoramas. After importing pictures, adjust the parameters to make PTgui stitch and combine them into a panorama. The software can produce multiple views, and the functions and mapping are also very complete and diverse. Not only can the stitching accuracy be modified, but also control points can be added. The image file input is very diverse, and the output has a high dynamic range of image selection. The traces of image combination will not be visible, and the brightness is consistent. Therefore, the panorama generated by PTgui is very comprehensive.

[0075] S104: fusing the panoramic environment image with the SLAM map to generate a fused SLAM map with the panoramic environment image;

[0076] In the specific implementation process of the present invention, the panoramic environment image and the SLAM map are fused to generate a fused SLAM map with a panoramic environment image, including: extracting obstacle targets in the panoramic environment image to obtain first extracted obstacle target data corresponding to the panoramic environment image; extracting obstacle targets in the SLAM map to obtain second extracted obstacle target data corresponding to the SLAM map; based on the first extracted obstacle target data and the second extracted obstacle target data, the panoramic environment image and the SLAM map are superimposed and fused using an orthogonalization mechanism to form a fused SLAM map with a panoramic environment image.

[0077] Specifically, obstacle targets in the panoramic environment image are first extracted and processed to obtain first extracted obstacle target data corresponding to the panoramic environment image; then, obstacle targets in the SLAM map are extracted and processed to obtain second extracted obstacle target data corresponding to the SLAM map; the first extracted obstacle target data is used as the feature of the panoramic environment image, and the second extracted obstacle target data is used as the feature of the SLAM map, and these two features overlap with each other, but direct superposition and fusion may cause the independence between the first extracted obstacle target data and the second extracted obstacle target data to be reduced, thereby causing feature collapse, so the panoramic environment image and the SLAM map are superimposed and fused by the first extracted obstacle target data and the second extracted obstacle target data using an orthogonalization mechanism to form a fused SLAM map with a panoramic environment image.

[0078] S105: When the mobile robot performs navigation operations in the indoor area to be navigated, real-time VR navigation information is generated using the fused SLAM map based on sensor data collected by sensors carried by the mobile robot, wherein the sensor data includes current positioning information, current odometer position information, and motion trend information of the mobile robot.

[0079] In the specific implementation process of the present invention, the sensor data collected by the sensors carried by the mobile robot is used to generate real-time VR navigation information using the fused SLAM map, including: performing posture estimation processing of the mobile robot based on the sensor data collected by the sensors carried by the mobile robot to obtain current posture data of the mobile robot; controlling the panoramic camera carried by the mobile robot to perform image acquisition processing on the current environment in the forward direction to obtain current environment image data in the forward direction; based on the current posture data, using the panoramic environment image index matching processing of the current environment image data in the fused SLAM map to obtain a panoramic environment image area matching the current environment image data; based on the current posture data, the motion trend information and the panoramic environment image area matching the current environment image data, real-time VR navigation generation processing is performed in the fused SLAM map to form real-time VR navigation information.

[0080] Furthermore, the posture estimation processing of the mobile robot is performed based on the sensor data collected by the sensors carried by the mobile robot to obtain the current posture data of the mobile robot, including: performing data collection processing based on the sensors carried by the mobile robot to obtain the current positioning information and current odometer position data of the mobile robot; estimating the current posture of the mobile robot based on the current positioning information and the current odometer position data to obtain the current posture data of the mobile robot.

[0081] When the mobile robot performs navigation operations in the indoor area to be navigated, the first step is to estimate the posture of the mobile robot through the sensor data collected by the sensor carried by the mobile robot to obtain the current posture data of the mobile robot; then the panoramic camera carried by the mobile robot is controlled to collect and process the current environment in the forward direction to obtain the current environment image data in the forward direction; then the panoramic environment image index matching processing is performed using the current environment image data in the fused SLAM map through the current posture data to obtain the panoramic environment image area matching the current environment image data; finally, the real-time VR navigation generation processing is performed on the fused SLAM map according to the current posture data, motion trend information and the panoramic environment image area matching the current environment image data to form real-time VR navigation information; the real-time VR navigation generation processing here is to generate VR navigation through Jiushang VR Cloud. The tools of this VR panoramic production platform have many personalized features, including background music, special effects, hot spots, navigation, navigation maps, etc., which can realize the upload of high-definition pictures and pictures that occupy a large memory area, with a high speed. A single picture can be up to 300M, and there will be no watermarks or advertisements on the page of the panoramic work, nor will it be compressed, and it will be regularly upgraded and updated every week.

[0082] When estimating the posture of a mobile robot, data collection and processing will first be performed based on the sensors carried by the mobile robot to obtain the current positioning information and current odometer position data of the mobile robot; then the current posture of the mobile robot will be estimated based on the current positioning information and current odometer position data, and finally the current posture data of the mobile robot will be obtained.

[0083] In an embodiment of the present invention, a mobile robot performs mapping processing on an indoor area to be navigated based on SLAM technology to obtain a SLAM map; performs image acquisition processing on the indoor area to be navigated based on a panoramic camera carried by the mobile robot to obtain an environmental image set; performs splicing processing on environmental image data in the environmental image set to generate a panoramic environmental image; fuses the panoramic environmental image with the SLAM map to generate a fused SLAM map with the panoramic environmental image; when performing navigation operations, real-time VR navigation information is generated using the fused SLAM map based on sensor data collected by sensors carried by the mobile robot; thereby improving the autonomous navigation accuracy of the mobile robot in high-risk indoor environments, and realizing the presentation of high-definition, three-dimensional, immersive environmental maps to users; and providing users with a better user experience.

[0084] For example 2, please refer to Figure 2 , Figure 2 It is a flowchart of a high-risk indoor environment VR navigation method for a mobile robot in another embodiment of the present invention.

[0085] like Figure 2 As shown, a high-risk indoor environment VR navigation method for a mobile robot, the method comprising:

[0086] S201: Obtain an indoor area to be navigated, and construct a motion model of the mobile robot in a plane coordinate system when the mobile robot moves, wherein the motion model represents the position of the mobile robot in terms of a horizontal coordinate x, a vertical coordinate y, and an angle θ between the orientation of the mobile robot and the square of the x-axis of the plane coordinate system;

[0087] S202: Obtaining predicted position information of the mobile robot at the current moment by calculating the probability value at the conditional probability density function based on the motion model;

[0088] S203: Mapping the indoor area to be navigated using SLAM technology based on the predicted position information of the mobile robot at the current moment, to obtain a SLAM map of the mobile robot in the indoor area to be navigated;

[0089] S204: During the mapping process of the mobile robot, image acquisition and processing are performed on the indoor area to be navigated based on the panoramic camera carried by the mobile robot to obtain an environment image set corresponding to the indoor area to be navigated;

[0090] S205: performing stitching processing based on the environmental image data in the environmental image set corresponding to the indoor area to be navigated, to generate a panoramic environmental image corresponding to the indoor area to be navigated;

[0091] S206: Fusing the panoramic environment image with the SLAM map to generate a fused SLAM map with the panoramic environment image;

[0092] S207: When the mobile robot performs navigation in the indoor area to be navigated, data collection and processing are performed based on sensors carried by the mobile robot to obtain current positioning information and current odometer position data of the mobile robot;

[0093] S208: estimating the current position and posture of the mobile robot based on the current positioning information and the current odometer position data to obtain current position and posture data of the mobile robot;

[0094] S209: controlling the panoramic camera carried by the mobile robot to perform image acquisition and processing on the current environment in the forward direction to obtain image data of the current environment in the forward direction;

[0095] S210: performing a panoramic environment image index matching process in the fused SLAM map using the current environment image data based on the current posture data to obtain a panoramic environment image area that matches the current environment image data;

[0096] S211: Based on the current posture data, the motion trend information and the panoramic environment image area matching the current environment image data, real-time VR navigation generation processing is performed on the fused SLAM map to form real-time VR navigation information.

[0097] Specifically, the specific implementation of the second embodiment can be referred to the above embodiment, which will not be described in detail here.

[0098] For example 3, please refer to Figure 3 , Figure 3 It is a schematic diagram of the structural composition of the high-risk indoor environment VR navigation device of the mobile robot in an embodiment of the present invention.

[0099] like Figure 3 As shown, a high-risk indoor environment VR navigation device for a mobile robot, the device comprising:

[0100] Mapping module 301: used to obtain an indoor area to be navigated, and the mobile robot performs mapping processing on the indoor area to be navigated based on the simultaneous localization and mapping SLAM technology to obtain a SLAM map of the mobile robot in the indoor area to be navigated;

[0101] In the specific implementation process of the present invention, the mobile robot maps the indoor area to be navigated based on the simultaneous positioning and map creation SLAM technology to obtain the SLAM map of the mobile robot in the indoor area to be navigated, including: constructing a motion model of the mobile robot in a plane coordinate system when the mobile robot moves, the motion model using a horizontal coordinate x, a vertical coordinate y, and an angle θ between the orientation of the mobile robot and the x-axis square of the plane coordinate system to represent the position of the mobile robot; obtaining the predicted posture information of the mobile robot at the current moment based on the probability value calculated by the conditional probability density function based on the motion model; mapping the indoor area to be navigated based on the predicted posture information of the mobile robot at the current moment using the SLAM technology to obtain the SLAM map of the mobile robot in the indoor area to be navigated.

[0102] Specifically, the simultaneous localization and mapping (SLAM) technology of mobile robots can be described as follows: the robot starts to move from an unknown posture in an unknown environment, and during the movement, it locates itself according to the posture estimation value and sensor observation value, and simultaneously creates an environment map. According to different sensors, mainstream SLAM is divided into laser SLAM and visual SLAM: laser SLAM uses laser radar sensors to observe the environment and uses the principle of laser ranging to perceive the position coordinates, reflectivity and other information of the surface points of objects in the environment; visual SLAM uses visual sensors and can be further divided into monocular visual SLAM, binocular visual SLAM and RGB-D SLAM. In addition to perceiving the position information of objects in the environment, visual sensors can also perceive richer information, such as color and texture. Mobile robots can also be equipped with ultrasonic sensors, inertial measurement units (IMUs), and wireless communication modules such as global positioning systems (GPS), wireless Internet access (WIFI), Bluetooth and ultra-wideband (UWB) [4] to assist in completing positioning and map creation.

[0103] When a mobile robot is mapping, the first step is to build a motion model of the mobile robot using the plane coordinate system when the mobile robot is moving. The motion model represents the position of the mobile robot using the horizontal coordinate x, the vertical coordinate y, and the angle θ between the orientation of the mobile robot and the x-axis square of the plane coordinate system. These three variables are then expressed in vector form, and the vector is called the robot's posture x. t (x,y,θ).

[0104] Using the conditional density function P(x t |u t ,x t-1 ), the probability value given represents the robot’s current position, x t-1 is the position of the mobile robot at the previous moment, u t is the control variable of the mobile robot, including the translation control variable v t , and the rotation control variable ω t , thereby obtaining the predicted posture information of the mobile robot at the current moment by calculating the probability value; and using the predicted posture information of the mobile robot at the current moment to map the indoor area to be navigated using SLAM technology to obtain the SLAM map of the mobile robot in the indoor area to be navigated.

[0105] In this embodiment, a SLAM algorithm based on the extended Kalman filter (EKF) algorithm is selected. The EKF algorithm is one of the most widely used data filtering methods. Data filtering can update and process system data in real time, and is easy to implement through computer programming. The EKF algorithm uses a linearization method based on Taylor series expansion to approximate the nonlinear system, and then recursively estimates and corrects the system state based on the measurement value of the sensor.

[0106] Under the probability-based velocity motion model, assuming that the position x of the mobile robot at time t is t The uncertainty can be expressed as μ t , the variance is Σ t Gaussian distribution bel(x t ). The state equation and measurement equation of the mobile robot are:

[0107] x t =g(u t ,x t-1 )+ε t

[0108] z t =h(x t )+δ t ;

[0109] In the formula, the function g(u t ,x t-1 )、h(x t ) are nonlinear equations, which are applicable to the motion models of most mobile robots in real environments; ε t , δ t They are used to describe the uncertainty of the state model and the measurement model respectively. Assume that ε t The mean is 0 and the variance is R t Gaussian distribution, δ t The mean is 0 and the variance is Q t Gaussian distribution.

[0110] The EKF algorithm is based on the posture x at the previous moment. t-1 and the control quantity u at the current moment t , get the random pose x at the current moment t Estimated value of Then according to the current measurement z t Estimated value Correction is performed, and finally a relatively accurate random pose x at the current moment is obtained t ; In the algorithm, K t It is called Kalman gain, which is a key parameter for state prediction and update. t and H t They are the function g(ut ,x t-1 )、h(x t ) is expanded by Taylor series and the first-order term is taken to approximate the linearization.

[0111] Compared with other algorithms, the EKF algorithm has good timeliness, and the time required for each update is O(k 2.4 +n 2 ), k is the dimension of the measured variable zt, n is the state variable x t The dimension of the particle filter, for example, the update time of the particle filter is exponential in n; the EKF algorithm also has certain limitations. The Taylor series expansion takes the first-order term to obtain an approximate linear equation, which is still different from the nonlinear equation of the actual system. In some cases, the approximate error will cause the EKF algorithm to be unable to accurately estimate the posture of the mobile robot.

[0112] In the algorithm, it is assumed that the known environment map m is feature-based and the measurement value z obtained by the robot's measurement sensor is t At least two-dimensional, including a distance variable r t and an angle variable φ t .

[0113] Based on the EKF SLAM algorithm, a new variable is first introduced, the joint variable y t , which contains the robot's pose variables and map information of the working environment:

[0114]

[0115] x t is the position variable of the mobile robot, and m represents the map of the working environment, where (m N,x ,m N,y ) represents the plane coordinates of the Nth feature point in the map; the SLAM problem is to study how to obtain the joint state variable y t process.

[0116] Image acquisition module 302: used for performing image acquisition and processing on the indoor area to be navigated based on the panoramic camera carried by the mobile robot during the mapping process of the mobile robot, so as to obtain an environment image set corresponding to the indoor area to be navigated;

[0117] In the specific implementation process of the present invention, the panoramic camera carried by the mobile robot is used to perform image acquisition and processing on the indoor area to be navigated to obtain an environmental image set corresponding to the indoor area to be navigated, including: when the mobile robot moves a preset interval distance in the indoor area to be navigated, the panoramic camera carried by the mobile robot is started to perform image acquisition and processing on the indoor area to be navigated to obtain a certain number of environmental image data; after marking the certain number of environmental image data with the acquisition position and the acquisition time, they are placed in a pre-constructed folder in order to form an environmental image set corresponding to the indoor area to be navigated.

[0118] Specifically, since the mobile robot is equipped with a panoramic camera, when the mobile robot is controlled to move a preset interval distance in the indoor area to be navigated, the panoramic camera carried by the mobile robot is started to perform image acquisition and processing on the indoor area to be navigated, and a certain number of environmental image data are obtained; at the same time, the certain number of environmental image data are marked with the acquisition position and the acquisition time, and are placed in a pre-constructed folder in sequence, so as to form an environmental image set corresponding to the indoor area to be navigated.

[0119] Image stitching module 303: used to perform stitching processing based on the environmental image data in the environmental image set corresponding to the indoor area to be navigated, so as to generate a panoramic environmental image corresponding to the indoor area to be navigated;

[0120] In the specific implementation process of the present invention, stitching processing is performed based on the environmental image data in the environmental image set corresponding to the indoor area to be navigated to generate a panoramic environmental image corresponding to the indoor area to be navigated, including: importing each environmental image data in the environmental image set corresponding to the indoor area to be navigated into a multifunctional panoramic production software in sequence; after the multifunctional panoramic production software receives the imported environmental image data, stitching processing is performed on the received environmental image data based on preset adjustment parameters to generate a panoramic environmental image corresponding to the indoor area to be navigated.

[0121] Specifically, each environmental image data in the environmental image set corresponding to the indoor area to be navigated is imported into the multi-functional panoramic production software in turn, wherein the multi-functional panorama is Panorama Tools software, and PTgui is a graphical user interface of Panorama Tools (multi-functional panoramic production tool); the software is free and widely used but complicated to operate; PTgui provides a graphical user interface for Panorama Tools (multi-functional panoramic production tool) for graphic stitching, so that the panorama can be presented to the user in a high-quality form.

[0122] That is, after the multifunctional panoramic production software receives the imported environmental image data, it splices the received environmental image data through preset adjustment parameters to generate a panoramic environmental image corresponding to the indoor area to be navigated.

[0123] Using PTgui, you can easily create more dazzling panoramas. After importing pictures, adjust the parameters to make PTgui stitch and combine them into a panorama. The software can produce multiple views, and the functions and mapping are also very complete and diverse. Not only can the stitching accuracy be modified, but also control points can be added. The image file input is very diverse, and the output has a high dynamic range of image selection. The traces of image combination will not be visible, and the brightness is consistent. Therefore, the panorama generated by PTgui is very comprehensive.

[0124] Fusion module 304: used for fusing the panoramic environment image with the SLAM map to generate a fused SLAM map with the panoramic environment image;

[0125] In the specific implementation process of the present invention, the panoramic environment image and the SLAM map are fused to generate a fused SLAM map with a panoramic environment image, including: extracting obstacle targets in the panoramic environment image to obtain first extracted obstacle target data corresponding to the panoramic environment image; extracting obstacle targets in the SLAM map to obtain second extracted obstacle target data corresponding to the SLAM map; based on the first extracted obstacle target data and the second extracted obstacle target data, the panoramic environment image and the SLAM map are superimposed and fused using an orthogonalization mechanism to form a fused SLAM map with a panoramic environment image.

[0126] Specifically, obstacle targets in the panoramic environment image are first extracted and processed to obtain first extracted obstacle target data corresponding to the panoramic environment image; then, obstacle targets in the SLAM map are extracted and processed to obtain second extracted obstacle target data corresponding to the SLAM map; the first extracted obstacle target data is used as the feature of the panoramic environment image, and the second extracted obstacle target data is used as the feature of the SLAM map, and these two features overlap with each other, but direct superposition and fusion may cause the independence between the first extracted obstacle target data and the second extracted obstacle target data to be reduced, thereby causing feature collapse, so the panoramic environment image and the SLAM map are superimposed and fused by the first extracted obstacle target data and the second extracted obstacle target data using an orthogonalization mechanism to form a fused SLAM map with a panoramic environment image.

[0127] VR navigation generation module 305: is used to generate real-time VR navigation information using the fused SLAM map based on sensor data collected by sensors carried by the mobile robot when the mobile robot performs navigation operations in the indoor area to be navigated, wherein the sensor data includes the current positioning information, current odometer position information and motion trend information of the mobile robot.

[0128] In the specific implementation process of the present invention, the sensor data collected by the sensors carried by the mobile robot is used to generate real-time VR navigation information using the fused SLAM map, including: performing posture estimation processing of the mobile robot based on the sensor data collected by the sensors carried by the mobile robot to obtain current posture data of the mobile robot; controlling the panoramic camera carried by the mobile robot to perform image acquisition processing on the current environment in the forward direction to obtain current environment image data in the forward direction; based on the current posture data, using the panoramic environment image index matching processing of the current environment image data in the fused SLAM map to obtain a panoramic environment image area matching the current environment image data; based on the current posture data, the motion trend information and the panoramic environment image area matching the current environment image data, real-time VR navigation generation processing is performed in the fused SLAM map to form real-time VR navigation information.

[0129] Furthermore, the posture estimation processing of the mobile robot is performed based on the sensor data collected by the sensors carried by the mobile robot to obtain the current posture data of the mobile robot, including: performing data collection processing based on the sensors carried by the mobile robot to obtain the current positioning information and current odometer position data of the mobile robot; estimating the current posture of the mobile robot based on the current positioning information and the current odometer position data to obtain the current posture data of the mobile robot.

[0130] When the mobile robot performs navigation operations in the indoor area to be navigated, the first step is to estimate the posture of the mobile robot through the sensor data collected by the sensor carried by the mobile robot to obtain the current posture data of the mobile robot; then the panoramic camera carried by the mobile robot is controlled to collect and process the current environment in the forward direction to obtain the current environment image data in the forward direction; then the panoramic environment image index matching processing is performed using the current environment image data in the fused SLAM map through the current posture data to obtain the panoramic environment image area matching the current environment image data; finally, the real-time VR navigation generation processing is performed on the fused SLAM map according to the current posture data, motion trend information and the panoramic environment image area matching the current environment image data to form real-time VR navigation information; the real-time VR navigation generation processing here is to generate VR navigation through Jiushang VR Cloud. The tools of this VR panoramic production platform have many personalized features, including background music, special effects, hot spots, navigation, navigation maps, etc., which can realize the upload of high-definition pictures and pictures that occupy a large memory area, with a high speed. A single picture can be up to 300M, and there will be no watermarks or advertisements on the page of the panoramic work, nor will it be compressed, and it will be regularly upgraded and updated every week.

[0131] When estimating the posture of a mobile robot, data collection and processing will first be performed based on the sensors carried by the mobile robot to obtain the current positioning information and current odometer position data of the mobile robot; then the current posture of the mobile robot will be estimated based on the current positioning information and current odometer position data, and finally the current posture data of the mobile robot will be obtained.

[0132] In an embodiment of the present invention, a mobile robot performs mapping processing on an indoor area to be navigated based on SLAM technology to obtain a SLAM map; performs image acquisition processing on the indoor area to be navigated based on a panoramic camera carried by the mobile robot to obtain an environmental image set; performs splicing processing on environmental image data in the environmental image set to generate a panoramic environmental image; fuses the panoramic environmental image with the SLAM map to generate a fused SLAM map with the panoramic environmental image; when performing navigation operations, real-time VR navigation information is generated using the fused SLAM map based on sensor data collected by sensors carried by the mobile robot; thereby improving the autonomous navigation accuracy of the mobile robot in high-risk indoor environments, and realizing the presentation of high-definition, three-dimensional, immersive environmental maps to users; and providing users with a better user experience.

[0133] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, a high-risk indoor environment VR navigation method of any one of the above embodiments is implemented. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (EraSable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (for example, a computer, a mobile phone), which can be a read-only memory, a disk or an optical disk, etc.

[0134] An embodiment of the present invention further provides a computer application program that runs on a computer and is used to execute the high-risk indoor environment VR navigation method of any one of the above embodiments.

[0135] also, Figure 4 It is a schematic diagram of the structural composition of the controller of the mobile robot in an embodiment of the present invention.

[0136] The present invention also provides a controller for a mobile robot. Figure 4 As shown. The controller of the mobile robot includes devices such as a processor 402, a memory 403, an input unit 404, and a display unit 405. Those skilled in the art will understand that Figure 4The structural components of the controller of the mobile robot shown do not constitute a limitation on all devices, and may include more or fewer components than shown, or combine certain components. The memory 403 can be used to store the application 401 and various functional modules, and the processor 402 runs the application 401 stored in the memory 403, thereby executing various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include both internal and external memories. The internal memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random access memory. The external memory may include a hard disk, a floppy disk, a ZIP disk, a U disk, a magnetic tape, etc. The memory disclosed in the present invention includes but is not limited to these types of memories. The memory disclosed in the present invention is only used as an example and not as a limitation.

[0137] The input unit 404 is used to receive the input of the signal and the keyword input by the user. The input unit 404 may include a touch panel and other input devices. The touch panel can collect the user's touch operation on or near it (such as the user's operation on or near the touch panel using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program; other input devices may include but are not limited to one or more of a physical keyboard, a function key (such as a playback control key, a switch key, etc.), a trackball, a mouse, a joystick, etc. The display unit 405 can be used to display the information input by the user or the information provided to the user and various menus of the terminal device. The display unit 405 can be in the form of a liquid crystal display, an organic light emitting diode, etc. The processor 402 is the control center of the terminal device, which uses various interfaces and lines to connect the various parts of the entire device, and executes various functions and processes data by running or executing software programs and / or modules stored in the memory 403, and calling the data stored in the memory.

[0138] As an embodiment, the controller of the mobile robot includes: one or more processors 402, a memory 403, and one or more applications 401, wherein the one or more applications 401 are stored in the memory 403 and are configured to be executed by the one or more processors 402, and the one or more applications 401 are configured to execute the high-risk indoor environment VR navigation method corresponding to any one of the above-mentioned embodiments.

[0139] In an embodiment of the present invention, a mobile robot performs mapping processing on an indoor area to be navigated based on SLAM technology to obtain a SLAM map; performs image acquisition processing on the indoor area to be navigated based on a panoramic camera carried by the mobile robot to obtain an environmental image set; performs splicing processing on environmental image data in the environmental image set to generate a panoramic environmental image; fuses the panoramic environmental image with the SLAM map to generate a fused SLAM map with the panoramic environmental image; when performing navigation operations, real-time VR navigation information is generated using the fused SLAM map based on sensor data collected by sensors carried by the mobile robot; thereby improving the autonomous navigation accuracy of the mobile robot in high-risk indoor environments, and realizing the presentation of high-definition, three-dimensional, immersive environmental maps to users; and providing users with a better user experience.

[0140] In addition, the above is a detailed introduction to the high-risk indoor environment VR navigation method and related devices of a mobile robot provided by an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in the field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A VR navigation method for a mobile robot in a high-risk indoor environment, characterized in that: The method comprises: An indoor area to be navigated is obtained, and a mobile robot performs mapping processing on the indoor area to be navigated based on a simultaneous localization and mapping (SLAM) technology to obtain a SLAM map of the mobile robot in the indoor area to be navigated; During the mapping process of the mobile robot, image acquisition and processing are performed on the indoor area to be navigated based on the panoramic camera carried by the mobile robot to obtain an environmental image set corresponding to the indoor area to be navigated; Generate a panoramic environment image corresponding to the indoor area to be navigated by performing stitching processing based on the environment image data in the environment image set corresponding to the indoor area to be navigated; The panoramic environment image is fused with the SLAM map to generate a fused SLAM map with the panoramic environment image; When the mobile robot performs navigation operations in the indoor area to be navigated, the fused SLAM map is used to generate real-time VR navigation information based on sensor data collected by sensors carried by the mobile robot, and the sensor data includes the current positioning information, current odometer position information and motion trend information of the mobile robot.

2. The high-risk indoor environment VR navigation method according to claim 1, characterized in that: The mobile robot performs mapping processing on the indoor area to be navigated based on the simultaneous localization and mapping SLAM technology to obtain a SLAM map of the mobile robot in the indoor area to be navigated, including: Constructing a motion model of the mobile robot in a plane coordinate system when the mobile robot moves, wherein the motion model represents the position of the mobile robot in terms of a horizontal coordinate x, a vertical coordinate y, and an angle θ between the orientation of the mobile robot and the square of the x-axis of the plane coordinate system; Obtaining predicted position information of the mobile robot at the current moment by calculating the probability value at the position using the conditional probability density function based on the motion model; Based on the predicted position information of the mobile robot at the current moment, the indoor area to be navigated is mapped using the SLAM technology to obtain a SLAM map of the mobile robot in the indoor area to be navigated.

3. The high-risk indoor environment VR navigation method according to claim 1, characterized in that: The method of performing image acquisition and processing on the indoor area to be navigated based on the panoramic camera carried by the mobile robot to obtain an environmental image set corresponding to the indoor area to be navigated includes: When the mobile robot moves a preset interval distance in the indoor area to be navigated, the panoramic camera carried by the mobile robot is started to perform image acquisition and processing on the indoor area to be navigated to obtain a certain amount of environmental image data; After marking a number of environmental image data with the acquisition location and the acquisition time, the data are sequentially placed into a pre-constructed folder to form an environmental image set corresponding to the indoor area to be navigated.

4. The high-risk indoor environment VR navigation method according to claim 1, characterized in that: Based on the environmental image data in the environmental image set corresponding to the indoor area to be navigated, a panoramic environmental image corresponding to the indoor area to be navigated is generated, including: Importing each piece of environmental image data in the environmental image set corresponding to the indoor area to be navigated into the multifunctional panoramic production software in sequence; After the multifunctional panoramic production software receives the imported environmental image data, it performs splicing processing on the received environmental image data based on preset adjustment parameters to generate a panoramic environmental image corresponding to the indoor area to be navigated.

5. The high-risk indoor environment VR navigation method according to claim 1, characterized in that: The step of fusing the panoramic environment image with the SLAM map to generate a fused SLAM map with the panoramic environment image comprises: Extracting and processing the obstacle target in the panoramic environment image to obtain first extracted obstacle target data corresponding to the panoramic environment image; Extracting and processing the obstacle target in the SLAM map to obtain second extracted obstacle target data corresponding to the SLAM map; Based on the first extracted obstacle target data and the second extracted obstacle target data, the panoramic environment image and the SLAM map are superimposed and fused using an orthogonalization mechanism to form a fused SLAM map with a panoramic environment image.

6. The high-risk indoor environment VR navigation method according to claim 1, characterized in that: The method of generating real-time VR navigation information based on the sensor data collected by the sensor carried by the mobile robot using the fused SLAM map includes: Performing position and posture estimation processing of the mobile robot based on sensor data collected by the sensor carried by the mobile robot to obtain current position and posture data of the mobile robot; Controlling the panoramic camera carried by the mobile robot to perform image acquisition and processing on the current environment in the forward direction to obtain image data of the current environment in the forward direction; Based on the current posture data, a panoramic environment image index matching process is performed using the current environment image data in the fused SLAM map to obtain a panoramic environment image area that matches the current environment image data; Based on the current posture data, the motion trend information and the panoramic environment image area matching the current environment image data, real-time VR navigation generation processing is performed on the fused SLAM map to form real-time VR navigation information.

7. The high-risk indoor environment VR navigation method according to claim 6, characterized in that: The step of performing posture estimation processing of the mobile robot based on sensor data collected by the sensor carried by the mobile robot to obtain current posture data of the mobile robot includes: Performing data collection and processing based on the sensors carried by the mobile robot to obtain current positioning information and current odometer position data of the mobile robot; The current position and posture of the mobile robot is estimated based on the current positioning information and the current odometer position data to obtain the current position and posture data of the mobile robot.

8. A VR navigation device for a mobile robot in a high-risk indoor environment, characterized in that: The device comprises: Mapping module: used to obtain the indoor area to be navigated, and the mobile robot performs mapping processing on the indoor area to be navigated based on the simultaneous localization and map creation SLAM technology to obtain the SLAM map of the mobile robot in the indoor area to be navigated; Image acquisition module: used for performing image acquisition and processing on the indoor area to be navigated based on the panoramic camera carried by the mobile robot during the mapping process of the mobile robot, so as to obtain an environmental image set corresponding to the indoor area to be navigated; An image stitching module is used to stitch the environmental image data in the environmental image set corresponding to the indoor area to be navigated, so as to generate a panoramic environmental image corresponding to the indoor area to be navigated; Fusion module: used for fusing the panoramic environment image with the SLAM map to generate a fused SLAM map with the panoramic environment image; VR navigation generation module: used to generate real-time VR navigation information using the fused SLAM map based on sensor data collected by sensors carried by the mobile robot when the mobile robot performs navigation operations in the indoor area to be navigated, wherein the sensor data includes the current positioning information, current odometer position information and motion trend information of the mobile robot.

9. A controller for a mobile robot, comprising a processor and a memory, characterized in that: The processor runs a computer program or code stored in the memory to implement the high-risk indoor environment VR navigation method according to any one of claims 1 to 7.

10. A computer-readable storage medium for storing a computer program or code, characterized in that: When the computer program or code is executed by a processor, the high-risk indoor environment VR navigation method according to any one of claims 1 to 7 is implemented.

Images