Indoor positioning and posture determination method and device based on panoramic video

Abstract

The present invention relates to a method, device, and electronic device for indoor positioning and attitude determination based on panoramic video. The method comprises: acquiring a panoramic image, dividing the panoramic image according to a set rule to obtain cylindrical fisheye images; converting the cylindrical fisheye images into spherical fisheye images, determining position and attitude based on the spherical fisheye images; and performing positioning and attitude determination based on the position and attitude. The indoor positioning and attitude determination method based on panoramic video provided by the present invention achieves precise indoor navigation.

Description

Technical Field

[0001] The present invention relates to the field of navigation and positioning technology, and in particular to an indoor positioning and attitude determination method, device and electronic equipment based on panoramic video. Background Art

[0002] Pedestrian indoor navigation is an important branch of navigation and positioning. Existing navigation and positioning methods can be mainly divided into the following three categories: the first is satellite positioning navigation (GNSS) technology, but the lack of indoor GNSS signals makes GNSS-based pedestrian navigation methods unusable indoors; the second is indoor navigation and positioning based on wireless communication technologies such as WIFI, Bluetooth, and UWB, which require the deployment of corresponding equipment indoors, and the positioning accuracy is greatly affected by the environment and is susceptible to interference; the third is indoor pedestrian navigation and positioning based on inertial sensors (IMUs), but IMU equipment is expensive and the positioning accuracy is unstable, so it cannot be used in pedestrian indoor navigation. Summary of the Invention

[0003] In view of this, it is necessary to provide an indoor positioning and attitude determination method, device and electronic equipment based on panoramic video to solve the problem that accurate indoor navigation cannot be achieved in the existing technology.

[0004] In order to solve the above problems, the present invention provides an indoor positioning and posture determination method based on panoramic video, comprising:

[0005] Acquire a panoramic image, and divide the panoramic image into cylindrical fisheye images according to a set rule;

[0006] Converting the cylindrical fisheye image into a spherical fisheye image, and determining the position and posture according to the spherical fisheye image;

[0007] Positioning and posture determination are performed according to the position and posture.

[0008] Furthermore, the panoramic image is divided according to a set rule to obtain a cylindrical fisheye image, including:

[0009] The panoramic image is expanded according to a cylinder to obtain a corresponding cylindrical panoramic image, and the cylindrical panoramic image is divided into cylindrical fisheye images of front perspective, rear perspective, left perspective and right perspective according to a set rule.

[0010] Furthermore, converting the cylindrical fisheye image into a spherical fisheye image comprises:

[0011] Construct cylindrical projection coordinate system, spherical projection coordinate system, spherical coordinate system and spherical image coordinate system;

[0012] Taking any point on the cylindrical fisheye image and projecting it onto the cylindrical projection coordinate system to obtain cylindrical projection coordinates, and converting the cylindrical projection coordinates into spherical projection coordinates on the spherical projection coordinate system using a first coordinate conversion formula;

[0013] The spherical projection coordinates are converted into spherical coordinates on a spherical coordinate system using a second coordinate conversion formula, and the spherical coordinates are converted into rectangular coordinates on a spherical coordinate system using a third coordinate conversion formula;

[0014] The rectangular coordinates are converted into image coordinates in a spherical image coordinate system using a fourth coordinate conversion formula, and a spherical fisheye image is obtained according to the image coordinates.

[0015] Furthermore, determining the position and attitude according to the spherical fisheye image includes:

[0016] Performing distortion calibration on the cylindrical fisheye images of the front view, the rear view, the left view, and the right view, respectively, to obtain calibrated spherical fisheye images;

[0017] Performing aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view.

[0018] Furthermore, the cylindrical fisheye images of the front view, the rear view, the left view, and the right view are calibrated for distortion respectively to obtain calibrated spherical fisheye images, including:

[0019] The KB model is used to perform distortion calibration on the cylindrical fisheye images of the front view, the rear view, the left view, and the right view, respectively, to obtain calibrated spherical fisheye images.

[0020] Furthermore, the calibrated spherical fisheye image is subjected to aerial triangulation to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view, including:

[0021] The ORB-SLAM algorithm is used to perform aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view.

[0022] Furthermore, before positioning and determining the posture according to the position and posture, the method includes:

[0023] Determine whether the adjacent heading angle deviation exceeds the set threshold.

[0024] Furthermore, positioning and determining the posture according to the position and posture include:

[0025] When the adjacent heading angle offsets do not exceed a set threshold, performing adjustment processing on the aerial triangulation position and attitude of the front view, and the aerial triangulation position and attitude of the rear view, and obtaining adjustment processing results as the position and attitude for positioning and attitude determination;

[0026] When the adjacent heading angle offset exceeds a set threshold, the aerial triangulation position and attitude of the left view, as well as the aerial triangulation position and attitude of the right view are adjusted to obtain the adjustment results as the position and attitude for positioning and attitude determination.

[0027] The present invention also provides an indoor positioning and attitude determination device based on panoramic video, comprising a data acquisition module, a data processing module and a positioning and attitude determination module;

[0028] The data acquisition module is used to acquire a panoramic image and divide the panoramic image into cylindrical fisheye images according to a set rule;

[0029] The data processing module is used to convert the cylindrical fisheye image into a spherical fisheye image, and determine the position and posture according to the spherical fisheye image;

[0030] The positioning and posture determination module is used to perform positioning and posture determination according to the position and posture.

[0031] The present invention also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the indoor positioning and attitude determination method based on panoramic video as described in any of the above technical solutions is implemented.

[0032] The beneficial effect of adopting the above embodiment is: the indoor positioning and attitude determination method based on panoramic video provided by the present invention obtains a panoramic image, obtains a cylindrical fisheye image based on the panoramic image, converts the cylindrical fisheye image into a spherical fisheye image, uses the spherical fisheye image to determine the position and attitude, and performs positioning and attitude determination based on the position and attitude, thereby realizing accurate indoor navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of an application scenario of the indoor positioning and posture determination device based on panoramic video provided by the present invention;

[0034] Figure 2 A schematic diagram of a flow chart of an embodiment of an indoor positioning and posture determination method based on panoramic video provided by the present invention;

[0035] Figure 3 A schematic diagram of a panoramic camera provided in an embodiment of the present invention;

[0036] Figure 4 A schematic diagram of a cylindrical fisheye image provided in an embodiment of the present invention;

[0037] Figure 5 A schematic diagram of the front, back, left, and right viewing angle division rules provided in an embodiment of the present invention;

[0038] Figure 6 A schematic diagram of coordinate conversion between a cylindrical projection coordinate system and a spherical projection coordinate system provided in an embodiment of the present invention;

[0039] Figure 7 A schematic diagram of a spherical coordinate system provided in an embodiment of the present invention;

[0040] Figure 8 A schematic diagram of coordinate conversion between a spherical coordinate system and a spherical image coordinate system provided in an embodiment of the present invention;

[0041] Figure 9 A schematic diagram of a spherical fisheye image provided in an embodiment of the present invention;

[0042] Figure 10 This is a structural block diagram of an embodiment of an indoor positioning and posture determination device based on panoramic video provided by the present invention;

[0043] Figure 11 This is a structural block diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0044] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0045] The present invention provides a method, device and electronic equipment for indoor positioning and posture determination based on panoramic video, which are described in detail below.

[0046] Figure 1 This is a schematic diagram of an application scenario of the indoor positioning and posture determination device based on panoramic video provided by the present invention. The system may include a server 100, in which the indoor positioning and posture determination device based on panoramic video is integrated, such as Figure 1 Server in.

[0047] In the embodiment of the present invention, the server 100 is mainly used for:

[0048] Acquire a panoramic image, and divide the panoramic image into cylindrical fisheye images according to a set rule;

[0049] Converting the cylindrical fisheye image into a spherical fisheye image, and determining the position and posture according to the spherical fisheye image;

[0050] Positioning and posture determination are performed according to the position and posture.

[0051] In the embodiments of the present invention, the server 100 may be a standalone server, or a server network or server cluster composed of servers. For example, the server 100 described in the embodiments of the present invention includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. A cloud server is composed of a large number of computers or network servers based on cloud computing.

[0052] It is understood that the terminal 200 used in the embodiments of the present invention can be a device that includes both receiving and transmitting hardware, that is, a device that has receiving and transmitting hardware capable of performing two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices that have a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the terminal 200 can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, an embedded device, etc. This embodiment does not limit the type of terminal 200.

[0053] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present invention and does not constitute a limitation on the application scenario of the present invention. Other application environments may also include Figure 1 More or fewer terminals as shown in , e.g. Figure 1 Only two terminals are shown in the figure. It can be understood that the indoor positioning and posture determination device based on panoramic video can also include one or more other terminals, which are not limited here.

[0054] In addition, refer to Figure 1 As shown, the indoor positioning and attitude determination device based on panoramic video can also include a memory 300 for storing data, such as panoramic images, cylindrical projection coordinates and other data.

[0055] It should be noted that Figure 1 The scene diagram of the indoor positioning and attitude determination device based on panoramic video shown is only an example. The indoor positioning and attitude determination device based on panoramic video and the scene described in the embodiment of the present invention are for the purpose of more clearly illustrating the technical solution of the embodiment of the present invention, and do not constitute a limitation on the technical solution provided by the embodiment of the present invention. Ordinary technicians in this field can know that with the evolution of the indoor positioning and attitude determination device based on panoramic video and the emergence of new business scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.

[0056] The embodiment of the present invention provides an indoor positioning and posture determination method based on panoramic video, and its flow chart is as follows: Figure 2 As shown, the indoor positioning and posture determination method based on panoramic video includes:

[0057] Step S201: Acquire a panoramic image, and divide the panoramic image into cylindrical fisheye images according to a set rule;

[0058] Step S202: converting the cylindrical fisheye image into a spherical fisheye image, and determining the position and posture according to the spherical fisheye image;

[0059] Step S203: Positioning and posture determination are performed according to the position and posture.

[0060] Specifically, a video acquisition device is used to obtain a 360-degree panoramic video of the room. It is easy to understand that the video acquisition device can be a panoramic camera, a mobile phone camera, or other device that can shoot video. The video acquisition device is placed at a height where it can clearly capture a 360-degree panoramic video.

[0061] In a specific embodiment, a panoramic camera is used to capture indoor 360-degree panoramic video in real time. The schematic diagram of the panoramic camera is as follows: Figure 3 As shown, Figure 3 The panoramic zero direction line is used as a reference, and the panoramic camera is placed on the pedestrian's clothing or hat to ensure that the height is sufficient to clearly capture the 360-degree panoramic video. The panoramic camera is fixed, and the front and rear fisheye lenses of the panoramic camera are consistent with the pedestrian's walking direction. The pedestrian's walking speed is not higher than 2m / s, and the resolution of the collected panoramic video is not lower than 4K, so as to ensure the clarity of the collected 360-degree panoramic video. Each frame of the panoramic image in the panoramic video is screened, and the dwell frames with a similarity of more than 99% between the previous and next frames are removed. The time interval for continuously removing the dwell frames is no more than 0.5 seconds, and a panoramic image without dwell frames is obtained. A panoramic image set is constructed, and a panoramic image is obtained from the panoramic image set.

[0062] It should be noted that eliminating dwell frames can reduce the data set, speed up the data processing process, and make indoor positioning and posture determination real-time.

[0063] As a preferred embodiment, the panoramic image is divided according to a set rule to obtain a cylindrical fisheye image, including:

[0064] The panoramic image is expanded according to a cylinder to obtain a corresponding cylindrical panoramic image, and the cylindrical panoramic image is divided into cylindrical fisheye images of front perspective, rear perspective, left perspective and right perspective according to a set rule.

[0065] In a specific embodiment, a panoramic image is expanded cylindrically to obtain a corresponding cylindrical fisheye image. The schematic diagram of the cylindrical fisheye image is as follows: Figure 4 As shown, the cylindrical fisheye image is divided according to the front, back, left, and right viewing angle division rules. The schematic diagram of the front, back, left, and right viewing angle division rules is as follows: Figure 5 As shown, a cylindrical fisheye image with a left viewing angle of 180 degrees, a cylindrical fisheye image with a front viewing angle of 180 degrees, a cylindrical fisheye image with a right viewing angle of 180 degrees, and a cylindrical fisheye image with a rear viewing angle of 180 degrees are obtained. The cylindrical fisheye image set is constructed by using the cylindrical fisheye images of the front, back, left, and right viewing angles obtained after the above processing of the panoramic image set. The purpose of constructing the cylindrical fisheye image set is to be able to instantly retrieve the required cylindrical fisheye images, thereby speeding up the data processing process.

[0066] As a preferred embodiment, converting the cylindrical fisheye image into a spherical fisheye image includes:

[0067] Construct cylindrical projection coordinate system, spherical projection coordinate system, spherical coordinate system and spherical image coordinate system;

[0068] Taking any point on the cylindrical fisheye image and projecting it onto the cylindrical projection coordinate system to obtain cylindrical projection coordinates, and converting the cylindrical projection coordinates into spherical projection coordinates on the spherical projection coordinate system using a first coordinate conversion formula;

[0069] The spherical projection coordinates are converted into spherical coordinates on a spherical coordinate system using a second coordinate conversion formula, and the spherical coordinates are converted into rectangular coordinates on a spherical coordinate system using a third coordinate conversion formula;

[0070] The rectangular coordinates are converted into image coordinates in a spherical image coordinate system using a fourth coordinate conversion formula, and a spherical fisheye image is obtained according to the image coordinates.

[0071] In a specific embodiment, a schematic diagram of coordinate conversion between a cylindrical projection coordinate system and a spherical projection coordinate system is shown as follows: Figure 6 As shown, Figure 6 Where ioj is the cylindrical projection coordinate system, XOY is the spherical projection coordinate system, and any point P on the cylindrical fisheye image is projected onto the cylindrical projection coordinate system ioj to obtain the cylindrical projection coordinates (i, j), i = 1, ..., r, j = 0, ..., r, r is the radius of the sphere. The first coordinate conversion formula is

[0072]

[0073] Among them, (X, Y) is the spherical projection coordinate;

[0074] Convert the spherical projection coordinates into spherical coordinates on the spherical coordinate system. The schematic diagram of the spherical coordinate system is as follows: Figure 7 As shown, the second coordinate transformation formula is

[0075]

[0076] in, is the spherical coordinate;

[0077] Convert the spherical coordinates to rectangular coordinates, and the third coordinate conversion formula is:

[0078]

[0079] Among them, (x, y, z) are rectangular coordinates, and r is the radius of the sphere;

[0080] Convert the rectangular coordinates in the spherical coordinate system to the image coordinates in the spherical image coordinate system. The coordinate conversion diagram between the spherical coordinate system and the spherical image coordinate system is as follows: Figure 8 As shown, Figure 8 In rowsocols, it is the spherical image coordinate system, and the fourth coordinate transformation formula is

[0081]

[0082] Where (rows, cols) is the image coordinate, height is the height of the spherical fisheye image, width is the width of the spherical fisheye image, and r is the radius of the sphere;

[0083] According to the image coordinates, the spherical fisheye image is obtained. The schematic diagram of the spherical fisheye image is as follows: Figure 9 shown.

[0084] As a preferred embodiment, determining the position and posture according to the spherical fisheye image includes:

[0085] Performing distortion calibration on the cylindrical fisheye images of the front view, the rear view, the left view, and the right view, respectively, to obtain calibrated spherical fisheye images;

[0086] Performing aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view.

[0087] It should be noted that the use of the ORB-SLAM algorithm to perform aerial triangulation on fisheye images requires spherical fisheye images. By converting cylindrical fisheye images into spherical fisheye images, the algorithm can more accurately calculate the position and attitude, achieving precise indoor navigation.

[0088] As a preferred embodiment, the cylindrical fisheye images of the front view, the rear view, the left view, and the right view are calibrated for distortion respectively to obtain calibrated spherical fisheye images, including:

[0089] The KB model is used to perform distortion calibration on the cylindrical fisheye images of the front view, the rear view, the left view, and the right view, respectively, to obtain calibrated spherical fisheye images.

[0090] It should be noted that the distortion calibration of cylindrical fisheye images can reduce errors, avoid image distortion caused by image processing, and improve the accuracy of positioning and attitude determination.

[0091] As a preferred embodiment, performing aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view includes:

[0092] The ORB-SLAM algorithm is used to perform aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view.

[0093] In a specific embodiment, the spatial three-dimensional position of the front view is and posture The aerotriangulation position of the rear view is and posture The left perspective's three-dimensional position is and posture The right perspective's three-dimensional position is and posture

[0094] As a preferred embodiment, before positioning and determining the posture according to the position and posture, the method includes:

[0095] Determine whether the adjacent heading angle deviation exceeds the set threshold.

[0096] In a specific embodiment, the threshold is set to 5 degrees. Setting this threshold is consistent with real life and can perform indoor positioning and posture determination more accurately.

[0097] As a preferred embodiment, positioning and determining the posture according to the position and posture include:

[0098] When the adjacent heading angle offsets do not exceed a set threshold, performing adjustment processing on the aerial triangulation position and attitude of the front view, and the aerial triangulation position and attitude of the rear view, and obtaining adjustment processing results as the position and attitude for positioning and attitude determination;

[0099] When the adjacent heading angle offset exceeds a set threshold, the aerial triangulation position and attitude of the left view, as well as the aerial triangulation position and attitude of the right view are adjusted to obtain the adjustment results as the position and attitude for positioning and attitude determination.

[0100] In a specific embodiment, when the adjacent heading angle changes of the pedestrian's turns do not exceed 5 degrees, the adjustment results of the aerial triangulation positions and attitudes of the front and rear view angles are used as the position and attitude outputs for positioning and posing. The steps for adjusting the aerial triangulation positions and attitudes of the front and rear view angles are as follows:

[0101]

[0102] Among them, (x′ i , y′ i , z′ i ) is the position in the adjustment processing result of the front and back perspectives, (yaw′ i , pitch′ i , row′ i ) is the posture in the adjustment result of the front and back perspectives;

[0103] When the adjacent heading angles of the pedestrian turn change by more than 5 degrees, the adjustment results of the aerial triangulation positions and attitudes of the left and right views are taken as the position and attitude outputs for positioning and attitude determination. The steps for adjusting the aerial triangulation positions and attitudes of the left and right views are as follows:

[0104]

[0105] Among them, (x i , y″ i , z″ i ) is the position in the adjustment processing result of the left and right perspectives, (yaw″ i , pitch″ i ,row″ i ) is the posture of the adjustment result of the left and right perspectives.

[0106] The embodiment of the present invention provides an indoor positioning and posture determination device based on panoramic video, and its structural block diagram is as follows: Figure 10 As shown, the indoor positioning and attitude determination device based on panoramic video includes a data acquisition module 1001, a data processing module 1002 and a positioning and attitude determination module 1003;

[0107] The data acquisition module 1001 is used to acquire a panoramic image and divide the panoramic image into cylindrical fisheye images according to a set rule;

[0108] The data processing module 1002 is configured to convert the cylindrical fisheye image into a spherical fisheye image, and determine the position and posture based on the spherical fisheye image;

[0109] The positioning and posture determination module 1003 is used to perform positioning and posture determination according to the position and posture.

[0110] like Figure 11 As shown, the present invention also provides an electronic device for the indoor positioning and posture determination method based on panoramic video. The electronic device can be a computing device such as a mobile terminal, a desktop computer, a notebook, a palmtop computer, and a server. The electronic device includes a processor 10, a memory 20, and a display 30.

[0111] In some embodiments, the memory 20 may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory 20 may also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Furthermore, the memory 20 may also include both an internal storage unit of the computer device and an external storage device. The memory 20 is used to store application software and various types of data installed on the computer device, such as program codes installed on the computer device. The memory 20 may also be used to temporarily store data that has been output or is about to be output. In one embodiment, the memory 20 stores an indoor positioning and attitude determination program 40 based on panoramic video, and the indoor positioning and attitude determination program 40 based on panoramic video can be executed by the processor 10, thereby realizing the indoor positioning and attitude determination method based on panoramic video of each embodiment of the present invention.

[0112] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes or process data stored in the memory 20, such as executing an indoor positioning and posture determination program based on panoramic video.

[0113] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the computer device and to display a visual user interface. The components 10-30 of the computer device communicate with each other via a system bus.

[0114] In one embodiment, when the processor 10 executes the panoramic video-based indoor positioning and posture determination program 40 in the memory 20, the following steps are implemented:

[0115] Acquire a panoramic image, and divide the panoramic image into cylindrical fisheye images according to a set rule;

[0116] Converting the cylindrical fisheye image into a spherical fisheye image, and determining the position and posture according to the spherical fisheye image;

[0117] Positioning and posture determination are performed according to the position and posture.

[0118] The present invention discloses a method, device and electronic equipment for indoor positioning and attitude determination based on panoramic video. The method acquires a panoramic image, obtains a cylindrical fisheye image based on the panoramic image, converts the cylindrical fisheye image into a spherical fisheye image, uses the spherical fisheye image to determine the position and attitude, and performs positioning and attitude determination based on the position and attitude, thereby achieving precise indoor navigation.

[0119] This technical solution collects 360-degree panoramic video data in real time and processes the panoramic images to accurately calculate the position and posture data, thereby improving the autonomy and reliability of pedestrian indoor navigation calculations. It is also a new pedestrian indoor navigation method.

[0120] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0121] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for indoor positioning and posture determination based on panoramic video, characterized in that: include: Obtaining a panoramic image captured by a panoramic camera, and dividing the panoramic image according to a set rule to obtain a cylindrical fisheye image; Converting the cylindrical fisheye image into a spherical fisheye image, and determining the position and posture according to the spherical fisheye image; Performing positioning and posture determination according to the position and posture; The step of converting the cylindrical fisheye image into a spherical fisheye image includes: Construct cylindrical projection coordinate system, spherical projection coordinate system, spherical coordinate system and spherical image coordinate system; Taking any point on the cylindrical fisheye image and projecting it onto the cylindrical projection coordinate system to obtain cylindrical projection coordinates, and converting the cylindrical projection coordinates into spherical projection coordinates on the spherical projection coordinate system using a first coordinate conversion formula; The spherical projection coordinates are converted into spherical coordinates on a spherical coordinate system using a second coordinate conversion formula, and the spherical coordinates are converted into rectangular coordinates on a spherical coordinate system using a third coordinate conversion formula; Converting the rectangular coordinates into image coordinates in a spherical image coordinate system using a fourth coordinate conversion formula, and obtaining a spherical fisheye image according to the image coordinates; The step of dividing the panoramic image according to a set rule to obtain a cylindrical fisheye image includes: Expanding the panoramic image according to a cylinder to obtain a corresponding cylindrical panoramic image, and dividing the cylindrical panoramic image into cylindrical fisheye images of front perspective, rear perspective, left perspective, and right perspective according to a set rule; Before positioning and determining the posture according to the position and posture, the following steps are included: Determine whether the adjacent heading angle deviation exceeds the set threshold; Determining the position and posture according to the spherical fisheye image includes: Performing distortion calibration on the cylindrical fisheye images of the front view, the rear view, the left view, and the right view, respectively, to obtain calibrated spherical fisheye images; Performing aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view; The positioning and posture determination according to the position and posture includes: When the adjacent heading angle offsets do not exceed a set threshold, the aerial triangulation position and attitude of the front view and the aerial triangulation position and attitude of the rear view are adjusted to obtain adjustment results as the position and attitude for positioning and attitude determination; When the adjacent heading angle offset exceeds a set threshold, the aerial triangulation position and attitude of the left view and the aerial triangulation position and attitude of the right view are adjusted to obtain the adjustment results as the position and attitude for positioning and attitude determination.

2. The indoor positioning and posture determination method based on panoramic video according to claim 1, characterized in that: The cylindrical fisheye images of the front view, the rear view, the left view, and the right view are calibrated for distortion to obtain calibrated spherical fisheye images, including: The KB model is used to perform distortion calibration on the cylindrical fisheye images of the front view, the rear view, the left view, and the right view, respectively, to obtain calibrated spherical fisheye images.

3. The indoor positioning and posture determination method based on panoramic video according to claim 1, characterized in that: Performing aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view, including: The ORB-SLAM algorithm is used to perform aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view.

4. An indoor positioning and posture determination device based on panoramic video, characterized in that: It includes data acquisition module, data processing module and positioning and attitude determination module; The data acquisition module is used to acquire a panoramic image captured by a panoramic camera, and divide the panoramic image into cylindrical fisheye images according to a set rule; The data processing module is used to convert the cylindrical fisheye image into a spherical fisheye image, and determine the position and posture according to the spherical fisheye image; The positioning and posture determination module is used to perform positioning and posture determination according to the position and posture; The step of converting the cylindrical fisheye image into a spherical fisheye image includes: Construct cylindrical projection coordinate system, spherical projection coordinate system, spherical coordinate system and spherical image coordinate system; Taking any point on the cylindrical fisheye image and projecting it onto the cylindrical projection coordinate system to obtain cylindrical projection coordinates, and converting the cylindrical projection coordinates into spherical projection coordinates on the spherical projection coordinate system using a first coordinate conversion formula; The spherical projection coordinates are converted into spherical coordinates on a spherical coordinate system using a second coordinate conversion formula, and the spherical coordinates are converted into rectangular coordinates on a spherical coordinate system using a third coordinate conversion formula; Converting the rectangular coordinates into image coordinates in a spherical image coordinate system using a fourth coordinate conversion formula, and obtaining a spherical fisheye image according to the image coordinates; The step of dividing the panoramic image according to a set rule to obtain a cylindrical fisheye image includes: Expanding the panoramic image according to a cylinder to obtain a corresponding cylindrical panoramic image, and dividing the cylindrical panoramic image into cylindrical fisheye images of front perspective, rear perspective, left perspective, and right perspective according to a set rule; Before positioning and determining the posture according to the position and posture, the following steps are included: Determine whether the adjacent heading angle deviation exceeds the set threshold; Determining the position and posture according to the spherical fisheye image includes: Performing distortion calibration on the cylindrical fisheye images of the front view, the rear view, the left view, and the right view, respectively, to obtain calibrated spherical fisheye images; Performing aerial triangulation on the calibrated spherical fisheye image to obtain the corresponding aerial triangulation positions and postures of the front view, rear view, left view, and right view; The positioning and posture determination according to the position and posture includes: When the adjacent heading angle offsets do not exceed a set threshold, the aerial triangulation position and attitude of the front view and the aerial triangulation position and attitude of the rear view are adjusted to obtain adjustment results as the position and attitude for positioning and attitude determination; When the adjacent heading angle offset exceeds a set threshold, the aerial triangulation position and attitude of the left view and the aerial triangulation position and attitude of the right view are adjusted to obtain the adjustment results as the position and attitude for positioning and attitude determination.

5. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the indoor positioning and attitude determination method based on panoramic video according to any one of claims 1 to 3 is implemented.

Images