Coordinate system conversion relationship determination method and apparatus, device, and storage medium

Abstract

Embodiments of the present disclosure disclose a coordinate system conversion relationship determination method and device, equipment and a storage medium. The method comprises: acquiring an indoor picture and collecting terminal sensor information when collecting the indoor picture; determining pose information of the terminal when collecting the indoor picture based on the indoor picture and an indoor visual map coordinate system; determining a pose alignment relationship between coordinate systems based on the pose information, accelerometer information and magnetometer information of the indoor picture; determining a translation alignment relationship and a scale alignment relationship between coordinate systems based on position information of the indoor picture and network positioning information; and determining a pose alignment relationship and a translation alignment relationship between two coordinate systems based on a scale alignment relationship corresponding to two or more indoor pictures, and the pose alignment relationship and the translation alignment relationship corresponding to the indoor pictures. The information is acquired by a sensor, and the determination of the conversion relationship between the coordinate systems does not require human participation, thereby reducing data costs and labor costs.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of geographic information, and in particular to a coordinate system conversion relationship determination method and device, equipment and a storage medium. BACKGROUND

[0002] With the increasing demand for travel convenience, users have more and more demand for indoor navigation. For example, in the online car-hailing scenario, after a user takes a car indoors, the user needs to go to the pickup point to meet the online car driver. When the user takes a car in a large building such as an airport terminal, a train waiting room or a shopping mall, the user needs to go from indoors to the pickup point to meet the driver. Since the indoor GPS signal is poor, it is difficult to obtain the user's location through GPS positioning indoors. If the user's location cannot be obtained, the user cannot be guided to the pickup point through navigation. To solve this problem, the user usually needs to take pictures of the indoor surroundings at the current location, determine the user's location by using visual positioning technology, and then plan a navigation path for the user to go to the pickup point.

[0003] The existing visual positioning technology relies on an indoor visual map. The indoor visual map is established based on an indoor visual map coordinate system. Therefore, the user's location obtained based on the visual positioning technology is also a location in the indoor visual map coordinate system. Since the indoor map data used for path planning by navigation is made based on a geographic coordinate system, in order to plan an accurate navigation path, the indoor visual map coordinate system needs to be aligned with the geographic coordinate system, and then the user's location in the indoor visual map coordinate system needs to be converted into a location in the geographic coordinate system. However, in the prior art, the conversion relationship between the indoor visual map coordinate system and the geographic coordinate system is usually manually implemented by humans, and needs to rely on a large amount of data, such as an indoor two-dimensional plan, manually marked control points, etc., resulting in high labor costs and data costs.

[0004] Therefore, it is necessary to propose a solution to the above technical problems, so as to reduce the labor costs and data costs. SUMMARY

[0005] The present disclosure provides a coordinate system conversion relationship determination method, device, equipment and storage medium.

[0006] In a first aspect, a coordinate system conversion relationship determination method is provided in the embodiments of the present disclosure, which comprises:

[0007] An indoor picture and sensor information of a terminal collecting the indoor picture are acquired, the sensor information comprising network positioning information, accelerometer information and magnetometer information;

[0008] determine, in the indoor visual map coordinate system, pose information of the collection terminal when the indoor picture is collected based on the indoor picture and a pre-generated indoor visual map;

[0009] determine a pose alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and a geographic coordinate system based on the pose information in the pose information of the indoor picture, the accelerometer information and the magnetometer information;

[0010] determine a translation alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system based on the position information of the indoor picture in the indoor visual map coordinate system and the network positioning information corresponding to the indoor picture;

[0011] determine a scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system based on the position information in the pose information of two or more indoor pictures and the network positioning information corresponding to the two or more indoor pictures;

[0012] determine a pose alignment relationship and a translation alignment relationship for aligning the indoor visual map coordinate system and the geographic coordinate system based on one scale alignment relationship corresponding to two or more indoor pictures and the pose alignment relationship and the translation alignment relationship corresponding to the indoor pictures.

[0013] Further, the determination of the pose alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system based on the pose information in the pose information of the indoor picture, the accelerometer information and the magnetometer information comprises:

[0014] determine a rotation alignment relationship between the z-axis of the indoor visual map coordinate system corresponding to the indoor picture and the z-axis of the geographic coordinate system based on the pose information in the pose information of the indoor picture and the accelerometer information;

[0015] determine a rotation alignment relationship between the x-axis and the y-axis of the indoor visual map coordinate system corresponding to the indoor picture and the x-axis and the y-axis of the geographic coordinate system based on the pose information in the pose information of the indoor picture and the magnetometer information.

[0016] Further, the determination of the rotation alignment relationship between the z-axis of the indoor visual map coordinate system corresponding to the indoor picture and the z-axis of the geographic coordinate system based on the pose information in the pose information of the indoor picture and the accelerometer information comprises:

[0017] determine a z-axis rotation alignment relationship of the collection terminal in the geographic coordinate system when the indoor picture is collected based on the accelerometer information of the indoor picture;

[0018] determining, based on the z-axis rotation alignment relationship in the pose information in the indoor visual map coordinate system of the indoor picture and the z-axis rotation alignment relationship of the collection terminal in the geographic coordinate system, a z-axis rotation alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system.

[0019] Further, determining, based on the accelerometer information of the indoor picture, a z-axis rotation alignment relationship of the collection terminal in the geographic coordinate system when the indoor picture is collected, comprises:

[0020] determining, based on the accelerometer information of the indoor picture, a first direction vector of gravity in a device coordinate system of the collection terminal;

[0021] determining, based on the first direction vector and a second direction vector of gravity in the geographic coordinate system, a z-axis rotation alignment relationship of the collection terminal relative to the geographic coordinate system.

[0022] Further, determining, based on the attitude information in the pose information of the indoor picture and the magnetometer information, a rotation alignment relationship between an x-axis and a y-axis of the indoor visual map coordinate system corresponding to the indoor picture and an x-axis and a y-axis of the geographic coordinate system, respectively, comprises:

[0023] determining, based on the magnetometer information of the indoor picture, an x-axis and y-axis rotation alignment relationship of the collection terminal relative to the geographic coordinate system when the indoor picture is collected;

[0024] determining, based on the x-axis and y-axis rotation alignment relationship in the pose information in the indoor visual map coordinate system of the indoor picture and the x-axis and y-axis rotation alignment relationship of the collection terminal relative to the geographic coordinate system, an x-axis and y-axis rotation alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system, respectively.

[0025] Further, determining, based on position information in the pose information of two or more indoor pictures and the network positioning information corresponding to the two or more indoor pictures, a scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system, comprises:

[0026] determining, based on the plurality of network positioning information of the plurality of indoor pictures, a plurality of first three-dimensional position coordinates of the collection terminal in the geographic coordinate system when the plurality of indoor pictures are collected;

[0027] determining, based on the plurality of first three-dimensional position coordinates and a plurality of second three-dimensional position coordinates of the plurality of indoor pictures in the indoor visual map coordinate system, scale information between the indoor visual map coordinate system and the geographic coordinate system.

[0028] In a second aspect, an indoor positioning method is provided in embodiments of the present application, which comprises:

[0029] acquiring pictures collected indoors by a user and uploaded by the user;

[0030] determining position information of the user in an indoor visual map coordinate system based on the pictures and the indoor visual map;

[0031] determining position information of the user in a geographic coordinate system based on a conversion relationship between the indoor visual map coordinate system and the geographic coordinate system, wherein the conversion relationship between the indoor visual map coordinate system and the geographic coordinate system is determined based on the method of the first aspect.

[0032] In a third aspect, an indoor path planning method is provided in embodiments of the present application, which comprises:

[0033] determining position information of a user in a geographic coordinate system based on the method of the second aspect;

[0034] planning a navigation path of the user from a position corresponding to the position information to a terminal point based on the position information.

[0035] In a fourth aspect, a coordinate system conversion relationship determining apparatus is provided in embodiments of the present application, which comprises:

[0036] an acquisition module configured to acquire indoor pictures and sensor information of a collection terminal when the indoor pictures are collected, wherein the sensor information comprises network positioning information, accelerometer information and magnetometer information, the indoor pictures and the sensor information correspond to each other in one-to-one manner;

[0037] a first determination module configured to determine, based on the indoor pictures and a pre-generated indoor visual map, pose information of the collection terminal when the indoor pictures are collected in an indoor visual map coordinate system, wherein the indoor pictures and the pose information correspond to each other in one-to-one manner;

[0038] a second determination module configured to determine, based on attitude information in the pose information, the accelerometer information and the magnetometer information, an attitude alignment relationship between the indoor visual map coordinate system corresponding to the indoor pictures and a geographic coordinate system;

[0039] a third determination module configured to determine, based on position information of the indoor pictures in the indoor visual map coordinate system and the network positioning information corresponding to the indoor pictures, a translation alignment relationship between the indoor visual map coordinate system corresponding to the indoor pictures and the geographic coordinate system;

[0040] The fourth determining module is configured to determine a scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system based on position information in pose information of the two or more indoor pictures and the network positioning information corresponding to the two or more indoor pictures.

[0041] The fifth determining module is configured to determine a pose alignment relationship and a translation alignment relationship for aligning the indoor visual map coordinate system and the geographic coordinate system based on one scale alignment relationship corresponding to the two or more indoor pictures and the pose alignment relationship and the translation alignment relationship corresponding to the indoor pictures.

[0042] The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above.

[0043] In one possible design, the apparatus includes a memory and a processor. The memory is configured to store one or more computer instructions for supporting the apparatus to perform the corresponding method described above. The processor is configured to execute the computer instructions stored in the memory. The apparatus can further include a communication interface configured to enable the apparatus to communicate with other devices or communication networks.

[0044] In a fifth aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the method in any of the aspects described above.

[0045] In a sixth aspect, a computer-readable storage medium is provided, which is configured to store computer instructions for the apparatus described above. The computer instructions are executed by a processor to implement the method in any of the aspects described above.

[0046] In a seventh aspect, a computer program product is provided, which includes computer instructions. The computer instructions are executed by a processor to implement the method in any of the aspects described above.

[0047] The technical solutions provided in the embodiments of the present disclosure can have the following beneficial effects:

[0048] The embodiment of the present disclosure provides a coordinate system conversion relationship determination method. The method comprises the following steps: acquiring an indoor picture and network positioning information, accelerometer information and magnetometer information of a terminal when the indoor picture is collected; determining, in an indoor visual map coordinate system, pose information of the terminal when the indoor picture is collected based on the indoor picture and a pre-generated indoor visual map; determining, based on attitude information in the pose information of the indoor picture, the accelerometer information and the magnetometer information, an attitude alignment relationship of the indoor visual map coordinate system corresponding to the indoor picture with a geographic coordinate system; determining, based on position information of the indoor picture in the indoor visual map coordinate system and network positioning information corresponding to the indoor picture, a translation alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system; and determining, based on position information in the pose information of more than two indoor pictures and network positioning information corresponding to the more than two indoor pictures, a scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system. Finally, based on one scale alignment relationship corresponding to the more than two indoor pictures and the attitude alignment relationship and the translation alignment relationship corresponding to the indoor picture, an attitude alignment relationship and a translation alignment relationship for aligning the indoor visual map coordinate system with the geographic coordinate system are determined. Since the network positioning information, the accelerometer information and the magnetometer information are acquired by the sensors of the terminal when the indoor picture is collected, the map coordinate system can be aligned with the geographic coordinate system based on the data, which reduces the manual participation link and reduces the data cost and the labor cost.

[0049] It should be understood that the general description above and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0050] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of the non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0051] Figure 1 A flow chart of a coordinate system conversion relationship determination method according to an embodiment of the present disclosure is shown.

[0052] Figure 2 A path navigation schematic diagram according to an embodiment of the present disclosure is shown.

[0053] Figure 3 A structure block diagram of a coordinate system conversion relationship determination device according to an embodiment of the present disclosure is shown.

[0054] Figure 4 A structure diagram of an electronic device suitable for implementing the coordinate system conversion relationship determination method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of the exemplary embodiments have been omitted from the drawings.

[0056] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and do not preclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0057] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0058] The details of the embodiments of this disclosure are described in detail below through specific examples.

[0059] Figure 1 A flowchart illustrating a method for determining coordinate system transformation relationships according to an embodiment of this disclosure is shown. Figure 1 As shown, the method for determining the coordinate system transformation relationship includes the following steps:

[0060] In step S101, indoor images are acquired and sensor information of the acquisition terminal is collected when acquiring the indoor images. The sensor information includes: network positioning information, accelerometer information and magnetometer information.

[0061] In step S102, based on the indoor image and the pre-generated indoor visual map, the pose information of the acquisition terminal when acquiring the indoor image is determined in the indoor visual map coordinate system.

[0062] In step S103, based on the pose information in the pose information of the indoor image, the accelerometer information, and the magnetometer information, the pose alignment relationship between the indoor visual map coordinate system and the geographic coordinate system corresponding to the indoor image is determined.

[0063] In step S104, based on the position information of the indoor image in the indoor visual map coordinate system and the network positioning information corresponding to the indoor image, the translational alignment relationship between the indoor visual map coordinate system corresponding to the indoor image and the geographic coordinate system is determined.

[0064] In step S105, based on the position information in the pose information of two or more indoor images and the network positioning information corresponding to the two or more indoor images, the scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system is determined.

[0065] In step S106, based on the one scale alignment relationship corresponding to the two or more indoor pictures and the pose alignment relationship, the translation alignment relationship corresponding to the indoor pictures, the pose alignment relationship, the translation alignment relationship for aligning the indoor visual map coordinate system and the geographic coordinate system are determined.

[0066] In this embodiment, the indoor pictures taken by the user can determine the current position of the user in the indoor, which is the position of the user in the indoor visual map coordinate system. The visual positioning technology relies on the indoor visual map, which is established based on the indoor visual map coordinate system. In order to convert the position of the user in the indoor visual map coordinate system into a geographic position, it is necessary to align the indoor visual map coordinate system with the geographic coordinate system, that is, to realize the conversion between the indoor visual map coordinate system and the geographic coordinate system. However, the conversion relationship between the indoor visual map coordinate system and the geographic coordinate system in the prior art is usually realized manually by artificial, and relies on a large amount of data, resulting in high labor cost and data cost. Therefore, the embodiment of the present disclosure proposes a coordinate system conversion relationship determination method.

[0067] The coordinate system conversion relationship determination method provided by the embodiment of the present disclosure can be implemented by a server, can be executed by a physical device capable of providing a server function, or can be executed by a component (such as a chip) configured in the physical device, or can be executed by a module capable of implementing part or all of the server function, etc. The present disclosure does not limit this. In order to facilitate understanding, the present disclosure will be described below as an example of the server to describe the method provided by the present disclosure.

[0068] Step S101

[0069] In the embodiment of the present disclosure, the indoor picture can be a picture collected from the indoor by using a collection terminal. The indoor can refer to the interior of a building, which can include but is not limited to the interior of an underground parking lot, the interior of a shopping mall, etc.

[0070] In some embodiments, the indoor picture needs to cover at least the area covered by the indoor visual map. In order to save costs, the pictures used to construct the indoor visual map can also be collected from the indoor to collect the indoor pictures used for coordinate alignment. The sensor information corresponding to the collection terminal can also be collected when collecting the indoor pictures.

[0071] The collection terminal can be a mobile phone, a smart television, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in a smart city, a wireless terminal in a smart home, and the like. The embodiments of the present disclosure do not limit the specific technology and specific device form adopted by the collection terminal. In some embodiments, the collection terminal can be a device deployed with an image sensing unit, an inertial measurement unit (IMU), a magnetometer, and a network positioning sensing unit.

[0072] The collection terminal can collect multiple indoor pictures when collecting indoor pictures.

[0073] In some embodiments, the posture of the collection terminal remains in a stationary state when the collection terminal collects indoor pictures. The stationary state means that the posture of the collection terminal remains unchanged in the initial state during the collection of an indoor picture. It can be understood that the postures of the collection terminal can be different when the collection terminal collects different indoor pictures.

[0074] The collection terminal can collect sensor information of the collection terminal when collecting each indoor picture. The sensor information can include network positioning information, accelerometer information, and magnetometer information. The network positioning information can be position information obtained based on a network point such as an indoor wifi transmitter through a network positioning service. The position information at least includes longitude and latitude. The accelerometer information can include gravity information, which can be a gravity value. The magnetometer information can include the geographical orientation of the collection terminal, such as the angle between the y-axis of the coordinate system of the collection terminal and the true north direction.

[0075] In some embodiments, the indoor pictures correspond to the sensor information one by one, that is, each indoor picture corresponds to one network positioning information, accelerometer information, and magnetometer information.

[0076] The server can obtain multiple indoor pictures and sensor information of the collection terminal when collecting each indoor picture.

[0077] Step S102

[0078] In the embodiments of the present disclosure, the indoor visual map is a map constructed according to visual data of indoor pictures. The indoor visual map is established based on an indoor visual map coordinate system. The indoor visual map coordinate system is different from a geographic coordinate system, and can be set based on actual needs when the map is constructed. For example, the coordinate system of a terminal used to collect the first indoor picture when the indoor visual map is constructed can be set as the indoor visual map coordinate system. Taking a mobile phone as an example, the coordinate system of the mobile phone can be established in the following manner: taking the center point of the screen of the mobile phone as the origin, the z-axis being perpendicular to the screen of the mobile phone, the x-axis being parallel to the short side of the screen of the mobile phone, and the y-axis being parallel to the long side of the screen of the mobile phone.

[0079] In some embodiments, the trajectory of the indoor visual map coincides with the trajectory of the terminal used to collect the indoor pictures. That is, the position points of the indoor pictures are located on the trajectory of the pictures used to construct the indoor visual map.

[0080] In some embodiments, the indoor visual map can be pre-constructed, and the server can directly obtain the indoor visual map.

[0081] In some embodiments, the indoor visual map coordinate system can be established based on the coordinate system of the terminal used to collect the pictures used to construct the indoor visual map. For example, the indoor visual map coordinate system can be constructed as the coordinate system of the terminal used to collect the first indoor picture among the indoor pictures used to construct the indoor visual map. It should be noted that the terminal used to collect the pictures used to construct the indoor visual map and the terminal used to collect the indoor pictures used to align the indoor visual map coordinate system and the geographic coordinate system can be different or the same.

[0082] The terminal coordinate system is a coordinate system established based on the terminal. When the terminal is rotated or translated, the terminal coordinate system also performs corresponding rotation or translation. That is, the terminal coordinate system can be a local coordinate established based on the terminal.

[0083] In some embodiments, the origin of the terminal coordinate system can be the center of the screen of the terminal, the long side of the screen of the terminal can be parallel to the y-axis of the terminal coordinate system, the short side of the screen of the terminal can be parallel to the x-axis of the terminal coordinate system, and the screen of the terminal can be perpendicular to the z-axis of the terminal coordinate system.

[0084] It should be understood that the above-mentioned construction manner of the terminal coordinate system is only exemplary, and the present disclosure does not limit the selection manner of the origin, the x-axis, the y-axis and the z-axis of the terminal coordinate system, as long as it is a coordinate system established based on the terminal itself, which is within the protection scope of the present disclosure.

[0085] The pose information of the collection terminal when collecting the indoor picture can be understood as: the pose information of the collection terminal coordinate system relative to the indoor visual map coordinate system when the collection terminal collects the indoor picture. The pose information can include rotation information, scale information, and translation information (also referred to as position information).

[0086] The rotation information can represent a rotation alignment relationship between coordinate axes of the collection terminal coordinate system and the indoor visual map coordinate system. Specifically, the rotation information can represent a rotation alignment relationship between a z-axis of the collection terminal coordinate system and a z-axis of the indoor visual coordinate system, and a rotation alignment relationship between an x-axis and a y-axis of the collection terminal coordinate system and an x-axis and a y-axis of the indoor visual coordinate system, respectively.

[0087] The translation information can represent a translation relationship between the collection terminal coordinate system and the indoor visual map coordinate system. The translation relationship can be a translation distance. For example, assuming that the translation distance is 5 cm, the collection terminal coordinate system can be moved 5 cm in a certain straight line direction from the current position to reach the position of the indoor visual map coordinate system.

[0088] The scale information can represent a scale ratio between the collection terminal coordinate system and the indoor visual map coordinate system. For example, the collection terminal coordinate system represents an actual object of 10 m with a length of 1 cm on a coordinate axis, and the indoor visual map coordinate system represents the actual object of 10 m with a length of 10 cm on a coordinate axis, and the scale ratio is 1:10.

[0089] In some embodiments, the indoor picture and the pose information correspond to each other. Because the pose of the collection terminal is different when the collection terminal collects different indoor pictures, the pose information of the collection terminal coordinate system relative to the indoor visual map coordinate system is also different when different indoor pictures are collected.

[0090] After the server obtains the pre-generated indoor visual map, the plurality of indoor pictures, and the network positioning information, the accelerometer information, and the magnetometer information of the collection terminal when the collection terminal collects each indoor picture, the server can determine, in the indoor visual map coordinate system, the pose information of the collection terminal coordinate system relative to the indoor visual map coordinate system when the collection terminal collects each indoor picture.

[0091] Step S103

[0092] In the embodiments of the present disclosure, the geographic coordinate system can be an East-North-Sky coordinate system. The x-axis of the East-North-Sky coordinate system points east, the y-axis points north, and the z-axis points skyward. The origin of the coordinate system can be different based on different positioning systems.

[0093] The server can determine the posture alignment relationship between the indoor visual map coordinate system and the geographic coordinate system, i.e., the rotation alignment relationship between the coordinate axes of the indoor visual map coordinate system and the coordinate axes of the geographic coordinate system, based on the posture information of the collection terminal when collecting the indoor picture and the accelerometer information and the magnetometer information of the collection terminal when collecting the indoor picture obtained by the server.

[0094] In some embodiments, when determining the posture alignment relationship between the coordinate axes of the indoor visual map coordinate system and the coordinate axes of the geographic coordinate system, the server can determine the posture alignment relationship by the accelerometer information, the magnetometer information of the collection terminal corresponding to each indoor picture in the plurality of indoor pictures collected by the collection terminal, and the posture information of the collection terminal when collecting each indoor picture. In this way, a plurality of posture alignment relationships can be obtained, and the one with the smallest error can be selected as the final posture alignment relationship.

[0095] In other embodiments, the server can also determine the rotation information by the accelerometer information, the magnetometer information of the collection terminal corresponding to any one of the plurality of indoor pictures collected by the collection terminal, and the posture information of the collection terminal when collecting the indoor picture.

[0096] Step S104

[0097] In the embodiments of the present disclosure, the network positioning information corresponding to the indoor picture can be understood as the position information of the collection terminal in the geographic coordinate system when the collection terminal collects the indoor picture.

[0098] The position information in the pose information of the indoor picture can be the translation information of the collection terminal coordinate system relative to the indoor visual map coordinate system when the collection terminal collects the indoor picture.

[0099] For the indoor picture obtained by the server, the server can obtain the translation information of the collection terminal coordinate system relative to the indoor visual map coordinate system through the pose information of the collection terminal when collecting the indoor picture, i.e., can determine the position information of the indoor picture in the indoor visual map coordinate system. Then, the server can determine the translation alignment relationship between the indoor visual map coordinate system and the geographic coordinate system in combination with the position information of the collection terminal in the geographic coordinate system when collecting the indoor picture, i.e., the network positioning information corresponding to the indoor picture.

[0100] Step S105

[0101] In the embodiments of the present disclosure, the position information in the pose information of the indoor picture can be scale information of a coordinate system of the collection terminal relative to a coordinate system of the indoor visual map when the collection terminal collects the indoor picture.

[0102] The server can determine the scale alignment relationship between the coordinate system of the indoor visual map and the geographic coordinate system according to the scale information of the coordinate system of the collection terminal relative to the coordinate system of the indoor visual map when the collection terminal collects the indoor picture, and the position information of the collection terminal in the geographic coordinate system when the collection terminal collects the indoor picture.

[0103] In some embodiments, the scale alignment relationship between the coordinate system of the indoor visual map and the geographic coordinate system can be determined by more than two indoor pictures.

[0104] Step S106

[0105] As introduced above, the server can determine one pose alignment relationship between the coordinate system of the indoor visual map and the geographic coordinate system for more than two indoor pictures by step S103, and can determine two or more pose alignment relationships and two or more translation alignment relationships by step S104 and step S105. The pose alignment relationship with the minimum error can be selected as the final determined pose alignment relationship for aligning the coordinate system of the indoor visual map and the geographic coordinate system from the two or more pose alignment relationships, and the translation alignment relationship with the minimum error can be selected as the final determined translation alignment relationship for aligning the coordinate system of the indoor visual map and the geographic coordinate system from the two or more translation alignment relationships.

[0106] The embodiment of the present disclosure provides a coordinate system conversion relationship determination method. The method comprises the following steps: acquiring an indoor picture and network positioning information, accelerometer information and magnetometer information of a terminal when the indoor picture is collected; determining, in an indoor visual map coordinate system, pose information of the terminal when the indoor picture is collected based on the indoor picture and a pre-generated indoor visual map; determining, based on attitude information in the pose information of the indoor picture, the accelerometer information and the magnetometer information, an attitude alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and a geographic coordinate system; determining, based on position information of the indoor picture in the indoor visual map coordinate system and network positioning information corresponding to the indoor picture, a translation alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system; and determining, based on position information in the pose information of more than two indoor pictures and network positioning information corresponding to the more than two indoor pictures, a scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system. Finally, based on one scale alignment relationship corresponding to the more than two indoor pictures and the attitude alignment relationship and the translation alignment relationship corresponding to the indoor picture, an attitude alignment relationship and a translation alignment relationship for aligning the indoor visual map coordinate system and the geographic coordinate system are determined. Since the network positioning information, the accelerometer information and the magnetometer information are acquired by the sensors of the terminal when the indoor picture is collected, the map coordinate system can be aligned with the geographic coordinate system based on the data, the manual participation link is reduced, and the data cost and the labor cost are reduced.

[0107] In an optional implementation of the embodiment, the step S103, i.e., determining, based on the attitude information in the pose information of the indoor picture, the accelerometer information and the magnetometer information, an attitude alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and a geographic coordinate system, comprises:

[0108] determining, based on the attitude information in the pose information of the indoor picture and the accelerometer information, a rotation alignment relationship between a z-axis of the indoor visual map coordinate system corresponding to the indoor picture and a z-axis of the geographic coordinate system;

[0109] determining, based on the attitude information in the pose information of the indoor picture and the magnetometer information, a rotation alignment relationship between an x-axis and a y-axis of the indoor visual map coordinate system corresponding to the indoor picture and an x-axis and a y-axis of the geographic coordinate system, respectively.

[0110] In the optional implementation, the server can determine the posture alignment relationship between the z-axis of the indoor visual coordinate system and the z-axis of the geographic coordinate system through S102, which can obtain the posture alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the indoor visual coordinate system, the posture alignment relationship between the x-axis of the terminal coordinate system and the x-axis of the indoor visual coordinate system, and the posture alignment relationship between the y-axis of the terminal coordinate system and the y-axis of the indoor visual coordinate system. In addition, the server can also obtain the rotation alignment relationship of the z-axis of the terminal relative to the geographic coordinate system through the accelerometer information, and obtain the rotation alignment relationship of the x-axis and the y-axis of the terminal relative to the geographic coordinate system through the magnetometer information. Therefore, the server can determine the rotation alignment relationship between the z-axis of the indoor visual coordinate system and the z-axis of the geographic coordinate system through the posture alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the indoor visual coordinate system and the rotation alignment relationship of the z-axis of the terminal relative to the geographic coordinate system. The rotation alignment relationship between the x-axis of the indoor visual coordinate system and the x-axis of the geographic coordinate system can also be determined through the posture alignment relationship between the x-axis of the terminal coordinate system and the x-axis of the indoor visual coordinate system and the rotation alignment relationship of the x-axis of the terminal relative to the geographic coordinate system. The rotation alignment relationship between the y-axis of the indoor visual coordinate system and the y-axis of the geographic coordinate system can also be determined through the posture alignment relationship between the y-axis of the terminal coordinate system and the y-axis of the indoor visual coordinate system and the rotation alignment relationship of the y-axis of the terminal relative to the geographic coordinate system.

[0111] In an optional implementation of the embodiment, the step of determining the rotation alignment relationship between the z-axis of the indoor visual coordinate system corresponding to the indoor picture and the z-axis of the geographic coordinate system based on the posture information in the position information of the indoor picture and the accelerometer information of the indoor picture further includes:

[0112] Determining the rotation alignment relationship of the z-axis of the terminal in the geographic coordinate system when collecting the indoor picture based on the accelerometer information of the indoor picture.

[0113] Determining the rotation alignment relationship between the z-axis of the indoor visual coordinate system corresponding to the indoor picture and the z-axis of the geographic coordinate system based on the z-axis rotation alignment relationship in the posture information in the indoor visual coordinate system of the indoor picture and the z-axis rotation alignment relationship of the terminal in the geographic coordinate system.

[0114] In the optional implementation, when the server acquires the acceleration information of the terminal, i.e., the gravity information, when acquiring an indoor picture, the server can determine the rotational alignment relationship of the z-axis of the terminal coordinate system relative to the z-axis of the geographic coordinate system, i.e., the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the geographic coordinate system. Since the server can also determine the rotational alignment relationship of the z-axis of the terminal coordinate system relative to the z-axis of the indoor visual map coordinate system, i.e., the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the indoor visual map coordinate system, when the terminal acquires the indoor picture, the server can determine the rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system according to the two rotational alignment relationships. That is, each indoor picture can correspond to the determination of the rotational alignment relationship between the z-axes of the indoor visual map coordinate system and the geographic coordinate system.

[0115] In some embodiments, the rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system can be as follows:

[0116]

[0117] wherein, is the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the indoor visual map coordinate system, is the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the geographic coordinate system, is the rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system, and T is a transpose operation.

[0118] For ease of understanding, the following is an exemplary description.

[0119] In order to improve the processing efficiency, the final rotational information between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system can be determined by any one of the multiple indoor pictures acquired by the terminal.

[0120] In some embodiments, it is assumed that the multiple indoor pictures acquired by the terminal include indoor picture 1, and the gravity value of the terminal is G1 when acquiring indoor picture 1. Then, the server can determine the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the indoor visual map coordinate system, and also determine the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the geographic coordinate system according to the gravity value G1 when the terminal acquires indoor picture 1. Further, the rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system is determined according to the two rotational alignment relationships.

[0121] In order to improve the accuracy, the final rotation information between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system can also be determined by the multiple indoor pictures collected by the collection terminal.

[0122] In some other embodiments, it is assumed that the collection terminal collects three indoor pictures, indoor picture 1, indoor picture 2 and indoor picture 3 in sequence, and the gravity values of the collection terminal when collecting the three indoor pictures are G1, G2 and G3 respectively. The indoor picture 1 is the first picture collected by the collection terminal.

[0123] For the indoor picture 1, the rotation alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system and the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the geographic coordinate system. Therefore, the server can determine the rotation alignment relationship R z1 between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system according to the gravity value G1.

[0124] For the indoor picture 2, the server can determine the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the geographic coordinate system, and the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the indoor visual map coordinate system according to the gravity value G2, and then determine the rotation alignment relationship R z2 between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system.

[0125] For the indoor picture 3, the server can determine the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the geographic coordinate system, and the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the indoor visual map coordinate system according to the gravity value G3, and then determine the rotation alignment relationship R z3 between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system.

[0126] In order to determine the rotation alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system with the highest accuracy, the rotation alignment relationship with the smallest error can be selected from the rotation alignment relationships R z1 , R z2 and R z3 as the final rotation alignment relationship. For example, the server can determine the final rotation alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system in the following way.

[0127] Specifically, the server can determine the absolute value of the difference between each two rotation alignment relationships R z1 , R z2 and R z3 , and it is assumed that R z1 and R z2The absolute value of the difference is r1, R z2 and R z3 The absolute value of the difference is r2, R z3 and R z1 The absolute value of the difference is r3. Then, the server can determine a target difference absolute value from the plurality of difference absolute values, and determine the final rotation alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system based on the two rotation alignment relationships corresponding to the target difference absolute value, which can be the smallest in the plurality of difference absolute values. For example, r 1＜ r 2＜ r3, r1 is the smallest, and the rotation alignment relationship between the z-axis of the indoor visual map coordinate system corresponding to R z1 and the z-axis of the geographic coordinate system corresponding to R z2 of indoor picture 2 can be taken as the final rotation alignment relationship, or the average of the two rotation alignment relationships between the z-axis of the indoor visual map coordinate system corresponding to R z1 and the z-axis of the geographic coordinate system corresponding to R z2 of indoor picture 2 can also be taken as the final rotation alignment relationship.

[0128] In an optional implementation of the embodiment, the step of determining the z-axis rotation alignment relationship of the collection terminal in the geographic coordinate system based on the accelerometer information of the indoor picture can further include:

[0129] determining a first direction vector of gravity in the device coordinate system of the collection terminal based on the accelerometer information of the indoor picture;

[0130] determining the z-axis rotation alignment relationship of the collection terminal relative to the geographic coordinate system based on the first direction vector and a second direction vector of gravity in the geographic coordinate system.

[0131] In this optional implementation, since the direction of gravity is known, the coordinates in the geographic coordinate system are [0, 0, 9.8], therefore, for any indoor picture, the server can determine the first direction vector of gravity in the collection terminal coordinate system and the second direction vector of gravity in the geographic coordinate system based on the accelerometer information when determining the z-axis rotation alignment relationship of the collection terminal coordinate system relative to the z-axis of the geographic coordinate system, and then determine the z-axis rotation alignment relationship of the collection terminal coordinate system corresponding to the indoor picture relative to the z-axis of the geographic coordinate system according to the first direction vector and the second direction vector. It can be understood that each indoor picture can correspond to a determined z-axis rotation alignment relationship of the collection terminal relative to the geographic coordinate system. It should be noted that the coordinate system of the collection terminal can be referred to as the collection terminal coordinate system in the present disclosure.

[0132] In an optional implementation of the embodiment, the step of determining the rotation alignment relationship between the x-axis and the y-axis of the indoor visual map coordinate system corresponding to the indoor picture and the x-axis and the y-axis of the geographic coordinate system based on the attitude information in the pose information of the indoor picture and the magnetometer information, further comprises:

[0133] determining the rotation alignment relationship between the x-axis and the y-axis of the indoor visual map coordinate system corresponding to the indoor picture and the x-axis and the y-axis of the geographic coordinate system based on the attitude information in the pose information of the indoor picture and the magnetometer information, further comprises:

[0134] determining the rotation alignment relationship between the x-axis and the y-axis of the indoor visual map coordinate system corresponding to the indoor picture and the x-axis and the y-axis of the geographic coordinate system based on the attitude information in the pose information of the indoor picture and the magnetometer information, further comprises:

[0135] In the optional implementation, after the server obtains the magnetometer information of the collection terminal when collecting an indoor picture, that is, the geographic orientation of the collection terminal, the server can determine the rotation alignment relationship between the x-axis and the y-axis of the collection terminal relative to the x-axis and the y-axis of the geographic coordinate system, that is, the rotation alignment relationship between the x-axis of the collection terminal coordinate system and the x-axis of the geographic coordinate system, and the rotation alignment relationship between the y-axis of the collection terminal coordinate system and the y-axis of the geographic coordinate system. The server can also determine the rotation alignment relationship between the x-axis of the collection terminal coordinate system and the x-axis of the indoor visual map coordinate system, and the rotation alignment relationship between the y-axis of the collection terminal coordinate system and the y-axis of the indoor visual map coordinate system when the collection terminal collects the indoor picture. Therefore, the server can determine the rotation alignment relationship between the x-axis of the indoor visual map coordinate system corresponding to the indoor picture and the x-axis of the geographic coordinate system based on the rotation alignment relationship between the x-axis of the collection terminal coordinate system and the x-axis of the geographic coordinate system, and the rotation alignment relationship between the x-axis of the collection terminal coordinate system and the x-axis of the indoor visual map coordinate system. The server can also determine the rotation alignment relationship between the y-axis of the indoor visual map coordinate system corresponding to the indoor picture and the y-axis of the geographic coordinate system based on the rotation alignment relationship between the y-axis of the collection terminal coordinate system and the y-axis of the geographic coordinate system, and the rotation alignment relationship between the y-axis of the collection terminal coordinate system and the y-axis of the indoor visual map coordinate system. That is, each indoor picture can correspond to determine the rotation alignment relationship between the x-axis of the indoor visual map coordinate system and the x-axis of the geographic coordinate system, and the rotation alignment relationship between the y-axis of the indoor visual map coordinate system and the y-axis of the geographic coordinate system.

[0136] In some embodiments, the rotation alignment relationship between the x-axis, y-axis of the indoor visual map coordinate system and the x-axis, y-axis of the geographic coordinate system can be as follows:

[0137]

[0138] wherein, is a rotation alignment relationship between the x-axis or y-axis of the acquisition terminal coordinate system and the x-axis or y-axis of the indoor visual map coordinate system, is a rotation alignment relationship between the x-axis or y-axis of the acquisition terminal coordinate system and the x-axis or y-axis of the geographic coordinate system, is a rotation alignment relationship between the x-axis or y-axis of the indoor visual map coordinate system and the x-axis or y-axis of the geographic coordinate system, and T is a transpose operation.

[0139] Similarly, the server can also determine the rotation alignment relationship between the x-axis and y-axis of the final indoor visual map coordinate system and the x-axis and y-axis of the geographic coordinate system based on any one of the plurality of indoor pictures, or all or part of the plurality of indoor pictures. For details, please refer to the foregoing description, which will not be repeated here.

[0140] In an optional implementation of the embodiment, the step S105, i.e., the step of determining the scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system based on the position information in the pose information of the two or more indoor pictures and the network positioning information corresponding to the two or more indoor pictures, further includes the following steps:

[0141] determining a plurality of first three-dimensional position coordinates of the acquisition terminal in the geographic coordinate system when the plurality of indoor pictures are acquired based on the plurality of network positioning information of the plurality of indoor pictures;

[0142] determining the scale information between the indoor visual map coordinate system and the geographic coordinate system based on the plurality of first three-dimensional position coordinates and a plurality of second three-dimensional position coordinates of the plurality of indoor pictures in the indoor visual map coordinate system.

[0143] In this optional implementation, the server can determine multiple second 3D position coordinates of the acquisition terminal's coordinate system relative to the indoor visual map coordinate system when acquiring multiple indoor images. These multiple second 3D position coordinates can constitute a set of 3D position coordinates. Since the server can also obtain the network positioning information of the acquisition terminal when acquiring each indoor image through network positioning, including latitude and longitude information, it can determine multiple first 3D position coordinates of the acquisition terminal's coordinate system relative to the geographic coordinate system when acquiring multiple indoor images based on the acquired latitude and longitude information. These multiple first 3D position coordinates can constitute another set of 3D position coordinates. Furthermore, based on these two sets of 3D position coordinates, the scale information between the indoor visual map coordinate system and the geographic coordinate system can be determined using a preset iterative closest point (ICP) algorithm. This ICP algorithm can obtain the transformation relationship between the coordinate systems of the two sets of point cloud data by aligning them, including rotation information, translation information, and scale information.

[0144] In this embodiment, by applying the aforementioned ICP algorithm to a set of three-dimensional position coordinates composed of multiple second three-dimensional position coordinates of the acquisition terminal in the indoor visual map coordinate system and another set of three-dimensional position coordinates composed of multiple first three-dimensional position coordinates of the acquisition terminal in the geographic coordinate system, rotation information, translation information, and scale information between the indoor visual map coordinate system and the geographic coordinate system can be obtained. Considering that indoor network positioning information is not accurate enough, and the rotation information obtained using ICP is not accurate enough, this embodiment uses a magnetometer and accelerometer to determine the rotation information between the indoor visual map coordinate system and the geographic coordinate system, while using the ICP algorithm to obtain scale and translation information.

[0145] According to one embodiment of this disclosure, an indoor positioning method is also proposed, the method comprising:

[0146] Acquire images collected and uploaded by users indoors;

[0147] Based on the image and the indoor visual map, determine the user's location information in the indoor visual map coordinate system;

[0148] The user's location information in the geographic coordinate system is determined based on the transformation relationship between the indoor visual map coordinate system and the geographic coordinate system; wherein, the transformation relationship between the indoor visual map coordinate system and the geographic coordinate system is determined based on the above-mentioned coordinate system transformation relationship determination method.

[0149] The indoor positioning method provided by the embodiments of the present disclosure can be implemented by a terminal device. The terminal device is deployed with an indoor visual map and an electronic map, or the terminal device can obtain the indoor visual map and the electronic map from a server. The coordinate system adopted by the electronic map is a geographic coordinate system, and the coordinate system adopted by the indoor visual map is an indoor visual coordinate system.

[0150] In the embodiment, when the user has a positioning requirement, the user can use the terminal device to capture indoor pictures around the location of the user. When the terminal device obtains the indoor pictures captured by the user, the terminal device can determine the location coordinates of the user in the indoor visual coordinate system according to the captured indoor pictures and the indoor visual map. The terminal device can obtain the conversion relationship between the indoor visual coordinate system and the geographic coordinate system from the server, and then determine the location coordinates of the user in the geographic coordinate system according to the conversion relationship. The conversion relationship between the indoor visual coordinate system and the geographic coordinate system can be determined based on the above-mentioned coordinate system conversion relationship determination method, and specific details can be referred to the description of the coordinate system conversion relationship determination method in the foregoing, which will not be described herein again.

[0151] According to an embodiment of the present disclosure, an indoor path planning method is also provided. The method comprises:

[0152] determining the location information of the user in the geographic coordinate system based on the above-mentioned indoor positioning method;

[0153] planning a navigation path for the user from the location corresponding to the location information to a destination based on the location information.

[0154] The indoor path planning method provided by the embodiments of the present disclosure can be implemented by a server. In the embodiment, when the user is taking a taxi or performing other navigation actions in an indoor environment, the user can use the terminal device to capture indoor pictures around the location of the user. When the terminal device obtains the indoor pictures captured by the user, the terminal device can determine the location coordinates of the user in the indoor visual coordinate system according to the captured indoor pictures and the indoor visual map. The terminal device can obtain the conversion relationship between the indoor visual coordinate system and the geographic coordinate system from the server, and then determine the location coordinates of the user in the geographic coordinate system according to the conversion relationship. The user can upload the location coordinates to a navigation server. Based on the location coordinates and the destination location input by the user in a navigation APP, the navigation server can plan a navigation path for the user by using an electronic map, and send the planned navigation path to the terminal device of the user, so that the user can go to the destination according to the navigation path.

[0155] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0156] Figure 2 A schematic diagram of path navigation according to an embodiment of the present disclosure is shown. For example... Figure 2 As shown, suppose a user is trying to hail a taxi inside shopping mall A but cannot find exit C, which is closest to the driver. In this situation, the user can take a picture of the surrounding indoor area using their device. The device can then determine that the user's location is a nail salon based on the indoor visual map, and determine the nail salon's coordinates in the indoor visual map coordinate system. The device can then obtain the conversion relationship between the indoor visual map coordinate system and the geographic coordinate system from the server. Based on this conversion relationship, the device can determine the nail salon's coordinates in the geographic coordinate system. Next, the user inputs the coordinates of exit C into the navigation app on their device. The device then uploads these coordinates to the server. Based on the coordinates of exit C and the nail salon, the server uses an electronic map to plan a navigation route for the user and displays it on the user's screen. The user can then easily find exit C by following this navigation route.

[0157] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein.

[0158] Figure 3 This diagram illustrates a structural block diagram of a coordinate system transformation relationship determination device according to an embodiment of the present disclosure. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 3 As shown, the coordinate system transformation relationship determination device includes:

[0159] The acquisition module 301 is configured to acquire indoor images and acquire sensor information of the terminal when acquiring the indoor images. The sensor information includes: network positioning information, accelerometer information and magnetometer information. The indoor images correspond one-to-one with the sensor information.

[0160] The first determining module 302 is configured to determine the pose information of the acquisition terminal when acquiring the indoor image based on the indoor image and a pre-generated indoor visual map in the indoor visual map coordinate system, wherein the indoor image and the pose information correspond one-to-one.

[0161] The second determining module 303 is configured to determine a pose alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system based on the pose information in the pose information of the indoor picture, the accelerometer information and the magnetometer information.

[0162] The third determining module 304 is configured to determine a translation alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system based on the position information of the indoor picture in the indoor visual map coordinate system and the network positioning information corresponding to the indoor picture.

[0163] The fourth determining module 305 is configured to determine a scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system based on the position information in the pose information of two or more indoor pictures and the network positioning information corresponding to the two or more indoor pictures.

[0164] The fifth determining module 306 is configured to determine a pose alignment relationship and a translation alignment relationship for aligning the indoor visual map coordinate system and the geographic coordinate system based on one scale alignment relationship corresponding to two or more indoor pictures and the pose alignment relationship and the translation alignment relationship corresponding to the indoor picture.

[0165] In this embodiment, the indoor picture taken by the user can determine the position of the user in the indoor picture in combination with the visual positioning technology. The position is the position of the user in the indoor visual map coordinate system. The visual positioning technology relies on the indoor visual map, and the indoor visual map is established based on the indoor visual map coordinate system. In order to convert the position of the user in the indoor visual map coordinate system into a geographic position, it is necessary to align the indoor visual map coordinate system with the geographic coordinate system, that is, to realize the conversion between the indoor visual map coordinate system and the geographic coordinate system. However, the conversion relationship between the indoor visual map coordinate system and the geographic coordinate system in the prior art is usually realized manually by artificial, and relies on a large amount of data, resulting in high labor cost and data cost. Therefore, the present embodiment of the present disclosure proposes a coordinate system conversion relationship determining device.

[0166] The coordinate system conversion relationship determining device provided by the present embodiment of the present disclosure can be implemented by a server, can be executed by a physical device capable of providing a server function, or can be executed by a component (such as a chip) configured in the physical device, or can be executed by a module capable of implementing part or all of the server function, etc. The present disclosure does not limit this. In order to facilitate understanding, the present disclosure will be described below as an example of a server to describe the device provided by the present disclosure.

[0167] In the embodiments of the present disclosure, the indoor picture can be a picture collected from an indoor space by a collection terminal. The indoor space can refer to the interior of a building, and can include but is not limited to the interior of an underground parking lot, the interior of a shopping mall, and the like.

[0168] In some embodiments, the indoor picture needs to cover at least the area covered by the indoor visual map. In order to save costs, the indoor picture used for coordinate alignment can be collected from the indoor space at the same time when the picture used for constructing the indoor visual map is collected. In addition, the sensor information of the mobile collection terminal needs to be collected when the indoor picture is collected.

[0169] The collection terminal can be a mobile phone, a smart television, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in smart city, a wireless terminal in smart home, and the like. The embodiments of the present disclosure do not limit the specific technology and specific device form of the collection terminal. In some embodiments, the collection terminal can be a device deployed with an image sensing unit, an inertial measurement unit (IMU), a magnetometer, and a network positioning sensing unit.

[0170] The collection terminal can collect multiple indoor pictures when collecting the indoor pictures.

[0171] In some embodiments, the posture of the collection terminal remains in a static state when the collection terminal collects the indoor pictures. The static state refers to that the posture of the collection terminal remains unchanged in the initial state during the process of collecting an indoor picture. It can be understood that the postures of the collection terminal can be different when the collection terminal collects different indoor pictures.

[0172] The collection terminal can collect sensor information of the collection terminal when collecting each indoor picture. The sensor information can include network positioning information, accelerometer information, and magnetometer information. The network positioning information can be position information obtained based on a network point such as an indoor wifi transmitter through a network positioning service, and the position information at least includes longitude and latitude. The accelerometer information can include gravity information, and the gravity information can be a gravity value. The magnetometer information can include the geographical orientation of the collection terminal, such as the angle between the y-axis of the coordinate system of the collection terminal and the true north direction.

[0173] In some embodiments, the indoor pictures correspond to the sensor information one-to-one, i.e., each indoor picture corresponds to one network positioning information, accelerometer information and magnetometer information.

[0174] The server can obtain multiple indoor pictures, and the sensor information of the terminal when each indoor picture is collected.

[0175] In the embodiments of the present disclosure, the indoor visual map is a map constructed according to visual data of the indoor pictures. The indoor visual map is established based on an indoor visual map coordinate system. The indoor visual map coordinate system is different from the geographic coordinate system, and can be set based on actual needs when the map is constructed. For example, the coordinate system of the terminal when the first indoor picture is collected for constructing the indoor visual map can be set as the indoor visual map coordinate system. Taking a mobile phone as an example, the coordinate system of the mobile phone can be established in the following manner: taking the center point of the mobile phone screen as the origin, the z-axis is perpendicular to the mobile phone screen, the x-axis is parallel to the short side of the mobile phone screen, and the y-axis is parallel to the long side of the mobile phone screen.

[0176] In some embodiments, the trajectory of the indoor visual map coincides with the trajectory of the terminal when the indoor pictures are collected. That is, the position points of the indoor pictures are located on the trajectory of the pictures used for constructing the indoor visual map.

[0177] In some embodiments, the indoor visual map can be pre-constructed, and the server can directly obtain the indoor visual map.

[0178] In some embodiments, the indoor visual map coordinate system can be established based on the coordinate system of the terminal when the pictures used for constructing the indoor visual map are collected. For example, the indoor visual map coordinate system can be constructed as the coordinate system of the terminal when the first indoor picture in the multiple indoor pictures used for constructing the indoor visual map is collected. It should be noted that the terminal for collecting the pictures for constructing the indoor visual map and the terminal for collecting the indoor pictures for aligning the indoor visual map coordinate system and the geographic coordinate system in the embodiments can be different or the same.

[0179] The terminal coordinate system is a coordinate system established based on the terminal. When the terminal performs a rotation or translation operation, the terminal coordinate system also performs a corresponding rotation or translation operation. That is, the terminal coordinate system can be a local coordinate established based on the terminal.

[0180] In some embodiments, the origin of the terminal coordinate system can be the center of the screen of the terminal, the long side of the screen of the terminal can be parallel to the y-axis of the terminal coordinate system, the short side of the screen of the terminal can be parallel to the x-axis of the terminal coordinate system, and the screen of the terminal is perpendicular to the z-axis of the terminal coordinate system.

[0181] It should be understood that the above construction of the acquisition terminal coordinate system is only an exemplary description, and the disclosure does not limit the selection of the origin, x-axis, y-axis and z-axis of the acquisition terminal coordinate system, as long as it is a coordinate system established based on the acquisition terminal itself, which is within the protection scope of the disclosure.

[0182] The pose information of the acquisition terminal when collecting the indoor picture can be understood as the pose information of the acquisition terminal coordinate system relative to the indoor visual map coordinate system when the acquisition terminal collects the indoor picture. The pose information can include rotation information, scale information and translation information (also referred to as position information).

[0183] The rotation information can represent the rotational alignment relationship between the coordinate axes of the acquisition terminal coordinate system and the coordinate axes of the indoor visual map coordinate system. Specifically, the rotation information can represent the rotational alignment relationship between the z-axis of the acquisition terminal coordinate system and the z-axis of the indoor visual coordinate system, and the rotational alignment relationship between the x-axis and y-axis of the acquisition terminal coordinate system and the x-axis and y-axis of the indoor visual coordinate system, respectively.

[0184] The translation information can represent the translation relationship between the acquisition terminal coordinate system and the indoor visual map coordinate system, which can be a translation distance. For example, assuming that the translation distance is 5 centimeters, the acquisition terminal coordinate system can be moved 5 centimeters in a certain straight line direction from the current position to reach the position of the indoor visual map coordinate system.

[0185] The scale information can represent the scale ratio between the acquisition terminal coordinate system and the indoor visual map coordinate system. For example, the acquisition terminal coordinate system represents an actual object of 10 meters with a length of 1 centimeter on the coordinate axis, and the indoor visual map coordinate system represents an actual object of 10 meters with a length of 10 centimeters on the coordinate axis, so the scale ratio is 1:10.

[0186] In some embodiments, the indoor picture and the pose information correspond to each other. Since the acquisition terminal has different poses when collecting different indoor pictures, the pose information of the acquisition terminal coordinate system relative to the indoor visual map coordinate system is also different when collecting different indoor pictures.

[0187] After the server obtains the pre-generated indoor visual map, the multiple indoor pictures, and the network positioning information, the accelerometer information and the magnetometer information of the acquisition terminal when collecting each indoor picture, the server can determine the pose information of the acquisition terminal coordinate system relative to the indoor visual map coordinate system when the acquisition terminal collects each indoor picture in the indoor visual map coordinate system.

[0188] In the embodiments of the present disclosure, the geographic coordinate system can be an East-North-Sky coordinate system. The x-axis of the East-North-Sky coordinate system points east, the y-axis points north, and the z-axis points skyward. The origin of the coordinate system can be different based on different positioning systems.

[0189] The server can determine the pose information of the collection terminal when collecting any one of the indoor pictures. Then, the server can determine the pose alignment relationship between the indoor visual map coordinate system and the geographic coordinate system based on the pose information of the collection terminal when collecting the indoor picture, and the accelerometer information and the magnetometer information of the collection terminal when collecting the indoor picture obtained by the server, that is, determine the rotation alignment relationship between the coordinate axes of the indoor visual map coordinate system and the coordinate axes of the geographic coordinate system.

[0190] In some embodiments, when determining the pose alignment relationship between the coordinate axes of the indoor visual map coordinate system and the coordinate axes of the geographic coordinate system, the server can determine the pose alignment relationship by the accelerometer information, the magnetometer information of the collection terminal corresponding to each of the plurality of indoor pictures collected by the collection terminal, and the pose information of the collection terminal when collecting each of the plurality of indoor pictures. This way can obtain multiple pose alignment relationships, from which the one with the smallest error can be selected as the final pose alignment relationship.

[0191] In other embodiments, the server can also determine the rotation information by the accelerometer information, the magnetometer information of the collection terminal corresponding to any one of the plurality of indoor pictures collected by the collection terminal, and the pose information of the collection terminal when collecting the indoor picture.

[0192] In the embodiments of the present disclosure, the network positioning information corresponding to the indoor picture can be understood as the position information of the collection terminal in the geographic coordinate system when the collection terminal collects the indoor picture.

[0193] The position information in the pose information of the indoor picture can be the translation information of the collection terminal coordinate system relative to the indoor visual map coordinate system when the collection terminal collects the indoor picture.

[0194] The server can obtain the translation information of the collection terminal coordinate system relative to the indoor visual map coordinate system by the pose information of the collection terminal when collecting the indoor picture, that is, determine the position information of the indoor picture in the indoor visual map coordinate system. Then, the server can determine the translation alignment relationship between the indoor visual map coordinate system and the geographic coordinate system in combination with the position information of the collection terminal in the geographic coordinate system when the collection terminal collects the indoor picture, that is, the network positioning information corresponding to the indoor picture.

[0195] In the embodiments of the present disclosure, the position information in the pose information of the indoor picture can be scale information of a coordinate system of the collection terminal relative to a coordinate system of the indoor visual map when the collection terminal collects the indoor picture.

[0196] The server can determine the scale alignment relationship between the coordinate system of the indoor visual map and the geographic coordinate system based on the scale information of the coordinate system of the collection terminal relative to the coordinate system of the indoor visual map when the collection terminal collects the indoor picture, and the position information of the collection terminal in the geographic coordinate system when the collection terminal collects the indoor picture.

[0197] In some embodiments, the scale alignment relationship between the coordinate system of the indoor visual map and the geographic coordinate system can be determined based on more than two indoor pictures.

[0198] As introduced above, the server can determine one pose alignment relationship between the coordinate system of the indoor visual map and the geographic coordinate system for more than two indoor pictures, determine two or more pose alignment relationships and two or more translation alignment relationships corresponding to the more than two indoor pictures, and select the pose alignment relationship with the smallest error from the two or more pose alignment relationships as the final determined pose alignment relationship for aligning the coordinate system of the indoor visual map and the geographic coordinate system, and select the translation alignment relationship with the smallest error from the two or more translation alignment relationships as the final determined translation alignment relationship for aligning the coordinate system of the indoor visual map and the geographic coordinate system.

[0199] The embodiments of the present disclosure provide a coordinate system conversion relationship determination apparatus, which obtains an indoor picture and network positioning information, accelerometer information and magnetometer information of a collection terminal when the collection terminal collects the indoor picture through an obtaining module, determines pose information of the collection terminal when the collection terminal collects the indoor picture through a first determining module, determines a pose alignment relationship between a coordinate system of an indoor visual map corresponding to the indoor picture and a geographic coordinate system through a second determining module, determines a translation alignment relationship between the coordinate system of the indoor visual map corresponding to the indoor picture and the geographic coordinate system through a third determining module, and determines a scale alignment relationship between the coordinate system of the indoor visual map and the geographic coordinate system through a fourth determining module. Finally, a pose alignment relationship and a translation alignment relationship for aligning the coordinate system of the indoor visual map and the geographic coordinate system are determined based on a fifth determining module. Since the network positioning information, the accelerometer information and the magnetometer information are obtained through the sensors of the collection terminal when the collection terminal collects the indoor picture, the map coordinate system can be aligned with the geographic coordinate system based on these data, which reduces the manual participation link and reduces the data cost and the labor cost.

[0200] In an optional implementation of the present embodiment, the second determining module 303 comprises:

[0201] The first determining sub-module is configured to determine a rotation alignment relationship between a z-axis of an indoor visual map coordinate system corresponding to the indoor picture and a z-axis of a geographic coordinate system based on the attitude information in the pose information of the indoor picture and the accelerometer information.

[0202] The second determining sub-module is configured to determine rotation alignment relationships between an x-axis and a y-axis of the indoor visual map coordinate system corresponding to the indoor picture and x-axes and y-axes of the geographic coordinate system, respectively, based on the attitude information in the pose information of the indoor picture and the magnetometer information.

[0203] In this optional implementation, the server can determine the attitude information of the collecting terminal when collecting an indoor picture, and can obtain the attitude alignment relationship between the z-axis of the collecting terminal coordinate system and the z-axis of the indoor visual coordinate system, the attitude alignment relationship between the x-axis of the collecting terminal coordinate system and the x-axis of the indoor visual coordinate system, and the attitude alignment relationship between the y-axis of the collecting terminal coordinate system and the y-axis of the indoor visual coordinate system. Moreover, the server can obtain the rotation alignment relationship of the collecting terminal relative to the z-axis of the geographic coordinate system through the accelerometer information, and obtain the rotation alignment relationships of the collecting terminal relative to the x-axis and the y-axis of the geographic coordinate system through the magnetometer information. Therefore, the server can determine the rotation alignment relationship between the z-axis of the indoor visual coordinate system and the z-axis of the geographic coordinate system based on the attitude alignment relationship between the z-axis of the collecting terminal coordinate system and the z-axis of the indoor visual coordinate system and the rotation alignment relationship of the collecting terminal relative to the z-axis of the geographic coordinate system. The rotation alignment relationship between the x-axis of the indoor visual coordinate system and the x-axis of the geographic coordinate system can be determined based on the attitude alignment relationship between the x-axis of the collecting terminal coordinate system and the x-axis of the indoor visual coordinate system and the rotation alignment relationship of the collecting terminal relative to the x-axis of the geographic coordinate system. The rotation alignment relationship between the y-axis of the indoor visual coordinate system and the y-axis of the geographic coordinate system can be determined based on the attitude alignment relationship between the y-axis of the collecting terminal coordinate system and the y-axis of the indoor visual coordinate system and the rotation alignment relationship of the collecting terminal relative to the y-axis of the geographic coordinate system.

[0204] In an optional implementation of the present embodiment, the first determining sub-module comprises:

[0205] The third determining sub-module is configured to determine a z-axis rotation alignment relationship of the collecting terminal in the geographic coordinate system when collecting the indoor picture based on the accelerometer information of the indoor picture.

[0206] The fourth determining sub-module is configured to determine a z-axis rotation alignment relationship between the indoor visual map coordinate system corresponding to the indoor picture and the geographic coordinate system based on the z-axis rotation alignment relationship in the pose information in the indoor visual map coordinate system of the indoor picture and the z-axis rotation alignment relationship of the collecting terminal in the geographic coordinate system.

[0207] In the optional implementation, when the server obtains the acceleration information of the terminal, i.e., the gravity information, when collecting an indoor picture, the server can determine the rotational alignment relationship of the z-axis of the terminal coordinate system relative to the z-axis of the geographic coordinate system, i.e., the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the geographic coordinate system. Since the server can also determine the rotational alignment relationship of the z-axis of the terminal coordinate system relative to the z-axis of the indoor visual map coordinate system, i.e., the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the indoor visual map coordinate system, when the terminal collects the indoor picture, the server can determine the rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system according to the two rotational alignment relationships. That is, each indoor picture can correspond to determine a rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system.

[0208] In some embodiments, the rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system can be as follows:

[0209]

[0210] wherein, is the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the indoor visual map coordinate system, is the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the geographic coordinate system, is the rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system, and T is a transpose operation.

[0211] For ease of understanding, the following is an exemplary description.

[0212] In order to improve the processing efficiency, the final rotational information between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system can be determined by any one of the multiple indoor pictures collected by the terminal.

[0213] In some embodiments, it is assumed that the multiple indoor pictures collected by the terminal include indoor picture 1, and the gravity value of the terminal is G1 when collecting indoor picture 1. Then, the server can determine the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the indoor visual map coordinate system, and also determine the rotational alignment relationship between the z-axis of the terminal coordinate system and the z-axis of the geographic coordinate system according to the gravity value G1 when the terminal collects indoor picture 1. Furthermore, the rotational alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system is determined according to the two rotational alignment relationships.

[0214] In order to improve the accuracy, the final rotation information between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system can also be determined by the multiple indoor pictures collected by the collection terminal.

[0215] In some other embodiments, it is assumed that the collection terminal collects three indoor pictures, indoor picture 1, indoor picture 2 and indoor picture 3 in sequence, and the gravity values of the collection terminal when collecting the three indoor pictures are G1, G2 and G3 respectively. The indoor picture 1 is the first picture collected by the collection terminal.

[0216] For the indoor picture 1, the rotation alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system and the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the geographic coordinate system. Therefore, the server can determine the rotation alignment relationship R z1 between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system according to the gravity value G1.

[0217] For the indoor picture 2, the server can determine the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the geographic coordinate system and the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the indoor visual map coordinate system according to the gravity value G2, and further determine the rotation alignment relationship R z2 between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system.

[0218] For the indoor picture 3, the server can determine the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the geographic coordinate system and the rotation alignment relationship between the z-axis of the collection terminal coordinate system and the z-axis of the indoor visual map coordinate system according to the gravity value G3, and further determine the rotation alignment relationship R z3 between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system.

[0219] In order to determine the rotation alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system with the highest accuracy, the rotation alignment relationship with the smallest error can be selected from the above rotation alignment relationships R z1 , R z2 and R z3 as the final rotation alignment relationship. For example, the server can determine the final rotation alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system in the following manner.

[0220] Specifically, the server can determine the absolute value of the difference between each two rotation alignment relationships R z1 , R z2 and R z3 , and it is assumed that R z1 and Rz2 The absolute value of the difference is r1, R z2 and R z3 The absolute value of the difference is r², R z3 and R z1 The absolute value of the difference is r3. Then, the server can determine a target absolute value among multiple absolute values ​​of the difference, and based on the two rotation alignment relationships corresponding to this target absolute value, determine the final rotation alignment relationship between the z-axis of the indoor visual map coordinate system and the z-axis of the geographic coordinate system. This target absolute value of the difference can be the smallest among multiple absolute values ​​of the difference. For example, r 1＜ r 2＜ If r3 is the smallest, then r1 is the smallest, and the R corresponding to indoor image 1 can be... z1 R corresponding to indoor image 2 z2 The rotation alignment relationship between the z-axis of the indoor visual map coordinate system corresponding to image 1 and the z-axis of the geographic coordinate system can be used as the final rotation alignment relationship. Alternatively, the R-axis corresponding to indoor image 1 can be used as the final rotation alignment relationship. z1 R corresponding to indoor image 2 z2 The average of the two rotation alignment relationships between the z-axis of the corresponding indoor visual map coordinate system and the z-axis of the geographic coordinate system is used as the final rotation alignment relationship.

[0221] In an optional implementation of this embodiment, the third determining submodule includes:

[0222] The fifth determining submodule is configured to determine the first direction vector of gravity in the device coordinate system of the acquisition terminal based on the accelerometer information of the indoor image;

[0223] The sixth determining submodule is configured to determine the z-axis rotation alignment relationship of the acquisition terminal relative to the geographic coordinate system based on the first direction vector and the second direction vector of gravity in the geographic coordinate system.

[0224] In this optional implementation, since the direction of gravity is known, its coordinates in the geographic coordinate system are [0,0,9.8]. Therefore, for any indoor image, when determining the rotational alignment of the z-axis of the acquisition terminal coordinate system relative to the z-axis of the geographic coordinate system, the server can determine the first direction vector of gravity in the acquisition terminal coordinate system and the second direction vector of gravity in the geographic coordinate system using accelerometer information. Then, based on the first and second direction vectors, the rotational alignment of the z-axis of the acquisition terminal coordinate system relative to the z-axis of the geographic coordinate system corresponding to that indoor image can be determined. It can be understood that each indoor image can correspond to a specific z-axis rotational alignment of the acquisition terminal relative to the geographic coordinate system. It should be noted that the coordinate system of the acquisition terminal can be simply referred to as the acquisition terminal coordinate system in this disclosure.

[0225] In an optional implementation of the embodiment, the second determining sub-module comprises:

[0226] The seventh determining sub-module is configured to determine, based on the magnetometer information of the indoor picture, a rotation alignment relationship between the x-axis and the y-axis of the acquisition terminal relative to the geographic coordinate system when the indoor picture is acquired;

[0227] The eighth determining sub-module is configured to determine, based on the rotation alignment relationship between the x-axis and the y-axis in the pose information in the indoor visual map coordinate system of the indoor picture and the rotation alignment relationship between the x-axis and the y-axis of the acquisition terminal relative to the geographic coordinate system, a rotation alignment relationship between the x-axis and the y-axis of the indoor visual map coordinate system corresponding to the indoor picture and the x-axis and the y-axis of the geographic coordinate system.

[0228] In the optional implementation, after the server acquires the magnetometer information of the acquisition terminal when an indoor picture is acquired, that is, the geographic orientation of the acquisition terminal, the server can determine the rotation alignment relationship between the x-axis and the y-axis of the acquisition terminal relative to the geographic coordinate system according to the geographic orientation, that is, the rotation alignment relationship between the x-axis of the acquisition terminal coordinate system and the x-axis of the geographic coordinate system and the rotation alignment relationship between the y-axis of the acquisition terminal coordinate system and the y-axis of the geographic coordinate system. The server can also determine the rotation alignment relationship between the x-axis of the acquisition terminal coordinate system and the x-axis of the indoor visual map coordinate system and the rotation alignment relationship between the y-axis of the acquisition terminal coordinate system and the y-axis of the indoor visual map coordinate system when the acquisition terminal acquires the indoor picture. Therefore, the server can determine the rotation alignment relationship between the x-axis of the indoor visual map coordinate system corresponding to the indoor picture and the x-axis of the geographic coordinate system according to the rotation alignment relationship between the x-axis of the acquisition terminal coordinate system and the x-axis of the geographic coordinate system and the rotation alignment relationship between the x-axis of the acquisition terminal coordinate system and the x-axis of the indoor visual map coordinate system. The server can also determine the rotation alignment relationship between the y-axis of the indoor visual map coordinate system corresponding to the indoor picture and the y-axis of the geographic coordinate system according to the rotation alignment relationship between the y-axis of the acquisition terminal coordinate system and the y-axis of the geographic coordinate system and the rotation alignment relationship between the y-axis of the acquisition terminal coordinate system and the y-axis of the indoor visual map coordinate system. That is, each indoor picture can correspond to a determined rotation alignment relationship between the x-axis of the indoor visual map coordinate system and the x-axis of the geographic coordinate system and a determined rotation alignment relationship between the y-axis of the indoor visual map coordinate system and the y-axis of the geographic coordinate system.

[0229] In some embodiments, the rotation alignment relationship between the x-axis and the y-axis of the indoor visual map coordinate system and the x-axis and the y-axis of the geographic coordinate system can be as follows:

[0230]

[0231] wherein, to acquire a rotation alignment relationship between the x-axis or the y-axis of the terminal coordinate system and the x-axis or the y-axis of the indoor visual map coordinate system, to acquire a rotation alignment relationship between the x-axis or the y-axis of the terminal coordinate system and the x-axis or the y-axis of the geographic coordinate system, to acquire a rotation alignment relationship between the x-axis or the y-axis of the indoor visual map coordinate system and the x-axis or the y-axis of the geographic coordinate system, and T is a transposition operation.

[0232] Likewise, the server can also determine the rotation alignment relationship between the x-axis and the y-axis of the final indoor visual map coordinate system and the x-axis and the y-axis of the geographic coordinate system based on any one of the plurality of indoor pictures or all or part of the plurality of indoor pictures. For details, refer to the foregoing description, which will not be repeated here.

[0233] In an optional implementation of the embodiment, the fourth determining module 305 comprises:

[0234] a ninth determining sub-module configured to determine a plurality of first three-dimensional position coordinates of the acquisition terminal in the geographic coordinate system when the acquisition terminal acquires the plurality of indoor pictures based on the plurality of network positioning information of the plurality of indoor pictures;

[0235] a tenth determining sub-module configured to determine the scale information between the indoor visual map coordinate system and the geographic coordinate system based on the plurality of first three-dimensional position coordinates and a plurality of second three-dimensional position coordinates of the indoor visual map coordinate system of the plurality of indoor pictures.

[0236] In the optional implementation, the server can determine, based on the plurality of indoor pictures acquired, a plurality of second three-dimensional position coordinates of the terminal coordinate system of the acquisition terminal relative to the indoor visual map coordinate system when the acquisition terminal acquires the plurality of indoor pictures, and the plurality of second three-dimensional position coordinates can constitute a set of three-dimensional position coordinates. Since the server can also acquire, through network positioning, network positioning information of the acquisition terminal when the acquisition terminal acquires each indoor picture, the network positioning information includes latitude and longitude information, the server can determine, based on the acquired latitude and longitude information, a plurality of first three-dimensional position coordinates of the terminal coordinate system of the acquisition terminal relative to the geographic coordinate system when the acquisition terminal acquires the plurality of indoor pictures, and the plurality of first three-dimensional position coordinates can constitute another set of three-dimensional position coordinates. Further, based on a preset iterative closest point (ICP) algorithm, the scale information between the indoor visual map coordinate system and the geographic coordinate system can be determined according to the two sets of three-dimensional position coordinates. The ICP algorithm can obtain the conversion relationship between the coordinate systems of the two sets of three-dimensional point cloud data by aligning the two sets of three-dimensional point cloud data, including rotation information, translation information and scale information.

[0237] In the embodiment of the present disclosure, the rotation information, the translation information and the scale information between the indoor visual map coordinate system and the geographic coordinate system can be obtained by performing the ICP algorithm on a group of three-dimensional position coordinates composed of the plurality of second three-dimensional position coordinates of the acquisition terminal in the indoor visual map coordinate system and another group of three-dimensional position coordinates composed of the plurality of first three-dimensional position coordinates of the acquisition terminal in the geographic coordinate system. In the embodiment of the present disclosure, considering that the indoor network positioning information is not accurate enough, the rotation information obtained by the ICP is not accurate enough, therefore, the rotation information between the indoor visual map coordinate system and the geographic coordinate system is determined by using the magnetometer and the accelerometer, and the scale information and the translation information are obtained by using the ICP algorithm.

[0238] According to an embodiment of the present disclosure, an indoor positioning device is also provided, which comprises:

[0239] a picture acquisition module configured to acquire pictures collected and uploaded by a user indoors;

[0240] a first position determination module configured to determine position information of the user in an indoor visual map coordinate system based on the pictures and the indoor visual map;

[0241] a second position determination module configured to determine position information of the user in a geographic coordinate system based on a conversion relationship between the indoor visual map coordinate system and the geographic coordinate system, wherein the conversion relationship between the indoor visual map coordinate system and the geographic coordinate system is determined by the above-mentioned coordinate system conversion relationship determination device.

[0242] The indoor positioning device provided by the embodiment of the present disclosure can be implemented by a terminal device, the terminal device is deployed with an indoor visual map, and the terminal device is also deployed with an electronic map, or the terminal device can obtain the indoor visual map and the electronic map from a server. The coordinate system used by the electronic map is a geographic coordinate system, and the coordinate system used by the indoor visual map is an indoor visual coordinate system.

[0243] In this embodiment, when the user has a positioning requirement, the user can use the terminal device to take indoor pictures around the location of the user, and when the terminal device acquires the indoor pictures taken by the user, the position coordinates of the user in the indoor visual map coordinate system can be determined according to the taken indoor pictures and the indoor visual map. The terminal device can obtain the conversion relationship between the indoor visual map coordinate system and the geographic coordinate system from the server, and then the position coordinates of the user in the geographic coordinate system can be determined according to the conversion relationship. The conversion relationship between the indoor visual map coordinate system and the geographic coordinate system can be determined based on the above-mentioned coordinate system conversion relationship determination device, and the specific details can be referred to the description of the coordinate system conversion relationship determination device in the foregoing, which will not be described herein again.

[0244] According to an embodiment of the present disclosure, an indoor path planning device is also provided, which comprises:

[0245] a third position determining module configured to determine position information of the user in a geographical coordinate system based on the indoor positioning device;

[0246] a path planning module configured to plan a navigation path for the user from a position corresponding to the position information to a destination based on the position information.

[0247] The indoor path planning device provided by the embodiment of the present disclosure can be implemented by a server. In this embodiment, when the user is taking a car or performing other navigation actions in an indoor environment, the user can use a terminal device to take pictures of the surrounding indoor environment. When the terminal device obtains the indoor pictures taken by the user, the terminal device can determine the position coordinates of the user in an indoor visual map coordinate system according to the taken indoor pictures and the indoor visual map. The terminal device can obtain the conversion relationship between the indoor visual map coordinate system and the geographical coordinate system from the server, and then determine the position coordinates of the user in the geographical coordinate system according to the conversion relationship. The user can upload the position coordinates to a navigation server. The navigation server can plan a navigation path for the user based on the position coordinates and the destination position input by the user in a navigation APP, and send the planned navigation path to the terminal device of the user, so that the user can go to the destination according to the navigation path.

[0248] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0249] Figure 4 is a structural schematic diagram of an electronic device suitable for implementing the coordinate system conversion relationship determining method according to an embodiment of the present disclosure.

[0250] As Figure 4As shown, the electronic device 400 includes a processing unit 401, which can be implemented as a CPU, a GPU, a FPGA, a NPU, or the like. The processing unit 401 can perform various processes in the embodiments of any of the methods of the present disclosure according to a program stored in a read only memory (ROM) 402 or a program loaded into a random access memory (RAM) 403 from the storage section 408. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing unit 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0251] Connected to the I / O interface 405 are an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as necessary. A removable recording medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 410 as necessary, so that a computer program read therefrom is installed into the storage section 408 as necessary.

[0252] In particular, according to embodiments of the present disclosure, the above-mentioned reference to any of the methods in the embodiments of the present disclosure can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for executing any of the methods of the embodiments of the present disclosure. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 409, and / or installed from the removable recording medium 411.

[0253] The flow and block diagrams in the drawings represent possible architectural, functional, and operational architectures of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0254] The units or modules described in the embodiments of the present disclosure can be implemented by software, or by hardware. The described units or modules can also be arranged in a processor, and the name of the unit or module does not constitute a limitation on the unit or module itself in some cases.

[0255] As another aspect, the present disclosure also provides a computer readable storage medium, which can be the computer readable storage medium included in the apparatus described in the above embodiments, or can exist separately from the apparatus and not be assembled into the apparatus. The computer readable storage medium stores one or more programs for execution by one or more processors to perform the method described in the present disclosure.

[0256] The above description is merely preferred embodiments of the present disclosure and a description of principles of applied technologies. It should be understood by those skilled in the art that the scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also includes other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the inventive concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions.

Claims

1. A method for determining coordinate system transformation relationships, wherein, include: The system acquires indoor images and collects sensor information from the terminal during the acquisition of these images. The sensor information includes network positioning information, accelerometer information, and magnetometer information. Based on the indoor images and the pre-generated indoor visual map, the pose information of the acquisition terminal when acquiring the indoor images is determined in the indoor visual map coordinate system. Based on the pose information in the pose information of the indoor image, the accelerometer information, and the magnetometer information, the pose alignment relationship between the indoor visual map coordinate system and the geographic coordinate system corresponding to the indoor image is determined; the geographic coordinate system is the East-North-Sky coordinate system. Based on the location information of the indoor image in the indoor visual map coordinate system and the network positioning information corresponding to the indoor image, the translation alignment relationship between the indoor visual map coordinate system corresponding to the indoor image and the geographic coordinate system is determined. Based on the positional information in the pose information of two or more indoor images and the network positioning information corresponding to the two or more indoor images, the scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system is determined. Based on a scale alignment relationship corresponding to two or more indoor images and a pose alignment relationship and a translation alignment relationship corresponding to the indoor images, a pose alignment relationship and a translation alignment relationship are determined for aligning the indoor visual map coordinate system and the geographic coordinate system.

2. The method according to claim 1, wherein, Based on the pose information in the pose information of the indoor image, the accelerometer information, and the magnetometer information, the pose alignment relationship between the indoor visual map coordinate system and the geographic coordinate system corresponding to the indoor image is determined, including: Based on the pose information in the pose information of the indoor image and the accelerometer information, the rotational alignment relationship between the z-axis of the indoor visual map coordinate system corresponding to the indoor image and the z-axis of the geographic coordinate system is determined. Based on the pose information in the pose information of the indoor image and the magnetometer information, the rotational alignment relationship between the x-axis and y-axis of the indoor visual map coordinate system corresponding to the indoor image and the x-axis and y-axis of the geographic coordinate system is determined.

3. The method according to claim 2, wherein, Based on the pose information in the indoor image and the accelerometer information, the rotational alignment relationship between the z-axis of the indoor visual map coordinate system corresponding to the indoor image and the z-axis of the geographic coordinate system is determined, including: Based on the accelerometer information of the indoor images, determine the z-axis rotation alignment of the acquisition terminal in the geographic coordinate system when acquiring the indoor images; Based on the z-axis rotation alignment relationship in the pose information of the indoor visual map coordinate system of the indoor image and the z-axis rotation alignment relationship of the acquisition terminal in the geographic coordinate system, the z-axis rotation alignment relationship between the indoor visual map coordinate system corresponding to the indoor image and the geographic coordinate system is determined.

4. The method according to claim 3, wherein, Based on the accelerometer information of the indoor images, determining the z-axis rotation alignment of the acquisition terminal in the geographic coordinate system when acquiring the indoor images includes: Based on the accelerometer information of the indoor image, determine the first direction vector of gravity in the device coordinate system of the acquisition terminal; Based on the first direction vector and the second direction vector of gravity in the geographic coordinate system, the rotational alignment relationship of the acquisition terminal relative to the z-axis of the geographic coordinate system is determined.

5. The method according to any one of claims 2 to 4, wherein, Based on the pose information in the pose information of the indoor image and the magnetometer information, the rotational alignment relationship between the x-axis and y-axis of the indoor visual map coordinate system corresponding to the indoor image and the x-axis and y-axis of the geographic coordinate system is determined, including: Based on the magnetometer information of the indoor images, determine the rotational alignment relationship of the acquisition terminal relative to the x-axis and y-axis of the geographic coordinate system when acquiring the indoor images; Based on the x-axis and y-axis rotation alignment relationship in the pose information of the indoor visual map coordinate system of the indoor image and the x-axis and y-axis rotation alignment relationship of the acquisition terminal relative to the geographic coordinate system, the rotation alignment relationship between the x-axis and y-axis of the indoor visual map coordinate system corresponding to the indoor image and the x-axis and y-axis of the geographic coordinate system is determined.

6. The method according to any one of claims 1 to 4, wherein, Based on the positional information in the pose information of two or more indoor images and the network positioning information corresponding to the two or more indoor images, the scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system is determined, including: Based on the network positioning information of multiple indoor images, the multiple first three-dimensional position coordinates of the acquisition terminal in the geographic coordinate system are determined when acquiring multiple indoor images; Based on the multiple first three-dimensional position coordinates and the multiple second three-dimensional position coordinates of the multiple indoor images in the indoor visual map coordinate system, the scale information between the indoor visual map coordinate system and the geographic coordinate system is determined.

7. An indoor positioning method, wherein, include: Acquire images collected and uploaded by users indoors; Based on the image and the indoor visual map, determine the user's location information in the indoor visual map coordinate system; The user's location information in the geographic coordinate system is determined based on the transformation relationship between the indoor visual map coordinate system and the geographic coordinate system; wherein the transformation relationship between the indoor visual map coordinate system and the geographic coordinate system is determined based on the method of any one of claims 1-6.

8. An indoor path planning method, wherein, include: The user's location information in a geographic coordinate system is determined based on the method described in claim 7; Based on the location information, a navigation path is planned for the user from the location corresponding to the location information to the destination.

9. A coordinate system transformation relationship determination device, wherein, include: The acquisition module is configured to acquire indoor images and collect sensor information of the terminal when acquiring the indoor images. The sensor information includes: network positioning information, accelerometer information and magnetometer information. The indoor images and sensor information are in one-to-one correspondence. The first determining module is configured to determine the pose information of the acquisition terminal when acquiring the indoor image based on the indoor image and a pre-generated indoor visual map in the indoor visual map coordinate system, with the indoor image and pose information corresponding one-to-one. The second determining module is configured to determine the attitude alignment relationship between the indoor visual map coordinate system corresponding to the indoor image and the geographic coordinate system based on the attitude information, the accelerometer information and the magnetometer information in the pose information of the indoor image. The third determining module is configured to determine the translational alignment relationship between the indoor visual map coordinate system corresponding to the indoor image and the geographic coordinate system based on the position information of the indoor image in the indoor visual map coordinate system and the network positioning information corresponding to the indoor image. The fourth determining module is configured to determine the scale alignment relationship between the indoor visual map coordinate system and the geographic coordinate system based on the position information in the pose information of two or more indoor images and the network positioning information corresponding to the two or more indoor images. The fifth determining module is configured to determine the pose alignment relationship and translation alignment relationship for aligning the indoor visual map coordinate system and the geographic coordinate system based on a scale alignment relationship corresponding to two or more indoor images and the pose alignment relationship and translation alignment relationship corresponding to the indoor images.

10. An electronic device, wherein, It includes a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the method of any one of claims 1-8.

11. A computer-readable storage medium, wherein, The computer-readable storage medium is used to store computer instructions, which, when executed by a processor, are used to implement the method of any one of claims 1-8.

Images