Image-based localization methods and related devices

Abstract

This application relates to an image-based positioning method and related equipment. The method includes: acquiring a panoramic image and corresponding image orientation information using a first electronic device; sending the panoramic image and the image orientation information to a second electronic device, enabling a user corresponding to the second electronic device to locate the position of the first electronic device using the panoramic image and the image orientation information, thereby determining the position of a user holding the first electronic device. This application can improve positioning efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image-based positioning method and related equipment. Background Technology

[0002] With the rapid development of electronic devices and the mobile internet, more and more users are finding each other's locations through navigation and location sharing on electronic devices.

[0003] However, during navigation, inaccurate location information or complex environments often occur, making it impossible to determine the other party's location. Summary of the Invention

[0004] This application discloses an image-based positioning method and related equipment, which can improve the efficiency of locating the position of electronic devices.

[0005] The first aspect of this application discloses an image-based localization method applied in a first electronic device, the image-based localization method comprising:

[0006] Obtain the panoramic image and the image orientation information corresponding to the panoramic image;

[0007] The panoramic image and the image orientation information are sent to the second electronic device, so that the user corresponding to the second electronic device can locate the position of the first electronic device using the panoramic image and the image orientation information.

[0008] In some optional implementations, acquiring the panoramic image and the image orientation information corresponding to the panoramic image includes:

[0009] The camera device of the first electronic device captures multiple scene images from the initial direction, rotating clockwise or counterclockwise, wherein the sum of the viewing angles of the multiple scene images on the horizontal plane is greater than or equal to 360 degrees.

[0010] Record the initial orientation information, rotation orientation information, and rotation angle of each scene image in the multiple scene images acquired;

[0011] A panoramic image is generated based on the multiple scene images and the rotation angle of each scene image. The image orientation information includes the initial orientation information and the rotation orientation information.

[0012] In some optional implementations, acquiring the panoramic image and the image orientation information corresponding to the panoramic image includes:

[0013] The first electronic device acquires multiple scene images in the horizontal direction, and the sum of the viewing angles of the multiple scene images on the horizontal plane is greater than or equal to 360 degrees.

[0014] Record the orientation information of each scene image in the plurality of scene images;

[0015] A panoramic image is generated based on the multiple scene images and their orientation information. The orientation information of the panoramic image includes the orientation information of the image at each angle in the panoramic image.

[0016] In some optional implementations, sending the panoramic image and the image orientation information to the second electronic device includes:

[0017] Send a panoramic image based on a first-person perspective and the image orientation information to a second electronic device; or

[0018] The panoramic image based on a third-person perspective and the orientation information of the image are sent to a second electronic device.

[0019] In some optional implementations, before sending the panoramic image based on the third-person perspective and the image orientation information to the second electronic device, the image-based positioning method further includes:

[0020] Get user image;

[0021] The user image is rendered onto the panoramic image to obtain the panoramic image based on a third-person perspective.

[0022] In some alternative implementations, the panoramic image may also include the location information of the first electronic device.

[0023] In some optional implementations, when sending the panoramic image and the image orientation information to the second electronic device, the image-based positioning method further includes:

[0024] Obtain the location information and preset distance value of the first electronic device;

[0025] Receive the location information of the second electronic device;

[0026] The distance between the first electronic device and the second electronic device is determined based on the location information of the first electronic device and the location information of the second electronic device.

[0027] If the distance value is less than the preset distance value, a navigation path is determined based on the location information of the first electronic device and the second electronic device. A first relative direction information between the first electronic device and the second electronic device is determined based on the navigation path, and the first relative direction information is sent to the second electronic device; or

[0028] If the distance value is greater than or equal to the preset distance value, the second relative direction information between the first electronic device and the second electronic device is determined based on the location information of the first electronic device and the location information of the second electronic device, and the second relative direction information is sent to the second electronic device.

[0029] In some optional implementations, sending the panoramic image and the image orientation information to the second electronic device includes:

[0030] Based on the relative orientation information of the first electronic device and the second electronic device and the image orientation information, the panoramic image is cropped to obtain the cropped image;

[0031] The captured image is sent to the second electronic device.

[0032] A second aspect of this application discloses an image-based localization method applied in a second electronic device, the image-based localization method comprising:

[0033] Receive panoramic images and image orientation information corresponding to the panoramic images sent by the first electronic device;

[0034] Obtain the device orientation of the second electronic device;

[0035] Display a portion of the panoramic image based on the device orientation and the image orientation information, and locate the position of the first electronic device based on the portion of the image.

[0036] In some optional implementations, the device orientation for acquiring the second electronic device includes:

[0037] Obtain the orientation information of the second electronic device, and determine the orientation information of the second electronic device as the device orientation; or

[0038] Obtain the relative direction information between the second electronic device and the first electronic device, and determine the relative direction information as the device direction.

[0039] In some optional implementations, obtaining the relative orientation information between the second electronic device and the first electronic device includes:

[0040] Receive the relative direction information from the first electronic device; or

[0041] Receive the user-selected viewpoint and determine the user-selected viewpoint as the relative direction information; or

[0042] The location information of the first electronic device and the location information of the second electronic device are obtained, and the relative direction information between the first electronic device and the second electronic device is determined based on the location information of the first electronic device and the location information of the second electronic device.

[0043] In some optional implementations, determining the relative orientation information between the first electronic device and the second electronic device based on the location information of the first electronic device and the second electronic device includes:

[0044] The navigation path is determined based on the location information of the first electronic device and the location information of the second electronic device;

[0045] The relative direction information between the first electronic device and the second electronic device is determined based on the navigation path.

[0046] In some optional implementations, determining the relative orientation information between the first electronic device and the second electronic device based on the location information of the first electronic device and the second electronic device includes:

[0047] Get the preset distance value;

[0048] The distance between the first electronic device and the second electronic device is determined based on the location information of the first electronic device and the location information of the second electronic device.

[0049] If the distance value is less than the preset distance value, the initial travel direction of the second electronic device or the final travel direction of the second electronic device in the navigation path is determined as the relative direction information;

[0050] If the distance value is greater than or equal to the preset distance value, the user-selected viewing direction information is received, and the user-selected viewing direction information is determined as the relative direction information.

[0051] In some optional implementations, if the distance value is greater than or equal to the preset distance value, the relative direction information of the second electronic device pointing to the first electronic device is obtained, and the relative direction information is determined as the device direction.

[0052] In some optional implementations, displaying a portion of the panoramic image based on the device orientation and the image orientation information includes:

[0053] Display a portion of the panoramic image from a first-person or third-person perspective based on the device orientation and the image orientation information.

[0054] In some alternative implementations, the partial image may also include a user image.

[0055] In some optional implementations, displaying a portion of the panoramic image based on the device orientation and the image orientation information includes:

[0056] Update the device orientation;

[0057] Based on the updated device orientation and the image orientation information, a portion of the panoramic image is displayed in real time.

[0058] In some optional implementations, after displaying a portion of the panoramic image based on the device orientation and the image orientation information, the image-based positioning method further includes:

[0059] Receive user perspective change commands;

[0060] The display view of the panoramic image is switched according to the viewpoint switching operation to display partial images of the panoramic image from different viewpoints.

[0061] In some optional implementations, when displaying a portion of the panoramic image based on the device orientation and the image orientation information, the image-based positioning method further includes:

[0062] The system displays a navigation path from the second electronic device to the first electronic device, enabling the user of the second electronic device to locate the position of the first electronic device based on the partial image and the navigation path.

[0063] In some optional implementations, when displaying a portion of the panoramic image based on the device orientation and the image orientation information, the image-based positioning method further includes:

[0064] The location information of the second electronic device and the first electronic device is displayed, so that the user corresponding to the second electronic device can locate the location of the first electronic device based on the partial image and the location information.

[0065] A third aspect of this application discloses an image-based localization method applied in a first electronic device and a second electronic device, the image-based localization method comprising:

[0066] The first electronic device acquires a panoramic image and the image orientation information corresponding to the panoramic image;

[0067] The first electronic device sends the panoramic image and the image orientation information to the second electronic device;

[0068] The second electronic device acquires the device orientation of the second electronic device;

[0069] The second electronic device displays a portion of the panoramic image based on the device orientation and the image orientation information, and locates the position of the first electronic device based on the portion of the image.

[0070] In some optional implementations, the first electronic device sending the panoramic image and the image orientation information to the second electronic device includes:

[0071] The first electronic device acquires the device orientation of the second electronic device;

[0072] The first electronic device extracts a portion of the image from the panoramic image based on the device orientation and the image orientation information, thereby obtaining the extracted portion of the image;

[0073] The first electronic device sends the captured portion of the image to the second electronic device, causing the second electronic device to display the captured portion of the image.

[0074] A fourth aspect of this application discloses an electronic device, including a processor and a memory; the memory is used to store instructions; the processor is used to invoke the instructions in the memory, causing the electronic device to execute the image-based positioning method.

[0075] The fifth aspect of this application discloses a computer-readable storage medium storing at least one instruction that, when executed by a processor, implements the image-based localization method.

[0076] The sixth aspect of this application discloses a chip coupled to a memory in an electronic device, the chip being used to control the electronic device to perform the image-based positioning method described above.

[0077] The technical effects brought about by the second to sixth aspects can be found in the descriptions of the methods involved in the above-mentioned methods section, and will not be repeated here. Attached Figure Description

[0078] Figure 1 This is a schematic diagram of an image-based positioning device interaction provided in an embodiment of this application.

[0079] Figure 2 This is a flowchart of an image-based positioning method applied to a first electronic device, provided in an embodiment of this application.

[0080] Figure 3 This is a panoramic image provided in an embodiment of this application.

[0081] Figure 4 This is a preset photo data structure diagram provided in the embodiments of this application.

[0082] Figure 5 This is a flowchart of an image-based positioning method for a second electronic device provided in an embodiment of this application.

[0083] Figures 6A-6B This is a panoramic image displayed by the second electronic device provided in the embodiments of this application.

[0084] Figure 7 This is a schematic diagram of a portion of a panoramic image displayed by the second electronic device provided in this application embodiment.

[0085] Figure 8 This is a flowchart of an image-based positioning method applied to a first electronic device and a second electronic device, provided in an embodiment of this application.

[0086] Figure 9 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0087] For ease of understanding, some concepts related to the embodiments of this application are illustrated and explained by way of example for reference.

[0088] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0089] To better understand the image-based positioning method and related devices disclosed in the embodiments of this application, the application scenarios of image-based positioning of this application will be described first below.

[0090] Figure 1 This is a schematic diagram illustrating the interaction of an image-based positioning device disclosed in an embodiment of this application. Figure 1As shown, the image-based positioning is applied to multiple electronic devices, such as a first electronic device 10 and a second electronic device 11. The first electronic device 10 is equipped with image capture, direction acquisition, and rotation angle recording functions, while the second electronic device 11 is equipped with image display and direction acquisition functions. The first electronic device 10 and the second electronic device 11 also have positioning and data transmission functions.

[0091] Specifically, the first electronic device 10 and the second electronic device 11 can be mobile phones, tablets, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc.

[0092] Furthermore, the specific product forms of the first electronic device 10 and the second electronic device 11 can be the same or different. For example, the first electronic device 10 can be a mobile phone, and the second electronic device 11 can also be a mobile phone, or the first electronic device 10 can be a tablet computer or a computer, and the second electronic device 11 can be a mobile phone. This application embodiment does not impose specific limitations here.

[0093] In some embodiments of this application, the image-based positioning method acquires a panoramic image through the first electronic device 10; sends the panoramic image to a second electronic device 11 through the first electronic device 10; and displays a portion of the panoramic image through the second electronic device 11.

[0094] like Figure 2 The diagram shown is a flowchart of an image-based localization method provided in an embodiment of this application, applied in a first electronic device 10. Specifically, the image-based localization method includes:

[0095] S201, Obtain the panoramic image and the image orientation information corresponding to the panoramic image.

[0096] In one embodiment of this application, obtaining the panoramic image and the image orientation information corresponding to the panoramic image includes:

[0097] The first electronic device 10 uses a camera (e.g., a camera or other device capable of capturing images) to capture multiple scene images starting from an initial direction, rotating clockwise or counterclockwise, wherein the sum of the viewing angles of the multiple scene images on the horizontal plane is greater than or equal to 360 degrees.

[0098] Record the initial orientation information, rotation orientation information, and rotation angle of each scene image in the multiple scene images acquired;

[0099] A 360-degree panoramic image is generated based on the multiple scene images and the rotation angle of each scene image. The image orientation information includes the initial orientation information and the rotation orientation information.

[0100] Specifically, in response to a user's shooting operation, the first electronic device 10, starting from an initial direction, captures multiple scene images via its camera (e.g., a TOF (Time-Offlight) camera) in a clockwise or counter-clockwise rotation direction. The sum of the viewing angles of these multiple scene images on the horizontal plane is greater than or equal to 360 degrees. The initial direction can be any direction, such as due north or due east, or 0 degrees can represent due north, 90 degrees due east, 180 degrees due south, 270 degrees due west, and so on. For example, if the viewing angle of the first electronic device 10's camera is 180 degrees, at least two scene images are captured; if the viewing angle is 90 degrees, at least four scene images are captured, and so on.

[0101] In one embodiment of this application, the initial orientation information can be acquired through the magnetic sensor of the first electronic device 10, and the rotation orientation information can be acquired through the gyroscope of the first electronic device 10. The orientation corresponding to each scene image can be determined using the initial orientation information, the rotation orientation information, and the rotation angle of each scene image. For example, if the initial orientation information is 0 degrees and the rotation orientation information is clockwise, the orientation corresponding to the first scene image is 0 degrees; if the rotation angle of the second scene image is 90 degrees, the orientation corresponding to the second scene image is 90 degrees; if the rotation angle of the third scene image is 90 degrees, the orientation corresponding to the third scene image is 180 degrees; and if the rotation angle of the fourth scene image is 90 degrees, the orientation corresponding to the fourth scene image is 270 degrees.

[0102] Based on the multiple scene images and the rotation angle of each scene image, different mapping models can be selected to generate different panoramic images. For example, the multiple scene images can be sequentially projected onto a cylindrical surface using cylindrical projection (perspective projection), and then a cylindrical panoramic image can be generated using a cylindrical mapping model. Alternatively, a spherical panoramic image can be generated using a spherical mapping model.

[0103] In embodiments of this application, panoramic images can be optimized through image warping. For example, image warping can be processed in parallel by the GPU of the first electronic device 10. Furthermore, the panoramic image can be jumped using the Reinhard method to reduce the correlation between the three primary colors in the panoramic image. The aspect ratio of the panoramic image is also corrected through image seam processing.

[0104] In another embodiment of this case, obtaining the panoramic image and the image orientation information corresponding to the panoramic image includes:

[0105] The first electronic device 10 acquires multiple scene images in the horizontal direction, and the sum of the viewing angles of the multiple scene images on the horizontal plane is greater than or equal to 360 degrees.

[0106] Record the orientation information of each scene image in the plurality of scene images;

[0107] A 360-degree panoramic image is generated based on the multiple scene images and their orientation information, wherein the image orientation information includes the orientation information of the image at each angle in the panoramic image.

[0108] The panoramic image also includes the location information of the first electronic device 10. The location information may include GPS information, BeiDou positioning information, etc.

[0109] Furthermore, in other embodiments of this application, the panoramic image further includes background location information corresponding to the panoramic image. Specifically, the first electronic device 10 may include an image library or be connected to an external database (e.g., via a network connection), match images corresponding to the panoramic image from the image library, and determine the location information of the image matching the panoramic image as the background location information. The image library may be an image library that includes various types of images and image information.

[0110] For example, if a panoramic image includes building B at location A, an image matching the panoramic image can be found in an image library. If the image matching the panoramic image includes building B at location A, the location information (such as the latitude and longitude of location A) corresponding to the image matching the panoramic image can be determined as the background location information of the panoramic image.

[0111] like Figure 3 The image shown is a panoramic image provided in an embodiment of this application. Here, 0 degrees represents due north, 90 degrees represents due east, 180 degrees represents due south, and 270 degrees represents due west. When the first electronic device acquires a scene image including trees in the east direction, it records the initial direction information as due east (90 degrees). By recording the rotation direction information and rotation angle, scene images including doors in the south direction, streetlights in the west direction, and towers in the north direction can be acquired respectively. A cylindrical panoramic image is then generated based on multiple scene images and the rotation angle of each scene image.

[0112] In one embodiment of this application, the panoramic image and the image orientation information can be stored using a preset photo data structure. This preset photo data structure includes an image header (used to store data related to the panoramic image source user, panoramic image size, etc.), panoramic image data, initial orientation information, and rotation orientation information. The initial orientation information can occupy 8 bits of storage space, with a certain direction (e.g., due north) designated as 0 degrees, for a total of 360 degrees. The rotation orientation information can occupy 4 bits of storage space, where 0 represents clockwise rotation and 1 represents counter-clockwise rotation. Figure 4 The diagram shown is a preset photo data structure provided in an embodiment of this application.

[0113] S202, the panoramic image and the image orientation information are sent to the second electronic device 11, so that the user corresponding to the second electronic device 11 can locate the position of the first electronic device 10 through the panoramic image and the image orientation information.

[0114] In one specific embodiment, sending the panoramic image and the image orientation information to the second electronic device 11 includes:

[0115] Send the panoramic image based on the first-person perspective and the image orientation information to the second electronic device 11; or

[0116] The panoramic image based on a third-person perspective and the image orientation information are sent to the second electronic device 11.

[0117] In one embodiment of this application, before sending the panoramic image based on a third-person perspective and the image orientation information to the second electronic device 11, the image-based positioning method further includes:

[0118] Get user image;

[0119] The user image is rendered onto the panoramic image to obtain the panoramic image based on a third-person perspective.

[0120] Specifically, a user depth image can be acquired via the TOF (Time of Flight) camera of the first electronic device 10, or a user depth image can be read from the first electronic device 10. A foreground person image is extracted from the user depth image based on edge information, and the extracted foreground person image is determined as the user image. Optionally, the user image can also be schematic (a preset user image). The user image can be rendered as foreground in the panoramic image, placing the user image directly in front of the virtual lens corresponding to the panoramic image. Understandably, the position of the user image will change according to the viewing angle of the virtual lens; when the viewing angle of the virtual lens changes, the user image is directly in front of the virtual lens corresponding to the panoramic image. This allows the user of the second electronic device 11 to find the user of the first electronic device 10 based on the position of the user of the first electronic device 10 in the panoramic image.

[0121] The user image can be rendered at a preset angle within the panoramic image. This preset angle can be 300 degrees, and a 300-degree field of view of the user image can be captured using a virtual lens. In other words, the user image can be rotated 300 degrees within the panoramic image to capture user images from different angles using a virtual lens.

[0122] In one specific embodiment, when sending the panoramic image and the image orientation information to the second electronic device 11, the image-based positioning method further includes:

[0123] Obtain the location information and preset distance value of the first electronic device 10;

[0124] Receive the location information of the second electronic device 11;

[0125] The distance between the first electronic device 10 and the second electronic device 11 is determined based on the location information of the first electronic device 10 and the location information of the second electronic device 11.

[0126] If the distance value is less than the preset distance value, a navigation path is determined based on the location information of the first electronic device 10 and the second electronic device 11. A first relative direction information between the first electronic device 10 and the second electronic device 11 is determined based on the navigation path, and the first relative direction information is sent to the second electronic device 11; or

[0127] If the distance value is greater than or equal to the preset distance value, the second relative direction information between the first electronic device 10 and the second electronic device 11 is determined based on the location information of the first electronic device 10 and the location information of the second electronic device 11, and the second relative direction information is sent to the second electronic device 11.

[0128] The preset distance value may include 3 kilometers, 300 meters, 100 meters, etc. The initial direction of travel of the second electronic device 11 or the final direction of travel of the second electronic device 11 in the navigation path can be determined as the first relative direction information.

[0129] In one specific embodiment, sending the panoramic image and the image orientation information to the second electronic device 11 includes:

[0130] Based on the relative orientation information of the first electronic device 10 and the second electronic device 11 and the image orientation information, the panoramic image is cropped to obtain the cropped image;

[0131] The captured image is sent to the second electronic device 11.

[0132] The image corresponding to the relative direction information can be obtained through the viewing direction of the virtual lens corresponding to the panoramic image, and the viewing direction of the virtual lens corresponding to the panoramic image is consistent with the relative direction information.

[0133] This application can send the panoramic image and the image orientation information to a second electronic device 11, enabling the user corresponding to the second electronic device 11 to locate the position of the first electronic device 10 using the panoramic image and the image orientation information, thereby determining the position of the user holding the first electronic device 10. This improves positioning efficiency and accuracy.

[0134] like Figure 5 The diagram shown is a flowchart of an image-based localization method provided in an embodiment of this application, applied in a second electronic device 11. Specifically, the image-based localization method includes:

[0135] S501, receive the panoramic image and the image direction information corresponding to the panoramic image sent by the first electronic device 10.

[0136] Specifically, when the user corresponding to the second electronic device 11 needs to go to the location of the first electronic device 10, the second electronic device 11 receives the panoramic image sent by the first electronic device 10 and the image direction information corresponding to the panoramic image, so as to determine the specific location of the first electronic device 10 through the panoramic image.

[0137] The panoramic image may also include the location information of the first electronic device 10.

[0138] S502, Obtain the device orientation of the second electronic device 11.

[0139] In one specific embodiment, the device orientation for acquiring the second electronic device 11 includes:

[0140] Obtain the orientation information of the second electronic device 11, and determine the orientation information of the second electronic device 11 as the device orientation; or

[0141] Obtain the relative direction information of the second electronic device 11 pointing to the first electronic device 10, and determine the relative direction information as the device direction.

[0142] Specifically, the signal strength corresponding to the location information of the second electronic device 11 can be detected. If the signal strength corresponding to the location information of the second electronic device 11 is lower than a preset signal strength or there is no signal corresponding to the location information of the second electronic device 11, the second electronic device 11 collects direction information and determines the direction information of the second electronic device 11 as the device direction. The signal strength corresponding to the location information of the second electronic device 11 can be the GPS signal strength of the second electronic device 11.

[0143] In one specific embodiment, obtaining the relative direction information between the second electronic device 11 and the first electronic device 10 includes:

[0144] Receive the relative direction information from the first electronic device 10; or

[0145] Receive the user-selected viewpoint and determine the user-selected viewpoint as the relative direction information; or

[0146] The location information of the first electronic device 10 and the location information of the second electronic device 11 are obtained, and the relative direction information between the first electronic device 10 and the second electronic device 11 is determined based on the location information of the first electronic device 10 and the second electronic device 11.

[0147] In one specific embodiment, determining the relative direction information between the first electronic device 10 and the second electronic device 11 based on the position information of the first electronic device 10 and the second electronic device 11 includes:

[0148] The navigation path is determined based on the location information of the first electronic device 10 and the location information of the second electronic device 11;

[0149] The relative direction information between the first electronic device 10 and the second electronic device 11 is determined based on the navigation path.

[0150] Specifically, the initial direction of travel of the second electronic device 11 or the final direction of travel of the second electronic device 11 in the navigation path can be determined as the relative direction information.

[0151] Further, determining the relative direction information between the first electronic device 10 and the second electronic device 11 based on the position information of the first electronic device 10 and the second electronic device 11 includes:

[0152] Get the preset distance value;

[0153] The distance between the first electronic device 10 and the second electronic device 11 is determined based on the location information of the first electronic device 10 and the location information of the second electronic device 11.

[0154] If the distance value is less than the preset distance value, the initial travel direction of the second electronic device 11 or the final travel direction of the second electronic device 11 in the navigation path is determined as the relative direction information;

[0155] If the distance value is greater than or equal to the preset distance value, the user-selected viewing direction information is received, and the user-selected viewing direction information is determined as the relative direction information.

[0156] Specifically, the initial direction of travel is the direction in which travel begins on the navigation path, and the final direction of travel is the direction in which travel approaches the final destination on the navigation path. For example... Figure 6A The initial direction of travel for the navigation path is 180 degrees (i.e., the direction of travel for vehicles on solid lines), and the final direction of travel for the navigation path is 90 degrees (i.e., the direction of travel for vehicles on dashed lines).

[0157] In another embodiment, if the distance value is greater than or equal to the preset distance value, the relative direction information of the second electronic device 11 pointing to the first electronic device 10 is obtained, and the relative direction information is determined as the device direction.

[0158] S503, display a portion of the panoramic image based on the device orientation and the image orientation information, so that the user corresponding to the second electronic device 11 can locate the position of the first electronic device 10 based on the portion of the image.

[0159] This application can obtain a portion of the image corresponding to the device orientation from the panoramic image based on the device orientation and the image orientation information, and display the portion of the image. The portion of the image corresponding to the device orientation can be obtained through the viewing angle of the virtual lens corresponding to the panoramic image, and the viewing angle of the virtual lens corresponding to the panoramic image can be consistent with the device orientation. The portion of the image corresponding to the device orientation can be obtained by cropping the panoramic image. The user corresponding to the second electronic device 11 can find the corresponding location, i.e., the location of the user corresponding to the first electronic device 10, through background elements (such as buildings, trees, etc.) in the portion of the image, thereby determining the location of the user holding the first electronic device 10. This application can improve positioning efficiency and accuracy by displaying a portion of the image from the panoramic image.

[0160] In one specific embodiment, displaying a portion of the panoramic image based on the device orientation and the image orientation information includes:

[0161] Display a portion of the panoramic image from a first-person or third-person perspective based on the device orientation and the image orientation information.

[0162] In one specific embodiment, the partial image further includes a user image rendered onto the panoramic image. The user image can be rendered into a third-person perspective panoramic image, and the partial image including the user image is displayed according to the device orientation and the image orientation information. The user corresponding to the second electronic device 11 can locate the position of the user corresponding to the first electronic device 10 through the positional relationship between the background in the partial image and the user image.

[0163] In one specific embodiment, displaying a portion of the panoramic image based on the device orientation and the image orientation information includes:

[0164] Update the device orientation;

[0165] Based on the updated device orientation and the image orientation information, a portion of the panoramic image is displayed in real time.

[0166] After displaying a portion of the panoramic image based on the device orientation and the image orientation information, the image-based positioning method further includes:

[0167] Receive user perspective change commands;

[0168] The display view of the panoramic image is switched according to the viewpoint switching operation to display partial images of the panoramic image from different viewpoints.

[0169] In another embodiment, when displaying a portion of the panoramic image based on the device orientation and the image orientation information, a navigation path from the second electronic device 11 to the first electronic device 10 is displayed, so that the user corresponding to the second electronic device 11 can locate the position of the user corresponding to the first electronic device 10 based on the portion of the image and the navigation path.

[0170] In another embodiment, when displaying a portion of the panoramic image based on the device orientation and the image orientation information, the location information of the second electronic device and the first electronic device is displayed, so that the user corresponding to the second electronic device can locate the location of the user corresponding to the first electronic device based on the portion of the image and the location information.

[0171] Figure 6 shows a panoramic image displayed by the second electronic device 11 provided in this embodiment of the application. Figure 6A The diagram shows the navigation path from the vehicle (solid line vehicle) carrying the user corresponding to the second electronic device 11 to the user (circled in the diagram) corresponding to the first electronic device 10. The user corresponding to the second electronic device 11 can travel to the location of the dashed line vehicle according to the navigation path. At the location of the dashed line vehicle, according to... Figure 6B The panoramic image shown depicts a portion of the image locating the user's position corresponding to the first electronic device 10. Among these, Figure 6A The first electronic device 10 corresponds to a tower, a tree, a door, and a street lamp located to the user's due north, due east, due south, and due west directions, respectively. The panoramic image acquired by the first electronic device 10 includes the tower, tree, door, street lamp, and their corresponding image orientation information. The second electronic device 11 acquires the image with a device orientation of 135 degrees. The second electronic device 11 displays a portion of the panoramic image corresponding to the 135-degree viewpoint, which includes the tree. Figure 6B The panoramic image contains a user image, which allows the user corresponding to the second electronic device 11 to quickly locate the position of the user corresponding to the first electronic device 10 based on a partial image and / or the user image.

[0172] like Figure 7 The diagram shown is a schematic representation of a portion of a panoramic image displayed by the second electronic device 11 provided in this embodiment of the application. The device orientation of the second electronic device 11 is 270 degrees, and the second electronic device 11 displays a 90-degree portion of the image, that is, it displays a portion of the panoramic image between 225 degrees and 315 degrees.

[0173] like Figure 8 The diagram shown is a flowchart of an image-based localization method provided in an embodiment of this application, which is applied in electronic devices, for example... Figure 1The first electronic device 10 and / or the second electronic device 11 shown. Specifically, the image-based localization method includes:

[0174] S801, the first electronic device 10 acquires the panoramic image and the image direction information corresponding to the panoramic image.

[0175] S802, the first electronic device 10 sends the panoramic image and the image direction information to the second electronic device 11.

[0176] S803, the second electronic device 11 obtains the device orientation of the second electronic device 11.

[0177] S804, the second electronic device 11 displays a portion of the panoramic image based on the device orientation and the image orientation information, so that the user corresponding to the second electronic device 11 can locate the position of the first electronic device 10 based on the portion of the image.

[0178] In another embodiment, the first electronic device 10 sends the panoramic image and the image orientation information to the second electronic device 11, including:

[0179] The first electronic device 10 acquires the device orientation of the second electronic device 11;

[0180] The first electronic device 10 extracts a portion of the image from the panoramic image based on the device orientation and the image orientation information, thereby obtaining the extracted portion of the image.

[0181] The first electronic device 10 sends the captured partial image to the second electronic device 11, so that the second electronic device 11 displays the captured partial image.

[0182] The second electronic device 11 displays a portion of the panoramic image based on the device orientation and the image orientation information, so that the user corresponding to the second electronic device 11 can locate the position of the first electronic device 10 based on the portion of the image, thereby determining the position of the user holding the first electronic device 10.

[0183] like Figure 9 The diagram shown is a schematic representation of an electronic device provided in an embodiment of this application. The electronic device 90 includes a memory 901, a processor 902, and computer-readable instructions stored in the memory 901 and executable on the processor 902, such as an image-based positioning program. When the processor 902 executes the computer-readable instructions, it implements the steps described in the image-based positioning embodiment.

[0184] Those skilled in the art will understand that the illustrations Figure 9This is merely an example of electronic device 90 and does not constitute a limitation on electronic device 90. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 90 may also include input / output devices, network access devices, buses, etc.

[0185] The processor 902 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor 902 may be any conventional processor. The processor 902 is the control center of the electronic device 90, connecting all parts of the electronic device 90 through various interfaces and lines.

[0186] The memory 901 can be used to store the computer-readable instructions. The processor 902 implements various functions of the electronic device 90 by running or executing the computer-readable instructions or modules stored in the memory 901 and calling the data stored in the memory 901. The memory 901 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 90, etc. In addition, the memory 901 may include a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, read-only memory (ROM), random access memory (RAM), or other non-volatile / volatile storage devices.

[0187] If the modules integrated in the electronic device 90 are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, they can implement the steps of the various method embodiments described above. The computer-readable instructions include computer-readable instruction code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying the computer-readable instruction code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), etc.

[0188] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the image-based positioning method in the above embodiment.

[0189] This embodiment also provides a computer program product that, when run on an electronic device, causes the electronic device to execute the aforementioned related method steps to achieve the image-based positioning method in the above embodiment.

[0190] This embodiment also provides a chip coupled to a memory in an electronic device. The chip is used to control the electronic device to execute the above-described related method steps to realize the image-based positioning method in the above embodiment.

[0191] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the image-based positioning method in the above-described method embodiments.

[0192] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0195] The unit described as a separate component may or may not be physically separate. The component shown as a unit can be one physical unit or multiple physical units, that is, it can be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0196] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0197] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0198] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image-based positioning method applied in a first electronic device, characterized in that, The image-based positioning method comprises: acquiring a panoramic image and image direction information corresponding to the panoramic image; sending the panoramic image and the image direction information to a second electronic device; the sending of the panoramic image and the image direction information to the second electronic device comprises: performing interception on the panoramic image according to the acquired relative direction information between the first electronic device and the second electronic device and the image direction information, to obtain an intercepted image; sending the intercepted image to the second electronic device, so that a user corresponding to the second electronic device locates the position of the first electronic device through the intercepted image.

2. The image-based positioning method of claim 1, wherein, The acquisition of the panoramic image and the image direction information corresponding to the panoramic image comprises: starting from an initial direction, collecting a plurality of scene images in a clockwise rotation direction or a counterclockwise rotation direction through a camera of the first electronic device, the sum of the viewing angles of the plurality of scene images on a horizontal plane being greater than or equal to 360 degrees; recording initial direction information, rotation direction information, and a rotation angle of each of the plurality of scene images; generating a panoramic image according to the plurality of scene images and the rotation angle of each of the plurality of scene images, the image direction information comprising the initial direction information and the rotation direction information.

3. The image-based positioning method of claim 1, wherein, The acquisition of the panoramic image and the image direction information corresponding to the panoramic image comprises: collecting a plurality of scene images in a horizontal direction through the first electronic device, the sum of the viewing angles of the plurality of scene images on a horizontal plane being greater than or equal to 360 degrees; recording direction information of each of the plurality of scene images; generating a panoramic image according to the plurality of scene images and the direction information of the plurality of scene images, the image direction information corresponding to the panoramic image comprising the direction information of the image at each angle in the panoramic image.

4. The image-based positioning method of claim 1, wherein, The sending of the panoramic image and the image direction information to the second electronic device comprises: sending a panoramic image based on a first-person perspective and the image direction information to the second electronic device; or sending a panoramic image based on a third-person perspective and the image direction information to the second electronic device.

5. The image-based positioning method of claim 4, wherein, Before the sending of the panoramic image based on the third-person perspective and the image direction information to the second electronic device, the image-based positioning method further comprises: acquiring a user image; rendering the user image to the panoramic image to obtain the panoramic image based on the third-person perspective.

6. The image-based positioning method of claim 1, wherein, The panoramic image further comprises position information of the first electronic device.

7. The image-based positioning method of claim 6, wherein, When the panoramic image and the image direction information are sent to the second electronic device, the image-based positioning method further comprises: acquiring position information of the first electronic device and a preset distance value; receiving position information of the second electronic device; determining a distance value between the first electronic device and the second electronic device according to the position information of the first electronic device and the position information of the second electronic device; If the distance value is less than the preset distance value, a navigation path is determined according to the position information of the first electronic device and the position information of the second electronic device, a first relative direction information of the first electronic device and the second electronic device is determined according to the navigation path, and the first relative direction information is sent to the second electronic device; or If the distance value is greater than or equal to the preset distance value, a second relative direction information of the first electronic device and the second electronic device is determined according to the position information of the first electronic device and the position information of the second electronic device, and the second relative direction information is sent to the second electronic device.

8. An image-based positioning method applied in a second electronic device, comprising: The image-based positioning method comprises: receiving a panoramic image and image direction information corresponding to the panoramic image sent by a first electronic device; obtaining a device direction of a second electronic device; displaying part of the panoramic image according to the device direction and the image direction information, and positioning a position of the first electronic device based on the part of the panoramic image.

9. The image-based positioning method of claim 8, wherein, The obtaining of the device direction of the second electronic device comprises: obtaining direction information of the second electronic device, and determining the direction information of the second electronic device as the device direction; or obtaining relative direction information of the second electronic device pointing to the first electronic device, and determining the relative direction information as the device direction.

10. The image-based positioning method of claim 9, wherein, The obtaining of the relative direction information of the second electronic device and the first electronic device comprises: receiving the relative direction information from the first electronic device; or receiving a user-selected view angle, and determining the user-selected view angle as the relative direction information; or obtaining position information of the first electronic device and position information of the second electronic device, and determining relative direction information of the first electronic device and the second electronic device according to the position information of the first electronic device and the position information of the second electronic device.

11. The image-based positioning method of claim 10, wherein, The determining of the relative direction information of the first electronic device and the second electronic device according to the position information of the first electronic device and the position information of the second electronic device comprises: determining a navigation path according to the position information of the first electronic device and the position information of the second electronic device; determining relative direction information of the first electronic device and the second electronic device according to the navigation path.

12. The image-based positioning method of claim 11, wherein, The determining of the relative direction information of the first electronic device and the second electronic device according to the position information of the first electronic device and the position information of the second electronic device comprises: obtaining a preset distance value; determining a distance value of the first electronic device and the second electronic device according to the position information of the first electronic device and the position information of the second electronic device; if the distance value is less than the preset distance value, determining an initial travel direction of the second electronic device or a terminal travel direction of the second electronic device in the navigation path as the relative direction information; if the distance value is greater than or equal to the preset distance value, receiving user-selected view angle direction information, and determining the user-selected view angle direction information as the relative direction information.

13. The image-based positioning method of claim 12, wherein, If the distance value is greater than or equal to the preset distance value, the relative direction information of the second electronic device pointing to the first electronic device is obtained, and the relative direction information is determined as the device direction.

14. The image-based positioning method of claim 8, wherein, The displaying the partial image in the panoramic image according to the device direction and the image direction information includes: The displaying the partial image in the panoramic image according to the device direction and the image direction information includes:

15. The image-based positioning method of claim 8, wherein, The partial image further includes a user image.

16. The image-based positioning method of claim 8, wherein, The displaying the partial image in the panoramic image according to the device direction and the image direction information includes: The device direction is updated; The partial image in the panoramic image is displayed in real time according to the updated device direction and the image direction information.

17. The image-based positioning method of claim 8, wherein, After the displaying the partial image in the panoramic image according to the device direction and the image direction information, the image-based positioning method further includes: A view conversion operation of a user is received; The display view angle of the panoramic image is switched according to the view conversion operation, so as to display a partial image of a different view angle in the panoramic image.

18. The image-based positioning method of claim 8, wherein, When the partial image in the panoramic image is displayed according to the device direction and the image direction information, the image-based positioning method further includes: A navigation path of the second electronic device pointing to the first electronic device is displayed, and the position of the first electronic device is located based on the partial image and the navigation path.

19. The image-based positioning method of claim 8, wherein, When the partial image in the panoramic image is displayed according to the device direction and the image direction information, the image-based positioning method further includes: Position information of the second electronic device and the first electronic device is displayed, and the position of the first electronic device is located based on the partial image and the position information.

20. An image-based positioning method, applied in a first electronic device and a second electronic device, characterized in that, The image-based positioning method includes: A first electronic device acquires a panoramic image and image direction information corresponding to the panoramic image; The first electronic device acquires a device direction of a second electronic device, so as to determine relative direction information of the first electronic device and the second electronic device; The first electronic device performs interception on the panoramic image according to the relative direction information and the image direction information, so as to obtain an intercepted image; The first electronic device sends the intercepted image to the second electronic device; The second electronic device locates the position of the first electronic device based on the intercepted image.

21. An electronic device, comprising: An electronic device includes a processor and a memory; the memory is used to store instructions; the processor is used to invoke the instructions in the memory, so that the electronic device performs the image-based positioning method in any one of claims 1 to 7, or the image-based positioning method in any one of claims 8 to 19.

22. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor to implement the image-based positioning method in any one of claims 1 to 7, or the image-based positioning method in any one of claims 8 to 19.

23. A chip coupled with a memory in an electronic device, the chip comprising: The chip is configured to control the electronic device to perform the image-based positioning method of any one of claims 1-7, or any one of claims 8-19.

Images