Positioning method and device and storage medium

Abstract

The invention provides a positioning method and device and a storage medium, and the method comprises the steps: a first cloud server obtains environment information uploaded by a terminal, carries out the environment reconstruction through the environment information, and obtains a map under a global coordinate system, the global coordinate system is a position coordinate system of map data after environment reconstruction; in response to starting of the terminal, the first cloud server obtains a first coordinate of the terminal in a local coordinate system every preset time interval, and determines a target coordinate difference value corresponding to the first coordinate in a global coordinate system; and the terminal receives the target coordinate difference value sent by the first cloud server, and corrects the first coordinate based on the target coordinate difference value to obtain a second coordinate positioned by the terminal. According to the method and the device, deviation accumulation or positioning loss of the positioning position is avoided, the positioning precision is improved, the situations of mode crossing and drifting in the virtual picture are further avoided, a user does not need to position through a computer backpack, and the user experience is improved.

Description

Technical Field

[0001] This disclosure relates to the field of virtual reality technology, and in particular to a positioning method, device and storage medium. Background Technology

[0002] In the field of large-scale spatial entertainment experiences (LBE), it is necessary to locate virtual devices (such as VR (Virtual Reality) devices) and display rendered images on the virtual devices based on the obtained location results, thereby achieving... Immersive virtual reality interaction.

[0003] In existing technologies, virtual reality interaction can be achieved through backpack-style solutions and lightweight all-in-one device local solutions. The backpack-style solution requires the user to carry a laptop backpack for local positioning, with the positioning computing resources residing in the backpack. However, the laptop backpack weighs approximately 10 kilograms, resulting in a poor user experience. The lightweight all-in-one device local solution utilizes the device's built-in positioning system, but prolonged positioning can lead to accumulated deviations (e.g., a 1-meter actual movement may only show 0.8 meters on the rendered screen) or positioning loss, reducing positioning accuracy and causing clipping and drifting in the virtual environment, resulting in a poor user experience. Summary of the Invention

[0004] The positioning method, apparatus, and storage medium disclosed herein aim to solve the technical problems of clipping and drifting in virtual images, low positioning accuracy, and poor user experience in the prior art.

[0005] According to a first aspect of the present disclosure, a positioning method is provided, the method comprising: The first cloud server obtains the environmental information uploaded by the terminal and reconstructs the environment using the environmental information to obtain a map in a global coordinate system. The environmental information is used to represent environmental information, and the global coordinate system is the location coordinate system of the map data after environmental reconstruction. In response to the terminal starting up, the first cloud server obtains the first coordinates of the terminal in the local coordinate system at preset time intervals, and determines the target coordinate difference corresponding to the first coordinates in the global coordinate system; The terminal receives the target coordinate difference sent by the first cloud server, and corrects the first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning.

[0006] Optionally, in this embodiment of the disclosure, determining the target coordinate difference corresponding to the first coordinate in the global coordinate system includes: Obtain the origin coordinates of the terminal in the local coordinate system, and determine the first coordinate difference of the origin coordinates in the global coordinate system, wherein the origin coordinates are the coordinates of the default position when the terminal starts up; Obtain the movement time of the terminal; If the movement time is less than or equal to the time threshold, then the first coordinate difference is determined as the target coordinate difference corresponding to the first coordinate; If the movement time is greater than the time threshold, then the fisheye image of the terminal is acquired, and the target coordinate difference corresponding to the first coordinate in the global coordinate system is determined based on the fisheye image.

[0007] Optionally, in this embodiment of the disclosure, determining the first coordinate difference of the origin coordinates in the global coordinate system includes: Determine the second coordinate of the origin in the global coordinate system; The difference between the origin coordinates and the second coordinates is determined as the first coordinate difference of the origin coordinates in the global coordinate system.

[0008] Optionally, in this embodiment of the disclosure, determining the target coordinate difference corresponding to the first coordinate in the global coordinate system based on the fisheye image includes: Based on the fisheye image, a conversion coefficient is determined, wherein the conversion coefficient is used to convert the pixels in the fisheye image onto a preset plane; Based on the transformation coefficient, the third coordinate of the terminal in the global coordinate system is determined; The difference between the first coordinate and the third coordinate is determined as the target coordinate difference corresponding to the first coordinate.

[0009] Optionally, in this embodiment of the disclosure, determining the conversion coefficient based on the fisheye image includes: Using the intrinsic parameters of the camera corresponding to the fisheye image, the pixel points in the fisheye image are projected onto the coordinates of the target point on the normalized plane. Based on the target point coordinates, determine the first radius and the corresponding incident angle of the center point of the normalized plane, wherein the first radius is the distance between the target point and the center point of the normalized plane; Based on the incident angle, the second radius after distortion removal on the normalized plane is determined using a fisheye model; The ratio of the second radius to the first radius is determined as the conversion coefficient.

[0010] Optionally, in this embodiment of the disclosure, determining the third coordinate of the terminal in the global coordinate system based on the transformation coefficient includes: Based on the conversion coefficient, the fisheye image is converted into a preset planar image; Extract feature points from the preset planar image; The third coordinate of the terminal in the global coordinate system is determined by comparing the feature points with the pixels of the environmental image in the first cloud server.

[0011] Optionally, the step of correcting the first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning includes: performing an addition operation on the difference between the first coordinate and the target coordinate, and determining the obtained addition calculation result as the second coordinate of the terminal positioning.

[0012] Optionally, the method further includes: The terminal synchronizes the second coordinates to the second cloud server; The second cloud server renders the image based on the second coordinates and sends the rendered image to the terminal. The terminal sends out the rendered screen.

[0013] According to a second aspect of the present disclosure, a positioning device is provided, the device comprising: The reconstruction module is used by the first cloud server to obtain environmental information uploaded by the terminal and to reconstruct the environment using the environmental information to obtain a map in a global coordinate system. The environmental information is used to represent environmental information, and the global coordinate system is the location coordinate system of the map data after environmental reconstruction. A determination module is configured to, in response to the terminal starting up, have the first cloud server acquire the first coordinates of the terminal in the local coordinate system at preset time intervals, and determine the target coordinate difference corresponding to the first coordinates in the global coordinate system; The correction module is used to receive the target coordinate difference sent by the first cloud server and correct the first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method described in the first aspect above.

[0015] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, the computer storage medium storing computer-executable instructions; the computer-executable instructions, when executed by a processor, are capable of implementing the method described in the first aspect above.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0017] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: This disclosure proposes a positioning method, apparatus, and storage medium. The method includes a first cloud server acquiring environmental information uploaded by a terminal and reconstructing the environment using the environmental information to obtain a map in a global coordinate system. The environmental information represents the environment, and the global coordinate system is the positional coordinate system of the reconstructed map data. In response to terminal startup, the first cloud server acquires the terminal's first coordinates in its local coordinate system at preset time intervals and determines the target coordinate difference corresponding to the first coordinates in the global coordinate system. The terminal receives the target coordinate difference sent by the first cloud server and corrects the first coordinates based on the target coordinate difference to obtain the terminal's second positioning coordinates. Therefore, this disclosure allows the first cloud server to determine the target coordinate difference corresponding to the terminal in the global coordinate system at preset time intervals, enabling the terminal to correct its first coordinates based on the target coordinate difference to obtain the terminal's second positioning coordinates. This avoids accumulated positioning deviations or positioning loss, improves positioning accuracy, and prevents clipping and drifting in virtual environments. Furthermore, it eliminates the need for users to locate the device using a computer backpack, enhancing the user experience.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0019] Figure 1 This is a flowchart illustrating a positioning method according to some embodiments of the present disclosure; Figure 2 This is a schematic diagram illustrating a walking path according to some embodiments of the present disclosure; Figure 3 This is a flowchart illustrating a positioning method according to some embodiments of the present disclosure; Figure 4 This is a schematic diagram illustrating a first coordinate difference according to some embodiments of the present disclosure; Figure 5 This is a schematic diagram illustrating pixel transformation according to some embodiments of the present disclosure; Figure 6This is a flowchart illustrating a positioning method according to some embodiments of the present disclosure; Figure 7 This is a schematic diagram of the structure of a positioning device according to some embodiments of the present disclosure; Figure 8 This is a block diagram illustrating an electronic device suitable for performing a positioning method according to some embodiments of the present disclosure. Detailed Implementation

[0020] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0021] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0022] Figure 1 This is a flowchart illustrating a positioning method according to some embodiments of the present disclosure, such as... Figure 1 As shown, the method may include the following steps: Step 101: The first cloud server obtains the environmental information uploaded by the terminal, and reconstructs the environment using the environmental information to obtain a map in the global coordinate system.

[0023] It should be noted that the entity executing the above positioning method is a positioning device, which can be implemented by software and / or hardware. In this embodiment, the positioning device can be configured in an electronic device.

[0024] In this example embodiment, the electronic device may include devices such as terminal devices and servers, and this embodiment does not limit the electronic device.

[0025] In this embodiment of the disclosure, the first cloud server can be a cloud positioning server; the terminal can be a head-mounted display device, that is, a head-mounted display device.

[0026] In this embodiment of the disclosure, the aforementioned environmental information can be used to represent environmental information, such as environmental data representing the environment. Furthermore, in this embodiment of the disclosure, a SLAM system can be initiated via a head-mounted display device to scan the surrounding environment, collect depth point cloud, visual features, and inertial measurement data (IMU data) in real time, and upload the scanned environmental data to a first cloud server.

[0027] In this embodiment of the disclosure, before the head-mounted display device scans the surrounding environment, an initial point (user's starting position), an end point (end point of the experience path), and environmental anchor points (such as feature positions of walls and obstacles) are fixed and calibrated within the experience area. A virtual walking path (e.g., ...) is then generated based on these calibration points. Figure 2 (As shown).

[0028] In this embodiment, after the first cloud server obtains the environmental information uploaded by the terminal, it can reconstruct the environment using the environmental information to obtain a map in a global coordinate system. In this embodiment, the global coordinate system refers to the location coordinate system of the map data after environmental reconstruction.

[0029] Specifically, in this embodiment of the disclosure, point cloud reconstruction can be performed using the obtained environmental data to obtain a map in a global coordinate system. For details regarding this part, please refer to the prior art; further elaboration is not provided here.

[0030] Step 102: In response to the terminal startup, the first cloud server obtains the terminal's first coordinates in the local coordinate system at preset time intervals, and determines the target coordinate difference corresponding to the first coordinates in the global coordinate system.

[0031] In this embodiment of the disclosure, in response to terminal startup, that is, when the user turns on the head-mounted display device in the default position, the first cloud server can obtain the first coordinates of the terminal in the local coordinate system at preset time intervals, determine the target coordinate difference corresponding to the first coordinates in the global coordinate system, and then send the target coordinate difference to the terminal. This allows the terminal to correct the first coordinates based on the target coordinate difference and reposition itself, thereby avoiding the accumulation of positioning deviations or positioning loss and improving positioning accuracy. In this embodiment of the disclosure, the preset time can be set as needed, such as 2 seconds.

[0032] Step 103: The terminal receives the target coordinate difference sent by the first cloud server, and corrects the first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning.

[0033] In this embodiment of the disclosure, after the terminal receives the target coordinate difference sent by the first cloud server, it can correct the first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning, thereby freeing up the terminal's computing resources. The user does not need to locate the position through a computer backpack, avoiding the accumulation of positioning deviations or positioning loss, improving positioning accuracy, and thus avoiding clipping and drifting in the virtual screen, thereby improving the user experience.

[0034] In this embodiment of the disclosure, the method for correcting the first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning may include: performing an addition operation on the difference between the first coordinate and the target coordinate, and determining the result of the addition calculation as the second coordinate of the terminal positioning.

[0035] For example, in this embodiment of the disclosure, assuming the first coordinate v1 is (2,2,0) and the target coordinate difference offset is (-3,-1,0), then the second coordinate of the terminal positioning v2 = v1 + offset = (2,2,0) + (-3,-1,0) = (-1,1,0).

[0036] In this embodiment of the disclosure, by periodically determining the target coordinate difference, the terminal corrects its first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning, thereby avoiding the accumulation of positioning deviation or positioning loss, improving positioning accuracy, and thus avoiding clipping and drifting in the virtual screen.

[0037] This disclosure proposes a positioning method, which includes a first cloud server acquiring environmental information uploaded by a terminal and reconstructing the environment using the environmental information to obtain a map in a global coordinate system. The environmental information represents the environment, and the global coordinate system is the positional coordinate system of the reconstructed map data. In response to terminal startup, the first cloud server acquires the terminal's first coordinates in its local coordinate system at preset time intervals and determines the target coordinate difference corresponding to the first coordinates in the global coordinate system. The terminal receives the target coordinate difference sent by the first cloud server and corrects the first coordinates based on the target coordinate difference to obtain the terminal's second positioning coordinates. Therefore, this disclosure allows the first cloud server to determine the target coordinate difference corresponding to the terminal in the global coordinate system at preset time intervals, enabling the terminal to correct its first coordinates based on the target coordinate difference to obtain the terminal's second positioning coordinates. This avoids accumulated positioning deviations or positioning loss, improves positioning accuracy, and prevents clipping and drifting in virtual environments. Furthermore, it eliminates the need for users to locate the device using a computer backpack, enhancing the user experience.

[0038] In this embodiment of the disclosure, as a detailed explanation of step 102, such as Figure 3As shown, it may also include: Step 1021: Obtain the origin coordinates of the terminal in the local coordinate system and determine the first coordinate difference between the origin coordinates and the global coordinate system.

[0039] In this embodiment of the disclosure, the above-mentioned origin coordinates can be the coordinates of the default position when the terminal starts up, that is, the origin of the terminal in the local coordinate system.

[0040] In this embodiment of the disclosure, the method for determining the first coordinate difference between the origin coordinates in the global coordinate system may include the following steps: Step 10212: Determine the second coordinate of the origin in the global coordinate system.

[0041] In this embodiment of the disclosure, the first cloud server acquires environmental data (such as visual features and IMU pose) of the origin coordinates and matches the environmental data with the map point cloud in the cloud to determine the second coordinates of the origin coordinates in the global coordinate system.

[0042] Step 10213: Determine the difference between the origin coordinates and the second coordinate as the first coordinate difference of the origin coordinates in the global coordinate system.

[0043] In this embodiment of the disclosure, after obtaining the origin coordinates and the second coordinates through the above steps, the difference between the origin coordinates and the second coordinates can be determined as the first coordinate difference of the origin coordinates in the global coordinate system.

[0044] Figure 4 This is a schematic diagram of a first coordinate difference proposed in an embodiment of this disclosure. Figure 4 As shown, the initial point of the global coordinate system is (0, 0, 0), the ending point of the global coordinate system is (-15, 1, 0), and the planned path between the initial point and the ending point of the global coordinate system is given. The second coordinate of the head-mounted device A in the global coordinate system corresponding to the origin coordinate of the local coordinate system is (-3, -1, 0). Based on this, the first coordinate difference = (-3, -1, 0) - (0, 0, 0) = (-3, -1, 0).

[0045] Step 1022: Obtain the terminal's movement time.

[0046] In this embodiment of the disclosure, after the first cloud server obtains the first coordinate of the terminal in the local coordinate system at preset time intervals, it can compare it with the first coordinate of the previous preset time. If there is a difference, the movement time of the terminal is accumulated; otherwise, the movement time is reset to zero until the terminal moves at the next preset time, and the accumulation is repeated.

[0047] Step 1023: If the movement time is less than or equal to the time threshold, then the first coordinate difference is determined as the target coordinate difference corresponding to the first coordinate.

[0048] In this embodiment of the disclosure, after obtaining the movement time through the above steps, the target coordinate difference corresponding to the first coordinate in the global coordinate system can be determined based on the movement time.

[0049] In this embodiment of the disclosure, if the movement time is less than or equal to a time threshold, the terminal movement time is relatively short, and the first coordinate difference can be determined as the target coordinate difference corresponding to the first coordinate. The time threshold can be set as needed, such as 2 seconds.

[0050] Step 1024: If the movement time is greater than the time threshold, then acquire the fisheye image of the terminal and determine the target coordinate difference in the global coordinate system corresponding to the first coordinate based on the fisheye image.

[0051] In this embodiment, if the movement time exceeds a time threshold, it indicates that the terminal has been moving continuously for a period of time. In this case, it is necessary to determine the coordinate difference of the terminal in real time to correct the first coordinate and prevent the accumulation of positioning deviations. Based on this, a fisheye image of the terminal can be acquired, and the target coordinate difference corresponding to the first coordinate in the global coordinate system can be determined based on the fisheye image.

[0052] In this embodiment of the disclosure, the fisheye image of the terminal can be obtained through the fisheye camera of the head-mounted display device.

[0053] In this embodiment of the disclosure, the method for determining the target coordinate difference in the global coordinate system based on the fisheye image may include the following steps: Step 10241: Based on the fisheye image, determine the conversion coefficient, wherein the conversion coefficient is used to convert the pixels in the fisheye image to a preset plane.

[0054] In this embodiment of the disclosure, after acquiring a fisheye image, a conversion coefficient can be determined based on the fisheye image. The pixels in the fisheye image are converted to a preset plane using the conversion coefficient. The preset plane can be a normalized plane camera imaging plane or an image plane.

[0055] In this embodiment of the disclosure, the method for determining the conversion coefficient based on a fisheye image may include the following steps: Step 1: Using the intrinsic parameters of the camera corresponding to the fisheye image, project the pixel points in the fisheye image onto the coordinates of the target point on the normalized plane.

[0056] In this embodiment of the disclosure, the aforementioned intrinsic parameters may include intrinsic parameters f (focal_length, focal length), dx (the physical length of a pixel in the X direction on the camera sensor plate) and dy (the physical length of a pixel in the Y direction on the camera sensor plate).

[0057] In this embodiment of the disclosure, for pixels in a fisheye image, the intrinsic parameters f, dx, and dy of the camera can be used to transform them into target coordinates on a normalized plane.

[0058] Figure 5 This is a schematic diagram illustrating a pixel transformation proposed in an embodiment of this disclosure. Figure 5 As shown, pixel p can be projected onto target point e in the normalized plane using the aforementioned intrinsic parameters.

[0059] Step 2: Based on the target point coordinates, determine the first radius and the corresponding incident angle of the center point of the normalized plane, where the first radius is the distance between the target point and the center point of the normalized plane.

[0060] In this embodiment of the disclosure, after determining the target point coordinates through the above steps, the distance between the target point coordinates and the center point of the normalized plane can be determined as the first radius, that is, Figure 5 The distance between the target point coordinate e and the center point O of the normalized plane is determined as the first radius (representing the radius before distortion).

[0061] In this embodiment of the disclosure, the incident angle corresponding to the projection of the pixel onto the normalized plane is determined, such as... Figure 5 The angle θ in the equation.

[0062] Step 3: Based on the incident angle, use the fisheye model to determine the second radius after distortion removal on the normalized plane.

[0063] In this embodiment of the disclosure, after obtaining the incident angle through the above steps, the second radius after distortion removal on the normalized plane can be determined based on the incident angle using a fisheye model.

[0064] In this embodiment of the disclosure, the fisheye model described above can be Kannala-Brandt.

[0065] In this embodiment of the disclosure, the method for determining the second radius after distortion removal in a normalized plane using a fisheye model may include: determining the second radius after distortion removal in a normalized plane using radial distortion coefficients and a fisheye model formula, wherein the fisheye model formula is: r(θ) = k0θ + k1θ 3 + k2θ 5 + k3θ 7 + k4θ 9 r(θ) represents the distorted radius, and k0, k1, k2, k3, and k4 are the radial distortion coefficients.

[0066] Step 4: Determine the ratio of the second radius to the first radius as the conversion coefficient.

[0067] In this embodiment of the disclosure, after determining the conversion coefficient through the above steps, the pixels in the fisheye image can be converted into pixels on a plane using the conversion coefficient, so that the coordinates in the global coordinate system can be determined based on the pixels on the plane, thereby improving the accuracy of positioning.

[0068] Step 10242: Based on the transformation coefficient, determine the third coordinate of the terminal in the global coordinate system.

[0069] In this embodiment of the disclosure, after obtaining the conversion coefficient through the above steps, the third coordinate of the terminal in the global coordinate system can be determined based on the conversion coefficient, so that the target coordinate difference corresponding to the first coordinate can be determined based on the third coordinate.

[0070] In this embodiment of the disclosure, the method for determining the third coordinate of the terminal in the global coordinate system based on the transformation coefficient may include the following steps: Step a: Based on the conversion coefficient, convert the fisheye image into a preset planar image.

[0071] In this embodiment of the disclosure, a preset planar image after fisheye image conversion can be obtained by dividing the position of the pixel on the fisheye image by the conversion coefficient.

[0072] Step b: Extract feature points from the preset planar image.

[0073] In this embodiment of the disclosure, after determining the preset planar image through the above steps, feature points in the preset planar image can be extracted. Specifically, in this embodiment of the disclosure, feature points in the preset planar image, such as a checkerboard pattern, can be extracted using existing technologies.

[0074] Step c: Based on the comparison between the feature points and the pixels of the environmental image in the first cloud server, determine the third coordinate of the terminal in the global coordinate system.

[0075] In this embodiment of the disclosure, after obtaining the feature points through the above steps, the feature points can be compared with the pixels of the environmental image in the first cloud server to determine the third coordinate of the terminal in the global coordinate system, so as to determine the target coordinate difference corresponding to the first coordinate through the third coordinate.

[0076] In this embodiment of the disclosure, existing technologies can be used to compare feature points with the pixels of the environmental image in the first cloud server to determine the third coordinate of the terminal in the global coordinate system, such as the PnP (Perspective-n-Point) algorithm.

[0077] Step 10243: Determine the difference between the first coordinate and the third coordinate as the target coordinate difference corresponding to the first coordinate.

[0078] In this embodiment of the disclosure, the first cloud server determines the target coordinate difference between the terminal's first coordinate and the target coordinate in the global coordinate system at preset time intervals through the above steps, so that the terminal corrects its first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal's positioning, thereby avoiding the accumulation of positioning deviation or positioning loss.

[0079] Figure 6 This is a flowchart illustrating a positioning method according to some embodiments of the present disclosure, such as... Figure 6 As shown, the method may include the following steps: Step 601: The first cloud server obtains the environmental information uploaded by the terminal, and reconstructs the environment using the environmental information to obtain a map in the global coordinate system.

[0080] Step 602: In response to the terminal startup, the first cloud server obtains the terminal's first coordinates in the local coordinate system at preset time intervals, and determines the target coordinate difference corresponding to the first coordinates in the global coordinate system.

[0081] Step 603: The terminal receives the target coordinate difference sent by the first cloud server, and corrects the first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning.

[0082] For a detailed description of steps 601 to 603 above, please refer to the detailed description in the above embodiments. This disclosure will not repeat the details here.

[0083] Step 604: The terminal synchronizes the second coordinates to the second cloud server.

[0084] In this embodiment of the disclosure, the second cloud server can be a cloud rendering server, thereby freeing up the rendering resources of the terminal and eliminating the need for the user to locate the device via a computer backpack, thus improving the user experience.

[0085] Step 605: The second cloud server renders the image based on the second coordinates and sends the rendered image to the terminal.

[0086] Step 606: The terminal sends out the rendered screen.

[0087] In this embodiment of the disclosure, the above steps can be used to render the image based on the second coordinates using the second cloud server, thereby avoiding the accumulation of positioning deviations or positioning loss, improving positioning accuracy, and thus avoiding clipping and drifting in the virtual image. Furthermore, it eliminates the need for users to locate the image using their computer backpack, thus enhancing the user experience.

[0088] Figure 7 This is a positioning device shown according to some embodiments of the present disclosure, such as Figure 7As shown, the device may include: The reconstruction module 701 is used by the first cloud server to obtain environmental information uploaded by the terminal and to reconstruct the environment using the environmental information to obtain a map in the global coordinate system. The environmental information is used to represent the information of the environment, and the global coordinate system is the location coordinate system of the map data after the environment is reconstructed. The determination module 702 is used to respond to the terminal startup, and the first cloud server obtains the first coordinate of the terminal in the local coordinate system at preset time intervals, and determines the target coordinate difference corresponding to the first coordinate in the global coordinate system; The correction module 703 is used for the terminal to receive the target coordinate difference sent by the first cloud server, and to correct the first coordinate based on the target coordinate difference to obtain the second coordinate of the terminal positioning.

[0089] In this embodiment of the disclosure, the determining module 702 is specifically used for: Obtain the origin coordinates of the terminal in the local coordinate system and determine the first coordinate difference between the origin coordinates and the global coordinate system. The origin coordinates are the coordinates of the terminal's default position when it starts up. Obtain the terminal's movement time; If the movement time is less than or equal to the time threshold, then the first coordinate difference is determined as the target coordinate difference corresponding to the first coordinate. If the movement time is greater than the time threshold, the fisheye image of the terminal is acquired, and the target coordinate difference corresponding to the first coordinate in the global coordinate system is determined based on the fisheye image.

[0090] Optionally, in this embodiment of the disclosure, the determining module 702 is further configured to: Determine the second coordinate of the origin in the global coordinate system; The difference between the origin coordinates and the second coordinate is determined as the first coordinate difference of the origin coordinates in the global coordinate system.

[0091] Optionally, in this embodiment of the disclosure, the determining module 702 is further configured to: Based on the fisheye image, a transformation coefficient is determined, which is used to transform the pixels in the fisheye image onto a preset plane. Based on the transformation coefficient, determine the third coordinate of the terminal in the global coordinate system; The difference between the first coordinate and the third coordinate is determined as the target coordinate difference corresponding to the first coordinate.

[0092] Optionally, in this embodiment of the disclosure, the determining module 702 is further configured to: By using the intrinsic parameters of the camera corresponding to the fisheye image, the pixel points in the fisheye image are projected onto the coordinates of the target point on the normalized plane. Based on the target point coordinates, determine the first radius of the center point of the normalized plane and the corresponding incident angle, where the first radius is the distance from the target point to the center point of the normalized plane; Based on the incident angle, the second radius after distortion removal in the normalized plane is determined using a fisheye model; The ratio of the second radius to the first radius is determined as the conversion coefficient.

[0093] Optionally, in this embodiment of the disclosure, the determining module 702 is further configured to: Based on the transformation coefficient, the fisheye image is transformed into a preset planar image; Extract feature points from a preset planar image; The third coordinate of the terminal in the global coordinate system is determined by comparing the feature points with the pixels of the environmental image in the first cloud server.

[0094] Optionally, in this embodiment of the disclosure, the above-mentioned correction module 703 is specifically used for: The difference between the first coordinate and the target coordinate is added together, and the result of the addition is determined as the second coordinate for terminal positioning.

[0095] Optionally, in this embodiment of the disclosure, the above-described apparatus is further used for: The terminal synchronizes the second coordinates to the second cloud server; The second cloud server renders the image based on the second coordinates and sends the rendered image to the terminal. The terminal sends out the rendered screen.

[0096] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0097] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other electronic devices through computer networks such as the Internet and / or various telecommunications networks.

[0098] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the positioning method. For example, in some embodiments, the positioning method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the positioning method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the positioning method by any other suitable means (e.g., by means of firmware).

[0099] Various embodiments of the apparatuses and techniques described above herein can be implemented in digital electronic circuit devices, integrated circuit devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), device-on-a-chip (SoC) devices, payload-programmable logic electronic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable device including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage device, at least one input device, and at least one output device, and transmitting data and instructions to the storage device, the at least one input device, and the at least one output device.

[0100] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution device, apparatus, or electronic device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or electronic device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage electronics, magnetic storage electronics, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the apparatus and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of apparatus can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0103] The apparatus and techniques described herein can be implemented in computing devices that include backend components (e.g., as a data server), or computing devices that include middleware components (e.g., an application server), or computing devices that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the apparatus and techniques described herein), or computing devices that include any combination of such backend, middleware, or frontend components. The components of the apparatus can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0104] Computer devices can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be distributed server devices or servers incorporating blockchain technology.

[0105] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0106] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A positioning method, characterized by, The method comprises: The first cloud server obtains the environment information uploaded by the terminal, and reconstructs the environment by using the environment information to obtain a map in a global coordinate system, wherein the environment information is used to represent information of the environment, and the global coordinate system is a position coordinate system of the map data after environment reconstruction; In response to the terminal starting, the first cloud server obtains a first coordinate of the terminal in a local coordinate system every interval of a preset time, and determines a target coordinate difference value corresponding to the first coordinate in the global coordinate system; The terminal receives the target coordinate difference value sent by the first cloud server, and corrects the first coordinate based on the target coordinate difference value to obtain a second coordinate of the terminal positioning.

2. The method of claim 1, wherein, The determination of the target coordinate difference value corresponding to the first coordinate in the global coordinate system comprises: Obtaining an origin coordinate of the terminal in the local coordinate system, and determining a first coordinate difference value of the origin coordinate in the global coordinate system, wherein the origin coordinate is a coordinate of a default position of the terminal when starting; Obtaining a moving time of the terminal; If the moving time is less than or equal to a time threshold, the first coordinate difference value is determined as the target coordinate difference value corresponding to the first coordinate; If the moving time is greater than the time threshold, an fisheye image of the terminal is obtained, and the target coordinate difference value corresponding to the first coordinate in the global coordinate system is determined based on the fisheye image.

3. The method of claim 2, wherein, The determination of the first coordinate difference value of the origin coordinate in the global coordinate system comprises: Determining a second coordinate of the origin coordinate in the global coordinate system; The difference value between the origin coordinate and the second coordinate is determined as the first coordinate difference value of the origin coordinate in the global coordinate system.

4. The method of claim 2, wherein, The determination of the target coordinate difference value corresponding to the first coordinate in the global coordinate system based on the fisheye image comprises: Based on the fisheye image, a conversion coefficient is determined, wherein the conversion coefficient is used to convert a pixel point in the fisheye image to a preset plane; Based on the conversion coefficient, a third coordinate of the terminal in the global coordinate system is determined; The difference value between the first coordinate and the third coordinate is determined as the target coordinate difference value corresponding to the first coordinate.

5. The method of claim 4, wherein, The determination of the conversion coefficient based on the fisheye image comprises: The pixel point in the fisheye image is projected to a target point coordinate in a normalized plane through an internal parameter of a camera corresponding to the fisheye image; Based on the target point coordinate, a first radius of a center point of the normalized plane and a corresponding incident angle are determined, wherein the first radius is a distance of the target point from the center point of the normalized plane; Based on the incident angle, a second radius after de-warping on the normalized plane is determined by using an fisheye model; The ratio of the second radius to the first radius is determined as the conversion coefficient.

6. The method of claim 4, wherein, The determination of the third coordinate of the terminal in the global coordinate system based on the conversion coefficient comprises: Based on the conversion coefficient, the fisheye image is converted into a preset plane image; Feature points in the preset plane image are extracted; Determine a third coordinate of the terminal in the global coordinate system based on a comparison between the feature points and pixel points of the environment image in the first cloud server.

7. The method of claim 1, wherein, The correction of the first coordinate based on the target coordinate difference value to obtain the second coordinate of the terminal positioning comprises: performing addition operation on the first coordinate and the target coordinate difference value, and determining the addition calculation result as the second coordinate of the terminal positioning.

8. The method of claim 1, wherein, The method further comprises: The terminal synchronizes the second coordinate to a second cloud server; The second cloud server performs rendering based on the second coordinate, and sends the obtained rendering picture to the terminal; The terminal issues the rendering picture.

9. A positioning device, characterized in that The device comprises: A reconstruction module configured to acquire environment information uploaded by a terminal by a first cloud server, and perform environment reconstruction based on the environment information to obtain a map in a global coordinate system, wherein the environment information is used to represent information of an environment, and the global coordinate system is a position coordinate system of the map data after environment reconstruction; A determination module configured to, in response to the terminal being started, acquire a first coordinate of the terminal in a local coordinate system by the first cloud server every interval of a preset time, and determine a target coordinate difference value corresponding to the first coordinate in the global coordinate system; A correction module configured to receive the target coordinate difference value sent by the first cloud server by the terminal, and correct the first coordinate based on the target coordinate difference value to obtain a second coordinate of the terminal positioning. 10.A non-transitory computer readable storage medium, the computer storage medium storing computer executable instructions;the computer executable instructions are executed by a processor, and the method of any one of claims 1-8 can be realized.

Images