Target absolute positioning method and device based on vehicle-mounted sequence panoramic images

Abstract

The present invention provides a method and device for absolutely locating a target based on a sequence of vehicle-mounted panoramic images. The method comprises: obtaining a sequence of vehicle-mounted panoramic images, the sequence of vehicle-mounted panoramic images comprising a plurality of consecutive panoramic images, each of the plurality of panoramic images including a target point; determining an initial target position of the target point in a geodetic coordinate system based on the plurality of panoramic images; determining the predicted pixel coordinates of the target point in a panoramic pixel coordinate system containing each panoramic image based on the initial target position; obtaining the actual pixel coordinates of the target point in the panoramic pixel coordinate system containing each panoramic image; constructing an objective function based on the actual pixel coordinates and the predicted pixel coordinates, and optimizing the objective function based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system. The present invention utilizes the vehicle-mounted sequential panoramic images to locate the target point, ensuring that the vehicle-mounted sequential panoramic images meet the accuracy requirements of urban surveying.

Description

Technical Field

[0001] The present invention relates to the field of visual positioning technology, and in particular to a method and device for absolute positioning of a target based on vehicle-mounted serial panoramic images. Background Art

[0002] With the development of computer technology, communication technology, sensor technology and Internet technology, the sequential 360° panoramic images obtained by multi-viewpoint combination cameras and then formed by image stitching processing have been widely used in street view maps, digital cities, public security emergency response, security deployment and other fields due to their all-round, immersive, high-resolution and realistic scene characteristics.

[0003] However, despite the ability to display 360° landscapes, current panoramic images often only have roaming and query functions, and lack measurement and analysis capabilities, which greatly limits their application in areas such as urban management component surveys and transportation infrastructure measurements. Summary of the Invention

[0004] In view of this, it is necessary to provide a method and device for absolute target positioning based on vehicle-mounted serial panoramic images to solve the technical problem in the prior art that panoramic images do not have measurement and analysis capabilities.

[0005] In order to solve the above technical problems, the present invention provides a method for absolute positioning of a target based on vehicle-mounted sequential panoramic images, comprising:

[0006] Acquire a vehicle-mounted sequence of panoramic images, wherein the vehicle-mounted sequence of panoramic images includes a plurality of continuous panoramic images, and each of the plurality of panoramic images includes a target point;

[0007] Determine the target initial position of the target point in the geodetic coordinate system based on the multiple panoramic images;

[0008] Determining the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image based on the target initial position;

[0009] Obtaining the real pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image;

[0010] An objective function is constructed based on the real pixel coordinates and the predicted pixel coordinates, and the objective function is optimized based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system.

[0011] In some possible implementations, determining the target initial position of the target point in the geodetic coordinate system based on the multiple panoramic images includes:

[0012] Acquire a plurality of vehicle body coordinates corresponding one-to-one to the plurality of panoramic images, the vehicle body coordinates being the coordinates of the vehicle body in the geodetic coordinate system at the moment of capturing the panoramic images;

[0013] The target initial position is determined according to a plurality of vehicle body coordinates.

[0014] In some possible implementations, determining the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image based on the initial target position includes:

[0015] Determining the panoramic coordinates of the target point in a panoramic spherical coordinate system based on the target initial position;

[0016] A coordinate conversion relationship between the panoramic spherical coordinate system and the panoramic pixel coordinate system is determined, and the predicted pixel coordinates are determined based on the panoramic coordinates and the coordinate conversion relationship.

[0017] In some possible implementations, determining the panoramic coordinates of the target point in a panoramic spherical coordinate system based on the target initial position includes:

[0018] Determining a first rotation matrix between the geodetic coordinate system and the panoramic spherical coordinate system;

[0019] Determine the conversion origin coordinates of the origin of the panoramic spherical coordinate system in the geodetic coordinate system;

[0020] The panoramic coordinates of the target point in the panoramic spherical coordinate system are determined according to the target initial position, the first rotation matrix and the conversion origin coordinates.

[0021] In some possible implementations, determining a first rotation matrix between the geodetic coordinate system and the panoramic spherical coordinate system includes:

[0022] Obtaining a second rotation matrix between the earth coordinate system and the carrier coordinate system;

[0023] Obtaining a third rotation matrix between the carrier coordinate system and the panoramic spherical coordinate system;

[0024] The first rotation matrix is ​​determined according to the second rotation matrix and the third rotation matrix.

[0025] In some possible implementations, determining a coordinate transformation relationship between the panoramic spherical coordinate system and the panoramic pixel coordinate system, and determining the predicted pixel coordinates based on the panoramic coordinates and the coordinate transformation relationship includes:

[0026] Acquire a projection transformation relationship between the panoramic spherical coordinate system and a panoramic spherical coordinate system, and determine the projection coordinates of the target point in the panoramic spherical coordinate system based on the panoramic coordinates and the projection transformation relationship;

[0027] A dimensionality reduction transformation relationship between the panoramic spherical coordinate system and the panoramic pixel coordinate system is acquired, and predicted pixel coordinates of the target point in the panoramic pixel coordinate system are determined based on the projection coordinates and the dimensionality reduction transformation relationship.

[0028] In some possible implementations, the projection coordinates are:

[0029]

[0030]

[0031] Where, is the projection coordinate; (X A S ,Y A S ,Z A S ) are the panoramic coordinates.

[0032] In some possible implementations, the predicted pixel coordinates are:

[0033]

[0034]

[0035] Where, (u c ,v c ) is the predicted pixel coordinate; imgW is the width of the panoramic image; imgH is the height of the panoramic image.

[0036] In some possible implementations, the preset optimization algorithm is a Levenberg-Marquardt algorithm;

[0037] The objective function is:

[0038]

[0039] Where, (u ci ,v ci ) is the predicted pixel coordinate of the target point in the i-th panoramic image; (u i ,v i ) is the real pixel coordinate of the target point in the i-th image.

[0040] On the other hand, the present invention also provides a target absolute positioning device based on vehicle-mounted sequential panoramic images, comprising:

[0041] A vehicle-mounted sequence panoramic image acquisition unit, configured to acquire a vehicle-mounted sequence panoramic image, wherein the vehicle-mounted sequence panoramic image includes a plurality of continuous panoramic images, and each of the plurality of panoramic images includes a target point;

[0042] a target initial position determining unit, configured to determine the target initial position of the target point in the geodetic coordinate system based on the multiple panoramic images;

[0043] A predicted pixel coordinate acquisition unit, configured to determine, based on the initial target position, the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image;

[0044] A real pixel coordinate acquisition unit, configured to acquire the real pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image;

[0045] An absolute positioning unit is used to construct an objective function based on the real pixel coordinates and the predicted pixel coordinates, and optimize the objective function based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system.

[0046] The beneficial effects of the above embodiment are as follows: the target absolute positioning method based on vehicle-mounted serial panoramic images provided by the present invention first obtains a vehicle-mounted serial panoramic image; then, based on multiple panoramic images, determines the target initial position of the target point in the geodetic coordinate system; and based on the target initial position, determines the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image; at the same time, obtains the real pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image; constructs an objective function based on the real pixel coordinates and the predicted pixel coordinates, and optimizes the objective function based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system. The vehicle-mounted serial panoramic image can be used to locate the target point, so that the vehicle-mounted serial panoramic image meets the accuracy requirements of urban measurement, which provides the possibility for the application of vehicle-mounted serial panoramic images in the fields of urban management component surveys and transportation infrastructure measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic flow chart of an embodiment of a method for absolute positioning of a target based on vehicle-mounted sequential panoramic images provided by the present invention;

[0049] Figure 2 A schematic structural diagram of an embodiment of the vehicle-mounted panoramic acquisition system provided by the present invention;

[0050] Figure 3 For the present invention Figure 1 A schematic flow chart of an embodiment of S102;

[0051] Figure 4 For the present invention Figure 1 A schematic flow chart of an embodiment of S103;

[0052] Figure 5 For the present invention Figure 4 A schematic flow chart of an embodiment of S401;

[0053] Figure 6 For the present invention Figure 5 A schematic flow chart of an embodiment of S501;

[0054] Figure 7 For the present invention Figure 4 A schematic flow chart of an embodiment of S402;

[0055] Figure 8 A schematic structural diagram of an embodiment of a target absolute positioning device based on vehicle-mounted sequential panoramic images provided by the present invention;

[0056] Figure 9 This is a schematic structural diagram of an embodiment of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, unless otherwise specified, "plurality" means two or more.

[0059] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0060] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or device comprising a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products or devices.

[0061] The naming or numbering of the steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0062] References to "embodiments" in this disclosure mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the disclosure. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0063] The present invention provides a method and device for absolute positioning of a target based on vehicle-mounted sequential panoramic images, which are described below.

[0064] Figure 1 This is a flow chart of an embodiment of a method for absolute positioning of a target based on vehicle-mounted sequence panoramic images provided by the present invention. Figure 1 As shown in FIG, the target absolute positioning method based on the vehicle-mounted sequence panoramic image includes:

[0065] S101: Acquire a vehicle-mounted sequence of panoramic images, where the vehicle-mounted sequence of panoramic images includes a plurality of consecutive panoramic images, each of which includes a target point;

[0066] S102, determining an initial target position of a target point in a geodetic coordinate system based on a plurality of panoramic images;

[0067] S103, determining the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image based on the initial target position;

[0068] S104, obtaining the real pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image;

[0069] S105 , constructing an objective function based on the real pixel coordinates and the predicted pixel coordinates, and optimizing the objective function based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system.

[0070] Compared with the prior art, the method for absolute target positioning based on vehicle-mounted sequential panoramic images provided by an embodiment of the present invention first obtains a vehicle-mounted sequential panoramic image; then, based on multiple panoramic images, determines the target's initial position in the geodetic coordinate system; and based on the target's initial position, determines the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image; simultaneously, obtains the target's true pixel coordinates in the panoramic pixel coordinate system of each panoramic image; constructs an objective function based on the true pixel coordinates and the predicted pixel coordinates, and optimizes the objective function based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system. The vehicle-mounted sequential panoramic image can be used to locate the target point, ensuring that the vehicle-mounted sequential panoramic image meets the accuracy requirements of urban measurement, thus facilitating the application of vehicle-mounted sequential panoramic images in areas such as urban management component surveys and transportation infrastructure measurement.

[0071] In some embodiments of the present invention, step S101 specifically involves integrating a panoramic capture device, a positioning and orientation system (POS system), a synchronizer, and other sensors onto a mobile carrier such as a vehicle to form an on-board panoramic capture system. While the vehicle is traveling at high speed, the on-board panoramic capture system continuously captures panoramic images to obtain a sequence of on-board panoramic images. Furthermore, the on-board panoramic capture system can simultaneously record the position and attitude information of the vehicle at the time of each panoramic image capture while capturing the panoramic images.

[0072] In a specific embodiment of the present invention, Figure 1 As shown, the vehicle-mounted panoramic acquisition system can acquire a vehicle-mounted sequence of panoramic images including the target point A. The vehicle-mounted panoramic acquisition system includes four coordinate systems, namely the geodetic coordinate system M, the carrier coordinate system B, the panoramic spherical coordinate system S and the panoramic pixel coordinate system P. Step S103 needs to determine the predicted pixel coordinates of the target point A in the panoramic pixel coordinate system P. This correspondence requires a series of coordinate transformations, which are the transformation relationship from the geodetic coordinate system M to the carrier coordinate system B, the transformation relationship from the carrier coordinate system B to the panoramic spherical coordinate system S, and the transformation relationship from the panoramic spherical coordinate system S to the panoramic pixel coordinate system P.

[0073] In some embodiments of the present invention, Figure 3 As shown, step S102 includes:

[0074] S301, obtaining a plurality of vehicle body coordinates corresponding one-to-one to a plurality of panoramic images, where the vehicle body coordinates are the coordinates of the vehicle body in the geodetic coordinate system at the time of panoramic image acquisition;

[0075] S302: Determine the initial position of the target according to multiple vehicle body coordinates.

[0076] Specifically, the initial target position is:

[0077]

[0078] In the formula, (X A M ,Y A M ,Z A M ) is the initial position of the target; N is the total number of panoramic images; (X1, Y1, Z1), (X2, Y2, Z2), ..., (X N ,Y N ,Z N ) are the coordinates of the vehicle body in the geodetic coordinate system at the time of capturing multiple panoramic images.

[0079] In some embodiments of the present invention, Figure 4 As shown, step S103 includes:

[0080] S401, determining the panoramic coordinates of the target point in the panoramic spherical coordinate system based on the initial target position;

[0081] S402: Determine a coordinate conversion relationship between the panoramic spherical coordinate system and the panoramic pixel coordinate system, and determine predicted pixel coordinates based on the panoramic coordinates and the coordinate conversion relationship.

[0082] In some embodiments of the present invention, Figure 5 As shown, step S401 includes:

[0083] S501, determining a first rotation matrix between a geodetic coordinate system and a panoramic spherical coordinate system;

[0084] S502, determining the coordinates of the conversion origin of the panoramic spherical coordinate system in the geodetic coordinate system;

[0085] S503: Determine the panoramic coordinates of the target point in the panoramic spherical coordinate system according to the initial target position, the first rotation matrix, and the conversion origin coordinates.

[0086] In some embodiments of the present invention, Figure 6 As shown, step S501 includes:

[0087] S601, obtaining a second rotation matrix between the earth coordinate system and the carrier coordinate system;

[0088] S602, obtaining a third rotation matrix between the carrier coordinate system and the panoramic spherical coordinate system;

[0089] S603: Determine a first rotation matrix according to the second rotation matrix and the third rotation matrix.

[0090] Specifically, the second rotation matrix is ​​the rotation matrix from the earth coordinate system to the vehicle coordinate system, which can be obtained by the vehicle body posture angle recorded by the POS system at the time of panoramic image acquisition. The third rotation matrix is ​​the rotation matrix from the vehicle coordinate system to the panoramic spherical coordinate system, which can be obtained through high-precision calibration in advance.

[0091] Specifically, the first rotation matrix is:

[0092]

[0093] Where R M S is the first rotation matrix; R M B is the second rotation matrix; R B S is the third rotation matrix.

[0094] Specifically, the coordinates of the converted origin of the panoramic spherical coordinate system in step S502 in the geodetic coordinate system are:

[0095]

[0096] In the formula, (X S M ,Y S M ,Z S M ) is the coordinate of the origin of the panoramic spherical coordinate system in the geodetic coordinate system; (X i ,Y i ,Z i ) is the coordinate of the vehicle body in the geodetic coordinate system at the time of capturing the i-th panoramic image, i∈(1,N); (X S B ,Y S B ,Z S B ) is the coordinate of the origin of the panoramic spherical coordinate system in the carrier coordinate system; R B M It is the rotation matrix from the carrier coordinate system to the earth coordinate system.

[0097] Among them, R B MThe vehicle posture angle at the time of panoramic image acquisition recorded by the POS system can be obtained; (X S B ,Y S B ,Z S B ) can be obtained in advance through high-precision calibration.

[0098] In some embodiments of the present invention, the panoramic coordinates of the target point in step S503 in the panoramic spherical coordinate system are:

[0099]

[0100] In the formula, (X A S ,Y A S ,Z A S ) is the panoramic coordinate of the target point in the panoramic spherical coordinate system.

[0101] In some embodiments of the present invention, Figure 7 As shown, step S402 includes:

[0102] S701, obtaining a projection transformation relationship between a panoramic spherical coordinate system and a panoramic spherical coordinate system, and determining the projection coordinates of a target point in the panoramic spherical coordinate system based on the panoramic coordinates and the projection transformation relationship;

[0103] S702: Acquire a dimensionality reduction transformation relationship between the panoramic spherical coordinate system and the panoramic pixel coordinate system, and determine the predicted pixel coordinates of the target point in the panoramic pixel coordinate system based on the projection coordinates and the dimensionality reduction transformation relationship.

[0104] It should be understood that the projection coordinates of the target point in the panoramic spherical coordinate system can be determined by the angle between the projection coordinates and the X axis of the panoramic spherical coordinate system. It is expressed as the angle θ formed by the plane with the X-axis and Y-axis of the panoramic spherical coordinate system. Specifically: the projection coordinates of the target point in the panoramic spherical coordinate system are:

[0105]

[0106]

[0107] Where, is the projection coordinate.

[0108] It should also be understood that since the panoramic spherical coordinate system is a three-dimensional sphere, it cannot be directly stored in the form of an image. In the actual storage process, it needs to be reduced in dimensionality to convert it into a two-dimensional image that is easy to store. That is, the projected coordinates of the target point in the panoramic spherical coordinate system need to be converted into two-dimensional predicted pixel coordinates. Specifically, the predicted pixel coordinates are:

[0109]

[0110]

[0111] Where, (u c ,v c ) are the predicted pixel coordinates; imgW is the width of the panoramic image; and imgH is the height of the panoramic image.

[0112] In some embodiments of the present invention, the optimization algorithm preset in step S105 is the Levenberg-Marquardt (LM) algorithm.

[0113] By setting the preset optimization algorithm to the LM algorithm, the speed and reliability of obtaining the absolute position of the target point in the geodetic coordinate system can be improved by utilizing its characteristics of being insensitive to initial values ​​and having a fast convergence speed.

[0114] In some embodiments of the present invention, the objective function is:

[0115]

[0116] Where, (u ci ,v ci ) is the predicted pixel coordinate of the target point in the i-th panoramic image; (u i ,v i ) is the real pixel coordinate of the target point in the i-th image.

[0117] To verify the effectiveness of the above method, the present invention used a calibrated vehicle-mounted panoramic measurement and acquisition system as the data acquisition platform. A set of panoramic image data and vehicle position and posture data at the moment of each image acquisition were collected. Five sets of sequential images were then selected from the entire driving route, each consisting of four consecutive panoramic images, with an interval of approximately 10 meters between adjacent images. Within the range of each sequential image, a target point was selected. The absolute position of the target point in the geodetic coordinate system was obtained using the vehicle-mounted sequential panoramic image absolute positioning method described in the present invention. The calculated results were then compared with the target point's field measurement results using the Global Navigation Satellite System (GNSS). The results are as follows:

[0118] Table 1 Plane accuracy statistics

[0119]

[0120] Table 2 Elevation accuracy statistics

[0121]

[0122] The above results show that the absolute position of the target point in the geodetic coordinate system obtained by the target absolute positioning method based on the vehicle-mounted panoramic image sequence has an average horizontal position error (absolute value) of 0.111m, a mean horizontal position error of 0.123m, an average elevation error (absolute value) of 0.116m, and a mean elevation error of 0.118m, compared with the GNSS measurement results. These results meet the relevant requirements of the "Urban Surveying Specification (CJJ / T8-2011)" and can be used for surveys of urban management component types and their attribute information, as well as for detailed urban planning and management.

[0123] In order to better implement the target absolute positioning method based on the vehicle-mounted sequence panoramic image in the embodiment of the present invention, based on the target absolute positioning method based on the vehicle-mounted sequence panoramic image, correspondingly, Figure 8 As shown, an embodiment of the present invention further provides a target absolute positioning device 800 based on vehicle-mounted sequential panoramic images, comprising:

[0124] The vehicle-mounted sequence panoramic image acquisition unit 801 is used to acquire a vehicle-mounted sequence panoramic image, wherein the vehicle-mounted sequence panoramic image includes a plurality of continuous panoramic images, and each of the plurality of panoramic images includes a target point;

[0125] A target initial position determining unit 802 is configured to determine the target initial position of a target point in a geodetic coordinate system based on a plurality of panoramic images;

[0126] The predicted pixel coordinate acquisition unit 803 is used to determine the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image based on the initial target position;

[0127] The real pixel coordinate acquisition unit 804 is used to acquire the real pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image;

[0128] The absolute positioning unit 805 is used to construct an objective function based on the real pixel coordinates and the predicted pixel coordinates, and optimize the objective function based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system.

[0129] The target absolute positioning device 800 based on vehicle-mounted serial panoramic images provided in the above embodiment can implement the technical solution described in the above embodiment of the target absolute positioning method based on vehicle-mounted serial panoramic images. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above embodiment of the target absolute positioning method based on vehicle-mounted serial panoramic images, which will not be repeated here.

[0130] like Figure 9 As shown, the present invention also provides an electronic device 900. The electronic device 900 includes a processor 901, a memory 902 and a display 903. Figure 9 Only some of the components of the electronic device 900 are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0131] In some embodiments, the memory 902 may be an internal storage unit of the electronic device 900, such as a hard disk or memory of the electronic device 900. In other embodiments, the memory 902 may also be an external storage device of the electronic device 900, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 900.

[0132] Furthermore, the memory 902 may include both an internal storage unit of the electronic device 900 and an external storage device. The memory 902 is used to store application software installed in the electronic device 900 and various data.

[0133] In some embodiments, the processor 901 can be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run the program code or process data stored in the memory 902, such as the target absolute positioning method based on vehicle-mounted serial panoramic images in the present invention.

[0134] In some embodiments, the display 903 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 903 is used to display information on the electronic device 900 and to display a visual user interface. Components 901-903 of the electronic device 900 communicate with each other via a system bus.

[0135] In one embodiment, when the processor 901 executes the target absolute positioning program based on the vehicle-mounted panoramic image sequence stored in the memory 902, the following steps may be implemented:

[0136] Acquire a vehicle-mounted sequence of panoramic images, where the vehicle-mounted sequence of panoramic images includes a plurality of continuous panoramic images, and each of the plurality of panoramic images includes a target point;

[0137] Determine the initial position of the target point in the geodetic coordinate system based on multiple panoramic images;

[0138] Determine the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image based on the initial position of the target;

[0139] Get the real pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image;

[0140] The objective function is constructed based on the real pixel coordinates and the predicted pixel coordinates, and the objective function is optimized based on the preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system.

[0141] It should be understood that, when the processor 901 executes the target absolute positioning program based on the vehicle-mounted serial panoramic image in the memory 902 , in addition to the above functions, it can also implement other functions. For details, please refer to the description of the corresponding method embodiment above.

[0142] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 900 mentioned. The electronic device 900 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, or the like. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with iOS, Android, Microsoft, or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 900 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0143] Accordingly, an embodiment of the present invention further provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, the method steps or functions provided in the above-mentioned method embodiments can be implemented.

[0144] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0145] The above is a detailed introduction to the target absolute positioning method and device based on vehicle-mounted serial panoramic images provided by the present invention. Specific examples are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for absolute target positioning based on vehicle-mounted panoramic image sequences, characterized in that: include: Acquire a vehicle-mounted sequence of panoramic images, wherein the vehicle-mounted sequence of panoramic images includes a plurality of continuous panoramic images, and each of the plurality of panoramic images includes a target point; Determine the target initial position of the target point in the geodetic coordinate system based on the multiple panoramic images; Determining the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image based on the target initial position; Obtaining the real pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image; An objective function is constructed based on the real pixel coordinates and the predicted pixel coordinates, and the objective function is optimized based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system.

2. The method for absolute positioning of a target based on vehicle-mounted sequential panoramic images according to claim 1, characterized in that: The determining the target initial position of the target point in the geodetic coordinate system based on the multiple panoramic images includes: Acquire a plurality of vehicle body coordinates corresponding one-to-one to the plurality of panoramic images, the vehicle body coordinates being the coordinates of the vehicle body in the geodetic coordinate system at the moment of capturing the panoramic images; The target initial position is determined according to a plurality of vehicle body coordinates.

3. The method for absolute positioning of a target based on vehicle-mounted sequential panoramic images according to claim 1, characterized in that: The step of determining the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image based on the initial target position includes: Determining the panoramic coordinates of the target point in a panoramic spherical coordinate system based on the target initial position; A coordinate conversion relationship between the panoramic spherical coordinate system and the panoramic pixel coordinate system is determined, and the predicted pixel coordinates are determined based on the panoramic coordinates and the coordinate conversion relationship.

4. The method for absolute positioning of a target based on vehicle-mounted sequential panoramic images according to claim 3, characterized in that: The determining of the panoramic coordinates of the target point in the panoramic spherical coordinate system based on the target initial position includes: Determining a first rotation matrix between the geodetic coordinate system and the panoramic spherical coordinate system; Determine the conversion origin coordinates of the origin of the panoramic spherical coordinate system in the geodetic coordinate system; The panoramic coordinates of the target point in the panoramic spherical coordinate system are determined according to the target initial position, the first rotation matrix and the conversion origin coordinates.

5. The method for absolute positioning of a target based on vehicle-mounted sequential panoramic images according to claim 4, characterized in that: The determining of a first rotation matrix between the geodetic coordinate system and the panoramic spherical coordinate system includes: Obtaining a second rotation matrix between the earth coordinate system and the carrier coordinate system; Obtaining a third rotation matrix between the carrier coordinate system and the panoramic spherical coordinate system; The first rotation matrix is ​​determined according to the second rotation matrix and the third rotation matrix.

6. The method for absolute positioning of a target based on vehicle-mounted panoramic image sequences according to claim 3, characterized in that: The determining of the coordinate conversion relationship between the panoramic spherical coordinate system and the panoramic pixel coordinate system, and determining the predicted pixel coordinates based on the panoramic coordinates and the coordinate conversion relationship, includes: Acquire a projection transformation relationship between the panoramic spherical coordinate system and a panoramic spherical coordinate system, and determine the projection coordinates of the target point in the panoramic spherical coordinate system based on the panoramic coordinates and the projection transformation relationship; A dimensionality reduction transformation relationship between the panoramic spherical coordinate system and the panoramic pixel coordinate system is acquired, and predicted pixel coordinates of the target point in the panoramic pixel coordinate system are determined based on the projection coordinates and the dimensionality reduction transformation relationship.

7. The method for absolute target positioning based on vehicle-mounted panoramic image sequences according to claim 6, characterized in that: The projection coordinates are: Where, is the projection coordinate; (X A S ,Y A S ,Z A S ) are the panoramic coordinates.

8. The method for absolute target positioning based on vehicle-mounted sequential panoramic images according to claim 7, characterized in that: The predicted pixel coordinates are: Where, (u c ,v c ) is the predicted pixel coordinate; imgW is the width of the panoramic image; imgH is the height of the panoramic image.

9. The method for absolute target positioning based on vehicle-mounted sequential panoramic images according to claim 1, characterized in that: The preset optimization algorithm is the Levenberg-Marquardt algorithm; The objective function is: Where, (u ci ,v ci ) is the predicted pixel coordinate of the target point in the i-th panoramic image; (u i ,v i ) is the real pixel coordinate of the target point in the i-th image.

10. A target absolute positioning device based on vehicle-mounted sequence panoramic images, characterized in that: include: A vehicle-mounted sequence panoramic image acquisition unit, configured to acquire a vehicle-mounted sequence panoramic image, wherein the vehicle-mounted sequence panoramic image includes a plurality of continuous panoramic images, and each of the plurality of panoramic images includes a target point; a target initial position determining unit, configured to determine the target initial position of the target point in the geodetic coordinate system based on the multiple panoramic images; A predicted pixel coordinate acquisition unit, configured to determine, based on the initial target position, the predicted pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image; A real pixel coordinate acquisition unit, configured to acquire the real pixel coordinates of the target point in the panoramic pixel coordinate system of each panoramic image; An absolute positioning unit is used to construct an objective function based on the real pixel coordinates and the predicted pixel coordinates, and optimize the objective function based on a preset optimization algorithm to obtain the absolute position of the target point in the geodetic coordinate system.

Images