Terminal initialization method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a terminal initialization method, device, electronic equipment and storage medium. The terminal initialization method comprises: in response to an initialization event of a terminal, acquiring image data of the terminal from an initial time to an end time and inertial data measured in an inertial coordinate system, the inertial coordinate system being fixed relative to the position of the terminal; establishing a linear constraint equation of a feature point according to a constraint equation of the coordinates of the feature point in a normalized plane and the coordinates of the feature point in a camera coordinate system, and a representation of the coordinates of the feature point in the camera coordinate system converted into the inertial system at the same time in the form of inertial data; projecting the linear constraint equation to a left null space of the corresponding feature point for dimension reduction; superimposing the linear constraint equations of each feature point to obtain an integrated constraint equation; and solving an initialization parameter of the terminal at the initial time according to the integrated constraint equation. The method of the present disclosure can improve the solving speed compared with the existing tight coupling initialization method, and takes into account robustness and efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a terminal initialization method, apparatus, electronic device, and storage medium. Background Technology

[0002] In extended reality devices such as virtual reality, augmented reality, and mixed reality, as well as drones and robots, visual inertial positioning systems are often used. Before normal use, visual inertial positioning systems need to be initialized, that is, to restore the initial values ​​of the state variables estimated by the visual inertial positioning system, which usually include the initial velocity of the terminal and the position of feature points.

[0003] Initialization methods are mainly divided into static initialization and dynamic initialization. Static initialization, due to its high requirements for device movement, is prone to limitations in practical applications. Dynamic initialization methods are mainly divided into tightly coupled and loosely coupled methods. The loosely coupled method estimates the initial state variables separately using the vision system and the inertial system, and then fuses them. This method does not fully utilize all the information from the vision system and the inertial system, resulting in low robustness. The tightly coupled method fuses the information from the vision system and the inertial system, providing better robustness, but it also suffers from drawbacks such as high computational cost leading to slower solution speed. Summary of the Invention

[0004] This disclosure provides a terminal initialization method, apparatus, electronic device, and storage medium.

[0005] The following technical solution is adopted in this disclosure.

[0006] In some embodiments, this disclosure provides a terminal initialization method, including:

[0007] In response to the terminal's initialization event, image data of the terminal from the initial time to the end time and inertial data measured in an inertial coordinate system are acquired, wherein the relative position of the inertial coordinate system and the terminal is fixed.

[0008] Based on the constraint equations of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the transformation of the coordinates of the feature point in the camera coordinate system to the inertial coordinate system at the same moment in the representation of inertial data, a linear constraint equation for the feature point is established.

[0009] The linear constraint equations are projected onto the left null space of the corresponding feature points to reduce dimensionality.

[0010] The integrated constraint equation is obtained by superimposing the linear constraint equations of each feature point.

[0011] The initialization parameters of the terminal at the initial time are solved according to the integrated constraint equation.

[0012] In some embodiments, this disclosure provides a terminal initialization apparatus, including:

[0013] The acquisition unit is used to acquire image data of the terminal from the initial time to the end time and inertial data measured in an inertial coordinate system in response to the initialization event of the terminal, wherein the relative position of the inertial coordinate system and the terminal is fixed.

[0014] The control unit is used to establish the linear constraint equation of the feature point based on the constraint equation of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the representation of the coordinates of the feature point in the camera coordinate system as inertial data after the coordinates of the feature point in the camera coordinate system are transformed to the inertial coordinate system at the same moment.

[0015] The control unit is also used to project the linear constraint equations onto the left null space of the corresponding feature points for dimensionality reduction;

[0016] The control unit is also used to superimpose the linear constraint equations of each of the feature points to obtain an integrated constraint equation;

[0017] The control unit is also used to solve the initialization parameters of the terminal at the initial time according to the integrated constraint equation.

[0018] In some embodiments, this disclosure provides an electronic device, including: at least one memory and at least one processor;

[0019] The memory is used to store program code, and the processor is used to call the program code stored in the memory to execute the above method.

[0020] In some embodiments, this disclosure provides a computer-readable storage medium for storing program code that, when executed by a processor, causes the processor to perform the methods described above.

[0021] The terminal initialization method provided in this embodiment is a tightly coupled initialization method, which can improve the calculation speed and balance robustness and efficiency compared with existing tightly coupled initialization methods. Attached Figure Description

[0022] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0023] Figure 1 This is a flowchart of a terminal initialization method according to an embodiment of the present disclosure.

[0024] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0025] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0026] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0027] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0028] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0029] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be understood that the various steps described in the method embodiments of this disclosure can be performed in sequence and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0034] It should be noted that the use of the word "a" in this disclosure is illustrative rather than restrictive, and those skilled in the art should understand that it should be understood as "one or more" unless otherwise expressly indicated in the context.

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] The solutions provided by the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0037] like Figure 1 As shown, Figure 1 This is a flowchart of a terminal initialization method according to an embodiment of the present disclosure, which includes the following steps.

[0038] S11. In response to the terminal's initialization event, acquire the image data of the terminal from the initial time to the end time and the inertial data measured in the inertial coordinate system.

[0039] In some embodiments, the terminal can be an extended reality device such as virtual reality, augmented reality, or mixed reality. Specifically, the terminal can be a head-mounted device within an extended reality device. The terminal's initialization event can be an event that sends an initialization signal, such as automatically sending an initialization signal after clicking the power button. The initial time can be the time after the initialization event, and the end time is the time after the initial time (denoted as time 0). After detecting the initialization event, the camera on the terminal can automatically capture image data (the image data can be a photograph). The image data contains feature points, which are points with specific information, such as corner points (bright and dark corner points) of objects in the terminal's environment, such as table corners or wall corners. The camera can automatically capture image data at regular intervals, for example, 30 frames per second, with each 1 / 30th of a second representing a moment. The end moment can be the 30th moment after the initial moment. Each image data capture moment corresponds to an inertial coordinate system at that moment. From the initial moment to the end moment, 30 image data captures are performed, corresponding to 30 inertial coordinate systems. During image data capture, the inertial measurement unit (accelerometer and angular velocity meter) in the terminal automatically measures inertial data, which can include acceleration and angular velocity. For better initialization, the terminal can be in a non-completely stationary state from the initial moment to the end moment, meaning that the terminal moves and / or rotates during this period. The relative position of the inertial coordinate system and the terminal is fixed. The origin of the inertial coordinate system can be set within the terminal, and the inertial coordinate system moves and / or rotates accordingly with the movement and / or rotation of the terminal. For example, taking the terminal as a head-mounted device in an extended reality device, the inertial coordinate system can be the head-mounted coordinate system, with its origin set within the head-mounted device. It can also be the coordinate system of the inertial measurement unit (IMU) itself in the terminal. Because the camera is located in the terminal, the relative positions of the inertial coordinate system and the camera coordinate system are also fixed. The camera coordinate system is the coordinate system in the image data.

[0040] In some embodiments, after acquiring inertial data, the changes in position, attitude, and velocity of the inertial coordinate system at different times can be calculated by integrating over time. The integral of inertial coordinate system I can be defined, where I0 represents the initial state of inertial coordinate system I. k Let I represent the inertial coordinate system at time k, and so on. At this point, the unobservable initial value R... I0I0 For the identity matrix, P I0I0 The zero vector. R represents rotation. InIm Let represent the rotation matrix from inertial coordinate system Im at time m to inertial coordinate system In at time n, where n and m refer to any time n and time m. P ​​represents the position vector. InIm Indicates from inertial coordinate system I m To inertial coordinate system I nThe position vector (translation vector). v represents the velocity vector, V InIm Indicating inertial coordinate system I m The velocity vector in the inertial coordinate system In is represented by the velocity vector in the inertial coordinate system In.

[0041] Based on their relationship, we can obtain the following formula (1):

[0042] Formula (1)

[0043] Among them, R I0I0 , △R I0Ik (Indicates from inertial coordinate system I) k The incremental rotation matrix to the inertial coordinate system I0, which can be obtained by integrating the angular velocity, Δt (the time difference between the times corresponding to different inertial coordinate systems, the specific value depends on the time corresponding to the calculated inertial coordinate system, here it is the time difference between the initial time and time k), P I0I0 , △P I0Ik (This indicates the distance from inertial coordinate system I0 to inertial coordinate system I) k The change in position, which can be obtained by integrating acceleration, ΔV I0Ik (This indicates the distance from inertial coordinate system I0 to inertial coordinate system I) k The change in velocity (obtained by integrating acceleration) is a known quantity. I0 Let I represent the gravitational acceleration in the inertial coordinate system I0 at the initial moment, whose direction in the inertial coordinate system at the initial moment is unknown. Equation (1) represents the inertial constraint.

[0044] S12. Based on the constraint equations of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the transformation of the coordinates of the feature point in the camera coordinate system to the inertial coordinate system at the same moment in the representation of inertial data, establish the linear constraint equations of the feature point.

[0045] In some embodiments, different image data are captured at different times, and the image data at any given time is processed according to the following steps:

[0046] First, define the visual constraints. Based on the relationship between observation and reprojection on the normalized plane, the constraint equations can be obtained. The constraint equations are shown below as formula (2):

[0047] Formula (2)

[0048] The normalized plane is a plane whose depth is always equal to 1, u n and v n Let P be the coordinates of a feature point on the normalized plane. CkfThis represents the coordinates of the feature point (f, feature) in the camera coordinate system. C represents the camera coordinate system in this disclosure. k Let k represent the camera coordinate system at time k, where k refers to the shooting time of the image data being processed. That is, the above formula (2) is an equation established for the image data captured at time k. Here, k refers to the time when an observation was performed (i.e., a frame of image data was captured), and the feature point is located in the observation (i.e., the captured image data) at time k. The coordinates of this feature point in the camera coordinate system can be transformed to the inertial coordinate system I at the same time (i.e., the shooting time k of the image data being processed is the same as the time corresponding to the inertial coordinate system) (assuming the feature point is in the image data captured at time k), which is the following formula (3).

[0049] Formula (3)

[0050] Among them, R CI Let be the extrinsic parameter, representing the rotation matrix from the inertial coordinate system I to the camera coordinate system C, ΔR. I0Ik See formula (1), P I0f Let ΔP represent the initial position of feature point f in the inertial coordinate system I0 at the initial moment. I0Ik See formula (1), P CI Let R represent the translation vector from the inertial coordinate system I to the camera coordinate system C at the same instant. Since the relative positions of the inertial coordinate system and the camera coordinate system are fixed, R... CI and P CI It is a fixed value and is independent of time.

[0051] After obtaining formulas (2) and (3), the problem of solving the initialization parameters can be defined as a linear problem of Ax=b based on inertial constraints and visual constraints, and the constraint equation for a single observation of the feature point (i.e., one frame of image data) can be established. The same steps are applied to all frames of image data to obtain the constraint equation for multiple observations (i.e., multiple frames of image data). Based on the constraint equation for multiple observations, the linear constraint equation for a single feature point can be obtained.

[0052] S13. Project the linear constraint equations onto the left null space of the corresponding feature points to reduce dimensionality.

[0053] In some embodiments, the linear constraint return path contains feature coordinates, which causes the dimension of the equation to depend mainly on the number of feature points. When the number of feature points is large, the matrix calculation will be huge. The main idea to reduce the calculation in this embodiment is dimensionality reduction. That is, in this embodiment, the linear constraint return path is projected onto the left null space of the feature points, thereby eliminating the related quantities of the feature points in the linear constraint equation, so that the linear constraint equation only includes the representation of the velocity of the terminal in the inertial coordinate system at the initial moment and the representation of the gravitational acceleration in the inertial coordinate system at the initial moment.

[0054] S14. Superimpose the linear constraint equations of each feature point to obtain the integrated constraint equation.

[0055] In some embodiments, for each feature point, steps S13 and S14 are performed to obtain the linear constraint equations corresponding to each feature point. Then, the linear constraint equations of each feature point are stacked to obtain the integrated constraint equation.

[0056] S15. Solve for the initialization parameters of the terminal at the initial moment based on the integrated constraint equations.

[0057] In some embodiments, the initialization parameters include one or more of the following: the direction of acceleration in the inertial coordinate system at the initial moment, the position of the feature point in the inertial coordinate system at the initial moment, and the velocity of the terminal device in the inertial coordinate system at the initial moment. The velocity vector V of the terminal in the inertial coordinate system I0 can be solved by integrating the constraint equations. I0 The expression of gravitational acceleration in inertial coordinate system I0: g I0 These two parameters allow the feature points to be triangulated and recovered using the obtained terminal pose, further determining the position P of the feature points in the inertial coordinate system I0. I0f For example, a head-mounted virtual reality device (VR) is used as the terminal. I0 This represents the speed of the head-mounted device at the start of initialization. This includes the initial linear velocity of the user's head, i.e., the velocity state before any additional movement.

[0058] In some embodiments of this disclosure, considering the low robustness of loosely coupled initialization and the low computational efficiency of tightly coupled initialization, this disclosure proposes a tightly coupled terminal initialization method. This method first constructs linear constraint equations based on visual and inertial constraints. Then, by projecting the coordinates of the feature points to be estimated onto the left null space, the dimensionality of the linear constraint equations is reduced, significantly increasing the solution efficiency. Finally, the feature points are triangulated using the obtained terminal pose to recover them, thereby achieving the goal of restoring all initialization parameters. Compared to conventional tightly coupled initialization methods, the tightly coupled terminal initialization method proposed in this disclosure can significantly increase the initialization solution speed and improve initialization efficiency.

[0059] In some embodiments of this disclosure, a linear constraint equation for the feature point is established based on the constraint equations of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the coordinates of the feature point in the camera coordinate system transformed into the inertial coordinate system at the same moment and represented as inertial data. This includes: substituting the coordinates of the feature point in the camera coordinate system transformed into the inertial coordinate system at the same moment and represented as inertial data, and the coordinates of the feature point in the normalized plane, into the constraint equations of the coordinates of the feature point in the camera coordinate system; organizing the initialization parameters to be calculated and the known inertial data on both sides of the constraint equations to form the constraint equations for one frame of image data for a single feature point; and stacking the constraint equations for all frames of image data for a single feature point to form the linear constraint equations for a single feature point.

[0060] In some embodiments, it is necessary to define the solution of the initialization parameters as a linear problem Ax=b based on inertial constraints and visual and visual constraints. Therefore, the constraint equation formula (2) is first simplified to form the following formula (4):

[0061] Formula (4)

[0062] In formula (2), the left-hand matrix is ​​Γ, and the right-hand matrix is ​​02. The coordinates of the feature points in the camera coordinate system are transformed to the inertial coordinate system at the same moment, and represented by inertial data, to establish the linear constraint equations for the feature points, i.e., P in formula (4). Ckf Replacing formula (3) with formula (5) results in the following formula (5):

[0063] Formula (5)

[0064] Then, according to formula (1), P in formula (5) I0Ik Substituting formula (1) into the expansion, we get the following:

[0065]

[0066] Then, organize the quantities that need to be estimated onto the left side and the known quantities onto the right side, forming the following formula (6):

[0067] Formula (6)

[0068] To further simplify formula (6), as shown above, the left side of formula (6) Use symbols Instead, the above formula (6) is the constraint equation for one side of the observation of a single feature point (i.e., in one frame of image data). Stacking the constraint equations of all frames of image data for a single feature point forms the linear constraint equation for that single feature point. It can be expressed as the following formula (7):

[0069] Formula (7)

[0070] In formula (7), the matrices on the left and right sides are replaced by general formulas. Each row in the leftmost matrix of formula (7) is a coefficient extracted from the constraint equation of a frame of image data of the feature point. Each row on the right side of formula (7) is the part on the right side of the constraint equation of a frame of image data of the feature point. The omitted parts in the matrices on the left and right sides of formula (7) are in the same form as the general formula shown. The calculation method is the same as the process of obtaining the above formula (6).

[0071] In some embodiments, solving the linear constraint equations requires overdetermining them; a single observation introduces two-dimensional constraints, and without considering the actual degrees of freedom, P... I0f V I0 and g I0 Dimensions are 9 (V) I0 That is, V I0I0 These three parameters are initialization parameters that need to be solved, so at least 5 observations are needed to solve the linear constraint equations. Considering that the gravitational modulus is fixed, the actual degree of freedom is 8 after adding the gravitational modulus constraint, so at least 4 observations are needed. Therefore, in some embodiments, a single feature point has at least 4 frames of image data and a single feature point has at least 4 constraint equations, that is, the same feature point needs to be captured in at least 4 frames of images in order to solve the linear constraint equations of the feature point.

[0072] In some embodiments, k frames of image data are captured from the initial time to the end time (assuming time k). N feature points are tracked in each image data, meaning that N feature points are observed during this time. Each feature point is observed k times. For each frame of image data for each feature point, constraint equations are constructed according to formulas (2) to (6). For any feature point, it is observed k times (k frames of image data), so the feature point has k constraint equations. The k constraint equations are superimposed to form the linear constraint equation of the feature point. The coordinates of the feature point on the normalized plane in formula (2) can be different depending on the feature point and the image data.

[0073] In some embodiments of this disclosure, projecting the linear constraint equations onto the left null space of the corresponding feature points for dimensionality reduction includes: rearranging the linear constraint equations to obtain the coordinates P of the feature points corresponding to the linear constraint equations in the inertial coordinate system at the initial time. I0f Organized, among which, P I0f The coefficients in the simplified linear constraint equations are H. p ; For H p Perform orthogonal triangular decomposition to obtain H p The column space basis Q1 and the null space basis Q2; the simplified linear constraint equations are projected onto H. p In the null basis Q2, P is eliminated using the orthogonality of Q1 and Q2. I0f .

[0074] In some embodiments, the goal of the method proposed in this disclosure is to transform the solution of the initialization parameters into a least squares problem. By stacking the linear constraint equations of all feature points that satisfy the observation count (i.e., feature points with at least 4 frames of image data), a least squares problem can be formed as shown in the following formula (8):

[0075] Formula (8)

[0076] In formula (8), X is represented by formula (9).

[0077] Formula (9)

[0078] In formula (9), N is the number of feature points that satisfy the observation count.

[0079] Because formula (9) contains feature point coordinates, the dimension of formula (8) mainly depends on the number of feature points. When the number of feature points is large, the matrix calculation will be extremely large. In some embodiments of this disclosure, the main idea to reduce the calculation is to reduce the dimension of x, thereby reducing the calculation by projecting the linear constraint equation into the left null space, thereby eliminating the feature point related quantities in the linear constraint equation, so that x in the final formula (9) only contains V. I0 and g I0 However, it does not include the coordinates of feature points.

[0080] The following describes in detail the processing procedure for the linear constraint equations of each feature point that satisfies the number of observations.

[0081] First, the constraint equations need to be rearranged so that, while stacking all observations of the feature point together, P can be... I0f Organize it and let P I0f With V I0 and g I0Separately, as shown in formula (10):

[0082]

[0083] Formula (10)

[0084] For simplification, V in formula (10) I0 and g I0 The matrix on the left is represented by Hx, P I0f The coefficient matrix on the left is denoted by Hp, and the matrix on the right side of the equation (10) is denoted by b. Υn and Γn are the parameters Υ and Γ in equation (6) calculated based on the nth observation of the feature point (i.e., the constraint in the nth frame of image data). Since the specific number of observations is uncertain, the middle part of the matrix is ​​represented by an ellipsis, and the number of rows in the matrix is ​​equal to the specific number of observations (i.e., the number of frames of image data captured for the feature point).

[0085] Then for H p Perform orthogonal triangular decomposition (QR decomposition) to obtain H p The column space basis Q1 and the null space basis Q2 are as shown in Equation (11);

[0086] Formula (11)

[0087] In the QR decomposition process, an orthogonal matrix Q is generated, from which column space basis Q1 and null space basis Q2 are separated, and R1 is an upper triangular matrix. Q1 and Q2 are orthogonal matrices.

[0088] Then, the simplified linear constraint equations are projected onto H. p In the null basis Q2, P is eliminated using the orthogonality of Q1 and Q2. I0f Specifically, as shown below, substitute formula (11) into formula (10) and multiply by Q on the left. T 2. We obtain the following formula (12):

[0089] Formula (12)

[0090] Because of the orthogonality of Q1 and Q2, the second term on the left side of equation (12) is eliminated, resulting in equation (13).

[0091] Formula (13)

[0092] To simplify formula (13), the symbols A, X, and B represent the partial matrices, respectively. Formula (13) above is full rank when the number of observations of the feature points is greater than 3.

[0093] In order to obtain formula (8), the linear constraint equation for each feature point needs to be processed according to formulas (10) to (13) shown below, so as to obtain formula (13) for N feature points.

[0094] In some embodiments of this disclosure, the linear constraint equations of each of the feature points are superimposed to obtain an integrated constraint equation. The specific process may include: obtaining formula (13) AX=B according to the aforementioned process, and multiplying formula (13) by A on the left. T A was obtained T AX=A T In the form of B, the formula (13) for each feature point that meets the observation number requirement (the number of feature points is denoted as N) is operated in the same way to obtain A corresponding to N feature points. T AX=A T B, then A corresponding to N feature points T AX=A T Bs stacked together form formula (8), that is, N A's. T The sum of A's is obtained by adding up the N A's in formula (8). T The superposition of B yields b in formula (8), which is the integrated constraint equation. The coefficient matrix of the integrated constraint equation obtained by superposition is 6*6, and the dimension of the integrated constraint equation is 6. The left null space projection not only achieves dimensionality reduction but also reduces the use of dynamic matrices during programming.

[0095] In some embodiments of this disclosure, solving the initialization parameters of the terminal at the initial moment according to the integrated constraint equation includes: introducing the constraint of the magnitude of gravitational acceleration into the integrated constraint equation using the Lagrange multiplier method; solving the integrated constraint equation to obtain the direction of gravitational acceleration in the inertial coordinate system at the initial moment and the initialization parameters.

[0096] In some embodiments, as described above, the integrated constraint equation shown in formula (8) is obtained by superimposing the left null space projection and the linear constraint equations, in order to solve V. I0 and g I0 Considering that the magnitude of gravitational acceleration g is known, the company (8) can be defined as a constrained least squares problem as shown in formulas (14) and (15), where x1 corresponds to V in formula (13). I0I0 It is unrestrained.

[0097]

[0098] Formula (14)

[0099] Formula (15)

[0100] "Subject to" indicates the constraint to be satisfied. In formula (15), g... I0 With the value fixed, find the least squares solution for x1 in formula (14), which is shown in formula (16) below:

[0101] Formula (16)

[0102] Substituting equation (16) back into equation (14), and defining C as shown in equation (17):

[0103] Formula (17)

[0104] C is defined as the part enclosed in parentheses above it. Expanding formula (17) yields formula (18):

[0105]

[0106] Formula (18)

[0107] D and d are defined in formula (18). T , which is the part enclosed in the corresponding parentheses, and constant represents the constant in the expanded formula (18). Formula (18) needs to satisfy the following constraint of formula (19):

[0108] Formula (19)

[0109] Define E=A1(A1) T A1) -1 A1 T E T =E and E T E=E. Since C=I-E, therefore C T Based on C=C, D and d in formula (18) can be simplified to the following formula (20):

[0110] Formula (20)

[0111] Based on the simplified formulas (18) and (19), by using the Lagrange multipliers to put the constraint term formula (19) into formula (18), we obtain the function shown in formula (21):

[0112] Formula (21)

[0113] Among them, L(g) I0,λ) represents the Lagrange function, and λ is the Lagrange multiplier, used to introduce the modulus constraint of gravitational acceleration. The right side of formula (21) includes two parts: the part enclosed in the first bracket is an objective function, and the product of λ and the part enclosed in the second bracket is the modulus constraint term.

[0114] Then let L(g) I0 ,λ) for g I0 With the derivative being zero, we can obtain the following formula (22):

[0115] Formula (22)

[0116] Furthermore, the above problem is equivalent to solving the following formula (23).

[0117] Formula (23)

[0118] That is, we need to minimize λ while satisfying the constraints of formula (23). λ and g I0 The solution satisfies the condition of the following formula (24) (where e ≠ 0):

[0119] Formula (24)

[0120] Where I3 represents the 3×3 identity matrix. e = is obtained from Theorem 5.1 on pages 832-833 of the article A constrained eigenvalue problem (Gander W, Golub GH, Von Matt U. A constrained eigenvalue problem[J].Linear Algebra and its applications, 1989, 114: 815-839.), and e≠0. Since e≠0, its coefficient matrix is ​​singular, and its determinant is zero, forming formula (25):

[0121] Formula (25)

[0122] λ is the smallest root of formula (25). The smallest λ can be obtained by performing eigenvalue decomposition on the adjoint matrix of the sixth-degree polynomial. After obtaining the smallest λ, g can be obtained according to formula (25). I0 The solution g * I0 (The * in the upper right corner indicates the solution) is formula (26):

[0123] Formula (26)

[0124] Then, substituting the obtained formula (26) into formula (16), we can solve for x1.* 1 is formula (27):

[0125] Formula (27)

[0126] Finally, the closed-form solution to this constrained problem, i.e., the solution x in formula (14), is obtained. * For formula (28):

[0127] Formula (28)

[0128] This yields the magnitude and direction of gravitational acceleration in the initial inertial coordinate system, as well as the terminal's velocity in the initial inertial coordinate system, thus determining the terminal's pose. Finally, the feature points are triangulated using the solved terminal pose to recover their positions P in the initial inertial coordinate system. I0f .

[0129] In some embodiments of this disclosure, a fast and tightly coupled terminal initialization method is proposed. By constructing linear constraint equations containing visual and inertial constraints, the coordinates of the feature points to be estimated are projected into the left null space, thereby reducing the dimensionality of the linear problem to 6 dimensions, significantly increasing the solution efficiency. Finally, the feature points are triangulated using the solved terminal pose to recover all initialization parameters. The method in some embodiments of this disclosure significantly increases the initialization solution speed, improves the terminal initialization speed, and can quickly determine the specific values ​​of the terminal initialization parameters.

[0130] Some embodiments of this disclosure also propose a terminal initialization device, including:

[0131] The acquisition unit is used to acquire image data of the terminal from the initial time to the end time and inertial data measured in an inertial coordinate system in response to the initialization event of the terminal, wherein the relative position of the inertial coordinate system and the terminal is fixed.

[0132] The control unit is used to establish the linear constraint equation of the feature point based on the constraint equation of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the representation of the coordinates of the feature point in the camera coordinate system as inertial data after the coordinates of the feature point in the camera coordinate system are transformed to the inertial coordinate system at the same moment.

[0133] The control unit is also used to project the linear constraint equations onto the left null space of the corresponding feature points for dimensionality reduction;

[0134] The control unit is also used to superimpose the linear constraint equations of each of the feature points to obtain an integrated constraint equation;

[0135] The control unit is also used to solve the initialization parameters of the terminal at the initial time according to the integrated constraint equation.

[0136] In some embodiments, based on the constraint equations of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the transformation of the coordinates of the feature point in the camera coordinate system to the inertial coordinate system at the same moment in the inertial data representation, a linear constraint equation for the feature point is established, including:

[0137] The coordinates of the feature point in the camera coordinate system are transformed to the inertial coordinate system at the same moment and represented by inertial data. The coordinates of the feature point in the normalized plane are then substituted into the constraint equation of the coordinates of the feature point in the camera coordinate system.

[0138] The initialization parameters to be calculated and the known inertial data are respectively arranged on both sides of the constraint equation to form the constraint equation of one frame of image data for a single feature point.

[0139] The constraint equations of all image data for a single feature point are stacked to form a linear constraint equation for that single feature point.

[0140] In some embodiments, a single feature point has at least 4 frames of image data and at least 4 constraint equations.

[0141] In some embodiments, projecting the linear constraint equations onto the left null space of the corresponding feature points for dimensionality reduction includes: rearranging the linear constraint equations to obtain the coordinates P of the feature points corresponding to the linear constraint equations in the inertial coordinate system at the initial time. I0f Organized, among which, P I0f The coefficients in the simplified linear constraint equations are H. p ;

[0142] For H p Perform orthogonal triangular decomposition to obtain H p The column space basis Q1 and the null space basis Q2;

[0143] Project the rearranged linear constraint equations onto H p In the null basis Q2, P is eliminated using the orthogonality of Q1 and Q2. I0f .

[0144] In some embodiments, the dimension of the superimposed integrated constraint equation is 6.

[0145] In some embodiments, solving the initialization parameters of the terminal at the initial time according to the integrated constraint equations includes:

[0146] The constraint on the modulus of gravitational acceleration is introduced into the integrated constraint equation using the Lagrange multiplier method.

[0147] Solving the integrated constraint equations yields the direction of gravitational acceleration in the inertial coordinate system at the initial moment, as well as the initialization parameters.

[0148] In some embodiments, the initialization parameters include one or more of the following: the direction of gravitational acceleration in the inertial coordinate system at the initial moment, the position of the feature point in the inertial coordinate system at the initial moment, and the velocity of the terminal device in the inertial coordinate system at the initial moment.

[0149] For embodiments of the apparatus, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments. The apparatus embodiments described above are merely illustrative, and the modules described as separate modules may or may not be separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0150] The methods and apparatus of this disclosure have been described above based on embodiments and application examples. Furthermore, this disclosure also provides an electronic device and a computer-readable storage medium, which are described below.

[0151] The following is for reference. Figure 2 The figure illustrates a structural schematic of an electronic device (e.g., a terminal device or server) 800 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in the figure is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.

[0152] Electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 802 or a program loaded from storage device 808 into random access memory (RAM) 803. RAM 803 also stores various programs and data required for the operation of electronic device 800. The processing device 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0153] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although an electronic device 800 with various devices is shown in the figure, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0154] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, it performs the functions defined in the methods of embodiments of this disclosure.

[0155] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0156] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0157] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0158] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods of the present disclosure.

[0159] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0161] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0162] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0163] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0164] According to one or more embodiments of this disclosure, a terminal initialization method is provided, comprising:

[0165] In response to the terminal's initialization event, image data of the terminal from the initial time to the end time and inertial data measured in an inertial coordinate system are acquired, wherein the relative position of the inertial coordinate system and the terminal is fixed.

[0166] Based on the constraint equations of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the transformation of the coordinates of the feature point in the camera coordinate system to the inertial coordinate system at the same moment in the representation of inertial data, a linear constraint equation for the feature point is established.

[0167] The linear constraint equations are projected onto the left null space of the corresponding feature points to reduce dimensionality.

[0168] The integrated constraint equation is obtained by superimposing the linear constraint equations of each feature point.

[0169] The initialization parameters of the terminal at the initial time are solved according to the integrated constraint equation.

[0170] According to one or more embodiments of this disclosure, a terminal initialization method is provided, which establishes a linear constraint equation for the feature point based on the constraint equation between the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the coordinates of the feature point in the camera coordinate system transformed to the inertial coordinate system at the same moment and represented by inertial data, including:

[0171] The coordinates of the feature point in the camera coordinate system are transformed to the inertial coordinate system at the same moment and represented by inertial data. The coordinates of the feature point in the normalized plane are then substituted into the constraint equation of the coordinates of the feature point in the camera coordinate system.

[0172] The initialization parameters to be calculated and the known inertial data are respectively arranged on both sides of the constraint equation to form the constraint equation of one frame of image data for a single feature point.

[0173] The constraint equations of all image data for a single feature point are stacked to form a linear constraint equation for that single feature point.

[0174] According to one or more embodiments of this disclosure, a terminal initialization method is provided, wherein a single feature point has at least 4 frames of image data and a single feature point has at least 4 constraint equations.

[0175] According to one or more embodiments of this disclosure, a terminal initialization method is provided, which projects the linear constraint equation onto the left null space of the corresponding feature points for dimensionality reduction, including:

[0176] The linear constraint equations are rearranged to obtain the coordinates P of the feature points corresponding to the linear constraint equations in the inertial coordinate system at the initial time. I0f Organized, among which, P I0f The coefficients in the simplified linear constraint equations are H. p ;

[0177] For H p Perform orthogonal triangular decomposition to obtain H p The column space basis Q1 and the null space basis Q2;

[0178] Project the rearranged linear constraint equations onto H p In the null basis Q2, P is eliminated using the orthogonality of Q1 and Q2. I0f .

[0179] According to one or more embodiments of this disclosure, a terminal initialization method is provided, wherein the superimposed integrated constraint equations have a dimension of 6.

[0180] According to one or more embodiments of this disclosure, a terminal initialization method is provided, which solves for the initialization parameters of the terminal at the initial time based on the integrated constraint equation, including:

[0181] The constraint on the modulus of gravitational acceleration is introduced into the integrated constraint equation using the Lagrange multiplier method.

[0182] Solving the integrated constraint equations yields the direction of gravitational acceleration in the inertial coordinate system at the initial moment, as well as the initialization parameters.

[0183] According to one or more embodiments of this disclosure, a terminal initialization method is provided, wherein the initialization parameters include one or more of the following: the direction of gravitational acceleration in the inertial coordinate system at the initial moment, the position of a feature point in the inertial coordinate system at the initial moment, and the velocity of the terminal device in the inertial coordinate system at the initial moment.

[0184] According to one or more embodiments of this disclosure, a terminal initialization apparatus is provided, comprising:

[0185] The acquisition unit is used to acquire image data of the terminal from the initial time to the end time and inertial data measured in an inertial coordinate system in response to the initialization event of the terminal, wherein the relative position of the inertial coordinate system and the terminal is fixed.

[0186] The control unit is used to establish the linear constraint equation of the feature point based on the constraint equation of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the representation of the coordinates of the feature point in the camera coordinate system as inertial data after the coordinates of the feature point in the camera coordinate system are transformed to the inertial coordinate system at the same moment.

[0187] The control unit is also used to project the linear constraint equations onto the left null space of the corresponding feature points for dimensionality reduction;

[0188] The control unit is also used to superimpose the linear constraint equations of each of the feature points to obtain an integrated constraint equation;

[0189] The control unit is also used to solve the initialization parameters of the terminal at the initial time according to the integrated constraint equation.

[0190] According to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one memory and at least one processor;

[0191] The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the method described in any one of the above.

[0192] According to one or more embodiments of the present disclosure, a computer-readable storage medium is provided for storing program code that, when executed by a processor, causes the processor to perform the methods described above.

[0193] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0194] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0195] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A terminal initialization method, characterized in that, include: In response to the terminal's initialization event, image data of the terminal from the initial time to the end time and inertial data measured in an inertial coordinate system are acquired. The relative position of the inertial coordinate system with the terminal is fixed, and the image data contains feature points. Based on the constraint equations of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the transformation of the coordinates of the feature point in the camera coordinate system to the inertial coordinate system at the same moment in the representation of inertial data, a linear constraint equation for the feature point is established. The linear constraint equations are projected onto the left null space of the corresponding feature points to reduce dimensionality. The integrated constraint equation is obtained by superimposing the linear constraint equations of each feature point. The initialization parameters of the terminal at the initial time are solved according to the integrated constraint equation.

2. The method according to claim 1, characterized in that, Based on the constraint equations relating the feature point's coordinates in the normalized plane to its coordinates in the camera coordinate system, and the transformation of the feature point's coordinates in the camera coordinate system to inertial coordinates at the same instant, represented as inertial data, a linear constraint equation for the feature point is established, including: The coordinates of the feature point in the camera coordinate system are transformed to the inertial coordinate system at the same moment and represented by inertial data. The coordinates of the feature point in the normalized plane are then substituted into the constraint equation of the coordinates of the feature point in the camera coordinate system. The initialization parameters to be calculated and the known inertial data are respectively arranged on both sides of the constraint equation to form the constraint equation of one frame of image data for a single feature point. The constraint equations of all image data for a single feature point are stacked to form a linear constraint equation for that single feature point.

3. The method according to claim 2, characterized in that, Each of the aforementioned feature points has at least 4 frames of image data and at least 4 constraint equations.

4. The method according to claim 1, characterized in that, Projecting the linear constraint equations onto the left null space of the corresponding feature points for dimensionality reduction includes: The linear constraint equations are rearranged to obtain the coordinates P of the feature points corresponding to the linear constraint equations in the inertial coordinate system at the initial time. I0f Organized, among which, P I0f The coefficients in the simplified linear constraint equations are H. p ; For H p Perform orthogonal trigonometric decomposition to obtain H p The column space basis Q1 and the null space basis Q2; Project the rearranged linear constraint equations onto H p In the null basis Q2, P is eliminated using the orthogonality of Q1 and Q2. I0f .

5. The method according to claim 1, characterized in that, The resulting integrated constraint equation has a dimension of 6.

6. The method according to claim 1, characterized in that, Solving for the initialization parameters of the terminal at the initial time based on the integrated constraint equations includes: The constraint on the modulus of gravitational acceleration is introduced into the integrated constraint equation using the Lagrange multiplier method. Solving the integrated constraint equations yields the direction of gravitational acceleration in the inertial coordinate system at the initial moment, as well as the initialization parameters.

7. The method according to claim 1, characterized in that, The initialization parameters include one or more of the following: the direction of gravitational acceleration in the inertial coordinate system at the initial moment, the position of the feature point in the inertial coordinate system at the initial moment, and the velocity of the terminal device in the inertial coordinate system at the initial moment.

8. A terminal initialization device, characterized in that, include: The acquisition unit is used to acquire image data of the terminal from the initial time to the end time and inertial data measured in an inertial coordinate system in response to the initialization event of the terminal. The relative position of the inertial coordinate system with the terminal is fixed, and the image data contains feature points. The control unit is used to establish the linear constraint equation of the feature point based on the constraint equation of the coordinates of the feature point in the normalized plane and the coordinates of the feature point in the camera coordinate system, and the representation of the coordinates of the feature point in the camera coordinate system as inertial data after the coordinates of the feature point in the camera coordinate system are transformed to the inertial coordinate system at the same moment. The control unit is also used to project the linear constraint equations onto the left null space of the corresponding feature points for dimensionality reduction; The control unit is also used to superimpose the linear constraint equations of each of the feature points to obtain an integrated constraint equation; The control unit is also used to solve the initialization parameters of the terminal at the initial time according to the integrated constraint equation.

9. An electronic device, comprising: At least one memory and at least one processor; The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the method of any one of claims 1 to 7.

10. A computer-readable storage medium for storing program code that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 7.

Images