Equipment initialization method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a device initialization method and device, an electronic device and a computer readable storage medium, and the method comprises the steps: obtaining inertial information of an augmented reality device and collecting a scene image through the augmented reality device in response to a starting operation of initializing an inertial vision parameter; identifying feature point information and scene plane information of the scene image; and according to the to-be-solved plane coefficient of the scene plane information, the inertial information and the feature point information, determining an inertial vision parameter and a target plane coefficient of the augmented reality device, the target plane coefficient being a solution corresponding to the to-be-solved plane coefficient. The inertial vision solving constraint based on the to-be-solved plane coefficient is realized, and the accuracy of the solved inertial vision parameter is improved. The to-be-solved plane coefficient is solved after initialization of the inertial vision parameters is completed, the environment understanding efficiency is improved, and the interaction efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of extended reality technology, and specifically to a device initialization method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] Extended Reality (XR) is a comprehensive term that covers virtual reality (VR), augmented reality (AR) and mixed reality (MR). XR technology expands the way humans perceive and interact, interacting with the real world through digital technology to create virtual, augmented and mixed reality experiences. XR devices are widely used in scenarios such as navigation, information display and augmented reality experience.

[0003] Among them, Visual-Inertial Odometry (VIO) is a key positioning and tracking algorithm in the field of extended reality. It achieves high-precision positioning of devices in three-dimensional space by fusing data from visual and inertial sensors.

[0004] However, in order to improve the accuracy of the inertial visual odometry, the inertial visual odometry needs to be initialized. However, the current initialization process lacks reference to environmental information, the accuracy of the initialization result needs to be improved, and usually, environmental understanding can only be performed after initialization is completed, which affects the interaction efficiency. Summary of the invention

[0005] The embodiments of the present application provide a device initialization method, apparatus, electronic device, and computer-readable storage medium, which can improve the accuracy of inertial vision initialization results and improve interaction efficiency.

[0006] In a first aspect, an embodiment of the present application provides a device initialization method, which is applied to an extended reality device, and the method includes: In response to a start-up operation of initializing inertial vision parameters, acquiring inertial information of the extended reality device and capturing a scene image through the extended reality device; Identifying feature point information and scene plane information of the scene image; The inertial visual parameters and the target plane coefficient of the extended reality device are determined according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, wherein the target plane coefficient is the solution corresponding to the plane coefficient to be solved.

[0007] In a second aspect, an embodiment of the present application further provides a device initialization apparatus, which is applied to an extended reality device, and the device includes: A response module, configured to obtain inertial information of the extended reality device and capture a scene image through the extended reality device in response to a start-up operation of initializing inertial vision parameters; A recognition module, used to recognize feature point information and scene plane information of the scene image; A determination module is used to determine the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, wherein the target plane coefficient is the solution corresponding to the plane coefficient to be solved.

[0008] Optionally, in some embodiments of the present application, determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information includes: Determine the three-dimensional coordinate representation corresponding to the feature point information according to the plane coefficient to be solved of the scene plane information; The inertial visual parameters and the target plane coefficient of the extended reality device are determined according to the three-dimensional coordinate representation, the inertial information and the feature point information.

[0009] Optionally, in some embodiments of the present application, determining the three-dimensional coordinate representation corresponding to the feature point information according to the plane coefficient to be solved of the scene plane information includes: Determine a spatial plane expression model corresponding to the scene plane information, wherein the spatial plane expression model is constructed based on the plane coefficients to be solved of the scene plane information; The three-dimensional coordinate representation is determined according to the image coordinate representation of the feature point information corresponding to the scene plane information, the camera internal parameters corresponding to the extended reality device, and the spatial plane expression model.

[0010] Optionally, in some embodiments of the present application, determining the three-dimensional coordinate representation according to the image coordinate representation of the feature point information corresponding to the scene plane information, the camera internal parameters corresponding to the extended reality device, and the spatial plane expression model includes: According to the camera intrinsic parameters corresponding to the extended reality device, the image coordinate representation of the feature point information corresponding to the scene plane information is converted into a normalized coordinate representation; The three-dimensional coordinate representation is determined according to the conversion relationship between the normalized coordinate representation and the three-dimensional coordinate representation and the spatial plane expression model.

[0011] Optionally, in some embodiments of the present application, the inertial information is acquired based on an inertial measurement unit in the extended reality device, the scene image is acquired by an image acquisition module in the extended reality device, and the three-dimensional coordinate representation includes a three-dimensional representation of the feature point information in a coordinate system of the image acquisition module; The determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the three-dimensional coordinate representation, the inertial information and the feature point information includes: Determine a three-dimensional coordinate conversion relationship and a three-dimensional coordinate projection relationship, wherein the three-dimensional coordinate conversion relationship includes a conversion relationship between a first coordinate system and a second coordinate system at the same time, and the three-dimensional coordinate projection relationship includes a conversion relationship between a first coordinate system and a second coordinate system at different times, wherein the first coordinate system corresponds to the inertial measurement unit, and the second coordinate system corresponds to the image acquisition module; Constructing a target model to be solved according to the three-dimensional coordinate representation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship; The target model to be solved is solved according to the inertial information and the feature point information, and the inertial visual parameters and the target plane coefficient of the extended reality device are determined.

[0012] Optionally, in some embodiments of the present application, constructing the target model to be solved according to the three-dimensional coordinate representation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship includes: Determine a first three-dimensional representation from the three-dimensional coordinate representation, where the first three-dimensional representation is a three-dimensional representation of the feature point information at the initial moment of the start-up operation in a second coordinate system at the initial moment of the image acquisition module; Determine a second three-dimensional representation according to the first three-dimensional representation and the three-dimensional coordinate conversion relationship, wherein the second three-dimensional representation represents a three-dimensional representation of a feature point at an initial moment of the start-up operation in the first coordinate system of the inertial measurement unit; A target model to be solved is constructed according to the second three-dimensional representation and the three-dimensional coordinate projection relationship.

[0013] Optionally, in some embodiments of the present application, determining the three-dimensional coordinate projection relationship includes: Determining an inertial vision observation model corresponding to the inertial measurement unit; Performing pre-integration processing on the inertial visual observation model to obtain a pre-integration representation of inertial parameters; Determining a three-dimensional coordinate transformation relationship for the inertial measurement unit according to the inertial parameter pre-integration representation, wherein the three-dimensional coordinate transformation relationship includes a transformation relationship of a coordinate system of feature point information at different times; A three-dimensional coordinate projection relationship is constructed according to the three-dimensional coordinate transformation relationship and the external parameters between the inertial measurement unit and the image acquisition module.

[0014] Optionally, in some embodiments of the present application, solving the target model to be solved according to the inertial information and the feature point information to determine the inertial visual parameters and the target plane coefficient of the extended reality device includes: Transforming the target model to be solved according to the coupling relationship to obtain a simplified model, wherein the coupling relationship includes a coupling relationship between the plane coefficient to be solved and the inertial information to be solved; The simplified model is solved according to the inertial information and the feature point information to obtain the inertial vision parameters of the inertial measurement unit at the initial moment of the startup operation and the target plane coefficient of the scene image at the initial moment.

[0015] Optionally, in some embodiments of the present application, the inertial vision parameter includes an initial inertial vision parameter at the initial moment of the startup operation, and the target plane coefficient includes an initial plane coefficient of the scene plane information at the initial moment; After determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, the method further includes: Determine the three-dimensional coordinates of the feature point information at the initial moment according to the initial plane coefficients; Solving a nonlinear optimization model according to the three-dimensional coordinates, the initial inertial vision parameters, the inertial information, and the feature point information to obtain optimized plane coefficients corresponding to the scene plane information at the initial moment and optimized inertial vision parameters at the initial moment; The nonlinear optimization model is constructed based on a spatial plane expression model corresponding to a three-dimensional coordinate projection relationship, a pre-integrated representation of inertial parameters, and scene plane information at an initial moment.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and which implements the steps in the above-mentioned device initialization method when the computer program is executed by the processor.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned device initialization method are implemented.

[0018] In a fifth aspect, the embodiments of the present application further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0019] The extended reality device of the embodiment of the present application responds to the startup operation of initializing inertial vision parameters, obtains the inertial information of the extended reality device and captures a scene image through the extended reality device, identifies the feature point information and scene plane information of the scene image, and determines the inertial vision parameters and target plane coefficients of the extended reality device according to the plane coefficients to be solved, inertial information and feature point information of the scene plane information, wherein the target plane coefficients are the solutions corresponding to the plane coefficients to be solved.

[0020] Among them, the embodiment of the present application initializes the inertial vision parameters by combining the plane coefficients to be solved with the scene plane information, realizes the inertial vision solution constraints based on the plane coefficients to be solved, and improves the accuracy of the solved inertial vision parameters.

[0021] Among them, by initializing the inertial vision parameters in combination with the plane coefficients to be solved, the plane coefficients to be solved are solved after the inertial vision parameter initialization is completed, and the plane detection is completed, so that the plane detection is no longer dependent on the completion of the initialization of the inertial vision parameters, thereby improving the efficiency of environmental understanding and the interaction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 It is a schematic diagram of a scenario in which an extended reality device according to an embodiment of the present application executes the device initialization method; Figure 2 It is a flowchart of a device initialization method provided in an embodiment of the present application; Figure 3 It is a structural schematic diagram of a device initialization apparatus provided in an embodiment of the present application; Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0024] Description of Figure Numbers: 10-extended reality device; 201-response module; 202-identification module; 203-determination module; 301-processor; 302-memory; 303-power supply; 304-input unit. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] The embodiments of the present application provide a device initialization method, device, electronic device and computer-readable storage medium. Specifically, the embodiments of the present application provide a device initialization device suitable for electronic devices, which is used to improve the accuracy of inertial vision initialization results and improve interaction efficiency. Specifically, the electronic device includes a terminal device with an inertial measurement unit and an image capture module, and the terminal device includes but is not limited to an extended reality device, and the extended reality device includes but is not limited to a head-mounted display, wearable glasses, and the terminal device can also be an integrated terminal device built into a computing processing unit, or a split terminal device external to the computing processing unit. Among them, the terminal device includes but is not limited to an airborne optical display system, i.e., a head-up display system, applied to vehicles such as aircraft, cars, and ships, for example, an AR-HUD (Augmented reality hud-up display) mounted on an intelligent networked car, an extended reality application applied to handheld mobile devices such as mobile phones, laptops, and tablets (such as extended reality games, virtual tourism, telemedicine, or virtual experiments, etc.), and a near-eye display system applied to wearable devices such as head-mounted displays and smart glasses.

[0027] See also Figure 1 , Figure 1 : is a schematic diagram of a scenario in which an extended reality device according to an embodiment of the present application performs the device initialization method, wherein the specific execution process of the extended reality device performing the device initialization method is as follows: The extended reality device 10 responds to the navigation and positioning tasks and starts the operation of initializing the inertial vision parameters. In response to the operation, the extended reality device 10 obtains the inertial information in a short period of time and collects the scene image. Subsequently, the extended reality device recognizes the scene image to obtain the feature point information and the scene plane information, and determines the inertial vision parameters of the extended reality device and the target plane coefficients for the plane coefficients to be solved according to the plane coefficients to be solved of the scene plane information, the inertial information and the feature point information, thereby initializing the inertial vision parameters of the extended reality device.

[0028] Subsequently, the extended reality device can use the initialized inertial vision parameters to perform navigation and positioning tasks, and at the same time, can also use the target plane coefficient to perform extended reality interaction processing.

[0029] In summary, the embodiments of the present application initialize the inertial vision parameters by combining the plane coefficients to be solved with the scene plane information, thereby realizing the inertial vision solution constraints based on the plane coefficients to be solved and improving the accuracy of the solved inertial vision parameters.

[0030] Among them, by initializing the inertial vision parameters in combination with the plane coefficients to be solved, the plane coefficients to be solved are solved after the inertial vision parameter initialization is completed, and the plane detection is completed, so that the plane detection is no longer dependent on the completion of the initialization of the inertial vision parameters, thereby improving the efficiency of environmental understanding and the interaction efficiency.

[0031] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0032] See also Figure 2 , Figure 2 A flowchart of a device initialization method provided in an embodiment of the present application. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that shown in the flowchart. Specifically, the execution subject of the device initialization method is an extended reality device. Specifically, the process of the extended reality device executing the device initialization method specifically includes: 101. In response to a startup operation of initializing inertial vision parameters, acquiring inertial information of the extended reality device and capturing a scene image through the extended reality device.

[0033] It can be understood that the startup operation is the operation started at the first time of applying the inertial vision parameters or the inertial vision system, and the purpose is to complete the initialization of the inertial vision parameters at the first time, so as to perform high-precision positioning based on the initialized inertial vision parameters.

[0034] Among them, inertial information is data collected by visual and inertial sensors, for example, inertial data collected by an inertial measurement unit (IMU), and the inertial information includes acceleration information, angular velocity information, etc. It is understandable that in order to improve the accuracy of inertial vision parameter initialization, the inertial information or inertial data is data collected within a short time after the initialization operation is started.

[0035] Among them, the scene image is an image that is collected in the aforementioned short period of time and matches the inertial information. For example, the scene image is collected by an image acquisition module (such as a camera) on the extended reality device while collecting inertial information based on an inertial measurement unit.

[0036] 102. Identify feature point information and scene plane information of the scene image.

[0037] The feature point information is information that has features or belongs to key content in the scene image, also known as key point information. The feature point information includes the location of the feature point, which is usually expressed in coordinates.

[0038] The scene plane information refers to the planes contained in the scene image and information related to the planes, such as the number of planes, the size of the planes, and the positions of the planes.

[0039] It can be understood that the feature point information can be extracted from the scene image based on a deep learning model or a machine learning model; the scene plane information can be obtained by performing plane segmentation on the scene image through a lightweight plane semantic segmentation network. It can be understood that each scene image may include multiple planes.

[0040] 103. Determine the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, wherein the target plane coefficient is the solution corresponding to the plane coefficient to be solved.

[0041] It should be noted that the plane coefficient is a coefficient reflecting the characteristics of the plane, for example, the plane coefficient includes a plane normal vector and a plane distance. The plane coefficient to be solved is the plane coefficient to be solved corresponding to the scene plane information in the scene image.

[0042] In summary, the embodiments of the present application initialize the inertial vision parameters by combining the plane coefficients to be solved with the scene plane information, thereby realizing the inertial vision solution constraints based on the plane coefficients to be solved and improving the accuracy of the solved inertial vision parameters.

[0043] Among them, by initializing the inertial vision parameters in combination with the plane coefficients to be solved, the plane coefficients to be solved are solved after the inertial vision parameter initialization is completed, and the plane detection is completed, so that the plane detection is no longer dependent on the completion of the initialization of the inertial vision parameters, thereby improving the efficiency of environmental understanding and the interaction efficiency.

[0044] Among them, the accuracy of inertial vision parameter initialization can be improved based on the three-dimensional coordinates corresponding to the feature point information, wherein the three-dimensional coordinates of the feature point information can be expressed based on the plane coefficient of the scene plane information where the feature point information is located, that is, optionally, in some embodiments of the present application, the step of "determining the inertial vision parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information" includes: Determine the three-dimensional coordinate representation corresponding to the feature point information according to the plane coefficient to be solved of the scene plane information; The inertial visual parameters and the target plane coefficient of the extended reality device are determined according to the three-dimensional coordinate representation, the inertial information and the feature point information.

[0045] Among them, the inertial vision parameters are initialized by using a three-dimensional coordinate representation based on the plane coefficient to be solved, so that when the initialization is completed, the target plane coefficient corresponding to the plane coefficient to be solved is directly solved, thereby realizing the synchronous solution of the plane coefficient when the inertial vision parameters are initialized.

[0046] Among them, the three-dimensional coordinate representation of the feature point information can be determined based on the spatial plane expression model, that is, optionally, in some embodiments of the present application, the step of "determining the three-dimensional coordinate representation corresponding to the feature point information according to the plane coefficient to be solved of the scene plane information" includes: Determine a spatial plane expression model corresponding to the scene plane information, wherein the spatial plane expression model is constructed based on the plane coefficients to be solved of the scene plane information; The three-dimensional coordinate representation is determined according to the image coordinate representation of the feature point information corresponding to the scene plane information, the camera internal parameters corresponding to the extended reality device, and the spatial plane expression model.

[0047] Furthermore, the normalized coordinates corresponding to the feature point information may be determined based on the camera internal parameters of the extended reality device, and the three-dimensional coordinate representation may be determined based on the normalized coordinates.

[0048] For example, with Represents each frame of scene image and the coordinate system of each frame of scene image, where: represents the first frame scene image (also represents the coordinate system of the first frame scene image), that is, the scene image collected at the initial moment of the start operation, and the feature point information (i.e., feature point) on the scene image is represented by express, Representing scene images The kth feature point information or feature point extracted from The three-dimensional coordinates in the coordinate system are expressed as , and the spatial plane expression model corresponding to the plane in space is: , where the normal vector of the plane is expressed as , D represents the plane distance. Correspondingly, for the above feature point information, the spatial plane expression model is satisfied, that is, the following expression is satisfied: .

[0049] Among them, the camera intrinsic parameters of the extended reality device are expressed as , feature point information The corresponding normalized coordinates are , then there is always: , .

[0050] Based on the above and , We can get:

[0051] in, , , .

[0052] Then, the three-dimensional coordinate representation of the feature point information represented by the plane coefficient to be solved is obtained, that is:

[0053] That is, through the above processing, the representation of the three-dimensional coordinates by the plane coefficients to be solved based on the scene plane information is achieved.

[0054] Among them, in the embodiments of the present application, a target model to be solved based on inertial visual parameters and plane coefficients to be solved can be established, and the inertial visual parameters and target plane coefficients can be determined by solving the target model to be solved. Among them, since the inertial information is acquired based on the inertial measurement unit, and the scene image is acquired based on the image acquisition module, the coordinate systems of the inertial information and the scene image are different. Therefore, it is necessary to combine the coordinate transformation relationship to construct the target model to be solved, that is, optionally, in some embodiments of the present application, the inertial information is acquired based on the inertial measurement unit in the extended reality device, and the scene image is acquired by the image acquisition module in the extended reality device. The three-dimensional coordinate representation includes the three-dimensional representation of the feature point information in the coordinate system of the image acquisition module. The step of "determining the inertial visual parameters and target plane coefficients of the extended reality device according to the three-dimensional coordinate representation, the inertial information and the feature point information" includes: Determine a three-dimensional coordinate conversion relationship and a three-dimensional coordinate projection relationship, wherein the three-dimensional coordinate conversion relationship includes a conversion relationship between a first coordinate system and a second coordinate system at the same time, and the three-dimensional coordinate projection relationship includes a conversion relationship between a first coordinate system and a second coordinate system at different times, wherein the first coordinate system corresponds to the inertial measurement unit, and the second coordinate system corresponds to the image acquisition module; Constructing a target model to be solved according to the three-dimensional coordinate representation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship; The target model to be solved is solved according to the inertial information and the feature point information, and the inertial visual parameters and the target plane coefficient of the extended reality device are determined.

[0055] Among them, since the purpose of initializing the inertial measurement parameters is to obtain the inertial measurement parameters at the initial moment of the startup operation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship can be established based on the three-dimensional coordinate representation at the initial moment, that is, optionally, in some embodiments of the present application, the step of "constructing the target model to be solved according to the three-dimensional coordinate representation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship" includes: Determine a first three-dimensional representation from the three-dimensional coordinate representation, where the first three-dimensional representation is a three-dimensional representation of the feature point information at the initial moment of the start-up operation in a second coordinate system at the initial moment of the image acquisition module; Determine a second three-dimensional representation according to the first three-dimensional representation and the three-dimensional coordinate conversion relationship, wherein the second three-dimensional representation represents a three-dimensional representation of a feature point at an initial moment of the start-up operation in the first coordinate system of the inertial measurement unit; A target model to be solved is constructed according to the second three-dimensional representation and the three-dimensional coordinate projection relationship.

[0056] Among them, for the three-dimensional coordinate projection relationship, it can be obtained by pre-integration processing based on the inertial visual observation model. That is, optionally, in some embodiments of the present application, the step of "determining the three-dimensional coordinate projection relationship" includes: Determining an inertial vision observation model corresponding to the inertial measurement unit; Performing pre-integration processing on the inertial visual observation model to obtain a pre-integration representation of inertial parameters; Determining a three-dimensional coordinate transformation relationship for the inertial measurement unit according to the inertial parameter pre-integration representation, wherein the three-dimensional coordinate transformation relationship includes a transformation relationship of a coordinate system of feature point information at different times; A three-dimensional coordinate projection relationship is constructed according to the three-dimensional coordinate transformation relationship and the external parameters between the inertial measurement unit and the image acquisition module.

[0057] For example, the first three-dimensional representation is written as: , the second three-dimensional representation is recorded as: , a second three-dimensional representation based on the first three-dimensional representation and the three-dimensional coordinate transformation relationship Then record it as:

[0058] Among them, the above The formula representation is recorded as the first formula representation, and Represents the external parameters between the inertial measurement unit and the image acquisition module. The coordinate system of the inertial measurement unit is recorded as , the three-dimensional coordinate projection relationship is to transform the coordinate system of the inertial measurement unit To the coordinate system of the image acquisition module Therefore, the transformation based on the three-dimensional coordinate projection relationship is Towards The conversion is expressed as:

[0059] Among them, the above The formula of is denoted as the second formula, and Represents the external parameters between the image acquisition module and the inertial measurement unit, Representing coordinate system and The external parameters between Indicates that The feature point information in the coordinate system is converted into The feature point information in the coordinate system is The feature point information projection in the coordinate system is Feature point information in the coordinate system.

[0060] Among them, for The feature point information in the coordinate system is converted into The feature point information in the coordinate system can be obtained based on the pre-integration method. For example, for the inertial measurement unit, the inertial information collected includes accelerometer data and gyroscope data , accordingly, the inertial visual inertial model corresponding to the inertial measurement unit includes:

[0061] in, , is a random deviation, is the expression of gravity acceleration in the inertial world coordinate system. At the same time, the accelerometer and gyroscope measurements are also affected by white noise, which are Through Euler median integration, the position and velocity of two adjacent moments can be connected by the following formula:

[0062] in, Represents the attitude, corresponding to the rotation matrix from the world coordinate system G to the inertial measurement unit coordinate system B, represents the velocity in the world coordinate system G, Represents the position in the world coordinate system G.

[0063] When the state of the inertial vision measurement system changes, the observation data of the inertial measurement unit will be recalculated based on the above relationship. In order to avoid repeated calculations, the above relationship is sorted out by pre-integration, and then:

[0064] in, is a pre-integrated term that is only affected by the random bias.

[0065] Similarly, based on the above pre-integration design, the i-th frame can be obtained, that is, arrive The pre-integration between , that is, includes: Similarly, we can also get the representation between the first frame to the i-th frame, that is, arrive The pre-integration between , that is, includes: .

[0066] Furthermore, since only the three-dimensional coordinate representation of the first frame is required, based on the above pre-integration representation, we obtain:

[0067] Furthermore, assume that:

[0068] Based on The pre-integrated representation and the above assumptions can be rearranged to obtain the third formula:

[0069] Then, the first formula and the third formula are substituted into the second formula to obtain the target model to be solved, which is expressed as:

[0070] From the above formula, we can see that the only unknown variable is the plane coefficient. And the velocity and gravity in the coordinate system The expression below , .

[0071] Further assume that:

[0072] Then, based on , It can be seen that Therefore, multiply both sides of the above target model by , in order to eliminate the above-mentioned target model to be solved , and the transformed target model is obtained, which is expressed as:

[0073] Further assume that:

[0074] Based on the above assumptions, we can get:

[0075] For the above formula, multiply both sides of the equation by , then put the known quantity on the right side of the equation, and the part with the unknown quantity on the left side of the equation, and continue to simplify to get the fourth formula, then we have:

[0076] Among them, it can be seen from the above formula that the plane coefficient and inertia information or inertia coefficient , will be coupled with each other, so we can solve based on the coupling relationship, that is, optionally, in some embodiments of the present application, the step of "solving the target model to be solved according to the inertial information and the feature point information, and determining the inertial visual parameters and the target plane coefficient of the extended reality device" includes: Transforming the target model to be solved according to the coupling relationship to obtain a simplified model, wherein the coupling relationship includes a coupling relationship between the plane coefficient to be solved and the inertial information to be solved; The simplified model is solved according to the inertial information and the feature point information to obtain the inertial vision parameters of the inertial measurement unit at the initial moment of the startup operation and the target plane coefficient of the scene image at the initial moment.

[0077] For example, the mutually coupled parameters are treated as unknown quantities again, so the assumption (referred to as the coupling assumption) is:

[0078] Based on the above assumptions, the fourth formula is sorted out to obtain:

[0079] Arranging the above formula into a linear matrix expression, we have: .in,

[0080]

[0081]

[0082] Furthermore, from the linear matrix expression It can be seen that a feature point information in the scene image A 2*1 linear constraint will be generated. Assuming that there are K feature points on a plane and a total of L scene images, a total of 2*K*L equality constraints will be generated. As long as 2KL ≥ 21, the above linear matrix expression We can get the solution, that is, we get the above If we get The initial value of the solution can be indirectly solved by the above coupling assumption. and , and based on , , The relationship between the target plane coefficients corresponding to the plane coefficients to be solved is obtained. , and .

[0083] After solving the inertial visual parameters, the gravitational acceleration can be obtained. The value below , assuming that the magnitude of gravity in the inertial world coordinate system is , through Schmidt orthogonal decomposition, we can get , express The rotation in the inertial world coordinate system is simultaneously get The velocity in the inertial world coordinate system is Can be directly set to zero vector. Based on the obtained coordinate system Rotation in world coordinates ,Location ,speed , we can recover each one according to the above formula of posture and velocity at two adjacent moments The position and velocity of are expressed as: , and .

[0084] Furthermore, after solving the inertial visual parameters and target plane coefficients at the initial moment, the inertial visual parameters, target plane coefficients, and The posture and speed of the augmented reality device are used to establish a nonlinear optimization model, and then the inertial vision parameters and the target plane parameters at the initial moment are optimized. That is, optionally, in some embodiments of the present application, the inertial vision parameters include the initial inertial vision parameters at the initial moment of the startup operation, and the target plane coefficients include the initial plane coefficients of the scene plane information at the initial moment. The step of "determining the inertial vision parameters and the target plane coefficients of the augmented reality device according to the plane coefficients to be solved of the scene plane information, the inertial information and the feature point information" includes: Determine the three-dimensional coordinates of the feature point information at the initial moment according to the initial plane coefficients; Solving a nonlinear optimization model according to the three-dimensional coordinates, the initial inertial vision parameters, the inertial information, and the feature point information to obtain optimized plane coefficients corresponding to the scene plane information at the initial moment and optimized inertial vision parameters at the initial moment; The nonlinear optimization model is constructed based on a spatial plane expression model corresponding to a three-dimensional coordinate projection relationship, a pre-integrated representation of inertial parameters, and scene plane information at an initial moment.

[0085] For example, the variables to be optimized are:

[0086] in, .

[0087] A nonlinear optimization model is established, which is expressed as:

[0088] in, Represents the reprojection error constraint equation, corresponding to the formula:

[0089] and

[0090] in, Represents the inertial measurement unit (IMU) pre-integration constraint equation, corresponding to the formula:

[0091] in, Represents the plane constraint equation, corresponding formula: .

[0092] Among them, the nonlinear optimization library (ceres) is used to Solve and obtain more accurate optimized plane coefficients and optimized inertial vision parameters, for example, obtain the posture and speed at each moment, and then obtain the trajectory, and also obtain the three-dimensional coordinates of each feature point information.

[0093] In summary, the embodiments of the present application initialize the inertial vision parameters by combining the plane coefficients to be solved with the scene plane information, thereby realizing the inertial vision solution constraints based on the plane coefficients to be solved and improving the accuracy of the solved inertial vision parameters.

[0094] Among them, by initializing the inertial vision parameters in combination with the plane coefficients to be solved, the plane coefficients to be solved are solved after the inertial vision parameter initialization is completed, and the plane detection is completed, so that the plane detection is no longer dependent on the completion of the initialization of the inertial vision parameters, thereby improving the efficiency of environmental understanding and the interaction efficiency.

[0095] In order to better implement the device initialization method of the present application, the present application also provides a device initialization apparatus based on the device initialization method, wherein the meanings of the terms are the same as those in the device initialization method, and the specific implementation details can refer to the description in the method embodiment.

[0096] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of a device initialization device provided in an embodiment of the present application. The device initialization device is applied to an extended reality device. The device initialization device may be specifically as follows: A response module 201 is used for obtaining inertial information of the extended reality device and collecting scene images through the extended reality device in response to a start-up operation of initializing inertial vision parameters; The recognition module 202 is used to recognize feature point information and scene plane information of the scene image; The determination module 203 is used to determine the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, wherein the target plane coefficient is the solution corresponding to the plane coefficient to be solved.

[0097] Optionally, in some embodiments of the present application, determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information includes: Determine the three-dimensional coordinate representation corresponding to the feature point information according to the plane coefficient to be solved of the scene plane information; The inertial visual parameters and the target plane coefficient of the extended reality device are determined according to the three-dimensional coordinate representation, the inertial information and the feature point information.

[0098] Optionally, in some embodiments of the present application, determining the three-dimensional coordinate representation corresponding to the feature point information according to the plane coefficient to be solved of the scene plane information includes: Determine a spatial plane expression model corresponding to the scene plane information, wherein the spatial plane expression model is constructed based on the plane coefficients to be solved of the scene plane information; The three-dimensional coordinate representation is determined according to the image coordinate representation of the feature point information corresponding to the scene plane information, the camera internal parameters corresponding to the extended reality device, and the spatial plane expression model.

[0099] Optionally, in some embodiments of the present application, determining the three-dimensional coordinate representation according to the image coordinate representation of the feature point information corresponding to the scene plane information, the camera internal parameters corresponding to the extended reality device, and the spatial plane expression model includes: According to the camera intrinsic parameters corresponding to the extended reality device, the image coordinate representation of the feature point information corresponding to the scene plane information is converted into a normalized coordinate representation; The three-dimensional coordinate representation is determined according to the conversion relationship between the normalized coordinate representation and the three-dimensional coordinate representation and the spatial plane expression model.

[0100] Optionally, in some embodiments of the present application, the inertial information is acquired based on an inertial measurement unit in the extended reality device, the scene image is acquired by an image acquisition module in the extended reality device, and the three-dimensional coordinate representation includes a three-dimensional representation of the feature point information in a coordinate system of the image acquisition module; The determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the three-dimensional coordinate representation, the inertial information and the feature point information includes: Determine a three-dimensional coordinate conversion relationship and a three-dimensional coordinate projection relationship, wherein the three-dimensional coordinate conversion relationship includes a conversion relationship between a first coordinate system and a second coordinate system at the same time, and the three-dimensional coordinate projection relationship includes a conversion relationship between a first coordinate system and a second coordinate system at different times, wherein the first coordinate system corresponds to the inertial measurement unit, and the second coordinate system corresponds to the image acquisition module; Constructing a target model to be solved according to the three-dimensional coordinate representation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship; The target model to be solved is solved according to the inertial information and the feature point information, and the inertial visual parameters and the target plane coefficient of the extended reality device are determined.

[0101] Optionally, in some embodiments of the present application, constructing the target model to be solved according to the three-dimensional coordinate representation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship includes: Determine a first three-dimensional representation from the three-dimensional coordinate representation, where the first three-dimensional representation is a three-dimensional representation of the feature point information at the initial moment of the start-up operation in a second coordinate system at the initial moment of the image acquisition module; Determine a second three-dimensional representation according to the first three-dimensional representation and the three-dimensional coordinate conversion relationship, wherein the second three-dimensional representation represents a three-dimensional representation of a feature point at an initial moment of the start-up operation in the first coordinate system of the inertial measurement unit; A target model to be solved is constructed according to the second three-dimensional representation and the three-dimensional coordinate projection relationship.

[0102] Optionally, in some embodiments of the present application, determining the three-dimensional coordinate projection relationship includes: Determining an inertial vision observation model corresponding to the inertial measurement unit; Performing pre-integration processing on the inertial visual observation model to obtain a pre-integration representation of inertial parameters; Determining a three-dimensional coordinate transformation relationship for the inertial measurement unit according to the inertial parameter pre-integration representation, wherein the three-dimensional coordinate transformation relationship includes a transformation relationship of a coordinate system of feature point information at different times; A three-dimensional coordinate projection relationship is constructed according to the three-dimensional coordinate transformation relationship and the external parameters between the inertial measurement unit and the image acquisition module.

[0103] Optionally, in some embodiments of the present application, solving the target model to be solved according to the inertial information and the feature point information to determine the inertial visual parameters and the target plane coefficient of the extended reality device includes: Transforming the target model to be solved according to the coupling relationship to obtain a simplified model, wherein the coupling relationship includes a coupling relationship between the plane coefficient to be solved and the inertial information to be solved; The simplified model is solved according to the inertial information and the feature point information to obtain the inertial vision parameters of the inertial measurement unit at the initial moment of the startup operation and the target plane coefficient of the scene image at the initial moment.

[0104] Optionally, in some embodiments of the present application, the inertial vision parameter includes an initial inertial vision parameter at the initial moment of the startup operation, and the target plane coefficient includes an initial plane coefficient of the scene plane information at the initial moment; After determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, the method further includes: Determine the three-dimensional coordinates of the feature point information at the initial moment according to the initial plane coefficients; Solving a nonlinear optimization model according to the three-dimensional coordinates, the initial inertial vision parameters, the inertial information, and the feature point information to obtain optimized plane coefficients corresponding to the scene plane information at the initial moment and optimized inertial vision parameters at the initial moment; The nonlinear optimization model is constructed based on a spatial plane expression model corresponding to a three-dimensional coordinate projection relationship, a pre-integrated representation of inertial parameters, and scene plane information at an initial moment.

[0105] In the embodiment of the present application, the response module 201 responds to the startup operation of initializing the inertial vision parameters, obtains the inertial information of the extended reality device and captures the scene image through the extended reality device, the recognition module 202 recognizes the feature point information and scene plane information of the scene image, and the determination module 203 determines the inertial vision parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, wherein the target plane coefficient is the solution corresponding to the plane coefficient to be solved.

[0106] In summary, the embodiments of the present application initialize the inertial vision parameters by combining the plane coefficients to be solved with the scene plane information, thereby realizing the inertial vision solution constraints based on the plane coefficients to be solved and improving the accuracy of the solved inertial vision parameters.

[0107] Among them, by initializing the inertial vision parameters in combination with the plane coefficients to be solved, the plane coefficients to be solved are solved after the inertial vision parameter initialization is completed, and the plane detection is completed, so that the plane detection is no longer dependent on the completion of the initialization of the inertial vision parameters, thereby improving the efficiency of environmental understanding and the interaction efficiency.

[0108] In addition, the present application also provides an electronic device, such as Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in this application, specifically: The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will appreciate that Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. The processor 301 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0109] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and device initialization by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0110] The electronic device also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 303 can also include any components such as one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, and power status indicators.

[0111] The electronic device may further include an input unit 304, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0112] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby implementing the steps in any device initialization method provided in the embodiments of the present application.

[0113] Among them, in the embodiment of the present application, the electronic device (terminal device) can also be an extended reality device, and the distance measurement of the object to be measured is performed through the inertial measurement module and the camera module on the extended reality device. Among them, the extended reality device includes smart glasses in the form of natural glasses, and the smart glasses have at least the functions of extended reality display, wearing status detection, biometric recognition, human-computer interaction, data processing, etc. It includes a frame, temples, processors, sensors, optical display components, microphones, speakers, etc. There are cavities inside the temples and frames, and circuits and electronic components are placed in the cavities. Sensors include cameras, eye trackers, iris meters, IMUs, gyroscopes, etc. The optical display components include micro-projection optical machines and optical couplers. The micro-projection optical machine can be based on Micro-Oled, Micro-Led, LCOS, or LBS. The optical coupler can be an optical lens or an optical waveguide. The processor can be an XR professional processor or an ordinary general-purpose processor. The frame and temples are the supporting structure of the entire glasses. The temples have a certain degree of elasticity, and the length and clamping force can be adjusted to suit users with different head shapes; the camera can take pictures of the user's hands, face, eyes, etc., the microphone can listen to the user's voice, and the processor can calculate and process various data.

[0114] The extended reality device of the embodiment of the present application responds to the startup operation of initializing inertial vision parameters, obtains the inertial information of the extended reality device and captures a scene image through the extended reality device, identifies the feature point information and scene plane information of the scene image, and determines the inertial vision parameters and target plane coefficients of the extended reality device according to the plane coefficients to be solved, inertial information and feature point information of the scene plane information, wherein the target plane coefficients are the solutions corresponding to the plane coefficients to be solved.

[0115] Among them, the embodiment of the present application initializes the inertial vision parameters by combining the plane coefficients to be solved with the scene plane information, realizes the inertial vision solution constraints based on the plane coefficients to be solved, and improves the accuracy of the solved inertial vision parameters.

[0116] Among them, by initializing the inertial vision parameters in combination with the plane coefficients to be solved, the plane coefficients to be solved are solved after the inertial vision parameter initialization is completed, and the plane detection is completed, so that the plane detection is no longer dependent on the completion of the initialization of the inertial vision parameters, thereby improving the efficiency of environmental understanding and the interaction efficiency.

[0117] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0118] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0119] To this end, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any device initialization method provided in the present application.

[0120] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0121] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0122] Since the instructions stored in the computer-readable storage medium can execute the steps in any device initialization method provided in the present application, the beneficial effects that can be achieved by any device initialization method provided in the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0123] The above is a detailed introduction to a device initialization method, apparatus, electronic device and computer-readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A device initialization method, characterized in that: Applied to an extended reality device, the method comprises: In response to a start-up operation of initializing inertial vision parameters, acquiring inertial information of the extended reality device and capturing a scene image through the extended reality device; Identifying feature point information and scene plane information of the scene image; The inertial visual parameters and the target plane coefficient of the extended reality device are determined according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, wherein the target plane coefficient is the solution corresponding to the plane coefficient to be solved.

2. The device initialization method according to claim 1, characterized in that: The determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information includes: Determine the three-dimensional coordinate representation corresponding to the feature point information according to the plane coefficient to be solved of the scene plane information; The inertial visual parameters and the target plane coefficient of the extended reality device are determined according to the three-dimensional coordinate representation, the inertial information and the feature point information.

3. The device initialization method according to claim 2, characterized in that: The determining the three-dimensional coordinate representation corresponding to the feature point information according to the plane coefficient to be solved of the scene plane information includes: Determine a spatial plane expression model corresponding to the scene plane information, wherein the spatial plane expression model is constructed based on the plane coefficients to be solved of the scene plane information; The three-dimensional coordinate representation is determined according to the image coordinate representation of the feature point information corresponding to the scene plane information, the camera internal parameters corresponding to the extended reality device, and the spatial plane expression model.

4. The device initialization method according to claim 3, characterized in that: The determining of the three-dimensional coordinate representation according to the image coordinate representation of the feature point information corresponding to the scene plane information, the camera internal parameters corresponding to the extended reality device, and the spatial plane expression model includes: According to the camera intrinsic parameters corresponding to the extended reality device, the image coordinate representation of the feature point information corresponding to the scene plane information is converted into a normalized coordinate representation; The three-dimensional coordinate representation is determined according to the conversion relationship between the normalized coordinate representation and the three-dimensional coordinate representation and the spatial plane expression model.

5. The device initialization method according to claim 2, characterized in that: The inertial information is acquired based on an inertial measurement unit in the extended reality device, the scene image is acquired by an image acquisition module in the extended reality device, and the three-dimensional coordinate representation includes a three-dimensional representation of the feature point information in a coordinate system of the image acquisition module; The determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the three-dimensional coordinate representation, the inertial information and the feature point information includes: Determine a three-dimensional coordinate conversion relationship and a three-dimensional coordinate projection relationship, wherein the three-dimensional coordinate conversion relationship includes a conversion relationship between a first coordinate system and a second coordinate system at the same time, and the three-dimensional coordinate projection relationship includes a conversion relationship between a first coordinate system and a second coordinate system at different times, wherein the first coordinate system corresponds to the inertial measurement unit, and the second coordinate system corresponds to the image acquisition module; Constructing a target model to be solved according to the three-dimensional coordinate representation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship; The target model to be solved is solved according to the inertial information and the feature point information, and the inertial visual parameters and the target plane coefficient of the extended reality device are determined.

6. The device initialization method according to claim 5, characterized in that: The step of constructing the target model to be solved according to the three-dimensional coordinate representation, the three-dimensional coordinate transformation relationship and the three-dimensional coordinate projection relationship includes: Determine a first three-dimensional representation from the three-dimensional coordinate representation, where the first three-dimensional representation is a three-dimensional representation of the feature point information at the initial moment of the start-up operation in a second coordinate system at the initial moment of the image acquisition module; Determine a second three-dimensional representation according to the first three-dimensional representation and the three-dimensional coordinate conversion relationship, wherein the second three-dimensional representation represents a three-dimensional representation of a feature point at an initial moment of the start-up operation in the first coordinate system of the inertial measurement unit; A target model to be solved is constructed according to the second three-dimensional representation and the three-dimensional coordinate projection relationship.

7. The device initialization method according to claim 6, characterized in that: Determining the three-dimensional coordinate projection relationship includes: Determining an inertial vision observation model corresponding to the inertial measurement unit; Performing pre-integration processing on the inertial visual observation model to obtain a pre-integration representation of inertial parameters; Determining a three-dimensional coordinate transformation relationship for the inertial measurement unit according to the inertial parameter pre-integration representation, wherein the three-dimensional coordinate transformation relationship includes a transformation relationship of a coordinate system of feature point information at different times; A three-dimensional coordinate projection relationship is constructed according to the three-dimensional coordinate transformation relationship and the external parameters between the inertial measurement unit and the image acquisition module.

8. The device initialization method according to claim 6, characterized in that: Solving the target model to be solved according to the inertial information and the feature point information to determine the inertial visual parameters and the target plane coefficient of the extended reality device includes: Transforming the target model to be solved according to the coupling relationship to obtain a simplified model, wherein the coupling relationship includes a coupling relationship between the plane coefficient to be solved and the inertial information to be solved; The simplified model is solved according to the inertial information and the feature point information to obtain the inertial vision parameters of the inertial measurement unit at the initial moment of the startup operation and the target plane coefficient of the scene image at the initial moment.

9. The device initialization method according to claim 1, characterized in that: The inertial vision parameters include initial inertial vision parameters at the initial moment of the startup operation, and the target plane coefficients include initial plane coefficients of scene plane information at the initial moment; After determining the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, the method further includes: Determine the three-dimensional coordinates of the feature point information at the initial moment according to the initial plane coefficients; Solving a nonlinear optimization model according to the three-dimensional coordinates, the initial inertial vision parameters, the inertial information, and the feature point information to obtain optimized plane coefficients corresponding to the scene plane information at the initial moment and optimized inertial vision parameters at the initial moment; The nonlinear optimization model is constructed based on a spatial plane expression model corresponding to a three-dimensional coordinate projection relationship, a pre-integrated representation of inertial parameters, and scene plane information at an initial moment.

10. A device initialization device, characterized in that: Applied to an extended reality device, the device comprises: A response module, configured to obtain inertial information of the extended reality device and capture a scene image through the extended reality device in response to a start-up operation of initializing inertial vision parameters; A recognition module, used to recognize feature point information and scene plane information of the scene image; A determination module is used to determine the inertial visual parameters and the target plane coefficient of the extended reality device according to the plane coefficient to be solved of the scene plane information, the inertial information and the feature point information, wherein the target plane coefficient is the solution corresponding to the plane coefficient to be solved.

11. An electronic device, characterized in that: The device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the device initialization method according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the device initialization method according to any one of claims 1 to 9 are implemented.

Images