Method for determining plane information and electronic equipment

Abstract

The invention discloses a plane information determination method and electronic equipment, and belongs to the technical field of computers. The method comprises the following steps: acquiring an object plane through a sensor to obtain initial position information of a plurality of acquisition points; based on the initial position information of the plurality of collection points, an initial distance function is constructed, and the initial distance function is used for describing the distance between the plurality of collection points and the object plane; determining reference attitude information of the sensor based on the initial distance function; based on the reference attitude information of the sensor, reference plane information of the object plane is determined. The reference attitude information of the sensor enables the distance between the acquisition point and the object plane to be shortest, and the accuracy of the reference attitude information is improved, so that the accuracy of the reference plane information is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method for determining plane information and an electronic device. Background Art

[0002] With the development of computer technology, the ways of collecting objects are becoming more and more diverse. At present, digital cameras can be used to collect objects to obtain object images, and depth cameras, lidar and other sensors can be used to collect objects to obtain point cloud data. Usually, there is at least one object plane on an object. By performing relevant operations on the point cloud data, the plane information of each object plane can be obtained. The plane information of each object plane reflects the appearance, size and other information of the object, and the three-dimensional model of the object can be reconstructed. Based on this, how to determine the plane information of each object plane on the object has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a method for determining plane information and an electronic device, which can be used to solve the problems in the related art. The technical solution includes the following contents.

[0004] In one aspect, a method for determining plane information is provided, the method comprising:

[0005] The object plane is collected by sensors to obtain initial position information of multiple collection points;

[0006] Based on the initial position information of the multiple acquisition points, construct an initial distance function, where the initial distance function is used to describe the distance between the multiple acquisition points and the object plane;

[0007] Based on the initial distance function, determining reference posture information of the sensor;

[0008] Based on the reference posture information of the sensor, reference plane information of the object plane is determined.

[0009] In a possible implementation, constructing an initial distance function based on the initial position information of the multiple acquisition points includes:

[0010] Acquiring initial posture information of the sensor;

[0011] Based on the initial posture information of the sensor, converting the initial position information of the designated collection point to obtain the converted initial position information of the designated collection point, wherein the designated collection point is one or more of the multiple collection points;

[0012] The initial distance function is constructed based on the converted initial position information.

[0013] In a possible implementation manner, there are multiple sensors, and determining the reference posture information of the sensors based on the initial distance function includes:

[0014] Acquire first partial derivative information and second partial derivative information, wherein the first partial derivative information is a partial derivative of a reference distance relative to initial posture information of one of the sensors, and the second partial derivative information is a partial derivative of the reference distance relative to initial posture information of two of the sensors, and the reference distance is determined according to the initial distance function;

[0015] Determining first adjustment information of each of the sensors based on the first partial guide information and the second partial guide information;

[0016] Determining first posture information of each sensor based on the first adjustment information and initial posture information of each sensor;

[0017] When the first posture information of each sensor satisfies a set condition, the first posture information of each sensor is determined as reference posture information of each sensor.

[0018] In a possible implementation manner, the obtaining the first partial guide information and the second partial guide information includes:

[0019] Acquire a reference coefficient, where the reference coefficient is used to constrain the reference distance;

[0020] Determine first sub-derivative information and second sub-derivative information based on the reference coefficient, wherein the first sub-derivative information is a derivative of the reference coefficient relative to the initial posture information of the one sensor, and the second sub-derivative information is a derivative of the reference coefficient relative to the initial posture information of the two sensors;

[0021] The first partial derivative information and the second partial derivative information are determined based on the first sub-partial derivative information and the second sub-partial derivative information.

[0022] In a possible implementation manner, determining the first sub-partial derivative information and the second sub-partial derivative information based on the reference coefficient includes:

[0023] Determine a first parameter based on initial posture information of the one sensor and initial position information of a designated collection point obtained by collecting data from the one sensor on the object plane;

[0024] Determine the first sub-partial derivative information based on the first parameter;

[0025] Determine a second parameter based on the initial posture information of the two sensors and the initial position information of the designated collection point obtained by each of the two sensors collecting the object plane respectively;

[0026] The second sub-partial derivative information is determined based on the second parameter.

[0027] In a possible implementation manner, the reference coefficient is related to an element in the first parameter, and determining the first sub-partial derivative information based on the first parameter includes:

[0028] Determine partial derivative information corresponding to the element, where the partial derivative information corresponding to the element is a partial derivative of the element relative to preset initial posture information of the sensor;

[0029] Based on the partial derivative information corresponding to the element, the first sub-partial derivative information is determined.

[0030] In a possible implementation, the reference coefficient includes a first sub-coefficient, a second sub-coefficient and a third sub-coefficient;

[0031] The determining the first partial guide information and the second partial guide information based on the first sub-partial guide information and the second sub-partial guide information includes:

[0032] Acquire a first element, a second element, and a third element, wherein the first element is constructed based on the reference distance, the second element is constructed based on the first sub-coefficient, the second sub-coefficient, and the third sub-coefficient, and the third element is constructed based on the first sub-coefficient and the second sub-coefficient;

[0033] The first partial derivative information and the second partial derivative information are determined based on the first element, the second element, the third element, the first sub-partial derivative information, and the second sub-partial derivative information.

[0034] In a possible implementation, the method further includes:

[0035] When the first posture information of each sensor does not satisfy the set condition, updating the initial distance function based on the first posture information of each sensor to obtain a first distance function;

[0036] Determine second posture information of each sensor based on the first distance function;

[0037] When the second posture information of each sensor satisfies the set condition, the second posture information of each sensor is determined as the reference posture information of each sensor.

[0038] In a possible implementation manner, determining the reference plane information of the object plane based on the reference posture information of the sensor includes:

[0039] Based on the reference posture information of the sensor, initial position information of the plurality of collection points is converted to obtain reference position information of the plurality of collection points;

[0040] Based on the reference position information of the multiple acquisition points, reference plane information of the object plane is determined.

[0041] In another aspect, a device for determining plane information is provided, the device comprising:

[0042] An acquisition module is used to collect the object plane through a sensor to obtain initial position information of multiple collection points;

[0043] A construction module, configured to construct an initial distance function based on the initial position information of the plurality of acquisition points, wherein the initial distance function is used to describe the distance between the plurality of acquisition points and the object plane;

[0044] A determination module, configured to determine reference posture information of the sensor based on the initial distance function;

[0045] The determination module is further used to determine the reference plane information of the object plane based on the reference posture information of the sensor.

[0046] In a possible implementation, the construction module is used to obtain initial posture information of the sensor; based on the initial posture information of the sensor, convert initial position information of a designated collection point to obtain the converted initial position information of the designated collection point, where the designated collection point is one or more of the multiple collection points; and based on the converted initial position information, construct the initial distance function.

[0047] In a possible implementation, there are multiple sensors, and the determination module is used to obtain first partial derivative information and second partial derivative information, wherein the first partial derivative information is the partial derivative of a reference distance relative to initial posture information of one of the sensors, and the second partial derivative information is the partial derivative of the reference distance relative to initial posture information of two of the sensors, and the reference distance is determined according to the initial distance function; based on the first partial derivative information and the second partial derivative information, the first adjustment information of each of the sensors is determined; based on the first adjustment information and the initial posture information of each of the sensors, the first posture information of each of the sensors is determined; when the first posture information of each of the sensors meets the set conditions, the first posture information of each of the sensors is determined as the reference posture information of each of the sensors.

[0048] In one possible implementation, the determination module is used to obtain a reference coefficient, wherein the reference coefficient is used to constrain the reference distance; determine first sub-derivative information and second sub-derivative information based on the reference coefficient, wherein the first sub-derivative information is the derivative of the reference coefficient relative to the initial posture information of the one sensor, and the second sub-derivative information is the derivative of the reference coefficient relative to the initial posture information of the two sensors; and determine the first derivative information and the second derivative information based on the first sub-derivative information and the second sub-derivative information.

[0049] In one possible implementation, the determination module is used to determine a first parameter based on initial posture information of the one sensor and initial position information of a designated acquisition point obtained by the one sensor acquiring the object plane; determine the first sub-partial derivative information based on the first parameter; determine a second parameter based on initial posture information of the two sensors and initial position information of a designated acquisition point obtained by each of the two sensors acquiring the object plane; and determine the second sub-partial derivative information based on the second parameter.

[0050] In a possible implementation, the reference coefficient is related to an element in the first parameter, and the determination module is used to determine the partial derivative information corresponding to the element, where the partial derivative information corresponding to the element is the partial derivative of the element relative to the preset initial posture information of the sensor; based on the partial derivative information corresponding to the element, the first sub-partial derivative information is determined.

[0051] In a possible implementation, the reference coefficient includes a first sub-coefficient, a second sub-coefficient and a third sub-coefficient; the determination module is used to obtain a first element, a second element and a third element, the first element is constructed based on the reference distance, the second element is constructed based on the first sub-coefficient, the second sub-coefficient and the third sub-coefficient, and the third element is constructed based on the first sub-coefficient and the second sub-coefficient; based on the first element, the second element, the third element, the first sub-partial derivative information and the second sub-partial derivative information, the first partial derivative information and the second partial derivative information are determined.

[0052] In a possible implementation, the determination module is also used to update the initial distance function based on the first posture information of each sensor to obtain a first distance function when the first posture information of each sensor does not meet the set conditions; determine the second posture information of each sensor based on the first distance function; and determine the second posture information of each sensor as the reference posture information of each sensor when the second posture information of each sensor meets the set conditions.

[0053] In a possible implementation, the determination module is used to convert the initial position information of the multiple acquisition points based on the reference posture information of the sensor to obtain the reference position information of the multiple acquisition points; and determine the reference plane information of the object plane based on the reference position information of the multiple acquisition points.

[0054] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device implements any of the above-mentioned methods for determining plane information.

[0055] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor so that an electronic device can implement any of the above-mentioned methods for determining plane information.

[0056] On the other hand, a computer program is also provided. The computer program is at least one, and the at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned methods for determining plane information.

[0057] On the other hand, a computer program product is also provided, wherein at least one computer program is stored in the computer program product, and the at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned methods for determining plane information.

[0058] The technical solution provided by this application brings at least the following beneficial effects:

[0059] The technical solution provided by the present application is to collect the object plane through the sensor to obtain the initial position information of multiple collection points. Based on the initial position information of the collection points, an initial distance function is constructed, and the distance between each collection point and the object plane is described by the initial distance function. The reference posture information of the sensor is determined by the initial distance function, so that the reference posture information of the sensor can make the distance between the collection point and the object plane the shortest, thereby improving the accuracy of the reference posture information. When the reference plane information of the object plane is determined based on the reference posture information of the sensor in the future, the accuracy of the reference plane information can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 It is a schematic diagram of an implementation environment of a method for determining plane information provided in an embodiment of the present application;

[0062] Figure 2 is a flow chart of a method for determining plane information provided by an embodiment of the present application;

[0063] Figure 3 It is a schematic diagram of a sensor provided in an embodiment of the present application collecting data on a plane of an object;

[0064] Figure 4 is a schematic diagram of point cloud data provided by an embodiment of the present application;

[0065] Figure 5 is a schematic diagram of disturbed point cloud data provided in an embodiment of the present application;

[0066] Figure 6 is a schematic diagram of disturbed point cloud data and an object plane determined based on the disturbed point cloud data provided by an embodiment of the present application;

[0067] Figure 7 It is a curve diagram of the convergence speed of a method of a related technology provided in an embodiment of the present application and a method of an embodiment of the present application;

[0068] Figure 8 It is a bar chart of the number of iterations of a method of a related technology provided in an embodiment of the present application and a method in an embodiment of the present application;

[0069] Fig. 9 is a structural schematic diagram of a device for determining plane information provided in an embodiment of the present application;

[0070] Fig.10 It is a structural diagram of a terminal device provided in an embodiment of the present application;

[0071] Fig.11 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0073] Figure 1 is a schematic diagram of an implementation environment of a method for determining plane information provided in an embodiment of the present application, such as Figure 1 As shown, the implementation environment includes a terminal device 101 and a server 102. The method for determining plane information in the embodiment of the present application can be executed by the terminal device 101, or by the server 102, or by the terminal device 101 and the server 102 together.

[0074] The terminal device 101 can be a smart phone, a game console, a desktop computer, a tablet computer, a laptop computer, a smart TV, a smart car device, an intelligent voice interaction device, a smart home appliance, etc. The server 102 can be a single server, or a server cluster consisting of multiple servers, or any one of a cloud computing platform and a virtualization center, which is not limited in the embodiments of the present application. The server 102 can be connected to the terminal device 101 through a wired network or a wireless network. The server 102 can have functions such as data processing, data storage, and data transmission and reception, which are not limited in the embodiments of the present application. The number of terminal devices 101 and servers 102 is not limited and can be one or more.

[0075] In the field of computer technology, point cloud data can be obtained by collecting each object plane through sensors such as depth cameras and laser radars. The point cloud data is subjected to relevant operations to obtain the plane information of each object plane, and the three-dimensional model of the object is reconstructed through the plane information of each object plane. Based on this, how to determine the plane information of each object plane on the object has become a technical problem that needs to be solved urgently.

[0076] The present application embodiment provides a method for determining plane information, which can be applied to the above implementation environment. Figure 2 Taking the flowchart of a method for determining plane information provided by an embodiment of the present application as an example, for ease of description, the terminal device 101 or the server 102 that executes the method for determining plane information in the embodiment of the present application is referred to as an electronic device, and the method can be executed by an electronic device. Figure 2 As shown, the method includes the following steps.

[0077] Step 201 : collect data on the object plane through sensors to obtain initial position information of multiple collection points.

[0078] The embodiment of the present application does not limit the sensor. Exemplarily, there is at least one sensor, any one of which is a depth camera or a laser radar, etc. Optionally, the depth camera is a three-primary color depth (Red-Green-Blue Depth, RGB-D) camera.

[0079] The embodiments of the present application do not limit the object. The object may be a building, an electrical appliance, a vehicle, etc., and the object includes at least one object plane. For any object plane, the object plane is collected by any sensor to obtain data of a point cloud block. Since the object plane includes multiple collection points, the data of the point cloud block includes the initial position information of each collection point on the object plane. Optionally, the initial position information of the collection point is a three-dimensional coordinate based on the Cartesian coordinate system (i.e., the sensor coordinate system) where the sensor is located. Since there is at least one sensor and at least one object plane, data of multiple point cloud blocks can be obtained in this way.

[0080] That is to say, the embodiments of the present application can obtain data of multiple point cloud blocks, and the data of any point cloud block includes the initial position information of multiple acquisition points, and the acquisition points included in any point cloud block are obtained by acquiring any object plane through any sensor. It should be noted that at least one point cloud block can be obtained by acquiring data from the same object plane through at least one sensor. Similarly, at least one point cloud block can be obtained by acquiring data from at least one object plane through the same sensor.

[0081] See also Figure 3 , Figure 3 Schematic diagram of a sensor provided in an embodiment of the present application collecting data on an object plane. Figure 3 (1) is a schematic diagram of collecting the object planes 1 to 4 through sensors 1 to 3. Figure 3 In (1), different rectangular boxes on the same object plane are acquisition areas of different sensors, and each point in the rectangular box represents each acquisition point obtained by the sensor acquiring the acquisition area. In other words, each object plane is acquired by the same sensor, and the same object plane is acquired by each sensor. Figure 3 (2) is in accordance with Figure 3 The actual collection process is not described in detail here.

[0082] Step 202: construct an initial distance function based on the initial position information of the multiple acquisition points, where the initial distance function is used to describe the distance between the multiple acquisition points and the object plane.

[0083] Optionally, based on data of multiple point cloud blocks, at least one initial distance function is constructed, and any initial distance function is used to describe the distance between multiple relevant acquisition points and any object plane. The multiple relevant acquisition points are obtained by collecting any object plane through at least one relevant sensor that collects any object plane.

[0084] In an embodiment of the present application, a point cloud block corresponds to an object plane, and different point cloud blocks correspond to the same object plane or different object planes. Optionally, for any object plane, the point cloud block corresponding to the object plane is determined from multiple point cloud blocks, and for ease of description, the point cloud block corresponding to the object plane is referred to as a related point cloud block. The number of related point cloud blocks is at least one, and the various acquisition points included in the related point cloud block are referred to as various related acquisition points, and the sensor used to acquire the related acquisition points is referred to as a related sensor. In other words, the various related acquisition points included in any related point cloud block are acquired by acquiring any object plane through any related sensor.

[0085] Based on the initial position information of each relevant acquisition point included in each relevant point cloud block, an initial distance function corresponding to any object plane can be constructed, and the distance between each relevant acquisition point and the object plane can be described by the initial distance function. Since the acquisition points on the object plane are located on the object plane, the embodiment of the present application minimizes the distance between each relevant acquisition point and the object plane by minimizing the initial distance function, so that plane information that is accurately used to describe the position, size, etc. of the object plane can be obtained.

[0086] In a possible implementation, step 202 includes steps 2021 to 2023 (not shown in the figure).

[0087] Step 2021, obtaining the initial posture information of the sensor.

[0088] For any initial distance function, the initial posture information of each related sensor can be obtained.

[0089] In the embodiment of the present application, the initial posture information of any relevant sensor is used to describe the transformation relationship between the sensor coordinate system of the relevant sensor and the world coordinate system. The embodiment of the present application does not limit the method for obtaining the initial posture information of the relevant sensor. Exemplarily, the initial posture information of the relevant sensor is set data, or manually input data, or randomly generated data.

[0090] The embodiments of the present application do not limit the initial posture information of any relevant sensor. For example, the initial posture information of the relevant sensor can be expressed as: Or, it can be expressed as: T = [R, t]. The initial posture information of the relevant sensor includes initial rotation information R and initial translation information t. The initial rotation information is used to describe the rotation relationship between the sensor coordinate system and the world coordinate system, and the initial translation information is used to describe the translation relationship between the sensor coordinate system and the world coordinate system. The initial rotation information can be recorded as R∈SO(3), and the initial translation information can be recorded as t∈R 3. Among them, SE(3) represents rigid body transformation motion, SO(3) represents rotation motion, and R3 represents an array of three real numbers.

[0091] Since R∈SO(3), the initial rotation information is parameterized data. Optionally, the initial rotation information is parameterized data using Cayley-Gibbs-Rodriguez (CGR). In this case, the initial rotation information R can be expressed as formula (1) shown below.

[0092]

[0093] Among them, s is a three-dimensional rotation matrix, s=[s1;s2;s3], s1;s2;s3 represent the three components of the rotation matrix. It is a parameter determined based on the rotation matrix s. T represents the matrix transpose symbol. I3 represents the set unit matrix, I3 = [0; 0; 0]. [s] × A diagonal matrix representing s.

[0094] Similarly, the initial posture information T may also be parameterized data. Optionally, T is a six-dimensional vector, and T may be equivalent to: x=[s; t], where s is a rotation matrix and t is initial translation information.

[0095] Step 2022: based on the initial posture information of the sensor, convert the initial position information of the designated collection point to obtain the converted initial position information of the designated collection point, where the designated collection point is one or more of the multiple collection points.

[0096] Optionally, for any relevant sensor, based on the initial posture information of any relevant sensor, the initial position information of the designated acquisition point is converted to obtain the converted initial position information of the designated acquisition point, and the designated acquisition point is obtained by collecting any object plane through any relevant sensor among multiple relevant acquisition points.

[0097] In the embodiment of the present application, multiple designated acquisition points can be obtained by collecting data on any object plane through any relevant sensor. For any designated acquisition point, the initial position information of the designated acquisition point is converted based on the initial posture information of the relevant sensor to obtain the converted initial position information of the designated acquisition point.

[0098] Optionally, the jth related sensor collects data of the i-th object plane to obtain the point cloud block Among them, N ij Indicates the number of specified acquisition points included in the point cloud block. In other words, the point cloud block includes N ijdesignated collection points, p ijk Characterize the initial position information of the kth designated collection point. Based on the initial posture information of the jth related sensor, the initial position information of the kth designated collection point is transformed to obtain the converted initial position information of the kth designated collection point as shown in the following formula (2).

[0099]

[0100] in, Represents the initial position information of the kth designated acquisition point after transformation. R j Represents the initial rotation information in the initial posture information of the jth related sensor. j Represents the initial translation information in the initial posture information of the jth related sensor. j Characterizes the initial posture information of the jth related sensor. Characterization ijk The homogeneous coordinates of

[0101] Step 2023, constructing an initial distance function based on the converted initial position information.

[0102] That is to say, any initial distance function is constructed based on the converted initial position information of each relevant acquisition point.

[0103] For any designated acquisition point, the distance between the designated acquisition point and the object plane can be determined based on the initial position information of the designated acquisition point after transformation and the initial plane information of any object plane. Optionally, the parameter is as shown in the following formula (3).

[0104]

[0105] Among them, d ijk (π i , X j ) represents the distance between the kth specified acquisition point and the i-th object plane. i The initial plane information representing the object plane is a four-dimensional vector. i =[n i ;d i ], where n i Characterizing the object plane π i The normal vector, d i The origin of the sensor coordinate system representing the jth related sensor and the object plane π i The distance between. j is the initial posture information of the jth related sensor. Characterization i The transpose of . Characterization of πi The transpose of . Characterization The homogeneous coordinates of The meanings of the remaining parameters in formula (3) can be found in the description of formula (2), and will not be repeated here.

[0106] In the embodiment of the present application, a designated acquisition point is a relevant acquisition point. According to the implementation principle of formula (3), the distance between each relevant acquisition point and any object plane can be determined. Based on these distances, an initial distance function can be constructed to describe the distance between each relevant acquisition point and any object plane through the initial distance function.

[0107] In an exemplary embodiment, the object includes M object planes, and the M object planes are collected by N sensors to obtain data of multiple point cloud blocks. The initial plane information of the M object planes is represented as {π i}, the initial posture information of N sensors is expressed as {x j Based on the implementation principle of formula (3), the initial distance function shown in the following formula (4) can be obtained.

[0108]

[0109] Among them, min represents the minimum value symbol. i Characterizes the number of relevant acquisition points. Among them, obs(π i ) represents the sensor set that collects the i-th object plane, and the sensor set includes at least one related sensor, j∈obs(π i ) represents the jth related sensor, N ij Characterizes the number of specified acquisition points obtained by the jth related sensor collecting the i-th object plane. i (π i , X i ) represents the initial distance function, which is used to represent the distance between each relevant acquisition point and the i-th object plane. The meanings of the remaining parameters in formula (4) can be found in the description of formula (3), which will not be repeated here.

[0110] Step 203: Determine reference posture information of the sensor based on the initial distance function.

[0111] In the embodiment of the present application, there are multiple sensors, and the reference posture information of each sensor is determined based on at least one initial distance function. Optionally, according to the implementation principle of methods such as Newton's Method or Hessian matrix, the reference posture information of each sensor is determined based on at least one initial distance function. The following takes Newton's method as an example to illustrate a possible implementation of step 203. Optionally, step 203 includes steps 2031 to 2034 (not shown in the figure).

[0112] Step 2031, obtaining first partial derivative information and second partial derivative information, the first partial derivative information is the partial derivative of the reference distance relative to the initial posture information of one sensor, the second partial derivative information is the partial derivative of the reference distance relative to the initial posture information of two sensors, and the reference distance is determined according to the initial distance function.

[0113] That is to say, for any initial distance function, the first partial derivative information and the second partial derivative information are obtained, the first partial derivative information is the partial derivative of the reference distance relative to the initial posture information of any related sensor, the second partial derivative information is the partial derivative of the reference distance relative to the initial posture information of any two related sensors, and the reference distance is the distance between multiple related acquisition points and any object plane under the condition of the reference plane information of any object plane.

[0114] It should be noted that any two related sensors may be the same or different. For ease of description, any two related sensors are referred to as the jth related sensor and the kth related sensor, respectively, where j=k or j≠k.

[0115] In the embodiment of the present application, for any initial distance function, the reference distance can be determined based on the initial distance function. i} is the position information of K acquisition points in the world coordinate system, p i Characterizes the position information of the i-th acquisition point in the world coordinate system. By minimizing the sum of the squares of the distances between the K acquisition points and the set plane, the plane information shown in the following formula (5) can be obtained. The plane information is used to describe the optimal plane with the minimum sum of the squares of the distances between the K acquisition points.

[0116]

[0117] in, Characterizes the plane information of the optimal plane, and the sum of the squares of the distances between the optimal plane and the K acquisition points is the smallest. argmin is used to characterize the reference function The value of the independent variable π when the value is minimum. π represents the plane information of the set plane, π = [n; d], n represents the normal vector of the set plane, and d represents the distance between the origin of the sensor coordinate system and the plane.T represents the transpose of n. st represents the symbol that makes A satisfy B, for example, Characterize so that n satisfies Characterizes the square of the 2-norm of n.

[0118] Plane information of the optimal plane There exists a closed-form solution, which is shown in the following formula (6).

[0119]

[0120] Among them, M represents The closed solution of . Represents the central position information of K collection points. Characterization The transpose of Represents the difference between the position information of the i-th acquisition point in the world coordinate system and the center position information. Represents the product of the position information of K acquisition points in the world coordinate system and the transpose of the corresponding position information, p i T Characterization i The transpose of . Characterization The transpose of .

[0121] Assuming λ3(M) is the minimum eigenvalue of M and ξ3(M) is the eigenvector of M, then the plane information of the optimal plane is It can be expressed as the following formula (7).

[0122]

[0123] in, The normal vector that represents the optimal plane. Characterizes the distance between the origin of the sensor coordinate system and the optimal plane. for The transpose of Represents the central position information of K collection points.

[0124] Based on formulas (6) and (7), formula (5) can be transformed to obtain formula (8) as shown below.

[0125]

[0126] Wherein, λ3(M) represents the plane information of the optimal plane, and the square sum of the distances between the optimal plane and the K acquisition points. min represents the minimum value symbol. The meanings of other parameters in formula (8) can be found in the description of formulas (5) to (6), which will not be repeated here.

[0127] According to the implementation principles of formulas (5) to (8), the following formula (9) can be obtained.

[0128]

[0129] Among them, M i (X i ) is a closed-form solution that represents the reference plane information of the object plane. Characterization N ij The product of the converted initial position information of the specified acquisition point and the transpose of the corresponding converted initial position information. Characterization N i The central position information of the initial position information after the conversion of the relevant collection points. Characterization The meaning of the remaining parameters in formula (9) can be found in the relevant description of formula (4), which will not be repeated here.

[0130] It should be noted that M i (X i ) is related to X i function, and S ij , Also about X i The function of X i Including the posture information of each related sensor. Based on formula (2), formula (9) can be transformed into formula (10) as shown below.

[0131]

[0132]

[0133] Among them, U ij Characterization N ij The product of the homogeneous coordinates of the initial position information of a specified acquisition point and the transpose of the corresponding homogeneous coordinates. Characterization N ij The homogeneous coordinates of the initial position information of the specified collection points.

[0134] According to formula (7), given X i, a closed solution of the i-th object plane can be obtained, and the closed solution corresponds to the plane information of the optimal plane, that is, the closed solution corresponds to the reference plane information of the object plane in the embodiment of the present application. Optionally, the reference plane information of the i-th object plane is Among them, ξ3(M i (X i )) is M i (X i ), represents the normal vector of the i-th object plane, represents the distance between the origin of the sensor coordinate system and the i-th object plane, Characterization The meanings of the other parameters are described above and will not be repeated here.

[0135] It should be noted that M i (X i ), It is about X i In addition, Also about X i For ease of description, in the embodiment of the present application, λ i，3 (X i )=λ3(M i (X i )). Among them, λ i，3 (X i ) is M i (X i ) is the sum of the squares of the distances between multiple relevant acquisition points and any object plane under the condition of the reference plane information of any object plane. i，3 (X i ) can represent the distance between multiple related acquisition points and any object plane under the condition of the reference plane information of any object plane. Therefore, λ i，3 (X i ) represents the reference distance.

[0136] In addition, you can set Based on this, formula (4) can be transformed into formula (11) as shown below.

[0137]

[0138] Where M represents the number of object planes. i，3 (X i) represents the distance between multiple related acquisition points and the i-th object plane under the condition of the reference plane information of the i-th object plane, which can be referred to as the reference distance corresponding to the i-th object plane. τ includes the reference distances corresponding to M object planes. argmin represents the independent variable {x j}, {x j} represents the initial posture information of each sensor. Among them, The reference posture information of each sensor can be characterized.

[0139] It can be seen from formula (11) that τ only depends on the posture information of each sensor. Formula (11) reduces the number of variables. For ease of description, in the embodiment of the present application, λ i，3 (X i ) is abbreviated as λ i，3 .

[0140] In the embodiment of the present application, for any initial distance function, the first partial derivative information can be obtained. The first partial derivative information is the partial derivative of the reference distance relative to the initial posture information of any relevant sensor. Optionally, the first partial derivative information can be expressed as Characterization reference distance λ i，3 The initial posture information x relative to the jth related sensor j The partial derivative of , and the first partial derivative information is the first-order partial derivative.

[0141] In addition, the embodiment of the present application can also obtain second partial derivative information. The second partial derivative information is the partial derivative of the reference distance relative to the initial posture information of any two related sensors. Optionally, the second partial derivative information can be expressed as Characterization reference distance λ i，3 The initial posture information x relative to the jth related sensor j , the initial posture information x of the kth related sensor k The partial derivative of , and the second partial derivative information is the second-order partial derivative.

[0142] In a possible implementation, step 2031 includes steps S1 to S3 (not shown in the figure).

[0143] Step S1, obtaining a reference coefficient, where the reference coefficient is used to constrain a reference distance.

[0144] In the embodiment of the present application, reference constraint information may be configured to describe the constraint conditions satisfied by the reference distance through the reference constraint information. Optionally, the reference constraint information may be expressed as: Among them, a i 、b i 、c i Characterization reference coefficient, λi，3 Characterizes the reference distance. The reference coefficient includes the first sub-coefficient, the second sub-coefficient and the third sub-coefficient. Among them, a i Characterize the first sub-coefficient, b i Characterize the second sub-coefficient, c i Characterize the third sub-coefficient. The reference coefficient is determined by the reference distance and the reference constraint information, and the reference coefficient is used to constrain the reference distance.

[0145] Step S2, determining first sub-derivative information and second sub-derivative information based on the reference coefficient, the first sub-derivative information is the derivative of the reference coefficient relative to the initial posture information of one sensor, and the second sub-derivative information is the derivative of the reference coefficient relative to the initial posture information of two sensors.

[0146] In other words, the first sub-derivative information and the second sub-derivative information are determined based on the reference coefficient, the first sub-derivative information is the derivative of the reference coefficient relative to the initial posture information of any related sensor, and the second sub-derivative information is the derivative of the reference coefficient relative to the initial posture information of any two related sensors.

[0147] In the embodiment of the present application, the reference coefficient includes a first sub-coefficient a i , the second sub-coefficient b i and the third sub-coefficient c i , the first sub-partial derivative information can be determined based on these three sub-coefficients, and the first sub-partial derivative information includes the first sub-coefficient a i The corresponding first partial derivative and second sub-coefficient b i The corresponding first partial derivative and third sub-coefficient c i The corresponding first partial derivative. Among them, the first sub-coefficient a i The corresponding first partial derivative is the partial derivative of the first sub-coefficient with respect to the initial attitude information of any relevant sensor, which can be expressed as or And this partial derivative is a first-order partial derivative. The second sub-coefficient b i The corresponding first partial derivative is the partial derivative of the second sub-coefficient with respect to the initial attitude information of any relevant sensor, which can be expressed as or And this partial derivative is a first-order partial derivative. The third sub-coefficient c i The corresponding first partial derivative is the partial derivative of the third sub-coefficient with respect to the initial attitude information of any relevant sensor, which can be expressed as or And the partial derivative is a first-order partial derivative. Among them, x j Represents the initial posture information of the jth related sensor, x k Characterizes the initial posture information of the kth related sensor. Optionally, the first sub-derivative information can be expressed as or

[0148] In the embodiment of the present application, the second sub-partial derivative information can also be determined based on the three sub-coefficients, and the second sub-partial derivative information includes the first sub-coefficient a i The corresponding second partial derivative and second sub-coefficient b i The corresponding second partial derivative and third sub-coefficient c i The corresponding second partial derivative. Among them, the first sub-coefficient a i The corresponding second partial derivative is the partial derivative of the first sub-coefficient with respect to the initial attitude information of any two related sensors, which can be expressed as And this partial derivative is a second-order partial derivative. The second sub-coefficient b i The corresponding second partial derivative is the partial derivative of the second sub-coefficient with respect to the initial attitude information of any two related sensors, which can be expressed as And this partial derivative is a second-order partial derivative. The third sub-coefficient c i The corresponding second partial derivative is the partial derivative of the third sub-coefficient with respect to the initial attitude information of any two related sensors, which can be expressed as And the partial derivative is a second-order partial derivative. Among them, x j Represents the initial posture information of the jth related sensor, x k Characterizes the initial posture information of the kth related sensor.

[0149] Optionally, step S2 includes steps S21 to S24 (not shown in the figure).

[0150] Step S21 , determining a first parameter based on initial posture information of a sensor and initial position information of a designated acquisition point obtained by a sensor acquiring data on an object plane.

[0151] In other words, the first parameter is determined based on the initial posture information of any relevant sensor, the number of relevant acquisition points, and the initial position information of the designated acquisition points obtained by acquiring any object plane through any relevant sensor.

[0152] In the embodiment of the present application, in formula (10) It can be expressed as: in, Characterizes the central position information of the initial position information after conversion of each relevant acquisition point obtained by acquiring the i-th object plane through the j-th relevant sensor and the k-th relevant sensor. j Represents the initial posture information of the jth related sensor, T k Characterizes the initial posture information of the kth related sensor. in, The homogeneous coordinates representing the initial position information of each designated acquisition point acquired by the jth related sensor on the i-th object plane. The homogeneous coordinates representing the initial position information of each designated acquisition point acquired by the kth related sensor on the i-th object plane. O jk Characterizes the other related sensors except the j-th related sensor and the k-th related sensor, that is, O jk = obs(π i )-{j,k}. n∈O jk represents the nth related sensor, Tn represents the initial posture information of the nth related sensor, The homogeneous coordinates representing the initial position information of each designated acquisition point obtained by acquiring the i-th object plane through the n-th related sensor. i Characterizes the number of relevant acquisition points.

[0153] In addition, in formula (10) It can also be expressed as: in, Represents the central position information of the initial position information after conversion of each relevant acquisition point obtained by acquiring the i-th object plane through the j-th relevant sensor. ij =T k q ik +c ijk The meaning of each parameter has been described above and will not be repeated here.

[0154] Next, based on the above content, formula (9) can be expressed as formula (12) shown below.

[0155]

[0156] Among them, M i (x j ) represents the first parameter. j Characterizes the initial posture information of the jth related sensor. S ij Represents each designated acquisition point (i.e., N ij The product of the initial position information after conversion (a specified acquisition point) and the transpose of the corresponding initial position information after conversion, Yes ij The transpose of . It is T j The transpose of . in, It is c ij The transpose of . yes The transpose of . is a bias term and can be eliminated during the calculation process.

[0157] Step S22, determining the first sub-partial derivative information based on the first parameter.

[0158] In the embodiment of the present application, by determining the and The first parameter M can be determined i (x j ). The first sub-partial derivative information can be determined by the first parameter, that is, the first sub-coefficient a mentioned above i The corresponding first partial derivative and second sub-coefficient b i The corresponding first partial derivative and third sub-coefficient c i Optionally, the first parameter is also used to determine the first sub-coefficient a i Relative to x j The second-order partial derivative and the second sub-coefficient b i Relative to x j The second-order partial derivative and the third sub-coefficient c i Relative to x j The second-order partial derivative of .

[0159] In an exemplary embodiment, the reference coefficient is associated with at least one element of the first parameter.

[0160] Assume that there exists a vector function a=[a1;…;a N ]∈R N , and the vector function b=[b1;…;b M ]∈R M , then there is the following definition:

[0161]

[0162] That is to say, is an M×N matrix, is the element in the i-th row and j-th column of the matrix. is an N-dimensional vector, is the i-th element in the vector. is an M-dimensional vector, is the jth element in the vector.

[0163] In the embodiment of the present application, the reference distance λ i，3 It is about X i function, and x j ∈X i 、x k ∈X i Assume x j m is x j The mth element in x kn is xk The nth element in , then the first partial derivative information include Second partial information include

[0164] In addition, the reference distance λ i,3 is the equation |M i (X i )-λ i I3|=0, where I3 represents the identity matrix. i is the independent variable of the equation, and || represents the determinant of the matrix. Assume that m ef It is M i (X i ) in the e-th row and f-th column, then according to |M i (X i )-λ i I3|=0, we can get the reference distance λ i，3 The cubic equation is the reference constraint information mentioned above.

[0165] Among them, the first sub-coefficient a i =m 11 +m 22 +m 33 , the second sub-coefficient The third sub-coefficient Due to m ef It is M i (X i ), and the reference coefficient includes the first sub-coefficient, the second sub-coefficient and the third sub-coefficient, so the reference coefficient is related to at least one element in the first parameter.

[0166] The reference coefficient is related to an element in the first parameter. Optionally, the reference coefficient is related to at least one element in the first parameter. In this case, step S22 includes steps S221 to S222 (not shown in the figure).

[0167] Step S221, determining partial derivative information corresponding to the element, where the partial derivative information corresponding to the element is the partial derivative of the element relative to the preset initial posture information of the sensor.

[0168] In other words, for any element in the first parameter related to the reference coefficient, the partial derivative information corresponding to any element is determined, and the partial derivative information corresponding to any element is the partial derivative of any element relative to the set posture information when the initial posture information of any related sensor is the set posture information.

[0169] It should be noted that, in the implementation of step S22, the partial derivative information corresponding to any element is the first-order partial derivative information corresponding to the element.

[0170] In the embodiment of the present application, the first parameter M i (x j ) in which any element related to the reference coefficient is m ef (x j ), and m ef (x j ) Characterization M i (x j ) in the e-th row and f-th column. i (x j ) is about T j The polynomial, in simple terms, m ef (x j ) is about T j A linear combination of monomials. ef (x j )=c ef ·h ef (x j ), c ef It is mentioned above and Coefficient of determination, h ef (x j ) is about x j Optionally, the element m ef (x j ) relative to x j The first-order partial derivative of can be expressed as: Due to c ef is a constant, so The value of Related, can focus on discussion The calculation method of .

[0171] In order to effectively calculate h ef (x j ) relative to x j The first-order partial derivative of x j As the setting posture information, calculate h ef (x j) is the first-order partial derivative with respect to the set posture information. Optionally, the set posture information T0 = [R0, t0]. Wherein, R0 represents the set rotation information, R0 = I3, and I3 represents the unit matrix. t0 represents the set translation information, t0 = [0; 0; 0]. The parameterized representation of T0 is recorded as x0, and in the CGR parameterization method, I3 = [0; 0; 0], based on which, it can be concluded that the set posture information x0 = [0; 0; 0; 0; 0; 0].

[0172] In the present application embodiment, h ef (x j ) The first-order partial derivative with respect to the set posture information x0 can be expressed as: Since x0 = [0; 0; 0; 0; 0; 0], the first-order partial derivative has multiple zero terms, which simplifies the calculation of the first-order partial derivative. ef (x j ) depends only on x j , h can be calculated only once ef (x j ) Relative to the first-order partial derivative of the set posture information x0, the first-order partial derivative is reused in subsequent iterative calculations to achieve the purpose of speeding up the calculation.

[0173] In calculating h ef (x j ) After the first-order partial derivative of the set posture information x0, it is equivalent to obtaining the element m ef (x j ) relative to the first-order partial derivative of the set posture information x0, that is, the element m is obtained ef (x j ) corresponds to the partial derivative information For the convenience of description, in the embodiments of the present application, Can be abbreviated as

[0174] Step S222: Determine the first sub-partial derivative information based on the partial derivative information corresponding to the element.

[0175] That is, based on the partial derivative information corresponding to each element related to the reference coefficient in the first parameter, the first sub-partial derivative information is determined.

[0176] In the embodiment of the present application, the reference coefficient is related to at least one element in the first parameter. According to the method of determining the partial derivative information corresponding to any element in the first parameter related to the reference coefficient, the partial derivative information corresponding to each element in the first parameter related to the reference coefficient can be determined, thereby determining the partial derivative of the reference coefficient relative to the initial posture information of any related sensor, that is, determining the first sub-partial derivative information.

[0177] Optionally, the reference coefficient includes a first sub-coefficient, a second sub-coefficient and a third sub-coefficient, and the first sub-partial derivative information includes a first partial derivative corresponding to the first sub-coefficient, a first partial derivative corresponding to the second sub-coefficient and a first partial derivative corresponding to the third sub-coefficient. i =m 11 +m 22 +m 33 For example, the first partial derivative corresponding to the first sub-coefficient can be expressed as: In step S221, the calculation is given Based on this implementation principle, it can be determined that Thereby, the first partial derivative corresponding to the first sub-coefficient is obtained, and then the first sub-partial derivative information is obtained.

[0178] Step S23, determining a second parameter based on the initial posture information of the two sensors and the initial position information of the designated acquisition point obtained by each of the two sensors acquiring data on the object plane.

[0179] That is to say, the second parameter is determined based on the initial posture information of any two related sensors, the number of related collection points, the initial position information of the designated collection points obtained by collecting any object plane through one of any two related sensors, and the initial position information of the designated collection points obtained by collecting any object plane through the other of any two related sensors.

[0180] In the embodiment of the present application, based on an implementation principle similar to that of formula (12), formula (9) can be expressed as formula (13) shown below.

[0181]

[0182] Among them, M i (x j , x k ) represents the second parameter. j Characterizes the initial posture information of the jth related sensor. Yes ik The transpose of . It is T k The transpose of k Characterizes the initial posture information of the kth related sensor. yes The transpose of It is T j The transpose of . is a bias term, which can be eliminated during the calculation process. The meaning of each parameter has been described above and will not be repeated here.

[0183] Step S24, determining the second sub-partial derivative information based on the second parameter.

[0184] In the embodiment of the present application, by determining the The second parameter M can be determined i (x j , x k ). The second sub-derivative information can be determined by the second parameter, that is, the first sub-coefficient a mentioned above i The corresponding second partial derivative and second sub-coefficient b i The corresponding second partial derivative and third sub-coefficient c i The corresponding second partial derivative.

[0185] In an exemplary embodiment, the reference coefficient is related to an element in the second parameter, that is, the reference coefficient is related to at least one element in the second parameter. In this case, step S24 includes steps S241 to S242 (not shown in the figure).

[0186] Step S241, determining the partial derivative information corresponding to the element, where the partial derivative information corresponding to the element is the partial derivative of the element relative to the preset initial posture information of the two sensors.

[0187] In other words, for any element in the second parameter related to the reference coefficient, the second-order partial derivative information corresponding to any element is determined. The second-order partial derivative information corresponding to any element is the partial derivative of any element relative to the set posture information when the initial posture information of any two related sensors is the set posture information.

[0188] In the embodiment of the present application, the second parameter M i (x j , x k ) in which any element related to the reference coefficient is me f (x j , x k ), and m ef (x j , x k ) Characterization M i (x j , x k ) in the e-th row and f-th column. i (x j , x k ) is about T j 、T k The second-order polynomial of m ef (x j , x k ) is about T j 、T kA linear combination of monomials. ef (x j , x k ) = d ef ·g ef (x j , x k ), d ef It is mentioned above Coefficient of determination, g ef (x j , x k ) is about x j 、x k Optionally, the element m ef (x j , x k ) relative to x j 、x k The second-order partial derivative of can be expressed as: Due to d ef is a constant, so The value of Related, can focus on discussion The calculation method of .

[0189] Optionally, the posture information x0 is set to [0; 0; 0; 0; 0; 0]. This simplifies the calculation of the second-order partial derivative. ef (x j , x k ) depends only on x j 、x k , we can calculate g only once ef (x j , x k ) Relative to the second-order partial derivative of the set posture information x0, the second-order partial derivative is reused in subsequent iterative calculations to achieve the purpose of speeding up the calculation.

[0190] In calculating g ef (x j , x k ) After taking the second-order partial derivative relative to the set posture information x0, it is equivalent to obtaining the element m ef (x j , x k ) relative to the second-order partial derivative of the set posture information x0, that is, the element m is obtained ef (x j , x k ) corresponds to the second-order partial derivative information For the convenience of description, in the embodiments of the present application, Can be abbreviated as

[0191] Step S242: Determine the second sub-partial derivative information based on the partial derivative information corresponding to the element.

[0192] That is, based on the second-order partial derivative information corresponding to each element related to the reference coefficient in the second parameter, the second sub-partial derivative information is determined.

[0193] In the embodiment of the present application, the reference coefficient is related to at least one element in the second parameter. According to the method of determining the second-order partial derivative information corresponding to any element in the second parameter related to the reference coefficient, the second-order partial derivative information corresponding to each element in the second parameter related to the reference coefficient can be determined, thereby determining the partial derivative of the reference coefficient relative to the initial posture information of any two related sensors, that is, determining the second sub-partial derivative information.

[0194] Step S3: Determine first partial derivative information and second partial derivative information based on the first sub-partial derivative information and the second sub-partial derivative information.

[0195] In the embodiment of the present application, the first sub-derivative information and the second sub-partial derivative information to determine the first partial derivative information and the second partial derivative information The first sub-partial derivative information includes the first partial derivative corresponding to the first sub-coefficient or The first partial derivative corresponding to the second sub-coefficient or The third sub-coefficient c i The corresponding first partial derivative or The second sub-partial derivative information includes the second partial derivative corresponding to the first sub-coefficient The second partial derivative corresponding to the second sub-coefficient The third sub-coefficient c i The corresponding second partial derivative

[0196] In a possible implementation, the reference coefficient includes a first sub-coefficient, a second sub-coefficient and a third sub-coefficient. Step S3 includes steps S31 to S32 (not shown in the figure).

[0197] Step S31, obtaining a first element, a second element and a third element, wherein the first element is constructed based on a reference distance, the second element is constructed based on a first sub-coefficient, a second sub-coefficient and a third sub-coefficient, and the third element is constructed based on the first sub-coefficient and the second sub-coefficient.

[0198] In the embodiment of the present application, based on the reference distance λ i,3 Construct the first element. Optionally, the first element is: The second element can be constructed based on the first sub-coefficient, the second sub-coefficient and the third sub-coefficient. Optionally, the second element is: The third element can be constructed based on the first sub-coefficient and the second sub-coefficient. Optionally, the third element is:

[0199] Step S32: Determine the first partial derivative information and the second partial derivative information based on the first element, the second element, the third element, the first sub-partial derivative information and the second sub-partial derivative information.

[0200] In the embodiment of the present application, for the first partial guide information Any element included The element can be transformed based on the first element, the second element and the third element to obtain the formula (14) shown below.

[0201]

[0202] Among them, · represents the dot product operation. Characterizes the reciprocal of the dot product between the first and third elements. Characterizes the second element relative to x j The first-order partial derivative of m.

[0203] Similarly, for the second partial derivative information Any element included The element can be transformed based on the first element, the second element and the third element to obtain the formula (15) shown below.

[0204]

[0205] in, Characterizes the first element relative to x kn The first-order partial derivative of Characterization Relative to x kn The first-order partial derivative of Characterization reference distance λ i,3 Relative to x j The first-order partial derivative of m, Characterization Relative to x kn The first-order partial derivative of .

[0206] Based on the above formula (14), formula (15), the first sub-partial derivative information, and the second sub-partial derivative information, the first partial derivative information and the second partial derivative information shown in the following formula (16) can be obtained.

[0207]

[0208]

[0209] in, Characterizes the first partial derivative information, which is the reference distance λ i,3 The initial posture information x relative to the jth related sensor j The partial derivative of . Characterizes the second partial derivative information, which is the reference distance λ i,3 The initial posture information x relative to the jth related sensor j , the initial posture information x of the kth related sensor k The partial derivative of . Among them, Characterizes the first partial derivative information, which is the reference distance λ i,3 The initial posture information x relative to the kth related sensor k The partial guide, The meaning of each parameter has been described above and will not be repeated here.

[0210] Through the implementation of step 2031, the first partial derivative information corresponding to any initial distance function can be determined. and the second partial derivative information Optionally, if j=k, that is, the jth related sensor is the kth related sensor, then x i =x k , and the above formula (16) can also be applied.

[0211] Step 2032: Determine first adjustment information for each sensor based on the first partial derivative information and the second partial derivative information.

[0212] That is, based on the first partial derivative information and the second partial derivative information corresponding to each initial distance function, the first adjustment information of each sensor is determined.

[0213] Since an initial distance function corresponds to an object plane, and Characterizes the partial derivative of the reference distance relative to the initial attitude information of the jth related sensor, where the reference distance is the distance between multiple related acquisition points and the i-th object plane under the condition of the reference plane information of the i-th object plane. Therefore, the number of first partial derivative information corresponding to any initial distance function is equal to the number of related sensors, and a first partial derivative information Corresponding to the i-th object plane and the j-th related sensor.

[0214] Optionally, the first partial derivative information corresponding to any relevant sensor is determined from the first partial derivative information corresponding to each initial distance function, and any relevant sensor corresponds to at least one first partial derivative information. The sum of each first partial derivative information corresponding to any relevant sensor is calculated, and the calculated sum value is called the partial derivative information corresponding to any relevant sensor. Optionally, the partial derivative information g corresponding to the jth relevant sensor j It can be expressed as: Among them, P j Characterize each object plane acquired by the jth associated sensor.

[0215] Based on the same principle, the number of second partial derivatives corresponding to any initial distance function is equal to the number of related sensor pairs. Corresponding to the i-th object plane, it also corresponds to the j-th related sensor and the k-th related sensor.

[0216] Optionally, the second partial derivative information corresponding to any two related sensors is determined from the second partial derivative information corresponding to each initial distance function, and any two related sensors correspond to at least one second partial derivative information. The sum of each second partial derivative information corresponding to any two related sensors is calculated, and the calculated sum value is called the partial derivative information corresponding to any two related sensors. Optionally, the partial derivative information H corresponding to the jth related sensor and the kth related sensor is jk It can be expressed as: Among them, P jk represents each object plane captured by the jth related sensor and the kth related sensor. It can be understood that if j=k, then P jk =P j .

[0217] Optionally, the partial derivative information g corresponding to the jth related sensor j It can be expressed as: g = (g j )∈R 6N , where g j is the gradient of τ in formula (11). (g j )∈R 6N Characterization j is a six-dimensional gradient vector. The partial derivative information H corresponding to the jth related sensor and the kth related sensor jk It can be expressed as: H = (H jk )∈R 6N×6N , where H jk is the Hessian matrix of τ in formula (11). jk )∈R 6N×6N Characterization H jk is a 6×6 Hessian matrix.

[0218] Next, based on the partial derivative information corresponding to each relevant sensor and the partial derivative information corresponding to every two relevant sensors, the first adjustment information of each sensor can be determined. Optionally, the first adjustment information of each sensor is determined according to formula (17) shown below.

[0219] (H f (z n )+μI)Δz=-g f (z n ) Formula (17)

[0220] Among them, H f (z n ) including the H mentioned above jk , represents the partial derivative information corresponding to each two related sensors. n Represents the posture information of each sensor corresponding to the nth iteration, where for the first iteration, z n Represents the initial posture information. For the second iteration, z n represents the first posture information, and so on. μ is a variable determined according to f(z) at each iteration, and f(z) represents the distance function corresponding to each iteration, where for the first iteration, f(z) represents each initial distance function, for the second iteration, f(z) represents each first distance function, and so on. I represents the unit matrix. g f (z n ) including the above mentioned g j , represents the partial derivative information corresponding to each relevant sensor. Δz represents the adjustment information corresponding to the nth iteration, where for the first iteration, Δz represents the first adjustment information of each sensor, for the second iteration, Δz represents the second adjustment information of each sensor, and so on.

[0221] Step 2033: Determine first posture information of each sensor based on the first adjustment information and initial posture information of each sensor.

[0222] For any sensor, the initial posture information of the sensor is adjusted based on the first adjustment information of the sensor to obtain the first posture information of the sensor. n+1 is: z n+1 =z n +Δz, where n represents the first iteration, zn represents the initial posture information, and Δz represents the first adjustment information. Alternatively, To determine the first posture information, where n represents the first iteration, Representing the first posture information, Characterizing the first adjustment information, Represents the initial posture information.

[0223] Step 2034: When the first posture information of each sensor meets the set condition, the first posture information of each sensor is determined as the reference posture information of each sensor.

[0224] The embodiment of the present application does not limit the first posture information of each sensor to satisfy the set condition. Exemplarily, the first posture information of each sensor satisfying the set condition means that the number of iterations corresponding to the first posture information of each sensor reaches the set number, or the first posture information of each sensor is within a set range.

[0225] When the first posture information of each sensor meets the set condition, the first posture information of any sensor is the reference posture information of the sensor.

[0226] In a possible implementation, after step 2033, steps 2035 to 2037 (not shown in the figure) are also included, wherein steps 2035 to 2037 and step 2034 are two possible implementations.

[0227] Step 2035: When the first posture information of each sensor does not meet the set condition, the initial distance function is updated based on the first posture information of each sensor to obtain a first distance function.

[0228] That is, based on the fact that the first posture information of each sensor does not satisfy the set condition, at least one initial distance function is updated based on the first posture information of each sensor to obtain at least one first distance function.

[0229] In the embodiment of the present application, if the first posture information of each sensor does not meet the set condition, the initial distance function shown in formula (4) can be updated based on the first posture information of each sensor to obtain an updated initial distance function. The first distance function is the updated initial distance function, and the content of the first distance function can be found in the content of the initial distance function, which will not be repeated here.

[0230] Step 2036: Determine second posture information of each sensor based on the first distance function.

[0231] That is, the second posture information of each sensor is determined based on at least one first distance function.

[0232] In the embodiment of the present application, for any first distance function, it can be For the parameter U in formula (10), ij Update and get the updated parameters Similarly, according to For the parameters in formula (10) Update and get the updated parameters By updating the parameters and Can be used for M i (X i ) to update and obtain the updated parameter M i (X i ) n+1 Based on M i (X i ) n+1 , the parameter τ in formula (11) can be updated to obtain the updated parameter τ n+1 Based on τ n+1 , the second adjustment information of each sensor can be determined.

[0233] Optionally, the second adjustment information of the jth sensor is The method for determining the second adjustment information of the sensor is similar to the method for determining the first adjustment information of the sensor, and will not be described in detail herein.

[0234] Based on the second adjustment information of each sensor, the first posture information of each sensor is updated to obtain the second posture information of each sensor. To determine the second posture information, where n represents the second iteration, Representing the second posture information, Characterizes the second adjustment information. Represents the first posture information. Or, the second posture information z n+1 is: z n+1 =z n +Δz, where z n represents the first posture information, and Δz represents the second adjustment information.

[0235] Step 2037: When the second posture information of each sensor meets the set condition, the second posture information of each sensor is determined as the reference posture information of each sensor.

[0236] The embodiment of the present application does not limit the second posture information of each sensor to satisfy the set condition. Exemplarily, the second posture information of each sensor satisfies the set condition when: the number of iterations corresponding to the second posture information of each sensor reaches the set number, or the second posture information of each sensor is within the set range.

[0237] When the second posture information of each sensor meets the set condition, the second posture information of any sensor is the reference posture information of the sensor.

[0238] In the embodiment of the present application, the parameters g and H mentioned above need to be constructed, and for the i-th object plane, the reference constraint information mentioned above needs to be solved and the reference distance λ needs to be selected. i，3 For the initial posture information x of the jth related sensor j , we need to construct M in formula (12) i (x j ), and calculate the a mentioned above i 、b i 、c i Equivalent to x j After that, we determine the parameters and Based on Update j ,based on Update H jj For x j and x k , we need to construct M in formula (13) i (x j , x k ), and calculate the a mentioned above i 、b i 、c i Equivalent to x j 、x k After that, we determine the parameters Based on Update H jk Among them, the parameter g includes g j , parameter H includes parameter H jj and H jk Based on the parameters g and H, the damped Newton stepping is performed as shown in formula (17) to obtain the first posture information of each sensor corresponding to this iteration. In each subsequent iteration, the parameter U is updated. ij , Thereby, the attitude information of the sensor is updated. The aforementioned update process can be summarized as the algorithm shown below.

[0239] Algorithm: Second-order plane adjustment algorithm for N pose information and M plane information

[0240] Execute when the algorithm does not converge

[0241] H = zeros (6N, 6N), g = zeros (6N, 1); / / construct a 6 × 6 Hessian matrix, and all elements in the matrix are 0, and construct a six-dimensional gradient vector, and all elements in the vector are 0

[0242] When i∈[1,M], execute

[0243] / * Calculate g and diagonal terms H* /

[0244] When j∈obs(π i )

[0245] Use formula (12) to calculate M i (x j );

[0246] calculate / / Calculate a i 、b i 、c i Equivalent to x j The partial derivative of

[0247] calculate and

[0248]

[0249] / * Calculate other forms of H* /

[0250] When j∈obs(π i )

[0251] When k∈obs(π i ) and k>j

[0252] calculate / / Calculate a i 、b i 、c i Equivalent to x j 、x k Calculation of partial derivatives

[0253]

[0254]

[0255] Use formula (17) to calculate the sensor's attitude information;

[0256] Refresh U ij and

[0257] Step 204 : determining reference plane information of the object plane based on the reference posture information of the sensor.

[0258] That is, based on the reference posture information of each sensor, the reference plane information of each object plane is determined.

[0259] In the embodiment of the present application, after determining the reference posture information of each sensor, the reference plane information of each object plane can be determined based on the reference posture information of each sensor and the data of multiple point cloud blocks. In the embodiment of the present application, according to the idea that the reference plane information is determined by the posture information of the sensor, the reference posture information of the sensor is first determined, and the reference plane information is determined after the accuracy of the posture information is ensured, which can reduce the number of parameters that need to be solved, speed up the convergence speed of the algorithm, and reduce the number of iterations of the algorithm.

[0260] In a possible implementation, step 204 includes steps 2041 to 2042 (not shown in the figure).

[0261] Step 2041: Based on the reference posture information of the sensor, the initial position information of the multiple acquisition points is converted to obtain the reference position information of the multiple acquisition points.

[0262] Optionally, for any point cloud block, based on the reference posture information of any sensor, the initial position information of each acquisition point included in any point cloud block is converted to obtain the reference position information of each acquisition point included in any point cloud block.

[0263] In the embodiment of the present application, for the initial position information of any acquisition point included in any point cloud block, the initial position information of the acquisition point can be converted based on the reference posture information of any sensor according to the implementation principle of formula (2) to obtain the reference position information of the acquisition point. Since formula (2) has been described in detail above, it will not be repeated here.

[0264] Step 2042: Determine reference plane information of the object plane based on the reference position information of the multiple acquisition points.

[0265] In other words, based on the reference position information of each acquisition point included in each point cloud block, the reference plane information of each object plane is determined.

[0266] In the embodiment of the present application, based on the reference position information of each acquisition point included in each point cloud block, the reference plane information of each object plane is determined by the implementation principle of formula (3) to formula (10). Since formula (3) to formula (10) have been described in detail above, they will not be repeated here.

[0267] In the embodiment of the present application, the laser radar is used to collect the object and obtain the following Figure 4 The point cloud data shown. Figure 4 (1) to (4) show point cloud data obtained by collecting four objects through laser radar, and the point cloud data of (1) to (4) include initial position information of 472, 1355, 1606, and 1184 collection points respectively.

[0268] Next, the attitude information of the laser radar is disturbed. Optionally, the translation information is disturbed by adding Gaussian noise to the translation information included in the attitude information, and the rotation information included in the attitude information is disturbed by randomly generating an angular axis vector from a Gaussian distribution.

[0269] In the embodiment of the present application, for the posture information after disturbance, through σ R The standard deviation of the Gaussian noise that characterizes the rotation information in the perturbed attitude information is expressed by σ t The standard deviation of the Gaussian noise that characterizes the translation information in the perturbed posture information. Based on this, we can use σ R , σ t To characterize the posture information after disturbance. In the embodiment of the present application, there are four types of posture information after disturbance. The first type of posture information after disturbance is: R =0.1°,σ t =0.01m, the attitude information after the second disturbance is: R =1°,σ t =0.1m, the attitude information after the third disturbance is: R =2°,σ t =0.2m, the attitude information after the fourth disturbance is: σ R =3°,σ t =0.3m.

[0270] Based on the above four types of perturbation posture information, Figure 4 The point cloud data shown in the figure is disturbed to obtain the disturbed point cloud data. Figure 5 , Figure 5 Shows Figure 4 The perturbed point cloud data corresponding to (3) in . Figure 5 (1) to (4) are the attitude information after the first to fourth disturbances. Figure 4 The point cloud data is obtained after perturbation of (3) in .

[0271] Next, the method for determining plane information provided in the embodiment of the present application is used to determine the reference plane information of the object plane based on the disturbed point cloud data. Figure 6 , Figure 6 (1) shows the attitude information after the fourth disturbance. Figure 4 The point cloud data obtained after perturbation of (4) is: Figure 6 (2) shows that the method for determining plane information provided by the embodiment of the present application is based on Figure 6 The object plane is determined by (1) in . Figure 6It can be seen that the object plane determined by the method of the embodiment of the present application has high accuracy.

[0272] See also Figure 7 , Figure 7 The following is a graph showing the convergence speed of the method of the related art and the method of the embodiment of the present application. Figure 7 The data shown on the horizontal axis corresponds to time in seconds. Figure 7 The data shown on the vertical axis corresponds to the logarithmic value of the distance between the acquisition point and the object plane mentioned above. The curve indicated by P corresponds to the method of the embodiment of the present application, and the curve indicated by R corresponds to the method of the related art. Figure 4 The four columns of curves correspond to the first to fourth types of posture information after disturbance. That is, based on the posture information after the first disturbance, Figure 4 The point cloud data shown in (1) is disturbed to obtain the disturbed point cloud data. When the reference plane information is determined based on the disturbed point cloud data by the method of the related art and the method of the embodiment of the present application, the curves of the convergence speed of the method of the related art and the method of the embodiment of the present application are as follows: Figure 7 Based on the posture information after the second disturbance, Figure 4 The point cloud data shown in (1) is disturbed to obtain the disturbed point cloud data. When the reference plane information is determined based on the disturbed point cloud data by the method of the related art and the method of the embodiment of the present application, the curves of the convergence speed of the method of the related art and the method of the embodiment of the present application are as follows: Figure 7 Similarly, based on the posture information after the fourth disturbance, Figure 4 The point cloud data shown in (4) is disturbed to obtain the disturbed point cloud data. When the reference plane information is determined based on the disturbed point cloud data by the method of the related art and the method of the embodiment of the present application, the curves of the convergence speed of the method of the related art and the method of the embodiment of the present application are as follows: Figure 7 As shown in (16) in .

[0273] Depend on Figure 7 It can be seen that the method of the embodiment of the present application converges faster than the method of the related art, indicating that the method of the embodiment of the present application has a small amount of calculation and a low algorithm complexity.

[0274] See also Figure 8 , Figure 8 A histogram showing the number of iterations of the method of the related art and the method of the embodiment of the present application is shown. Figure 8 The A to D shown on the horizontal axis correspond to Figure 4 The point cloud data shown in (1) to (4) in Figure 8 The data shown on the vertical axis corresponds to the number of iterations.

[0275] Figure 8 (1) in corresponds to the posture information after the third disturbance, that is, the posture information after the third disturbance is used to Figure 4 After disturbing the point cloud data shown in (1) to (4) in the figure, four disturbed point cloud data are obtained. When the reference plane information is determined based on the four disturbed point cloud data by the method of the related art and the method of the embodiment of the present application, the histogram of the number of iterations of the method of the related art and the method of the embodiment of the present application is as follows: Figure 8 As shown in (1) in . Figure 8 (2) in corresponds to the posture information after the fourth disturbance, that is, the posture information after the fourth disturbance is used to Figure 4 After disturbing the point cloud data shown in (1) to (4) in the figure, four disturbed point cloud data are obtained. When the reference plane information is determined based on the four disturbed point cloud data by the method of the related art and the method of the embodiment of the present application, the histogram of the number of iterations of the method of the related art and the method of the embodiment of the present application is as follows: Figure 8 As shown in (2) in .

[0276] from Figure 8 It can be seen that the method in the embodiment of the present application has fewer iterations than the method in the related art, which means that the method in the embodiment of the present application has a fast convergence speed, a small amount of calculation, and a low algorithm complexity.

[0277] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions. For example, the point cloud block data and the initial posture information of the sensor involved in this application are all obtained with full authorization.

[0278] The above method is to collect the object plane through the sensor to obtain the initial position information of multiple collection points. Based on the initial position information of the collection points, an initial distance function is constructed, and the distance between each collection point and the object plane is described by the initial distance function. The reference posture information of the sensor is determined by the initial distance function, so that the reference posture information of the sensor can make the distance between the collection point and the object plane the shortest, thereby improving the accuracy of the reference posture information. When the reference plane information of the object plane is determined based on the reference posture information of the sensor in the future, the accuracy of the reference plane information can be improved.

[0279] Fig. 9FIG. 1 is a schematic diagram of a structure of a device for determining plane information provided by an embodiment of the present application. Fig. 9 As shown, the device comprises:

[0280] An acquisition module 901 is used to collect the object plane through a sensor to obtain initial position information of multiple collection points;

[0281] A construction module 902 is used to construct an initial distance function based on initial position information of multiple acquisition points, where the initial distance function is used to describe the distance between the multiple acquisition points and the object plane;

[0282] A determination module 903, configured to determine reference posture information of the sensor based on the initial distance function;

[0283] The determination module 903 is further configured to determine reference plane information of the object plane based on the reference posture information of the sensor.

[0284] In a possible implementation, construction module 902 is used to obtain initial posture information of the sensor; based on the initial posture information of the sensor, initial position information of the designated collection point is converted to obtain the converted initial position information of the designated collection point, where the designated collection point is one or more of the multiple collection points; and based on the converted initial position information, an initial distance function is constructed.

[0285] In one possible implementation, there are multiple sensors, and a determination module 903 is used to obtain first partial derivative information and second partial derivative information, where the first partial derivative information is the partial derivative of a reference distance relative to initial posture information of one sensor, and the second partial derivative information is the partial derivative of the reference distance relative to initial posture information of two sensors, and the reference distance is determined according to an initial distance function; based on the first partial derivative information and the second partial derivative information, first adjustment information of each sensor is determined; based on the first adjustment information and initial posture information of each sensor, first posture information of each sensor is determined; when the first posture information of each sensor meets a set condition, the first posture information of each sensor is determined as the reference posture information of each sensor.

[0286] In one possible implementation, a determination module 903 is used to obtain a reference coefficient, which is used to constrain a reference distance; based on the reference coefficient, first sub-derivative information and second sub-derivative information are determined, the first sub-derivative information being the derivative of the reference coefficient relative to initial posture information of one sensor, and the second sub-derivative information being the derivative of the reference coefficient relative to initial posture information of two sensors; based on the first sub-derivative information and the second sub-derivative information, the first derivative information and the second derivative information are determined.

[0287] In one possible implementation, the determination module 903 is used to determine a first parameter based on initial posture information of a sensor and initial position information of a designated acquisition point obtained by collecting data on an object plane by one sensor; determine first sub-partial derivative information based on the first parameter; determine a second parameter based on initial posture information of two sensors and initial position information of a designated acquisition point obtained by collecting data on the object plane by each of the two sensors; and determine second sub-partial derivative information based on the second parameter.

[0288] In one possible implementation, the reference coefficient is related to an element in the first parameter, and a determination module 903 is used to determine partial derivative information corresponding to the element, where the partial derivative information corresponding to the element is the partial derivative of the element relative to a preset initial posture information of the sensor; based on the partial derivative information corresponding to the element, the first sub-partial derivative information is determined.

[0289] In a possible implementation, the reference coefficient includes a first sub-coefficient, a second sub-coefficient, and a third sub-coefficient;

[0290] Determination module 903 is used to obtain the first element, the second element and the third element, where the first element is constructed based on the reference distance, the second element is constructed based on the first sub-coefficient, the second sub-coefficient and the third sub-coefficient, and the third element is constructed based on the first sub-coefficient and the second sub-coefficient; based on the first element, the second element, the third element, the first sub-partial derivative information and the second sub-partial derivative information, the first partial derivative information and the second partial derivative information are determined.

[0291] In a possible implementation, the determination module 903 is also used to update the initial distance function based on the first posture information of each sensor to obtain a first distance function when the first posture information of each sensor does not meet the set conditions; determine the second posture information of each sensor based on the first distance function; and determine the second posture information of each sensor as the reference posture information of each sensor when the second posture information of each sensor meets the set conditions.

[0292] In a possible implementation, the determination module 903 is further used to convert the initial position information of multiple acquisition points based on the reference posture information of the sensor to obtain the reference position information of the multiple acquisition points; and determine the reference plane information of the object plane based on the reference position information of the multiple acquisition points.

[0293] The above device collects the object plane through the sensor to obtain the initial position information of multiple collection points. Based on the initial position information of the collection points, an initial distance function is constructed, and the distance between each collection point and the object plane is described by the initial distance function. The reference posture information of the sensor is determined by the initial distance function, so that the reference posture information of the sensor can make the distance between the collection point and the object plane the shortest, thereby improving the accuracy of the reference posture information. When the reference plane information of the object plane is determined based on the reference posture information of the sensor in the future, the accuracy of the reference plane information can be improved.

[0294] It should be understood that the above Fig. 9 When the device provided realizes its functions, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0295] Fig.10 The structure block diagram of a terminal device 1000 provided by an exemplary embodiment of the present application is shown. The terminal device 1000 includes: a processor 1001 and a memory 1002 .

[0296] The processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1001 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1001 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1001 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0297] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one computer program, which is used to be executed by the processor 1001 to implement the method for determining the plane information provided in the method embodiment of the present application.

[0298] In some embodiments, the terminal device 1000 may further optionally include: a peripheral device interface 1003 and at least one peripheral device. The processor 1001, the memory 1002 and the peripheral device interface 1003 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1003 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1004, a display screen 1005, a camera assembly 1006, an audio circuit 1007 and a power supply 1008.

[0299] The peripheral device interface 1003 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1001 and the memory 1002. In some embodiments, the processor 1001, the memory 1002, and the peripheral device interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1001, the memory 1002, and the peripheral device interface 1003 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0300] The radio frequency circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1004 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1004 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1004 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The radio frequency circuit 1004 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1004 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0301] The display screen 1005 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1005 is a touch display screen, the display screen 1005 also has the ability to collect touch signals on the surface or above the surface of the display screen 1005. The touch signal can be input to the processor 1001 as a control signal for processing. At this time, the display screen 1005 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1005 can be one, arranged on the front panel of the terminal device 1000; in other embodiments, the display screen 1005 can be at least two, respectively arranged on different surfaces of the terminal device 1000 or in a folding design; in other embodiments, the display screen 1005 can be a flexible display screen, arranged on a curved surface or a folding surface of the terminal device 1000. Even, the display screen 1005 can also be arranged as a non-rectangular irregular figure, that is, a special-shaped screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0302] The camera assembly 1006 is used to capture images or videos. Optionally, the camera assembly 1006 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0303] The audio circuit 1007 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1001 for processing, or input them into the radio frequency circuit 1004 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal device 1000. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1001 or the radio frequency circuit 1004 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1007 may also include a headphone jack.

[0304] The power supply 1008 is used to power various components in the terminal device 1000. The power supply 1008 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1008 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0305] In some embodiments, the terminal device 1000 further includes one or more sensors 1009 . The one or more sensors 1009 include, but are not limited to: an acceleration sensor 1011 , a gyroscope sensor 1012 , a pressure sensor 1013 , an optical sensor 1014 , and a proximity sensor 1015 .

[0306] The acceleration sensor 1011 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal device 1000. For example, the acceleration sensor 1011 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1001 can control the display screen 1005 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1011. The acceleration sensor 1011 can also be used to collect game or user motion data.

[0307] The gyro sensor 1012 can detect the body direction and rotation angle of the terminal device 1000, and the gyro sensor 1012 can cooperate with the acceleration sensor 1011 to collect the user's 3D actions on the terminal device 1000. The processor 1001 can implement the following functions based on the data collected by the gyro sensor 1012: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0308] The pressure sensor 1013 can be set on the side frame of the terminal device 1000 and / or the lower layer of the display screen 1005. When the pressure sensor 1013 is set on the side frame of the terminal device 1000, the user's holding signal of the terminal device 1000 can be detected, and the processor 1001 performs left and right hand recognition or shortcut operations according to the holding signal collected by the pressure sensor 1013. When the pressure sensor 1013 is set on the lower layer of the display screen 1005, the processor 1001 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1005. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0309] The optical sensor 1014 is used to collect the ambient light intensity. In one embodiment, the processor 1001 can control the display brightness of the display screen 1005 according to the ambient light intensity collected by the optical sensor 1014. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the display screen 1005 is reduced. In another embodiment, the processor 1001 can also dynamically adjust the shooting parameters of the camera assembly 1006 according to the ambient light intensity collected by the optical sensor 1014.

[0310] The proximity sensor 1015, also called a distance sensor, is usually arranged on the front panel of the terminal device 1000. The proximity sensor 1015 is used to collect the distance between the user and the front of the terminal device 1000. In one embodiment, when the proximity sensor 1015 detects that the distance between the user and the front of the terminal device 1000 is gradually decreasing, the processor 1001 controls the display screen 1005 to switch from the screen-on state to the screen-off state; when the proximity sensor 1015 detects that the distance between the user and the front of the terminal device 1000 is gradually increasing, the processor 1001 controls the display screen 1005 to switch from the screen-off state to the screen-on state.

[0311] Those skilled in the art will understand that Fig.10 The structure shown in the figure does not constitute a limitation on the terminal device 1000, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0312] Fig.11 This is a schematic diagram of the structure of the server provided in the embodiment of the present application. The server 1100 may have relatively large differences due to different configurations or performances, and may include one or more processors 1101 and one or more memories 1102, wherein the one or more memories 1102 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 1101 to implement the method for determining the plane information provided in the above-mentioned various method embodiments. Exemplarily, the processor 1101 is a CPU. Of course, the server 1100 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 1100 may also include other components for implementing device functions, which will not be described in detail here.

[0313] In an exemplary embodiment, a computer-readable storage medium is further provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned methods for determining plane information.

[0314] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0315] In an exemplary embodiment, a computer program is also provided. The computer program is at least one, and the at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned methods for determining plane information.

[0316] In an exemplary embodiment, a computer program product is further provided. The computer program product stores at least one computer program. The at least one computer program is loaded and executed by a processor to enable an electronic device to implement any of the above-mentioned methods for determining plane information.

[0317] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0318] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0319] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining plane information, characterized in that: The method comprises: The object plane is collected by sensors to obtain initial position information of multiple collection points; Based on the initial position information of the multiple acquisition points, construct an initial distance function, where the initial distance function is used to describe the distance between the multiple acquisition points and the object plane; Based on the initial distance function, determining reference posture information of the sensor; Based on the reference posture information of the sensor, reference plane information of the object plane is determined.

2. The method according to claim 1, characterized in that The constructing an initial distance function based on the initial position information of the multiple acquisition points includes: Acquiring initial posture information of the sensor; Based on the initial posture information of the sensor, converting the initial position information of the designated collection point to obtain the converted initial position information of the designated collection point, wherein the designated collection point is one or more of the multiple collection points; The initial distance function is constructed based on the converted initial position information.

3. The method according to claim 1, characterized in that: There are multiple sensors, and determining the reference posture information of the sensors based on the initial distance function includes: Acquire first partial derivative information and second partial derivative information, wherein the first partial derivative information is a partial derivative of a reference distance relative to initial posture information of one of the sensors, and the second partial derivative information is a partial derivative of the reference distance relative to initial posture information of two of the sensors, and the reference distance is determined according to the initial distance function; Determining first adjustment information of each of the sensors based on the first partial guide information and the second partial guide information; Determining first posture information of each sensor based on the first adjustment information and initial posture information of each sensor; When the first posture information of each sensor satisfies a set condition, the first posture information of each sensor is determined as reference posture information of each sensor.

4. The method according to claim 3, characterized in that The obtaining of the first partial guide information and the second partial guide information comprises: Acquire a reference coefficient, where the reference coefficient is used to constrain the reference distance; Determine first sub-derivative information and second sub-derivative information based on the reference coefficient, wherein the first sub-derivative information is a derivative of the reference coefficient relative to the initial posture information of the one sensor, and the second sub-derivative information is a derivative of the reference coefficient relative to the initial posture information of the two sensors; The first partial derivative information and the second partial derivative information are determined based on the first sub-partial derivative information and the second sub-partial derivative information.

5. The method according to claim 4, characterized in that The determining the first sub-partial derivative information and the second sub-partial derivative information based on the reference coefficient comprises: Determine a first parameter based on initial posture information of the one sensor and initial position information of a designated collection point obtained by collecting data from the one sensor on the object plane; Determine the first sub-partial derivative information based on the first parameter; Determine a second parameter based on the initial posture information of the two sensors and the initial position information of the designated collection point obtained by each of the two sensors collecting the object plane respectively; The second sub-partial derivative information is determined based on the second parameter.

6. The method according to claim 5, characterized in that The reference coefficient is related to an element in the first parameter, and the determining the first sub-partial derivative information based on the first parameter includes: Determine partial derivative information corresponding to the element, where the partial derivative information corresponding to the element is a partial derivative of the element relative to preset initial posture information of the sensor; Based on the partial derivative information corresponding to the element, the first sub-partial derivative information is determined.

7. The method according to claim 4, characterized in that The reference coefficient includes a first sub-coefficient, a second sub-coefficient and a third sub-coefficient; The determining the first partial guide information and the second partial guide information based on the first sub-partial guide information and the second sub-partial guide information includes: Acquire a first element, a second element, and a third element, wherein the first element is constructed based on the reference distance, the second element is constructed based on the first sub-coefficient, the second sub-coefficient, and the third sub-coefficient, and the third element is constructed based on the first sub-coefficient and the second sub-coefficient; The first partial derivative information and the second partial derivative information are determined based on the first element, the second element, the third element, the first sub-partial derivative information, and the second sub-partial derivative information.

8. The method according to claim 3, characterized in that The method further comprises: When the first posture information of each sensor does not satisfy the set condition, updating the initial distance function based on the first posture information of each sensor to obtain a first distance function; Determine second posture information of each sensor based on the first distance function; When the second posture information of each sensor satisfies the set condition, the second posture information of each sensor is determined as the reference posture information of each sensor.

9. The method according to claim 1, characterized in that: The determining, based on the reference posture information of the sensor, reference plane information of the object plane comprises: Based on the reference posture information of the sensor, initial position information of the plurality of collection points is converted to obtain reference position information of the plurality of collection points; Based on the reference position information of the multiple acquisition points, reference plane information of the object plane is determined.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the method for determining plane information as described in any one of claims 1 to 9.

Images