Methods, apparatuses, devices, and media for managing a collection device

Abstract

Methods, apparatuses, devices, and media for managing a capture device are provided. In one method, a capture device is fixed to a rotating device, and initial point cloud data is acquired, the initial point cloud data being captured by the capture device at an initial rotation angle of the rotating device. A plurality of point cloud data is acquired, the plurality of point cloud data being captured by the capture device at a plurality of rotation angles of the rotating device, respectively. Based on the initial point cloud data and the plurality of point cloud data, a pose relationship between the capture device and the rotating device is determined. With example implementations of the present disclosure, a pose relationship between a capture device and a rotating device can be determined based on a plurality of different point clouds associated with a plurality of different rotation angles of the rotating device without the need to use a dedicated calibration tool.

Description

TECHNICAL FIELD

[0001] Exemplary implementations of the present disclosure generally relate to device management, and in particular, to methods, apparatuses, devices, and computer-readable storage media for managing a three-dimensional acquisition device. BACKGROUND

[0002] With the development of digital technology, people can acquire environmental data in a physical environment. Specifically, three-dimensional data and image data of the physical environment can be acquired by using an acquisition device, and the acquired data can be reproduced in a virtual environment corresponding to the physical environment through computer technology. In the data acquisition process, in order to ensure that the acquisition device can acquire comprehensive environmental data, the acquisition device needs to be fixed to a rotating device (for example, a gimbal device or the like), and as the rotating device rotates, the acquisition device obtains environmental data in various directions.

[0003] However, the center of the acquisition device cannot always be accurately aligned with the rotation axis of the rotating device, which causes the pose of the acquisition device to change during the rotation of the rotating device, and thus the acquired environmental data is not data from the same acquisition point. This will cause positional deviation and the like when generating a virtual environment later. Therefore, how to determine the pose relationship between the acquisition device and the rotating device in a simpler and more effective manner has become a problem to be solved in the data acquisition process. SUMMARY

[0004] In a first aspect of the present disclosure, a method for managing an acquisition device is provided, where the acquisition device is fixed to a rotating device. In the method, initial point cloud data is acquired, which is acquired by the acquisition device at an initial rotation angle of the rotating device. A plurality of point cloud data is acquired, which is acquired by the acquisition device at a plurality of rotation angles of the rotating device, respectively. Based on the initial point cloud data and the plurality of point cloud data, a pose relationship between the acquisition device and the rotating device is determined.

[0005] In a second aspect of the present disclosure, an apparatus for managing an acquisition device is provided, where the acquisition device is fixed to a rotating device. The apparatus comprises: a first acquisition module configured to acquire initial point cloud data, which is acquired by the acquisition device at an initial rotation angle of the rotating device; a second acquisition module configured to acquire a plurality of point cloud data, which is acquired by the acquisition device at a plurality of rotation angles of the rotating device, respectively; and a determination module configured to determine a pose relationship between the acquisition device and the rotating device based on the initial point cloud data and the plurality of point cloud data.

[0006] In a third aspect of the disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit cause the device to perform the method according to the first aspect of the disclosure.

[0007] In a fourth aspect of the disclosure, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, causes the processor to implement the method according to the first aspect of the disclosure.

[0008] It is to be understood that the details set forth herein do not purport to be essential or limiting of the disclosed implementations. The disclosure is described in relation to the illustrated implementation as a non-limiting example only, which aspects and features will become more readily apparent upon review of the following detailed description and summary of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other features, aspects and advantages of various implementations of the present disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numbers represent like elements throughout. The drawings depict one or more implementations of the present disclosure and are not intended to limit the scope of the disclosure. The drawings serve to explain principles of the present disclosure.

[0010] Figure 1 A block diagram illustrating an example environment in which implementations of the present disclosure can be implemented is shown;

[0011] Figure 2 A block diagram illustrating a position relationship between a collection device and a rotating device according to one implementation is shown;

[0012] Figure 3 A block diagram illustrating a process of managing a collection device according to some implementations of the present disclosure is shown;

[0013] Figure 4 A block diagram illustrating a distribution of beams of a collection device according to some implementations of the present disclosure is shown;

[0014] Figure 5 A block diagram illustrating a field of view of a collection device according to some implementations of the present disclosure is shown;

[0015] Figure 6 A block diagram illustrating a plurality of positions of a collection device determined according to some implementations of the present disclosure is shown;

[0016] Figure 7 A block diagram illustrating a collection device coordinate system and a rotating device coordinate system according to some implementations of the present disclosure is shown;

[0017] Figure 8 A flowchart illustrating a method for managing a collection device according to some implementations of the present disclosure is shown;

[0018] Figure 9 a block diagram illustrating an apparatus for managing a collection device according to some implementations of the present disclosure; and

[0019] Figure 10 a block diagram illustrating an apparatus capable of implementing a number of implementations of the present disclosure. DETAILED DESCRIPTION

[0020] Implementations of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While several implementations of the present disclosure are described, it should be understood that the present disclosure can be embodied in various forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. It should be understood that the figures and implementations of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0021] In the description of implementations of the present disclosure, the term "includes" and its derivatives, shall be understood to be open-ended, i.e., "includes, but is not limited to". The term "based on" is understood as "based at least in part on". The term "one implementation" or "the implementation" is understood as "at least one implementation". The term "some implementations" is understood as "at least some implementations". Other explicit or implicit definitions can also be included below. As used herein, the term "model" can represent the relationship between various data. For example, the above-mentioned relationship can be obtained based on various technical solutions known at present and / or to be developed in the future.

[0022] It can be understood that the data involved in the technical solutions of the present disclosure (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0023] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the scope of use, the scene of use, etc. should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0024] For example, when responding to the active request of the user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be executed will require the acquisition and use of the personal information of the user. Thus, the user can voluntarily choose whether to provide personal information to the electronic device, application program, server or storage medium, etc. software or hardware that executes the operation of the technical solutions of the present disclosure according to the prompt information.

[0025] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0026] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0027] Example Environment

[0028] Figure 1 A block diagram of an example environment 100 in which the implementation of this disclosure can be implemented is shown. For example... Figure 1 As shown, environment 100 may include physical environment (e.g., building) 110. When collecting data from physical environment 110, user 122 (e.g., a data collector) can acquire raw model data of physical environment 110 via acquisition device 120 (e.g., radar equipment, etc.). Terminal device 130 can be used to control acquisition device 120 and perform preliminary processing on the acquired raw model data. For example, the acquired raw model data can be transmitted to terminal device 130. Operations such as stitching can be performed at terminal device 130 to stitch raw model data, including multiple local spaces of the building, into a model 140 of the entire physical environment 110. Furthermore, model 140 can be uploaded from terminal device 130 to server device 160 via network 150 for further processing.

[0029] Typically, to collect data from multiple angles of the physical environment 110, the acquisition device 120 can be fixed to a rotating device. For example, the acquisition device 120 can be fixed to a rotatable pan-tilt head of the rotating device, and the pan-tilt head can be rotated to orient the acquisition device 120 in various directions, thereby collecting data from a 360-degree radius around a certain location. However, due to manufacturing and installation precision requirements, the center of the acquisition device 120 cannot always be aligned with the rotation axis of the rotating device. This causes the position of the acquisition device 120 to change during pan-tilt head rotation; for example, it may rotate around the rotation axis by a certain radius.

[0030] Figure 2 A block diagram 200 illustrates the positional relationship between a data acquisition device 120 and a rotating device 210 according to a technical solution. (See diagram 200.) Figure 2A perspective view of the acquisition device 120 and the rotation device 210 is shown. The rotation device 210 can include a circular gimbal plane. When the acquisition device 120 is not aligned with the rotation axis 214 (i.e., the acquisition device 120 is located at a position other than the center 212 of the circular gimbal plane, and the coordinate system 220 of the acquisition device 120 is different from the coordinate system 216 of the rotation device 210), the acquisition device 120 will move around the center 212. For example, when the rotation device is at an initial angle, it is assumed that the coordinate system of the acquisition device 120 is as shown in the coordinate system 220. The rotation device 210 can be rotated along the direction 216. When the rotation device 210 reaches a certain angle, the acquisition device 120 will reach a new position (as shown in the acquisition device 120' on the left side of the coordinate system 220'). At this time, the positions and orientations of the coordinate axes in the corresponding coordinate system 220' will change, and thus the acquired data is not from a fixed position. The error in the acquired data will further cause model errors in the process of generating a virtual environment. Figure 2

[0031] Although a hardware-based calibration tool has been developed to align the center of the acquisition device and the rotation axis of the rotation device. However, the calibration tool involves complex operations and is largely dependent on the experience of professionals. At this time, it is desirable to determine the pose relationship between the acquisition device 120 and the rotation device 210 in a simpler and more effective manner, and to calibrate the data acquired by the acquisition device.

[0032] Summary process of managing an acquisition device

[0033] To solve the above technical problems, according to one example implementation of the present disclosure, a method for managing an acquisition device is proposed. In the method, the acquisition device 120 is fixed to the rotation device 210. For example, the acquisition device 120 can be aligned with the rotation axis of the rotation device 210 based on a conventional manner, and then the acquisition device 120 is fixed. Alternatively and / or additionally, the acquisition device 120 can be fixed to any position of the rotation device 210.

[0034] Reference is made to Figure 3 According to one example implementation of the present disclosure, Figure 3 A block diagram 300 of a process of managing an acquisition device 120 according to some implementations of the present disclosure is shown. Figure 3 ​A top view of the rotating device 210 is shown, and the acquisition device 120 can be fixed at a position 310 (i.e., P0) of the rotating device. The rotating device 210 can be placed at an initial rotation angle. For example, a direction from the center 212 to the position 310 can be defined as 0 degree, and the initial rotation angle is 0 degree when the acquisition device 120 is located at the position P0. An initial point cloud data can be obtained from the acquisition device 120, and the initial point cloud data is acquired by the acquisition device 120 from the rotating device 210 at the initial rotation angle (0 degree).

[0035] Further, the rotating device 210 can be rotated along the direction 216, and the rotating device 210 can be placed at a plurality of rotation angles to obtain a plurality of point cloud data. At this time, the plurality of point cloud data is acquired by the acquisition device 120 from the rotating device 210 at the plurality of rotation angles, respectively. Specifically, when the rotating device 210 is located at an angle 320, the acquisition device 120 is located at a position P1. The rotating device 210 can be further rotated, for example, the rotating device 210 can be placed at angles 322, 323, …, and 324 to obtain a plurality of point cloud data associated with each rotation angle, respectively. It will be understood that in this technical solution, only the rotating device 210 needs to be rotated and placed at a predetermined angle (for example, the initial rotation angle 0 degree and the subsequent angles 320, 322, 323, …, and 324) to automatically obtain the initial point cloud data and the subsequent plurality of point cloud data. This process does not involve complex manual operation, thereby reducing the error caused by manual operation in the calibration process.

[0036] Then, the pose relationship between the acquisition device 120 and the rotating device 210 can be determined based on the initial point cloud data acquired at the initial rotation angle and the plurality of point cloud data acquired at the plurality of subsequent rotation angles, respectively. At this time, only mathematical operations are involved in the process of determining the pose relationship by using the initial point cloud data and the plurality of point cloud data. Thus, the pose relationship between the acquisition device 120 and the rotating device 210 (i.e., the external parameters of the acquisition device 120) can be automatically determined without excessive manual workload, thereby realizing the calibration process.

[0037] Detailed procedure for managing a collection device

[0038] Having described the outline of one example implementation according to the present disclosure, more details about managing the acquisition device 120 will be described in the following. According to one example implementation of the present disclosure, the acquisition device 120 can be a multi-line lidar device, see Figure 4 More details of the acquisition device 120 are described. Figure 4 A block diagram 400 of the distribution of beams of the acquisition device 120 according to some implementations of the present disclosure is shown. As shown in FIG. 4, the acquisition device 120 can include a plurality of beams 410, 420, 430, 440, 450, 460, 470, 480, 490, and 500.Figure 4 As shown, the transmitter of the multi-line laser radar device can be installed at the center of the plane 420, and can emit multiple beams 410 in different directions, which can be diffusely reflected after encountering an object, and returned to the receiver of the multi-line laser radar device. At this time, based on the difference between the emission time and the reception time of the beams, the distance between the object and the multi-line laser radar device can be determined, and the point cloud data within a certain range around the multi-line laser radar device can be obtained.

[0039] Here, the rotating device 210 can be a gimbal device for supporting the acquisition device 120, and the rotating device 210 can be fixed to the plane on which the gimbal is located. By rotating the plane, the acquisition device 120 can collect environmental data in a horizontal direction of 360 degrees around the acquisition device 120. In this way, more abundant environmental data can be obtained, and a more realistic virtual environment can be constructed.

[0040] Based on different configurations, the multi-line laser radar device can collect point cloud data within different field of view ranges around the multi-line laser radar device. At this time, the angle of each rotation of the rotating device 210 can be determined based on the field of view range. Figure 5 A block diagram 500 of a field of view (FOV) of the acquisition device 120 according to some implementations of the present disclosure is shown. As shown, the field of view range 510 represents a cone of view of the acquisition device 120, and the field of view range 510 can be defined using angles in both horizontal and vertical directions. Figure 5 As shown, the field of view range 510 represents a cone of view of the acquisition device 120, and the field of view range 510 can be defined using angles in both horizontal and vertical directions. Figure 5 In particular, the angle 520 represents the range in the horizontal direction (i.e., the horizontal FOV angle), and the angle 530 represents the range in the vertical direction (i.e., the vertical FOV angle).

[0041] According to one exemplary implementation of the present disclosure, the angle 520 or 530 can be used to determine the rotation angle based on the direction in which the acquisition device 120 is fixed to the rotating device. For example, in order to obtain omnidirectional point cloud data, the acquisition device 120 can be fixed to the plane of the rotating device 210 in the vertical direction. In other words, at this time, the plane 420 of the acquisition device 210 is perpendicular to the gimbal plane of the rotating device 210.

[0042] In order to facilitate the later splicing operation, it is necessary to ensure that there is an overlapping area between each point cloud data collected. At this time, the threshold angle of each rotation of the rotating device 210 should not be too large. According to one exemplary implementation of the present disclosure, the number N of point clouds that need to be collected within a range of 360 degrees can be determined based on the FOV angle in the corresponding direction. Specifically, the number N of point clouds can be determined based on formula 1:

[0043] N ≥ 360 * 2 / FOV Formula 1

[0044] In Equation 1, N represents the number of point clouds that should be acquired in the 360-degree range, and FOV represents the horizontal FOV angle after the acquisition device 120 is fixed to the rotating device 210. When the acquisition device 120 is fixed vertically to the rotating device 210, the horizontal FOV angle at this time is the vertical FOV angle of the acquisition device 120 itself. When the acquisition device 120 is fixed horizontally to the rotating device 210, the horizontal FOV angle at this time is the horizontal FOV angle of the acquisition device 120 itself. With the exemplary implementation of the present disclosure, it can be ensured that there is an overlapping area between the acquired adjacent point cloud data, which facilitates the subsequent alignment operation.

[0045] In Equation 1, assuming that the FOV angle is 70 degrees, the number of point clouds that need to be acquired is: At this time, in addition to the initial point cloud data acquired at 0 degrees, n = N - 1 = 11 - 1 = 10 point cloud data need to be acquired. According to one exemplary implementation of the present disclosure, for the convenience of description, the acquired initial point cloud data can be denoted as C0, and the point cloud data acquired at the subsequent angles can be denoted as C1,..., C n At this time, the sequence of point cloud data C0, C1,..., C n At this time, after obtaining the initial point cloud data at 0 degrees, the rotating device 210 can be rotated by a threshold angle (for example, FOV / 2) each time to obtain the subsequent point cloud data. In this way, the predetermined threshold of the rotation angle can be determined based on simple mathematical operations.

[0046] According to one exemplary implementation of the present disclosure, based on the initial point cloud data C0and the plurality of point cloud data C1,..., C n , the initial position of the acquisition device 120 when the rotating device 210 is at the initial rotation angle and the plurality of rotation positions of the acquisition device when the rotating device is at the plurality of rotation angles can be determined. For the convenience of calculation, the pose of the acquisition device 120 when the rotating device 210 is at 0 degrees can be taken as the initial position. Figure 6 A block diagram 600 of the determined plurality of positions of the acquisition device according to some implementations of the present disclosure is shown. As Figure 6 shown, the acquisition device 120 is at the position 310 (i.e., P0) in the initial stage, the acquisition device 210 can be rotated, and the acquisition device 120 will be at the position 311 (P1), the position 610 (P2), the position 612 (P3),..., the position 614 (P n ) respectively.

[0047] According to one exemplary implementation of the present disclosure, based on the initial position, the acquisition device coordinate system of the acquisition device 120 can be determined. AsFigure 6 As shown, position 310 can be taken as the origin of the acquisition device coordinate system 620, and the vertical direction of the line connecting the center 212 and the position 310 can be taken as the x-axis direction of the acquisition device coordinate system 620, and the direction of the line connecting the center 212 and the position 310 can be taken as the z-axis direction of the acquisition device coordinate system 620. Further, the direction perpendicular to the xoz plane can be taken as the y-axis direction of the acquisition device coordinate system 620 based on the right-hand coordinate system. In this way, the acquisition device coordinate system 620 can be conveniently established, and at this time the coordinates of the position 310 under the acquisition device coordinate system 620 are (0, 0, 0).

[0048] Further, the coordinates of each position 311, 610, 612,..., 614 can be represented under the acquisition device coordinate system 620. For example, the relationship between each position can be represented in the form of a rotation matrix and a translation matrix. For example, the rotation matrix between the i-th position (i.e., point P i，(i-1) ) and the i-1-th position (i.e., point P i ) can be represented by a rotation matrix R i-1 . Specifically, R 1，0 represents the rotation matrix between point P1 and point P0, R 2，1 represents the rotation matrix between point P2 and point P1, R 3，2 represents the rotation matrix between point P3 and point P2,..., and R n，(n-1) represents the rotation matrix between point P n and point P n-1 .

[0049] According to one exemplary implementation of the present disclosure, the translation matrix between the i-th position (i.e., point P i，(i-1) ) and the i-1-th position (i.e., point P i ) can be represented by a translation matrix T i-1 . Specifically, T 1，0 represents the translation matrix between point P1 and point P0, T 2，1 represents the translation matrix between point P2 and point P1, T 3，2 represents the translation matrix between point P3 and point P2,..., and T n，(n-1) represents the translation matrix between point P n and point P n-1 . At this time, the values of each rotation matrix and translation matrix are unknown, and the above unknown matrices can be solved based on the overlapping regions in the aligned point cloud data. In other words, the overlapping regions in the initial point cloud data and the plurality of point cloud data can be aligned to determine the plurality of rotation positions.

[0050] According to one exemplary implementation of the present disclosure, the overlapping parts in the collected multiple point cloud data can be aligned based on an Iterative Closest Point (ICP) algorithm. In the aligned point cloud data, there is the following formula 2:

[0051] C0=R 1，0 C1+T 1，0

[0052] C1=R 2，1 C2+T 2，1

[0053] …

[0054] C n-1 =R n，(n-1) C n +T n，(n-1) Formula 2

[0055] In formula 2, C0, C1,..., C n (represented by known data) respectively represent the point cloud data obtained at respective points P0, P1,..., P n R i，(i-1) represents a rotation matrix between the i-th position (i.e., point P i ) and the i-1-th position (i.e., point P i-1 ), and T i(i-1) represents a translation matrix between the i-th position (i.e., point P i ) and the i-1-th position (i.e., point P i-1 ). At this time, based on the ICP algorithm and by solving formula 2, the corresponding rotation matrixes [R 1，0 , R 2，1 , R 3，2 ,..., R n，(n-1 ] and translation matrixes [T 1，0 , T 2，1 , T 3，2 ,..., T n，(n-1) ] can be obtained.

[0056] Further, the specific coordinates of the respective positions, i.e., Here, R i，0 represents a rotation matrix of point P i relative to the coordinate origin (i.e., point P0), and T i，0 represents a translation matrix of point P i relative to the coordinate origin (i.e., point P0). Thus, represents P iThe pose relative to the coordinate system 620 of the acquisition device. Using the exemplary implementation of this disclosure, the process of solving the pose of each point can be transformed into a mathematical operation, thereby determining the pose of each point (i.e., the initial position and multiple rotational positions) in the coordinate system 620 of the acquisition device in a simpler and more accurate way.

[0057] Furthermore, the rotation axis of the rotating device can be determined based on the initial position and multiple rotational positions. It will be understood that since the acquisition device 120 is fixed to the rotating device 210, the distance between the acquisition device 120 and the center 212 remains constant during rotation. Thus, the initial position (point P0) and multiple rotational positions (points P1, P2, P3, ..., P...) are determined. n All of these points are located on the circumference of a circle centered at point 212. In other words, multiple points P0, P1, ..., P2 can be determined. n The circumcircle of .

[0058] It will be understood that, due to P0, P1, ..., P n Since the coordinates are known, the radius R of the circumcircle can be determined using simple mathematical operations, and thus the coordinates of the center 212 in the coordinate system of the acquisition device can be determined. It will be understood that since the center 212 is located at the z-axis of the coordinate system of the acquisition device, the x and y coordinates of the center 212 are both 0. At this time, the coordinates of the center 212 can be represented as (0,0,-R).

[0059] Subsequently, the axis position of the rotation axis of the rotating device 210 can be determined as (0, 0, -R) based on the center 212 of the circumcircle. Further, the axis direction of the rotation axis of the rotating device 210 can be determined based on the normal direction of the circumcircle. For example, the upward direction perpendicular to the plane containing the circumcircle can be defined as the direction of the rotation axis. At this point, the pose relationship between the acquisition device 120 and the rotating device 210 in the initial state can be clearly defined based on the rotation axis of the rotating device 210 and the initial position of the acquisition device 120. Therefore, the pose relationship between the acquisition device 120 and the rotating device 210 can be determined using the rotation axis and the initial position.

[0060] According to an exemplary implementation of this disclosure, the rotation coordinate system of the rotating device 210 can be determined based on the rotation axis and the initial position. Specifically, one coordinate axis of the rotation coordinate system can be determined based on the axis position of the rotation axis and the initial position, and the other coordinate axis can be determined based on the axis direction of the rotation axis. Subsequently, the rotation coordinate system can be established based on the determined two coordinate axes and using a right-handed coordinate system. Figure 7 A block diagram 700 shows a data acquisition device coordinate system 620 and a rotating device coordinate system 710 according to some implementations of this disclosure. For example... Figure 7As shown, the axis position of the rotation axis (i.e., the center 212 of the circle) can be taken as the origin of the rotation device coordinate system 710, the direction from the center 212 to the position 310 can be taken as the x-axis of the rotation device coordinate system 710, the axis direction of the rotation axis can be taken as the z-axis direction, and the y-axis of the rotation device coordinate system 710 (i.e., perpendicular to the xoz plane) can be determined based on the right-hand system.

[0061] At this time, the distance 720 between the origin of the rotation device coordinate system 710 and the origin of the acquisition device coordinate system 620 is R, and the directions of the xyz axes of the two coordinate systems are as shown. At this time, the pose relationship between the acquisition device 120 and the rotation device 210 can be determined based on the definitions of the two coordinate systems. At this time, the rotation matrix and the translation matrix in the pose relationship are respectively as shown in the following formula 3.

[0062]

[0063] It will be understood that the above only shows a specific example of the coordinate system in an exemplary manner. Alternatively and / or additionally, the respective axes of the coordinate system can be determined based on other directions, and the relationship between the respective axes can also be defined based on, for example, a left-hand coordinate system. For example, when the defined directions of the coordinate axes in the two coordinate systems are different, the rotation matrix R can have different representations. For another example, when the coordinate origins of the two coordinate systems have different positional relationships, the translation matrix T can have different representations. In this way, the process of determining the pose relationship is converted into a mathematical operation process, so that the pose relationship can be automatically determined without the calibration operation of a professional technician.

[0064] After the pose relationship between the acquisition device 120 and the rotation device 210 has been determined, the point cloud data acquired by the acquisition device 120 can be calibrated by using the determined pose relationship. It is assumed that the point cloud data acquired by the acquisition device 120 is represented as C collecting device The point cloud data acquired by the acquisition device 120 can be calibrated based on the following formula 4.

[0065] C rotating device = RC collecting device + T Formula 4

[0066] In formula 4, C collecting device represents the point cloud data acquired by the acquisition device 120 in the acquisition device coordinate system 620, C rotating deviCeR and T represent the rotation matrix and the translation matrix in the determined pose relationship, respectively. With the exemplary implementation of the present disclosure, the point cloud captured by the capturing device 120 can be calibrated to the rotating device coordinate system 710. In this way, the calibrated point cloud is equivalent to the one captured when the capturing device 120 is placed at the center of the rotating device 210. At this time, as the rotating device rotates, the position of the center 212 of the rotating device will remain unchanged. Thus, the calibrated point cloud data can more accurately reflect the data of the physical environment 110 captured at a fixed point, and further construct a virtual environment in a more accurate and realistic manner.

[0067] According to one exemplary implementation of the present disclosure, the capturing device 120 can have an image capturing function, and at this time, the panoramic data of the physical environment 110 can be captured by the capturing device 120. Further, the panoramic data captured by the capturing device 120 can be calibrated by the determined pose relationship. At this time, the calibrated panoramic data is equivalent to the one taken at the center of the rotating device 210, and thus can more accurately reflect the positional relationship between the viewpoint of the panoramic data and the real physical environment. Further, the virtual environment of the physical environment can be constructed by the calibrated point cloud data and the calibrated panoramic image data.

[0068] It will be appreciated that although the above describes the case where the capturing device itself has an image capturing function. Alternatively and / or additionally, an image capturing device can be coupled to the capturing device 120. In the case where the pose relationship between the image capturing device and the capturing device 120 is known, the panoramic image can be calibrated based on the known pose relationship and the determined pose relationship based on the above described method. In this way, the virtual environment generated by the calibrated point cloud data and the panoramic data will be more consistent with the real physical environment 110, and thus the physical environment 110 can be more realistically simulated.

[0069] In the context of the present disclosure, it is not necessary to use a professional calibration tool to align the center of the capturing device 120 with the center of the rotating device 210. With the exemplary implementation of the present disclosure, the pose relationship between the capturing device 120 and the rotating device 210 can be automatically determined based on multiple point cloud data captured by the capturing device 120 at different rotation angles of the rotating device 210. In this way, the workload of manual labor of professionals can be reduced, and the performance and accuracy of the calibration operation can be improved.

[0070] Exemplary process

[0071] Figure 8A flowchart of a method 800 for managing a collection device is shown according to some implementations of the present disclosure. The collection device is fixed to a rotating device. At block 810, initial point cloud data is acquired, the initial point cloud data being collected by the collection device at an initial rotation angle of the rotating device. At block 820, a plurality of point cloud data is acquired, the plurality of point cloud data being collected by the collection device at a plurality of rotation angles of the rotating device respectively; and at block 830, a pose relationship between the collection device and the rotating device is determined based on the initial point cloud data and the plurality of point cloud data.

[0072] According to one example implementation of the present disclosure, an angle difference between the initial rotation angle and two adjacent rotation angles of the plurality of rotation angles has a predetermined threshold.

[0073] According to one example implementation of the present disclosure, the method 800 further includes determining a number of the initial point cloud data and the plurality of point cloud data based on a field of view range of the collection device.

[0074] According to one example implementation of the present disclosure, determining the pose relationship includes determining, based on the initial point cloud data and the plurality of point cloud data, an initial position of the collection device when the rotating device is at the initial rotation angle, and a plurality of rotation positions of the collection device when the rotating device is at the plurality of rotation angles respectively; determining a rotation axis of the rotating device based on the initial position and the plurality of rotation positions; and determining the pose relationship between the collection device and the rotating device based on the rotation axis and the initial position.

[0075] According to one example implementation of the present disclosure, determining the plurality of rotation positions respectively includes determining a collection device coordinate system of the collection device based on the initial position; and aligning overlapping regions of the initial point cloud data and the plurality of point cloud data to determine the plurality of rotation positions, the plurality of rotation positions being poses in the collection device coordinate system.

[0076] According to one example implementation of the present disclosure, determining the rotation axis of the rotating device includes determining a circumscribed circle associated with the initial position and the plurality of rotation positions, each position corresponding to the initial position and the plurality of rotation positions respectively being located on a circumference of the circumscribed circle; determining an axis position of the rotation axis of the rotating device based on a center of the circumscribed circle; and determining an axis direction of the rotation axis of the rotating device based on a normal direction of the circumscribed circle.

[0077] According to one example implementation of the present disclosure, determining the pose relationship between the collection device and the rotating device includes determining a rotating device coordinate system of the rotating device based on the rotation axis and the initial position; and determining the pose relationship between the collection device and the rotating device based on the rotating device coordinate system and the collection device coordinate system.

[0078] According to one example implementation of the present disclosure, determining the rotation device coordinate system comprises: determining a first coordinate axis of the rotation device coordinate system of the rotation device based on the axis position and the initial position of the rotation axis; determining a second coordinate axis of the rotation device coordinate system based on the axis direction of the rotation axis; and determining the rotation device coordinate system based on the first coordinate axis and the second coordinate axis.

[0079] According to one example implementation of the present disclosure, the acquisition device is a multi-line laser radar device, and the rotation device is a rotation holder device for supporting the acquisition device.

[0080] According to one example implementation of the present disclosure, the method 800 further comprises: calibrating point cloud data acquired by the acquisition device by using the pose relationship.

[0081] According to one example implementation of the present disclosure, the method 800 further comprises: calibrating panoramic data acquired by the acquisition device by using the pose relationship; and constructing a virtual environment of the physical environment by using the calibrated point cloud data and the calibrated panoramic image data.

[0082] Example apparatuses and devices

[0083] Figure 9 A block diagram 900 of an apparatus for managing an acquisition device according to some implementations of the present disclosure is shown. The apparatus 900 comprises: a first obtaining module 910 configured to obtain initial point cloud data, the initial point cloud data being acquired by an acquisition device at an initial rotation angle of a rotation device; a second obtaining module 920 configured to obtain a plurality of point cloud data, the plurality of point cloud data being acquired by the acquisition device at a plurality of rotation angles of the rotation device respectively; and a determining module 930 configured to determine a pose relationship between the acquisition device and the rotation device based on the initial point cloud data and the plurality of point cloud data.

[0084] According to one example implementation of the present disclosure, an angle difference between the initial rotation angle and two adjacent rotation angles in the plurality of rotation angles has a predetermined threshold.

[0085] According to one example implementation of the present disclosure, the apparatus 900 further comprises: a number determining module configured to determine a number of the initial point cloud data and the plurality of point cloud data based on a field of view range of the acquisition device.

[0086] According to one example implementation of the present disclosure, the determining module 930 includes: a position determining module configured to determine, based on the initial point cloud data and the plurality of point cloud data, an initial position of the collection device when the rotating device is at an initial rotation angle, and a plurality of rotating positions of the collection device when the rotating device is at a plurality of rotation angles respectively; and a rotation axis determining module configured to determine, based on the initial position and the plurality of rotating positions, a rotation axis of the rotating device; and a pose determining module configured to determine, based on the rotation axis and the initial position, a pose relationship between the collection device and the rotating device.

[0087] According to one example implementation of the present disclosure, the position determining module includes: a collection device coordinate system determining module configured to determine, based on the initial position, a collection device coordinate system of the collection device; and an alignment module configured to align overlapping regions in the initial point cloud data and the plurality of point cloud data to determine the plurality of rotating positions, the plurality of rotating positions being poses in the collection device coordinate system.

[0088] According to one example implementation of the present disclosure, the rotation axis determining module includes: a circumscribed circle determining module configured to determine a circumscribed circle associated with the initial position and the plurality of rotating positions, each position corresponding to the initial position and the plurality of rotating positions being located on a circumference of the circumscribed circle; an axis position determining module configured to determine, based on a center of the circumscribed circle, an axis position of the rotation axis of the rotating device; and an axis direction determining module configured to determine, based on a normal direction of the circumscribed circle, an axis direction of the rotation axis of the rotating device.

[0089] According to one example implementation of the present disclosure, the determining module 930 includes: a rotating device coordinate system determining module configured to determine, based on the rotation axis and the initial position, a rotating device coordinate system of the rotating device; and a pose relationship determining module configured to determine, based on the rotating device coordinate system and the collection device coordinate system, the pose relationship between the collection device and the rotating device.

[0090] According to one example implementation of the present disclosure, the rotating device coordinate system determining module includes: a first coordinate axis determining module configured to determine, based on the axis position of the rotation axis and the initial position, a first coordinate axis of the rotating device coordinate system of the rotating device; a second coordinate axis determining module configured to determine, based on the axis direction of the rotation axis, a second coordinate axis of the rotating device coordinate system; and a coordinate system determining module configured to determine, based on the first coordinate axis and the second coordinate axis, the rotating device coordinate system.

[0091] According to one example implementation of the present disclosure, the collection device is a multi-line laser radar device, and the rotating device is a rotating holder device for supporting the collection device.

[0092] According to one example implementation of the present disclosure, the apparatus further comprises a calibration module configured to calibrate the point cloud data captured by the capture device using the pose relationship.

[0093] According to one example implementation of the present disclosure, the apparatus further comprises a panoramic data calibration module configured to calibrate the panoramic data captured by the capture device using the pose relationship; and a construction module configured to construct a virtual environment of the physical environment using the calibrated point cloud data and the calibrated panoramic image data.

[0094] Figure 10 A block diagram of a device 1000 capable of implementing the implementations of the present disclosure is shown. It should be understood that Figure 10 The computing device 1000 shown is merely exemplary and should not be construed as limiting the functionality and scope of implementations described herein. Figure 10 The computing device 1000 shown can be used to implement the methods described above.

[0095] As Figure 10 The computing device 1000 is in the form of a general-purpose computing device as shown. Components of the computing device 1000 can include, but are not limited to, one or more processors or processing units 1010, a memory 1020, a storage device 1030, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060. The processing unit 1010 can be a real or virtual processor and is capable of executing various processing in accordance with programs stored in the memory 1020. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the computing device 1000.

[0096] The computing device 1000 typically includes a plurality of computer storage media. Such media can be any available media that is accessible by the computing device 1000 and includes both volatile and non-volatile media, removable and non-removable media. The memory 1020 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory), or some combination thereof. The storage device 1030 can be a removable or non-removable media and can include machine-readable media, such as a flash drive, a magnetic disk, or any other media that can be used to store information and / or data (e.g., training data for training) and that can be accessed by the computing device 1000.

[0097] The computing device 1000 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown, the computing device 1000 can further include a network-based storage device, such as the Internet or an intranet. Figure 10As shown in FIG. 10, a disk drive 1025 can be provided to read from or write to a removable, non- volatile magnetic media (e.g., a "floppy drive" within the disk drive 1025). In such a case, the disk drive 1025 can be connected to the bus - 1020 by one or more data media interfaces 1027. The memory 1020 can include a computer-readable product 1025 having one or more program modules configured to carry out the various methods or actions of the various implementations of the present disclosure.

[0098] The communication unit 1040 enables communication over a communication medium to other computing devices. Additionally, the functionality of the components of the computing device 1000 can be implemented in a single computing cluster or multiple computer machines that are capable of communicating over a communication connection. Thus, the computing device 1000 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.

[0099] The input device 1050 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 1060 can be one or more output devices, such as a display, a speaker, a printer, etc. The computing device 1000 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc. through the communication unit 1040, as needed, communicate with one or more devices that enable a user to interact with the computing device 1000, or communicate with any devices (e.g., a network card, a modem, etc.) that enable the computing device 1000 to communicate with one or more other computing devices. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0100] According to an example implementation of the present disclosure, a computer-readable storage medium is provided, having computer-executable instructions stored thereon, where the computer-executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, where the computer-executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is provided, having a computer program stored thereon, which when executed by a processor implements the method described above.

[0101] Various aspects of the disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, and computer program products according to implementations of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0102] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage viruses or other code or instructions implementing a functionally equivalent to that of the software manual reproduction process. The instructions can form an interface to other code or programs that or adapted at one time to implement specific processes or can form a component of another program, which can implement specific processes as conditions require.

[0103] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0104] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various implementations of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions ("instructions"). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0105] implementations have been described above, the description is intended to be illustrative, and not restrictive, of the implementations disclosed. Many modifications and variations of the implementations described herein will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The description herein is intended to be illustrative, and not restrictive, of the implementations described. The scope of the implementations described herein will be apparent from the appended claims, along with their equivalents.

Claims

1. A method for managing a collection device, the collection device being fixed to a rotating device, and the method comprising: acquiring initial point cloud data, the initial point cloud data being collected by the collection device at an initial rotation angle of the rotating device; acquiring a plurality of point cloud data, the plurality of point cloud data being collected by the collection device at a plurality of rotation angles of the rotating device respectively; and determining a pose relationship between the collection device and the rotating device based on the initial point cloud data and the plurality of point cloud data, comprising: determining an initial position of the collection device when the rotating device is at the initial rotation angle and a plurality of rotation positions of the collection device when the rotating device is at the plurality of rotation angles respectively based on the initial point cloud data and the plurality of point cloud data; determining a circumscribed circle associated with the initial position and the plurality of rotation positions, each position corresponding to the initial position and the plurality of rotation positions respectively being located on a circumference of the circumscribed circle; determining an axis position of a rotation axis of the rotating device based on a center of the circumscribed circle; and determining the pose relationship between the collection device and the rotating device based on the rotation axis and the initial position. 2.The method of claim 1, wherein an angle difference between the initial rotation angle and two adjacent rotation angles of the plurality of rotation angles has a predetermined threshold.

3. The method of claim 1, further comprising: determining a number of the initial point cloud data and the plurality of point cloud data based on a field of view range of the collection device. 4.The method of claim 1, wherein determining the plurality of rotation positions respectively comprises: determining a collection device coordinate system of the collection device based on the initial position; and aligning overlapping areas in the initial point cloud data and the plurality of point cloud data to determine the plurality of rotation positions, the plurality of rotation positions being poses in the collection device coordinate system. 5.The method of claim 1, further comprising: determining an axis direction of the rotation axis of the rotating device based on a normal direction of the circumscribed circle. 6.The method of claim 5, wherein determining the pose relationship between the collection device and the rotating device comprises: determining a rotating device coordinate system of the rotating device based on the rotation axis and the initial position; and determining the pose relationship between the collection device and the rotating device based on the rotating device coordinate system and the collection device coordinate system. 7.The method of claim 6, wherein determining the rotating device coordinate system comprises: determining a first coordinate axis of the rotating device coordinate system based on the axis position of the rotation axis and the initial position; determining a second coordinate axis of the rotating device coordinate system based on the axis direction of the rotation axis; and determining the rotating device coordinate system based on the first coordinate axis and the second coordinate axis. 8.The method of claim 1, wherein the collection device is a multi-line laser radar device, and the rotating device is a rotating holder device for supporting the collection device. ​ ​ ​ 9. The method of claim 1, further comprising: calibrate panoramic data captured by the capturing device using the pose relationship; 10.The method of claim 9, further comprising: calibrating panoramic data captured by the capturing device using the pose relationship; and constructing a virtual environment of a physical environment using the calibrated point cloud data and the calibrated panoramic image data. 11.An apparatus for managing a capturing device, the capturing device being fixed to a rotating device, and the apparatus comprising: a first obtaining module configured to obtain initial point cloud data, the initial point cloud data being captured by the capturing device when the rotating device is at an initial rotation angle; a second obtaining module configured to obtain a plurality of point cloud data, the plurality of point cloud data being captured by the capturing device when the rotating device is at a plurality of rotation angles respectively; and a determining module configured to determine a pose relationship between the capturing device and the rotating device based on the initial point cloud data and the plurality of point cloud data, the determining module being further configured to: determine an initial position of the capturing device when the rotating device is at the initial rotation angle and a plurality of rotation positions of the capturing device when the rotating device is at the plurality of rotation angles respectively based on the initial point cloud data and the plurality of point cloud data; determine a circumscribed circle associated with the initial position and the plurality of rotation positions, each position corresponding to the initial position and the plurality of rotation positions being located on a circumference of the circumscribed circle respectively; determine an axis position of the rotation axis of the rotating device based on a center of the circumscribed circle; and determine the pose relationship between the capturing device and the rotating device based on the rotation axis and the initial position. 12.An electronic device comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit causing the electronic device to carry out the method according to any one of claims 1 to 10. 13.A computer-readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 10.

Images