Point cloud data processing method, device, three-dimensional scanning system and storage medium

Abstract

The present application relates to a point cloud data processing method, device, three-dimensional scanning system and storage medium, wherein the point cloud data processing method includes: obtaining first point cloud data and second point cloud data obtained by scanning a scanning device to obtain a measured object; wherein the tracking device is in a first position when obtaining the first point cloud data, and is in a second position when obtaining the second point cloud data; based on the posture change information generated by the tracking device rotating from the first position to the second position, the first point cloud data and the second point cloud data are aligned. Through this application, the problem that the initial alignment of point cloud data cannot be achieved by using an alignment method that does not rely on marker points is solved, and the initial alignment of point cloud data is achieved without the need for additional marker points.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning technology, and in particular to a point cloud data processing method, device, three-dimensional scanning system and storage medium. Background Art

[0002] The tracking 3D scanner uses intelligent optical tracking measurement technology to complete high-precision dynamic 3D measurement. It is suitable for complex structures and high-demand application scenarios, such as in aerospace, automobile manufacturing, rail transportation and other fields.

[0003] In the existing tracking 3D scanning technology, markers are usually placed in the scanning environment in advance. The tracking head is transferred according to the common markers during the scanning process, and the common markers are used to unify the coordinate system to achieve the initial registration of point cloud data at different positions and different viewing angles. However, the above method cannot achieve the initial registration of point cloud data using a registration method that does not rely on markers, and it is difficult to cope with measurement scenarios where markers are not allowed to be placed.

[0004] With regard to the problem in related technologies that it is impossible to achieve initial registration of point cloud data using a registration method that does not rely on marker points, no effective solution has been proposed so far. Summary of the invention

[0005] In this embodiment, a point cloud data processing method, device, three-dimensional scanning system and storage medium are provided to solve the problem in the related art that the initial registration of point cloud data cannot be achieved by using a registration method that does not rely on marker points.

[0006] In a first aspect, a point cloud data processing method is provided in this embodiment, which is applicable to a three-dimensional scanning system; the system includes a scanning device, a tracking device and a rotating platform; the rotating platform is connected to the tracking device to drive the tracking device to rotate; the method includes:

[0007] Acquire first point cloud data and second point cloud data obtained by scanning the object under test by the scanning device; wherein the tracking device is at a first position when acquiring the first point cloud data, and is at a second position when acquiring the second point cloud data;

[0008] The first point cloud data and the second point cloud data are registered based on posture change information generated when the tracking device rotates from the first position to the second position.

[0009] In some embodiments, before registering the first point cloud data and the second point cloud data based on the posture change information generated by the tracking device rotating from the first position to the second position, the method further includes:

[0010] determining a rotation angle of the rotating platform and a position transfer vector of the tracking device in response to the tracking device rotating from the first position to the second position;

[0011] Determine an angle transfer matrix corresponding to the rotation angle;

[0012] Determine the posture change information generated by the tracking device rotating from the first position to the second position; the posture change information includes the position transfer vector and the angle transfer vector.

[0013] In some embodiments, determining the rotation angle of the rotating platform in response to the tracking device rotating from the first position to the second position includes:

[0014] When the tracking device is in the first position, obtaining a first encoding value of a motor for controlling the rotation of the rotating platform;

[0015] When the tracking device is in the second position, obtaining a second encoding value of the motor;

[0016] Based on the first code value and the second code value, a rotation angle of the rotating platform is determined.

[0017] In some embodiments, the system further comprises a first camera and a preset marker; wherein the first camera remains stationary and the preset marker rotates with the rotating platform; or, the first camera rotates with the rotating platform and the preset marker remains stationary;

[0018] The determining the rotation angle of the rotating platform in response to the tracking device switching from the first position to the second position further comprises:

[0019] When the tracking device is at the first position, capturing a first image of the preset marker by the first camera;

[0020] When the tracking device is at the second position, acquiring a second image of the preset marker by using the first camera;

[0021] Detecting the first image and the second image to obtain a displacement change of the preset marker relative to the first camera;

[0022] Based on a preset calibration relationship and the displacement change, the rotation angle of the rotating platform is determined; the calibration relationship is a corresponding relationship between the rotation angle and the displacement change.

[0023] In some embodiments, determining the position transfer vector of the tracking device includes:

[0024] Determine a first feature point cloud in the first point cloud data and a second feature point cloud in the second point cloud data; the first feature point cloud and the second feature point cloud are on the same ray;

[0025] Acquire first distance information between the first feature point cloud and the tracking device, and second distance information between the second feature point cloud and the tracking device;

[0026] A position transfer vector of the tracking device is determined based on the first distance information and the second distance information.

[0027] In some of the embodiments, the system further comprises a second camera; the second camera is disposed on the tracking device and is used to capture an image including the measured object and the current working environment;

[0028] Before registering the first point cloud data and the second point cloud data based on the posture change information generated by the tracking device being converted from the first position to the second position, the method further includes:

[0029] When the tracking device is in the first position, acquiring a third image by the second camera;

[0030] When the tracking device is at the second position, acquiring a fourth image by the second camera;

[0031] Determine relative position information corresponding to the second camera based on the target feature points with the same name in the third image and the fourth image; the relative position information is the position change of the second camera caused by the position change of the tracking device;

[0032] Based on the relative posture information corresponding to the second camera, posture change information generated by the tracking device rotating from the first position to the second position is determined.

[0033] In some of the embodiments, determining the target feature points with the same name in the third image and the fourth image includes:

[0034] Extracting initial feature points of the same name in the third image and the fourth image;

[0035] Based on the initial feature points with the same name, determining initial relative pose information corresponding to the second camera;

[0036] Based on the target geometric constraints and the initial relative pose information, the initial feature points with the same name are iteratively updated until convergence, so as to obtain the target feature points with the same name in the third image and the fourth image.

[0037] In a second aspect, a point cloud data processing device is provided in this embodiment, which is applicable to a three-dimensional scanning system; the system includes a scanning device, a tracking device and a rotating platform; the rotating platform is connected to the tracking device, and is used to drive the tracking device to rotate; the device includes an acquisition module and a registration module;

[0038] The acquisition module is used to acquire first point cloud data and second point cloud data obtained by the scanning device scanning the measured object; wherein the tracking device is at a first position when acquiring the first point cloud data, and is at a second position when acquiring the second point cloud data;

[0039] The registration module is used to register the first point cloud data and the second point cloud data based on the posture change information generated when the tracking device rotates from the first position to the second position.

[0040] In the third aspect, a three-dimensional scanning system is provided in this embodiment, and the system includes a scanning device, a tracking device and a processing device; the processing device is respectively connected to the scanning device and the tracking device, and is used to execute the point cloud data processing method described in the first aspect above.

[0041] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the point cloud data processing method described in the first aspect is implemented.

[0042] Compared with the related art, the point cloud data processing method, device, three-dimensional scanning system and storage medium provided in this embodiment obtain the first point cloud data and the second point cloud data by acquiring the scanning device scanning the object to be measured; wherein, when acquiring the first point cloud data, the tracking device is in the first position, and when acquiring the second point cloud data, the tracking device is in the second position; based on the posture change information generated by the rotation of the tracking device from the first position to the second position, the first point cloud data and the second point cloud data are aligned, which solves the problem that it is impossible to achieve the initial alignment of point cloud data using an alignment method that does not rely on marker points, and achieves the initial alignment of point cloud data without the need for additional marker points.

[0043] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 It is a hardware structure block diagram of a terminal device of a point cloud data processing method provided in an embodiment of the present application;

[0046] Figure 2 is a flow chart of a point cloud data processing method provided by an embodiment of the present application;

[0047] Figure 3 is a flow chart of a method for calculating a target transformation matrix provided by an embodiment of the present application;

[0048] Figure 4 is a flow chart of a point cloud data processing method provided by a preferred embodiment of the present application;

[0049] Figure 5 is a structural block diagram of a point cloud data processing device provided by an embodiment of the present application;

[0050] Figure 6 It is a structural block diagram of a point cloud data processing system provided in one embodiment of the present application.

[0051] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, acquisition module; 20, registration module; 100, scanning device; 200, tracking device; 300, processing device. DETAILED DESCRIPTION

[0052] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0053] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0054] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of the point cloud data processing method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0055] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the point cloud data processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0056] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0057] In this embodiment, a point cloud data processing method is provided, which is applicable to a three-dimensional scanning system. The system includes a scanning device, a tracking device and a rotating platform. The rotating platform is connected to the tracking device to drive the tracking device to rotate. Figure 2 is a flow chart of the point cloud data processing method of this embodiment. Figure 2 As shown, the process includes the following steps:

[0058] Step S210, obtaining first point cloud data and second point cloud data obtained by scanning the object under test by a scanning device; wherein the tracking device is at a first position when obtaining the first point cloud data, and is at a second position when obtaining the second point cloud data;

[0059] Step S220: aligning the first point cloud data and the second point cloud data based on the posture change information generated when the tracking device rotates from the first position to the second position.

[0060] Specifically, when the scanning area of ​​the object to be measured is large or the structure is fine and complex, the surface to be scanned in the object to be measured can be divided into multiple scanning areas, and the tracking scanning technology is used to scan each scanning area of ​​the object to be measured to obtain point cloud data corresponding to different scanning areas. For example, when the object to be measured is a complex structure such as an organism or a sculpture, or a large structure such as a bridge or an airplane, the surface of the object to be measured can be divided into n areas for scanning. It can be understood that the tracking scanning technology is usually combined with the transfer station technology of the tracking device, and the different transfer station positions of the tracking device correspond one-to-one to the scanning area of ​​the object to be measured, so as to obtain multi-perspective point cloud data through the transfer station to ensure the generation of a complete three-dimensional model.

[0061] Accordingly, the three-dimensional scanning system applicable to this embodiment includes a scanning device, a tracking device and a rotating platform, wherein the scanning device is used to scan the surface data of an object, the tracking device is used to locate the position of the scanning device, and the rotating platform is connected to the tracking device, for example, the rotating platform is fixed to the bottom of the tracking device or the corresponding tripod, and is used to drive the tracking device to rotate, so as to realize the transfer of the tracking device. The tracking device is placed at a first position, and the scanning area corresponding to the first position is scanned by the scanning device to obtain the first point cloud data, and then the rotating platform is transferred, and the tracking device is placed at a second position, and the scanning area corresponding to the second position is scanned to obtain the second point cloud data.

[0062] In response to the tracking device rotating from the first position to the second position, the position change information generated by the tracking device rotating from the first position to the second position is obtained. Among them, the angle change of the rotating platform before and after the transfer, that is, the rotation angle of the rotating platform, is obtained, the angle transfer matrix corresponding to the rotation angle is calculated, and the first feature point cloud in the first point cloud data and the second feature point cloud in the second point cloud data are determined. The first feature point cloud and the second feature point cloud are on the same ray. The first distance information between the first feature point cloud and the tracking device, and the second distance information between the second feature point cloud and the tracking device are obtained. Based on the first distance information and the second distance information, the position transfer vector of the tracking device is calculated. Based on this, the position change information generated by the tracking device rotating from the first position to the second position is determined, and the position change information includes the position transfer vector of the tracking device and the angle transfer matrix corresponding to the rotation angle.

[0063] Afterwards, the corresponding target transformation matrix is ​​calculated based on the position transfer vector and the angle transfer matrix, and the first point cloud data and the second point cloud data are aligned based on the target transformation matrix. It should be noted that the target transformation matrix describes the spatial transformation relationship between the point cloud data under different scanning perspectives, and is used to perform preliminary alignment of the point cloud data scanned before and after the station transfer. The preliminary alignment of the point cloud data based on the target transformation matrix can unify the coordinate system before and after the station transfer, and ensure the alignment of the point cloud data under different perspectives.

[0064] Among them, the rotation angle of the rotating platform before and after the transfer can be obtained in a variety of ways. When the tracking device is in the first position, the first code value of the motor used to control the rotation of the rotating platform is obtained, and when the tracking device transfers to the second position, the second code value of the motor is obtained, and the rotation angle of the rotating platform is calculated based on the first code value and the second code value; or, a first camera and a preset marker are provided in the three-dimensional scanning system, and the setting method can also be that the first camera remains stationary during the transfer process, and the preset marker rotates with the rotating platform, or the first camera rotates with the rotating platform during the transfer process, and the preset marker remains stationary, when the tracking device is in the first position, the first camera is used to collect a first image of the preset marker, and when the tracking device is in the second position, the first camera is used to collect a second image of the preset marker, and the first image and the second image are detected to obtain the displacement change of the preset marker relative to the first camera, and the rotation angle of the rotating platform is calculated based on the corresponding relationship between the rotation angle and the displacement change and the currently detected displacement change.

[0065] In the existing tracking 3D scanning technology, markers are usually placed in the scanning environment in advance. The tracking head is transferred according to the common markers during the scanning process, and the common markers are used to unify the coordinate system to achieve the initial registration of point cloud data at different positions and different viewing angles. However, the above method cannot achieve the initial registration of point cloud data using a registration method that does not rely on markers, and it is difficult to cope with measurement scenarios where markers are not allowed to be placed.

[0066] Compared with the prior art, the present application obtains the first point cloud data and the second point cloud data obtained by the scanning device scanning the measured object; wherein, when the first point cloud data is obtained, the tracking device is in the first position, and when the second point cloud data is obtained, the tracking device is in the second position; based on the posture change information generated by the tracking device rotating from the first position to the second position, the first point cloud data and the second point cloud data are aligned. Based on this, the posture change information generated by the tracking device rotating from the first position to the second position is obtained, and according to the posture change information of the tracking device, the spatial transformation relationship between the point cloud data under different scanning angles can be determined, so that the point cloud data scanned before and after the transfer can be initially aligned based on the posture change information, without relying on the mark points for alignment, which solves the problem that the initial alignment of the point cloud data cannot be achieved by using the alignment method that does not rely on the mark points, and realizes the initial alignment of the point cloud data without the need for additional mark points.

[0067] In some of the embodiments, Figure 3 As shown, before registering the first point cloud data with the second point cloud data based on the posture change information generated by the tracking device rotating from the first position to the second position, the following steps are also included:

[0068] Step S221, in response to the tracking device rotating from the first position to the second position, determining the rotation angle of the rotating platform and the position transfer vector of the tracking device;

[0069] Step S222, determining an angle transfer matrix corresponding to the rotation angle;

[0070] Step S223, determining the posture change information generated by the tracking device rotating from the first position to the second position; the posture change information includes a position transfer vector and an angle transfer vector.

[0071] Specifically, as the tracking device rotates from the first position to the second position, the angle change of the rotating platform before and after the rotation is obtained, that is, the rotation angle of the rotating platform. For example, when the rotating platform rotates only at an angle a on the xy plane and does not move on the z axis, the rotation angle of the rotating platform is expressed as (r x ,r y ,r z )=(0,0,a). It can be understood that, since the tracking device rotates along with the rotating platform, and the relative positions of the tracking device and the rotating platform are fixed, the rotation changes of the tracking device and the rotating platform are consistent.

[0072] Afterwards, the Rodriguez rotation formula, Euler angle and other calculation methods can be used to calculate the angle transfer matrix R corresponding to the rotation angle. The angle transfer matrix is ​​used to describe the rotation transformation from the coordinate system in the first position to the coordinate system in the second position.

[0073] Further, a first feature point cloud in the first point cloud data and a second feature point cloud in the second point cloud data are determined, the first feature point cloud and the second feature point cloud are on the same ray, and the depths of the two feature point clouds are inconsistent at different positions of the tracking device. This embodiment calculates the first distance information between the first feature point cloud and the tracking device according to the spatial position of the first feature point cloud and the tracking device, and at the same time calculates the second distance information between the second feature point cloud and the tracking device according to the spatial position of the second feature point cloud and the tracking device, so that the depth difference between the first feature point cloud and the second feature point cloud can be calculated based on the first distance information and the second distance information, and the depth difference is represented by a vector in the direction of the ray to obtain the position transfer vector T of the tracking device before and after the transfer, and the position transfer vector is used to describe the translation transformation of the tracking device from the first position to the second position.

[0074] It should be noted that, in this embodiment, the posture change information generated by the rotation of the tracking device from the first position to the second position includes the position transfer vector T and the angle transfer matrix R. The target transformation matrix RT corresponding to the posture change information is calculated to describe the spatial transformation relationship between the point cloud data under different scanning angles, which is used to perform initial alignment of the point cloud data scanned before and after the station transfer. The initial alignment of the point cloud data is performed based on the target transformation matrix, which can unify the coordinate systems before and after the station transfer to ensure the alignment of the point cloud data under different angles.

[0075] Through this embodiment, in response to the tracking device rotating from a first position to a second position, the rotation angle of the rotating platform and the position transfer vector of the tracking device are determined, and the angle transfer matrix corresponding to the rotation angle is determined. By utilizing the rotation angle of the rotating platform and the position transfer vector of the tracking device during the transfer process, the posture change information caused by the tracking device rotating from the first position to the second position can be accurately obtained.

[0076] In some embodiments, in response to the tracking device rotating from the first position to the second position, determining the rotation angle of the rotating platform in step S221 includes the following steps:

[0077] When the tracking device is in a first position, obtaining a first encoding value of a motor for controlling the rotation of the rotating platform;

[0078] When the tracking device is in a second position, obtaining a second encoding value of the motor;

[0079] Based on the first code value and the second code value, a rotation angle of the rotating platform is determined.

[0080] Specifically, when the tracking device is in the first position, the first code value of the motor used to control the rotation of the rotating platform is obtained, and when the tracking device is transferred to the second position, the second code value of the motor is obtained, and the code difference between the first code value and the second code value is calculated. After that, the rotation angle corresponding to each code value of the current motor is obtained, and the code difference is multiplied by the rotation angle corresponding to each code value to obtain the rotation angle of the rotating platform.

[0081] It should be noted that, in addition to calculating the rotation angle based on the motor encoding value, devices or instruments such as angle sensors can be installed in the rotating platform to facilitate obtaining the rotation angle of the rotating platform before and after the transfer station.

[0082] Through this embodiment, when the tracking device is in a first position, a first coding value of a motor used to control the rotation of a rotating platform is obtained, and when the tracking device is in a second position, a second coding value of the motor is obtained. Based on the first coding value and the second coding value, the rotation angle of the rotating platform is determined, thereby achieving accurate calculation of the rotation angle.

[0083] In some embodiments, the three-dimensional scanning system further comprises a first camera and a preset marker; wherein the first camera remains stationary and the preset marker rotates with the rotating platform; or, the first camera rotates with the rotating platform and the preset marker remains stationary;

[0084] In response to the tracking device switching from the first position to the second position, determining the rotation angle of the rotating platform, further comprising the steps of:

[0085] When the tracking device is in a first position, capturing a first image of a preset marker through a first camera;

[0086] When the tracking device is at the second position, acquiring a second image of the preset marker through the first camera;

[0087] Detecting the first image and the second image to obtain a displacement change of a preset marker relative to the first camera;

[0088] Based on the preset calibration relationship and displacement change, the rotation angle of the rotating platform is determined; the calibration relationship is the corresponding relationship between the rotation angle and the displacement change.

[0089] In this embodiment, the three-dimensional scanning system is provided with a first camera and a preset marker, and the preset marker can be any graphic or logo with a marking function, for example, a precision pattern with a high-precision and complex structure graphic or logo, etc. The setting method of the first camera and the preset marker can be to fix the first camera so that the first camera remains stationary during the transfer of the tracking device, and to set the preset marker on the rotating platform so that the preset marker will rotate with the rotating platform; or, to place the first camera on the rotating platform so that the first camera will rotate with the rotating platform, and to fix the preset marker so that the preset marker remains stationary during the transfer of the tracking device, for example, to set the preset marker on a fixed tripod. This embodiment does not limit the specific positions of the first camera and the preset marker, and the setting method of the first camera and the preset marker must satisfy the above-mentioned relative motion relationship and ensure that the first camera can capture the image of the preset marker.

[0090] It should be noted that, in the case where the preset markers are fixed, the preset markers in this embodiment are different from the common marking points used for transfer in the prior art. The common marking points in the prior art need to be distributed in each scanning area and are relatively large in number. In this embodiment, usually only one preset marker needs to be placed, and the common marking points in the prior art are used for transfer to ensure that point cloud data from different perspectives can be integrated into a unified coordinate system, resulting in a relatively strict placement of the common marking points to ensure that point cloud data from different perspectives can be integrated into a unified coordinate system. In this embodiment, the placement of the preset markers is relatively free, and only needs to satisfy the relative motion relationship with the first camera, and can be flexibly selected according to actual scanning requirements.

[0091] Specifically, when the tracking device is in a first position, a first image of a preset marker is captured by the first camera. When the tracking device is in a second position, a second image of the preset marker is captured by the first camera. The first image and the second image are detected to obtain the position difference of the preset marker in different images. The displacement change of the preset marker relative to the first camera before and after the transfer is determined based on the position difference.

[0092] Furthermore, a preset calibration relationship is obtained, where the calibration relationship refers to a corresponding relationship between a rotation angle of the rotating platform and a displacement change, and a rotation angle corresponding to a current displacement change is determined based on the calibration relationship.

[0093] Through this embodiment, when the tracking device is in the first position, the first camera captures a first image of the preset marker, and when the tracking device is in the second position, the first camera captures a second image of the preset marker. The first image and the second image are detected to obtain the displacement change of the preset marker relative to the first camera, and then based on the preset calibration relationship and the displacement change, the rotation angle of the rotating platform is determined to achieve accurate acquisition of the rotation angle.

[0094] In some of the embodiments, determining a position transfer vector of a tracking device comprises the following steps:

[0095] Determine a first characteristic point cloud in the first point cloud data and a second characteristic point cloud in the second point cloud data; the first characteristic point cloud and the second characteristic point cloud are on the same ray;

[0096] Acquire first distance information between the first feature point cloud and the tracking device, and second distance information between the second feature point cloud and the tracking device;

[0097] Based on the first distance information and the second distance information, a position transfer vector of the tracking device is determined.

[0098] Specifically, determine the feature point clouds on the same ray in the point cloud data before and after the transfer, obtain the first feature point cloud in the first point cloud data and the second feature point cloud in the second point cloud data, calculate the first distance information between the first feature point cloud and the tracking device according to the spatial position of the first feature point cloud and the tracking device, and at the same time calculate the second distance information between the second feature point cloud and the tracking device according to the spatial position of the second feature point cloud and the tracking device, calculate the depth difference between the first feature point cloud and the second feature point cloud based on the first distance information and the second distance information, express the depth difference as a vector in the direction of the ray, and obtain the position transfer vector of the tracking device before and after the transfer, and the position transfer vector is used to describe the translation transformation of the tracking device from the first position to the second position.

[0099] It should be noted that, at different positions of the tracking device, the depths of the first feature point cloud and the second feature point cloud on the same ray are inconsistent. Based on this, the depth difference of the point cloud data before and after the station transfer is represented by a vector, and the position transfer vector of the tracking device before and after the station transfer can be obtained.

[0100] Through this embodiment, a first feature point cloud in the first point cloud data and a second feature point cloud in the second point cloud data are determined, the first feature point cloud and the second feature point cloud are on the same ray, and first distance information between the first feature point cloud and the tracking device, as well as second distance information between the second feature point cloud and the tracking device are obtained, and a position transfer vector of the tracking device is determined based on the first distance information and the second distance information, so that the position transfer vector is accurately calculated according to the distance from the point cloud to the tracking device.

[0101] In some of the embodiments, the three-dimensional scanning system further comprises a second camera; the second camera is disposed on the tracking device and is used to collect images including the object to be measured and the current working environment;

[0102] Before initially registering the first point cloud data with the second point cloud data based on the posture change information generated when the tracking device is converted from the first position to the second position, the method further includes the following steps:

[0103] When the tracking device is in the first position, acquiring a third image by the second camera;

[0104] When the tracking device is in the second position, acquiring a fourth image by the second camera;

[0105] Determine relative position information corresponding to the second camera based on the target feature points with the same name in the third image and the fourth image; the relative position information is the position change of the second camera caused by the position change of the tracking device;

[0106] Based on the relative posture information corresponding to the second camera, posture change information generated by the tracking device rotating from the first position to the second position is determined.

[0107] In this embodiment, the three-dimensional scanning system further includes a second camera, which is fixed on the tracking device and is used to collect images including the object to be measured and the current working environment. When the tracking device is in the first position, the third image is collected by the second camera, and when the tracking device is transferred to the second position, the fourth image is collected by the second camera, and the second camera posture is restored from the images of different perspectives using a multi-view geometry algorithm such as Structure From Motion (SFM) to obtain the posture change of the second camera caused by the position change of the tracking device.

[0108] Specifically, feature detection algorithms such as scale-invariant feature transform are used to detect and process the third image and the fourth image respectively to obtain feature points in the third image and feature points in the fourth image, match the feature points in the third image and the feature points in the fourth image, and determine target feature points with the same name in the third image and the fourth image based on the matching results. The feature points with the same name refer to feature points corresponding to the same object in images from different perspectives.

[0109] Further, based on the target feature points with the same name in the third image and the fourth image, the corresponding basic matrix is ​​estimated, and the basic matrix is ​​used to extract the posture change caused by the second camera following the position change of the tracking device, that is, the relative posture information corresponding to the second camera, and the relative posture information includes the angle transfer matrix R and the position transfer vector T. It can be understood that since the second camera is fixed on the tracking device, the tracking device rotates synchronously with the second camera, and the rotation changes of the tracking device and the rotating platform are consistent.

[0110] Afterwards, the corresponding target transformation matrix RT can be calculated according to the relative posture information. The target transformation matrix is ​​used to perform preliminary registration on the first point cloud data and the second point cloud data scanned before and after the station transfer to unify the coordinate system before and after the station transfer.

[0111] Through this embodiment, when the tracking device is in the first position, the third image is captured by the second camera, and when the tracking device is in the second position, the fourth image is captured by the second camera. Based on the target feature points with the same name in the third image and the fourth image, the relative posture information corresponding to the second camera is determined, and then based on the relative posture information corresponding to the second camera, the posture change information caused by the rotation of the tracking device from the first position to the second position is determined, thereby using the image captured by the second camera to accurately calculate the posture change caused by the second camera following the position change of the tracking device.

[0112] In some of the embodiments, determining the target feature points with the same name in the third image and the fourth image comprises the following steps:

[0113] extracting initial feature points of the same name in the third image and the fourth image;

[0114] Based on the initial feature points with the same name, determining the initial relative pose information corresponding to the second camera;

[0115] Based on the target geometric constraints and the initial relative pose information, the initial feature points with the same name are iteratively updated until convergence, and the target feature points with the same name in the third image and the fourth image are obtained.

[0116] Specifically, feature detection is performed on the third image and the fourth image respectively to obtain feature points in the third image and feature points in the fourth image, the feature points in the third image and the feature points in the fourth image are matched, and initial feature points with the same name in the third image and the fourth image are determined according to the matching results. The feature points with the same name refer to feature points corresponding to the same object in images from different perspectives.

[0117] Furthermore, based on the preliminary feature points of the same name in the third image and the fourth image, the corresponding basic matrix is ​​estimated, and the pose change caused by the position change of the second camera following the tracking device is extracted using the basic matrix to obtain the initial relative pose information corresponding to the second camera, and based on the target geometric constraints and the initial relative pose information, the initial feature points of the same name are iteratively updated until convergence, and the target feature points of the same name in the third image and the fourth image are obtained. Among them, the target geometric constraints can be known scene structures, etc., and the consistency of the camera pose and the feature point position can be ensured by introducing geometric constraints.

[0118] Through this embodiment, initial feature points with the same name are extracted from the third image and the fourth image, and based on the initial feature points with the same name, initial relative pose information corresponding to the second camera is determined. Based on the target geometric constraints and the initial relative pose information, the initial feature points with the same name are iteratively updated until convergence, and target feature points with the same name in the third image and the fourth image are obtained. In this way, an iterative matching method is used to improve the accuracy of finding feature points with the same name, which helps to accurately calculate the relative pose information of the camera.

[0119] The present embodiment is described and illustrated below through preferred embodiments.

[0120] Figure 4 is a flow chart of the point cloud data processing method of this preferred embodiment. Figure 4 As shown, the point cloud data processing method includes the following steps:

[0121] Step S410, obtaining first point cloud data and second point cloud data obtained by scanning the object under test by a scanning device; wherein the tracking device is at a first position when obtaining the first point cloud data, and is at a second position when obtaining the second point cloud data;

[0122] Step S420, when the tracking device is in the first position, obtaining a first code value of the motor used to control the rotation of the rotating platform; when the tracking device is in the second position, obtaining a second code value of the motor; and determining the rotation angle of the rotating platform based on the first code value and the second code value;

[0123] Step S430, determining an angle transfer matrix corresponding to the rotation angle, and determining a corresponding target transformation matrix based on the position transfer vector and the angle transfer matrix;

[0124] Step S440: aligning the first point cloud data and the second point cloud data based on the target transformation matrix.

[0125] Through this embodiment, the first point cloud data and the second point cloud data obtained by scanning the object under test by the scanning device are obtained; wherein, when the first point cloud data is obtained, the tracking device is in the first position, and when the second point cloud data is obtained, the tracking device is in the second position. Afterwards, when the tracking device is in the first position, the first coding value of the motor used to control the rotation of the rotating platform is obtained, and when the tracking device is in the second position, the second coding value of the motor is obtained, and the rotation angle of the rotating platform is determined based on the first coding value and the second coding value.

[0126] Furthermore, the angle transfer matrix corresponding to the rotation angle is determined, and the corresponding target transformation matrix is ​​determined based on the position transfer vector and the angle transfer matrix, so that the first point cloud data and the second point cloud data can be registered using the target transformation matrix, which solves the problem that the initial registration of point cloud data cannot be achieved using a registration method that does not rely on marker points, and achieves initial registration of point cloud data without the need for additional marker points.

[0127] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0128] In this embodiment, a point cloud data processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0129] Figure 5 is a structural block diagram of the point cloud data processing device of this embodiment, such as Figure 5 As shown, the device comprises: an acquisition module 10 and a registration module 20;

[0130] An acquisition module 10 is used to acquire first point cloud data and second point cloud data obtained by scanning the object under test by a scanning device; wherein the tracking device is at a first position when acquiring the first point cloud data, and is at a second position when acquiring the second point cloud data;

[0131] The registration module 20 is used to register the first point cloud data and the second point cloud data based on the posture change information generated when the tracking device rotates from the first position to the second position.

[0132] Through the device provided in this embodiment, first point cloud data and second point cloud data obtained by scanning the object under test by a scanning device are obtained; wherein, when the first point cloud data is obtained, the tracking device is in a first position, and when the second point cloud data is obtained, the tracking device is in a second position; based on the posture change information generated by the rotation of the tracking device from the first position to the second position, the first point cloud data and the second point cloud data are aligned, which solves the problem that it is impossible to achieve initial alignment of point cloud data using an alignment method that does not rely on marker points, and achieves initial alignment of point cloud data without the need for additional marker points.

[0133] In some of the embodiments, the registration module 20 is further used to determine the rotation angle of the rotating platform and the position transfer vector of the tracking device in response to the tracking device rotating from a first position to a second position; determine the angle transfer matrix corresponding to the rotation angle; determine the posture change information generated by the tracking device rotating from the first position to the second position; the posture change information includes the position transfer vector and the angle transfer matrix.

[0134] In some of the embodiments, the alignment module 20 is further used to obtain a first coding value of a motor used to control the rotation of the rotating platform when the tracking device is in a first position; to obtain a second coding value of the motor when the tracking device is in a second position; and to determine the rotation angle of the rotating platform based on the first coding value and the second coding value.

[0135] In some of the embodiments, the registration module 20 is further used to capture a first image of a preset marker through the first camera when the tracking device is in the first position; to capture a second image of the preset marker through the first camera when the tracking device is in the second position; to detect the first image and the second image to obtain a displacement change of the preset marker relative to the first camera; and to determine a rotation angle of the rotating platform based on a preset calibration relationship and the displacement change; the calibration relationship is a corresponding relationship between the rotation angle and the displacement change.

[0136] In some of the embodiments, the registration module 20 is further used to determine a first feature point cloud in the first point cloud data and a second feature point cloud in the second point cloud data; the first feature point cloud and the second feature point cloud are on the same ray; obtain first distance information between the first feature point cloud and the tracking device, and second distance information between the second feature point cloud and the tracking device; based on the first distance information and the second distance information, determine the position transfer vector of the tracking device.

[0137] In some of the embodiments, the registration module 20 is further used to capture a third image through the second camera when the tracking device is in the first position; to capture a fourth image through the second camera when the tracking device is in the second position; to determine relative posture information corresponding to the second camera based on the target feature points with the same name in the third image and the fourth image; the relative posture information is the posture change caused by the second camera following the position change of the tracking device; based on the relative posture information corresponding to the second camera, determine the posture change information caused by the tracking device rotating from the first position to the second position.

[0138] In some of the embodiments, the registration module 20 is further used to extract initial feature points of the same name in the third image and the fourth image; based on the initial feature points of the same name, determine the initial relative pose information corresponding to the second camera; based on the target geometric constraints and the initial relative pose information, iteratively update the initial feature points of the same name until convergence, to obtain the target feature points of the same name in the third image and the fourth image.

[0139] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0140] In this embodiment, a three-dimensional scanning system is also provided. Figure 6 is a structural block diagram of the point cloud data processing device of this embodiment, such as Figure 6 As shown, the system includes a scanning device 100, a tracking device 200 and a processing device 300; the processing device 300 is connected to the scanning device 100 and the tracking device 200, respectively, and is used to execute the steps in any of the above method embodiments.

[0141] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0142] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0143] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0144] S1, obtaining first point cloud data and second point cloud data obtained by scanning the object under test by a scanning device; wherein the tracking device is at a first position when obtaining the first point cloud data, and is at a second position when obtaining the second point cloud data;

[0145] S2: Based on the posture change information generated when the tracking device rotates from the first position to the second position, the first point cloud data and the second point cloud data are registered.

[0146] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0147] In addition, in combination with the point cloud data processing method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the point cloud data processing method. The storage medium stores a computer program; when the computer program is executed by a processor, any point cloud data processing method in the above embodiments is implemented.

[0148] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0149] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0150] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0151] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A point cloud data processing method, characterized in that: Applicable to a three-dimensional scanning system; the system comprises a scanning device, a tracking device and a rotating platform; The rotating platform is connected to the tracking device and is used to drive the tracking device to rotate; the method includes: Acquire first point cloud data and second point cloud data obtained by scanning the object under test by the scanning device; wherein the tracking device is at a first position when acquiring the first point cloud data, and is at a second position when acquiring the second point cloud data; The first point cloud data and the second point cloud data are registered based on posture change information generated when the tracking device rotates from the first position to the second position.

2. The point cloud data processing method according to claim 1, characterized in that: Before registering the first point cloud data and the second point cloud data based on the posture change information generated by the tracking device rotating from the first position to the second position, the method further includes: determining a rotation angle of the rotating platform and a position transfer vector of the tracking device in response to the tracking device rotating from the first position to the second position; Determine an angle transfer matrix corresponding to the rotation angle; Determine the posture change information generated by the tracking device rotating from the first position to the second position; the posture change information includes the position transfer vector and the angle transfer vector.

3. The point cloud data processing method according to claim 2, characterized in that: The step of determining a rotation angle of the rotating platform in response to the tracking device rotating from the first position to the second position comprises: When the tracking device is in the first position, obtaining a first encoding value of a motor for controlling the rotation of the rotating platform; When the tracking device is in the second position, obtaining a second encoding value of the motor; Based on the first code value and the second code value, a rotation angle of the rotating platform is determined.

4. The point cloud data processing method according to claim 2, characterized in that: The system further comprises a first camera and a preset marker; wherein the first camera remains stationary and the preset marker rotates with the rotating platform; or, the first camera rotates with the rotating platform and the preset marker remains stationary; The step of determining the rotation angle of the rotating platform in response to the tracking device switching from the first position to the second position further comprises: When the tracking device is at the first position, capturing a first image of the preset marker by the first camera; When the tracking device is at the second position, acquiring a second image of the preset marker by using the first camera; Detecting the first image and the second image to obtain a displacement change of the preset marker relative to the first camera; Based on a preset calibration relationship and the displacement change, the rotation angle of the rotating platform is determined; the calibration relationship is a corresponding relationship between the rotation angle and the displacement change.

5. The point cloud data processing method according to any one of claims 2 to 4, characterized in that: The determining the position transfer vector of the tracking device comprises: Determine a first feature point cloud in the first point cloud data and a second feature point cloud in the second point cloud data; the first feature point cloud and the second feature point cloud are on the same ray; Acquire first distance information between the first feature point cloud and the tracking device, and second distance information between the second feature point cloud and the tracking device; A position transfer vector of the tracking device is determined based on the first distance information and the second distance information.

6. The point cloud data processing method according to claim 1, characterized in that: The system further comprises a second camera; the second camera is arranged on the tracking device and is used to collect images including the measured object and the current working environment; Before registering the first point cloud data and the second point cloud data based on the posture change information generated by the tracking device being converted from the first position to the second position, the method further includes: When the tracking device is in the first position, acquiring a third image by the second camera; When the tracking device is at the second position, acquiring a fourth image by the second camera; Determine relative position information corresponding to the second camera based on the target feature points with the same name in the third image and the fourth image; the relative position information is the position change of the second camera caused by the position change of the tracking device; Based on the relative posture information corresponding to the second camera, posture change information generated by the tracking device rotating from the first position to the second position is determined.

7. The point cloud data processing method according to claim 6, characterized in that: Determining target feature points with the same name in the third image and the fourth image includes: Extracting initial feature points of the same name in the third image and the fourth image; Based on the initial feature points with the same name, determining initial relative pose information corresponding to the second camera; Based on the target geometric constraints and the initial relative pose information, the initial feature points with the same name are iteratively updated until convergence, so as to obtain the target feature points with the same name in the third image and the fourth image.

8. A point cloud data processing device, characterized in that: Applicable to a three-dimensional scanning system; the system comprises a scanning device, a tracking device and a rotating platform; the rotating platform is connected to the tracking device to drive the tracking device to rotate; the device comprises an acquisition module and a registration module; The acquisition module is used to acquire first point cloud data and second point cloud data obtained by the scanning device scanning the measured object; wherein the tracking device is at a first position when acquiring the first point cloud data, and is at a second position when acquiring the second point cloud data; The registration module is used to register the first point cloud data and the second point cloud data based on the posture change information generated when the tracking device rotates from the first position to the second position.

9. A three-dimensional scanning system, characterized in that: The system includes a scanning device, a tracking device, and a processing device; The processing device is connected to the scanning device and the tracking device, respectively, and is used to execute the point cloud data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the point cloud data processing method according to any one of claims 1 to 7 are implemented.

Images