True value acquisition device and method based on three-dimensional reconstruction

Abstract

The invention discloses a truth value acquisition device and method based on three-dimensional reconstruction. The device comprises a target camera configured to collect data of a target scene; the positioning module is configured to collect first pose information corresponding to the target camera for the single frame of data of the target scene; and the cutting module is configured to perform data cutting on the three-dimensional reconstruction data of the whole scene including the target scene based on the data of the target scene and the first pose information to obtain a three-dimensional data true value of the target scene. The data acquisition efficiency can be improved, and the target camera can enter the narrow space to acquire the data of the narrow space.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a device and method for obtaining a true value based on three-dimensional reconstruction. Background Art

[0002] The current method for obtaining the true depth value is mainly to calibrate the external parameters between the target camera and the true value camera, collect the true three-dimensional data of the target scene through the true value camera, and then align the true three-dimensional data of the target scene to the coordinate system of the target camera according to the extrinsic parameters between the calibrated target camera and the true value camera.

[0003] However, this method of obtaining the true depth value has the following problems:

[0004] 1. Since both the target camera and the true value camera have a certain field of view, the acquisition of the true depth value of the target scene by the target camera usually requires multiple data collections by the target camera and the true value camera respectively, which results in low data collection efficiency.

[0005] 2. Since the target camera and the true value camera need to be combined together, the overall volume of the combination is larger than that of a single target camera, which makes it impossible to collect data in a narrow space. Summary of the invention

[0006] The three-dimensional reconstruction-based true value acquisition device and method provided in the embodiments of the present application can solve or partially solve the above-mentioned deficiencies in the prior art or other deficiencies in the prior art.

[0007] According to a first aspect of the present application, a true value acquisition device based on three-dimensional reconstruction is provided, comprising: a target camera, configured to collect data of a target scene; a positioning module, configured to collect first pose information corresponding to the target camera for a single frame of data of the target scene; and a cropping module, configured to perform data cropping on three-dimensional reconstructed data of a full scene including the target scene based on the data of the target scene and the first pose information, so as to obtain the three-dimensional data true value of the target scene.

[0008] In one embodiment of the present application, it also includes: a true value camera, configured to collect the three-dimensional data true value of the full scene; the positioning module, further configured to collect the second posture information corresponding to the true value camera for the three-dimensional data true value of the full scene in a single frame; and a stitching module, configured to perform data stitching on the three-dimensional data true value of the full scene based on the second posture information to obtain the three-dimensional reconstruction data of the full scene.

[0009] In one embodiment of the present application, the true value camera includes one of a laser radar, an active binocular structured light camera, a passive binocular camera, a monocular speckle structured light camera, an iTOF camera, a DTof camera, and a single-line structured light camera.

[0010] In one embodiment of the present application, the positioning module includes: a first infrared positioning code, which is affixed to the upper surface of the target camera and calibrates the first external parameter between the first infrared positioning code and the target camera; a second infrared positioning code, which is affixed to the upper surface of the true value camera and calibrates the second external parameter between the second infrared positioning code and the true value camera; and a plurality of infrared cameras, which are installed above the full scene and calibrate the third external parameter between every two infrared cameras among the plurality of infrared cameras, and the plurality of infrared cameras are configured to respectively collect data of the first infrared positioning code, and determine the three-dimensional posture information of the target camera based on the data of the first infrared positioning code, the first external parameter and the third external parameter, and respectively collect data of the second infrared positioning code, and determine the three-dimensional posture information of the true value camera based on the data of the second infrared positioning code, the second external parameter and the third external parameter.

[0011] In one embodiment of the present application, the positioning module also includes an external component; one end of the external component is connected to the target camera, and the first infrared positioning code is affixed to the other end of the external component; one end of the external component is connected to the true value camera, and the second infrared positioning code is affixed to the other end of the external component.

[0012] In one embodiment of the present application, the positioning module includes a single-line laser radar and a structural member; the single-line laser radar is connected to the target camera through the structural member, and the fourth external parameter between the single-line laser radar and the target camera is calibrated, and the single-line laser radar is configured to scan the surrounding environment to construct a first environment map, and determine the two-dimensional pose information of the target camera based on the first environment map and the fourth external parameter; the single-line laser radar is connected to the true value camera through the structural member, and the fifth external parameter between the single-line laser radar and the true value camera is calibrated, and the single-line laser radar is configured to scan the surrounding environment to construct a second environment map, and determine the two-dimensional pose information of the true value camera based on the second environment map and the fifth external parameter.

[0013] In one embodiment of the present application, the positioning module includes a multi-line laser radar and a structural member; the multi-line laser radar is connected to the target camera through the structural member, and the sixth external parameter between the multi-line laser radar and the target camera is calibrated, and the multi-line laser radar is configured to scan the surrounding environment to construct a third environment map, and determine the three-dimensional pose information of the target camera based on the third environment map and the sixth external parameter; the multi-line laser radar is connected to the true value camera through the structural member, and the seventh external parameter between the multi-line laser radar and the true value camera is calibrated, and the multi-line laser radar is configured to scan the surrounding environment to construct a fourth environment map, and determine the three-dimensional pose information of the true value camera based on the fourth environment map and the seventh external parameter.

[0014] In one embodiment of the present application, the positioning module and the target camera are configured to be time synchronized; the positioning module and the true value camera are configured to be time synchronized.

[0015] In one embodiment of the present application, the positioning module and the target camera are respectively configured to generate a timestamp based on a reference time generated by a computing unit, and perform time synchronization based on the timestamp alignment; the positioning module and the true value camera are respectively configured to generate a timestamp based on the reference time generated by the computing unit, and perform time synchronization based on the timestamp alignment.

[0016] In one embodiment of the present application, the positioning module includes the calculation unit.

[0017] In one embodiment of the present application, the target camera is configured to collect data of the target scene at a first acquisition frequency; the truth value camera is configured to collect the three-dimensional data truth of the full scene at a third acquisition frequency; the positioning module is configured to collect the first pose information of the target camera at a second acquisition frequency, and collect the second pose information of the truth value camera at a fourth acquisition frequency; wherein the second acquisition frequency is an integer multiple of the first acquisition frequency; and the fourth acquisition frequency is an integer multiple of the third acquisition frequency.

[0018] In one embodiment of the present application, the stitching module is configured to perform data stitching on the data that meets the target accuracy in the true value of the three-dimensional data of the full scene based on the second pose information to obtain the three-dimensional reconstructed data of the full scene.

[0019] In one embodiment of the present application, the stitching module is configured to determine a data interval in which the true value camera has the target accuracy, and perform data stitching on the data that meets the data interval in the true value of the three-dimensional data of the full scene based on the second pose information to obtain the three-dimensional reconstruction data of the full scene.

[0020] In one embodiment of the present application, the true value camera is further configured to set the data in the true value of the three-dimensional data of the full scene that does not conform to the data interval to zero based on the data interval with the target accuracy of the true value camera, so as to obtain data that conforms to the target accuracy.

[0021] In one embodiment of the present application, the truth camera is configured to collect data of a material with ordinary reflectivity attached to a surface with special reflectivity, as the three-dimensional data truth value of the surface with special reflectivity; the target camera is configured to collect data of the surface with special reflectivity after removing the material with ordinary reflectivity, as the data of the surface with special reflectivity.

[0022] In one embodiment of the present application, the stitching module is configured to correct the data of the surface with special reflectivity in the three-dimensional reconstruction data.

[0023] According to the second aspect of the present application, a method for obtaining a true value based on three-dimensional reconstruction is provided, which is applied to the device for obtaining a true value based on three-dimensional reconstruction described in the first aspect. The method comprises: acquiring data of a target scene through a target camera, and acquiring first position information corresponding to the target camera for a single frame of data of the target scene through a positioning module; and performing data cropping on the three-dimensional reconstructed data of the entire scene including the target scene based on the data of the target scene and the first position information through a cropping module to obtain the three-dimensional data true value of the target scene.

[0024] In one embodiment of the present application, it also includes: collecting the three-dimensional data truth value of the full scene through a true value camera, and collecting the second posture information corresponding to the truth value camera for a single frame of the three-dimensional data truth value of the full scene through the positioning module; and performing data stitching on the three-dimensional data truth value of the full scene based on the second posture information through a stitching module to obtain the three-dimensional reconstruction data of the full scene.

[0025] In one embodiment of the present application, the first pose information corresponding to the target camera is collected for the data of the target scene of a single frame through a positioning module, including: collecting data of the first infrared positioning code through a plurality of infrared cameras respectively; wherein the first infrared positioning code is attached to the upper surface of the target camera; the plurality of infrared cameras are installed above the entire scene; and based on the data of the first infrared positioning code, the calibrated first external parameter between the first infrared positioning code and the target camera, and the calibrated third external parameter between every two infrared cameras among the plurality of infrared cameras, the three-dimensional pose information of the target camera is determined; the second pose information corresponding to the true value camera is collected for the true value of the three-dimensional data of a single frame of the entire scene through the positioning module, including: collecting data of the second infrared positioning code through the plurality of infrared cameras respectively; wherein the second infrared positioning code is attached to the upper surface of the true value camera; and based on the data of the second infrared positioning code, the calibrated second external parameter between the second infrared positioning code and the true value camera, and the third external parameter, the three-dimensional pose information of the true value camera is determined.

[0026] In one embodiment of the present application, the first pose information corresponding to the target camera is collected for a single frame of the target scene data through a positioning module, including: scanning the surrounding environment through a single-line laser radar to construct a first environment map; wherein the single-line laser radar and the target camera are connected through a structural member; based on the first environment map, the calibrated fourth external parameter between the single-line laser radar and the target camera, the two-dimensional pose information of the target camera is determined; the second pose information corresponding to the true value camera is collected for the three-dimensional data truth value of a single frame of the full scene through the positioning module, including: scanning the surrounding environment through the single-line laser radar to construct a second environment map; wherein the single-line laser radar and the true value camera are connected through the structural member; based on the second environment map, the calibrated fifth external parameter between the single-line laser radar and the true value camera, the two-dimensional pose information of the true value camera is determined.

[0027] In one embodiment of the present application, the first pose information corresponding to the target camera is collected for a single frame of the target scene data through a positioning module, including: scanning the surrounding environment through a multi-line laser radar to construct a third environment map; wherein the multi-line laser radar and the target camera are connected through a structural member; based on the third environment map, the calibrated sixth external parameter between the multi-line laser radar and the target camera, the three-dimensional pose information of the target camera is determined; the second pose information corresponding to the true value camera is collected for the three-dimensional data truth value of a single frame of the full scene through the positioning module, including: scanning the surrounding environment through the multi-line laser radar to construct a fourth environment map; wherein the multi-line laser radar and the true value camera are connected through the structural member; based on the fourth environment map, the calibrated seventh external parameter between the multi-line laser radar and the true value camera, the three-dimensional pose information of the true value camera is determined.

[0028] In one embodiment of the present application, it also includes: generating a timestamp by the positioning module and the target camera based on the reference time generated by the calculation unit respectively; synchronizing the positioning module with the target camera based on the timestamp alignment; generating a timestamp by the positioning module and the true value camera based on the reference time generated by the calculation unit respectively; synchronizing the positioning module with the true value camera based on the timestamp alignment.

[0029] In one embodiment of the present application, it also includes: generating the reference time by the calculation unit in the positioning module.

[0030] In one embodiment of the present application, data of a target scene is collected by a target camera, and first position information corresponding to the target camera is collected for a single frame of data of the target scene by a positioning module, including: collecting data of the target scene by the target camera at a first collection frequency, and collecting first position information of the target camera by the positioning module at a second collection frequency; wherein the second collection frequency is an integer multiple of the first collection frequency; collecting a true value of three-dimensional data of the entire scene by a true value camera, and collecting second position information corresponding to the true value camera for a single frame of three-dimensional data true value of the entire scene by the positioning module, including: collecting the true value of three-dimensional data of the entire scene by the true value camera at a third collection frequency, and collecting the second position information of the true value camera by the positioning module at a fourth collection frequency; wherein the fourth collection frequency is an integer multiple of the third collection frequency.

[0031] In one embodiment of the present application, data stitching is performed on the three-dimensional data true value of the full scene based on the second posture information through a stitching module to obtain three-dimensional reconstruction data of the full scene, including: data stitching is performed on the three-dimensional data true value of the full scene that meets the target accuracy based on the second posture information through the stitching module to obtain the three-dimensional reconstruction data of the full scene.

[0032] In one embodiment of the present application, the stitching module performs data stitching on the data that meets the target accuracy in the true value of the three-dimensional data of the full scene based on the second posture information to obtain the three-dimensional reconstruction data of the full scene, including: determining that the data interval in which the true value camera has the target accuracy; performing data stitching on the data that meets the data interval in the true value of the three-dimensional data of the full scene based on the second posture information to obtain the three-dimensional reconstruction data of the full scene.

[0033] In one embodiment of the present application, it also includes: using the true value camera to set the data that does not conform to the data interval in the true value of the three-dimensional data of the full scene to zero based on the data interval with the target accuracy of the true value camera, so as to obtain data that conforms to the target accuracy.

[0034] In one embodiment of the present application, the three-dimensional data truth value of the entire scene is collected by a truth camera, including: collecting data of a material with ordinary reflectivity attached to a surface with special reflectivity, as the three-dimensional data truth value of the surface with special reflectivity; and collecting data of the target scene by a target camera, including: collecting data of the surface with special reflectivity after removing the material with ordinary reflectivity, as the data of the surface with special reflectivity.

[0035] In one embodiment of the present application, the method further includes: correcting the data of the surface with special reflectivity in the three-dimensional reconstruction data by using the stitching module.

[0036] According to the three-dimensional reconstruction-based truth value acquisition device and method of the implementation mode of the present application, since the three-dimensional data truth value of the target scene is cut and acquired from the three-dimensional reconstruction data of the whole scene including the target scene, only the target camera is needed to collect the data of the target scene, which can improve the efficiency of data collection. At the same time, since the target camera works alone and does not need to be combined with the truth value camera, the target camera can enter narrow spaces to collect data in narrow spaces.

[0037] The content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Other features, purposes and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments made with reference to the following drawings. The drawings are used to better understand the present solution and do not constitute a limitation of the present application. Among them:

[0039] Figure 1 is a schematic diagram of the composition structure of a true value acquisition device based on three-dimensional reconstruction according to an embodiment of the present application;

[0040] Figure 2 is a schematic diagram of the composition structure of a true value acquisition device based on three-dimensional reconstruction according to another embodiment of the present application;

[0041] Figure 3A and Figure 3B is a schematic diagram of the positional relationship among an infrared camera, a true value camera and a target camera according to an embodiment of the present application;

[0042] Figure 4 It is a flowchart of a method for obtaining a true value based on three-dimensional reconstruction according to an embodiment of the present application;

[0043] Figure 5 It is a flow chart of a method for obtaining a true value based on three-dimensional reconstruction according to another embodiment of the present application. DETAILED DESCRIPTION

[0044] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0045] In the drawings, the thickness, size and shape of the components have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not drawn strictly to scale. As used herein, the terms "substantially," "approximately," and similar terms are used as terms of approximation, not as terms of degree, and are intended to account for the inherent deviations in measurements or calculations that would be recognized by one of ordinary skill in the art.

[0046] It should also be understood that expressions such as "include", "including", "have", "contain" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.

[0047] Unless otherwise specified, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that, unless clearly stated in this application, words defined in common dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0048] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments in this application can be combined with each other. In addition, unless explicitly limited or contradictory to the context, the specific steps included in the method recorded in this application are not necessarily limited to the recorded order, but can be performed in any order or in parallel.

[0049] In addition, those skilled in the art will appreciate that the quantities shown in the drawings and below of the present application, such as the number of light films, are only shown for ease of illustration, and the specific quantities may be set according to actual needs without departing from the teachings of the disclosure of the present application.

[0050] The current method for obtaining the true depth value is mainly to calibrate the external parameters between the target camera and the true value camera, collect the true three-dimensional data of the target scene through the true value camera, and then align the true three-dimensional data of the target scene to the coordinate system of the target camera according to the extrinsic parameters between the calibrated target camera and the true value camera.

[0051] However, this method of obtaining the true depth value has the following problems:

[0052] 1. Since both the target camera and the true value camera have a certain field of view, the acquisition of the true depth value of the target scene by the target camera usually requires multiple data collections by the target camera and the true value camera respectively, which results in low data collection efficiency.

[0053] 2. Since the target camera and the true value camera need to be combined together, the overall volume of the combination is larger than that of a single target camera, which makes it impossible to collect data in a narrow space.

[0054] In order to solve the above problems, an embodiment of the present application provides a true value acquisition device 100 based on three-dimensional reconstruction.

[0055] Figure 1 FIG. 1 shows a schematic diagram of the structure of a true value acquisition device 100 based on three-dimensional reconstruction according to an embodiment of the present application. Figure 1 As shown, the three-dimensional reconstruction-based truth value acquisition device 100 of the embodiment of the present application may include: a target camera 110, a positioning module 120 and a cropping module 130. Among them, the target camera 110 is configured to collect data of the target scene. The positioning module 120 is configured to collect the first position information corresponding to the target camera 110 for the data of a single frame of the target scene. The cropping module 130 is configured to perform data cropping on the three-dimensional reconstruction data of the full scene including the target scene based on the data of the target scene and the first position information, and obtain the three-dimensional data truth value of the target scene.

[0056] The three-dimensional reconstruction-based truth value acquisition device 100 of the implementation mode of the present application collects data of the target scene through the target camera 110, collects the first position information corresponding to the target camera 110 when collecting single-frame data of the target scene through the positioning module 120, and performs data cropping on the three-dimensional reconstructed data of the whole scene including the target scene according to the target scene data collected by the target camera 110 and the first position information of the target camera 110 collected by the positioning module 120 through the cropping module 130, and aligns the cropped data to the target camera 110 to obtain the three-dimensional data truth value of the target scene.

[0057] The three-dimensional reconstruction-based truth value acquisition device 100 provided in the embodiment of the present application, since the three-dimensional data truth value of the target scene is obtained by cutting out the three-dimensional reconstruction data of the whole scene including the target scene, only the target camera 110 is needed to collect the data of the target scene, which can improve the efficiency of data collection. At the same time, since the target camera works alone and does not need to be combined with the truth value camera, the target camera can enter a narrow space to collect data in the narrow space.

[0058] It should be understood that the target camera 110 in the embodiments of the present application may be an independent imaging device with relatively low precision and low cost, such as a camera, a camcorder, etc., or may be a relatively low precision and low cost photographic module, a video module, etc. used in products such as automobiles and mobile phones. The embodiments of the present application do not limit the form of the target camera 110.

[0059] It should be noted that the implementation method of the present application does not limit the implementation method of the positioning module 120. For example, the positioning module 120 can be positioned through ultra-wide band (Ultra Wide Band, referred to as UWB), or it can also be positioned through an infrared camera, or it can also be positioned through a lidar.

[0060] It should be noted that the cutting module 130 in the implementation manner of the present application can adopt the existing data cutting method to perform data cutting, and can also adopt the cutting method designed according to actual requirements to perform data cutting. The implementation manner of the present application does not limit the way in which the cutting module 130 implements data cutting. For example, the cutting module 130 can perform data cutting through an artificial neural network (ANN for short).

[0061] It should be noted that the three-dimensional reconstruction data of the full scene in the implementation manner of the present application is obtained based on the true value of the three-dimensional data of the full scene. After obtaining the three-dimensional reconstruction data of the full scene, the three-dimensional reconstruction data of the full scene is stored. When a target camera 110 collects data of a target scene in the full scene, the true value of the three-dimensional data of the target scene is cut out from the pre-stored three-dimensional reconstruction data of the full scene.

[0062] Furthermore, if Figure 2 As shown, Figure 2 A schematic diagram of the composition structure of a three-dimensional reconstruction-based truth value acquisition device 100 according to another embodiment of the present application is shown. The three-dimensional reconstruction-based truth value acquisition device 100 of the embodiment of the present application may also include: a truth value camera 140 and a stitching module 150. Among them, the truth value camera 140 is configured to collect the three-dimensional data truth value of the entire scene. The positioning module 120 is also configured to collect the second posture information corresponding to the truth value camera 140 for the three-dimensional data truth value of a single frame of the entire scene. The stitching module 150 is configured to perform data stitching on the three-dimensional data truth value of the entire scene based on the second posture information to obtain the three-dimensional reconstruction data of the entire scene.

[0063] The three-dimensional reconstruction-based truth value acquisition device 100 of the implementation mode of the present application collects the three-dimensional data truth value of the entire scene through the truth value camera 140, collects the second posture information corresponding to the truth value camera 140 when collecting the single-frame three-dimensional data truth value of the entire scene through the positioning module 120, and performs data splicing on the three-dimensional data truth value of the entire scene collected by the truth value camera 140 according to the second posture information of the truth value camera 140 collected by the positioning module 120 through the stitching module 150, thereby completing the three-dimensional reconstruction of the entire scene and obtaining the three-dimensional reconstruction data of the entire scene.

[0064] It should be understood that the true value camera 140 in the embodiments of the present application can be an independent imaging device with higher precision and higher cost, such as a lidar, an active binocular structured light camera, a passive binocular camera, a monocular speckle structured light camera, an iTOF camera, a DTof camera, a single-line structured light camera, etc. The embodiments of the present application do not limit the form of the true value camera 140.

[0065] It should be noted that the splicing module 150 in the implementation manner of the present application can use existing data splicing methods to perform data splicing, or can use a splicing method designed according to actual requirements to perform data splicing. The implementation manner of the present application does not limit the way in which the splicing module 150 implements data splicing. For example, the splicing module 150 can perform data splicing through an artificial neural network (ANN for short).

[0066] Optionally, in order to ensure that the first pose information collected by the positioning module 120 is the instantaneous pose of the target camera 110, and the second pose information collected by the positioning module 120 is the instantaneous pose of the true value camera 140, it is necessary to synchronize the time of the positioning module 120 with the target camera 110, and the time of the positioning module 120 with the true value camera 140. Therefore, the positioning module 120 and the target camera 110 are configured to be time synchronized, and the positioning module 120 and the true value camera 140 are configured to be time synchronized. Considering the independence between the positioning module 120, the target camera 110, and the true value camera 140, time synchronization can be achieved through soft synchronization.

[0067] In an optional example, when the positioning module 120 collects the first pose information, the positioning module 120 and the target camera 110 can be respectively configured to generate a timestamp based on the reference time generated by the calculation unit, and perform time synchronization based on the timestamp alignment. When the positioning module 120 collects the second pose information, the positioning module 120 and the true value camera 140 can be respectively configured to generate a timestamp based on the reference time generated by the calculation unit, and perform time synchronization based on the timestamp alignment. Among them, the calculation unit that generates the reference time when collecting the first pose information and collecting the second pose information is the same calculation unit, which can be an external calculation unit independent of the device 100, or a built-in calculation unit included in the positioning module 120, and the embodiments of the present application do not limit this. The embodiments of the present application do not limit the implementation method of the calculation unit, the form of the generated reference time, and the generation method of the timestamp.

[0068] Optionally, the acquisition frequency of the positioning module 120 can be set to an integer multiple of the acquisition frequency of the target camera 110. Similarly, the acquisition frequency of the positioning module 120 can be set to an integer multiple of the acquisition frequency of the true value camera 140. When the multiple is higher, the result of the final spliced ​​data is more accurate. For example, the target camera 110 can be configured to acquire data of the target scene at a first acquisition frequency, the positioning module 120 can be configured to acquire the first position information of the target camera 110 at a second acquisition frequency, the true value camera 140 can be configured to acquire the true value of the three-dimensional data of the entire scene at a third acquisition frequency, and the positioning module 120 can be configured to acquire the second position information of the true value camera at a fourth acquisition frequency, wherein the second acquisition frequency is an integer multiple of the first acquisition frequency, and the fourth acquisition frequency is an integer multiple of the third acquisition frequency. The embodiments of the present application do not limit the specific values ​​of the first acquisition frequency, the second acquisition frequency, the third acquisition frequency, and the fourth acquisition frequency.

[0069] Optionally, since there is a field of view intersection part in the two frames of data, the field of view intersection part may be in different fields of view ranges of the true value camera 140 in the two frames of data, so the field of view intersection part may have different precision problems caused by different field of view ranges, that is, the precision of the true value of the same physical space is inconsistent, so that there is an error in the field of view intersection part. In the existing method for obtaining the true value of depth, since the true value camera only collects the data of the target scene, if the data collected by the true value camera has precision problems, it is difficult to correct it with the help of data of other scenes outside the target scene. In order to solve this problem, the implementation method of the present application can use the data of the data interval with higher data precision of the true value camera 140 to reconstruct the three-dimensional scene to improve the quality of the data and ensure that the data precision will not be affected by factors such as the field of view boundary and distance. That is, the splicing module 150 can be configured to perform data splicing on the data that meets the target precision in the true value of the three-dimensional data of the full scene based on the second posture information to obtain the three-dimensional reconstruction data of the full scene. The implementation method of the present application does not limit the implementation method of the data of the data interval with higher data precision of the true value camera 140.

[0070] In an optional example, during the data acquisition process, after the true value camera 140 acquires the true value of the three-dimensional data of the entire scene, the true value camera 140 may also be configured to set the data that does not conform to the data interval in the true value of the three-dimensional data of the entire scene to zero based on the data interval with target accuracy of the true value camera 140, so as to obtain data that conforms to the target accuracy. In another optional example, during the data splicing process, the splicing module 140 may be configured to determine the data interval with target accuracy of the true value camera, and perform data splicing on the data that conforms to the data interval in the true value of the three-dimensional data of the entire scene based on the second posture information, so as to obtain the three-dimensional reconstruction data of the entire scene.

[0071] Optionally, when there is a surface with a special reflectivity in the scene, such as a mirror with a high reflectivity, the data collected by the true value camera 140 may be erroneous. In the existing method for obtaining the true value of depth, since the true value camera 140 only collects data of the target scene, if the data collected by the true value camera is erroneous, it is difficult to correct it with the help of data from other scenes other than the target scene. The embodiments of the present application can obtain the correct data true value of the surface with a special reflectivity.

[0072] In an optional example, during the process of data acquisition by the true value camera 140, a very thin material with ordinary reflectivity, such as paper with ordinary reflectivity, may be attached to the surface with special reflectivity, so that the true value camera 140 can obtain correct data and ensure the quality of the data truth value. In this case, the true value camera 140 can be configured to acquire data of the material with ordinary reflectivity attached to the surface with special reflectivity as the three-dimensional data truth value of the surface with special reflectivity. During the process of data acquisition by the target camera 110, the material with ordinary reflectivity attached to the surface with special reflectivity can be removed, so that the target camera 110 can acquire the real data under the reflectivity. In this case, the target camera 110 can be configured to acquire data of the surface with special reflectivity after the material with ordinary reflectivity is removed as the data of the surface with special reflectivity. In another optional example, after the data truth value is collected, the data of the surface with special reflectivity can be corrected in combination with the data of other scenes so that the data truth value of the area is correct, and only one correction is required, and the data correction can be performed in an automated manner or manually. When the data correction is performed in an automated manner, the splicing module 150 can be configured to correct the data of the surface with special reflectivity in the three-dimensional reconstructed data. The embodiments of the present application do not limit the implementation method of obtaining the correct data truth value of the surface with special reflectivity.

[0073] The three-dimensional reconstruction-based true value acquisition device 100 provided in the implementation of the present application will be described below in conjunction with specific examples.

[0074] In some optional embodiments of the present application, the true value camera 140 is a multi-line laser radar. The positioning module 120 may include: a plurality of infrared cameras, a first infrared positioning code and a second infrared positioning code. Figure 3A and Figure 3BAs shown, the positioning module 120 includes three infrared cameras 121, 122 and 123, which are installed above the entire scene in a triangular arrangement, and accurately calibrate the third external parameter between every two infrared cameras in the three infrared cameras 121, 122 and 123. The first infrared positioning code is attached to the upper surface of the target camera 110, and the first external parameter between the first infrared positioning code and the target camera 110 is accurately calibrated. The second infrared positioning code is attached to the upper surface of the true value camera 140, and the second external parameter between the second infrared positioning code and the true value camera 140 is accurately calibrated. The three infrared cameras 121, 122 and 123 are configured to respectively collect data of the first infrared positioning code, and determine the three-dimensional posture information of the target camera 110 based on the data of the first infrared positioning code, the first external parameter and the third external parameter, and respectively collect data of the second infrared positioning code, and determine the three-dimensional posture information of the true value camera 140 based on the data of the second infrared positioning code, the second external parameter and the third external parameter.

[0075] When performing three-dimensional reconstruction of the entire scene, the multi-line laser radar performs a full-scene scan to obtain point cloud information of the entire scene. The three infrared cameras 121, 122 and 123 obtain three-dimensional pose information corresponding to the single-frame point cloud information when the multi-line laser radar obtains the point cloud information of the entire scene. The stitching module 150 performs data stitching on the point cloud information of the entire scene obtained by the multi-line laser radar according to the three-dimensional pose information obtained by the three infrared cameras 121, 122 and 123 to complete the three-dimensional reconstruction of the entire scene and obtain the three-dimensional reconstruction data of the entire scene.

[0076] When obtaining the true value of the three-dimensional data of the target scene, the target camera 110 shoots the target scene to obtain the data of the target scene. When the target camera 110 obtains the data of the target scene, the three infrared cameras 121, 122 and 123 obtain the three-dimensional pose information corresponding to the data of a single frame of the target scene. The cropping module 130 crops the true value of the three-dimensional data of the target scene from the three-dimensional reconstructed data of the entire scene based on the data of the target scene obtained by the target camera 110 and the three-dimensional pose information obtained by the three infrared cameras 121, 122 and 123.

[0077] Optionally, if a narrow scene is encountered, in order to ensure that the three infrared cameras 121, 122 and 123 can collect infrared positioning code data, the positioning module 120 can also include an external component, such as a long pole. When collecting data of the target scene, one end of the external component can be connected to the target camera 110, and the first infrared positioning code can be attached to the other end of the external component. When collecting the true value of the data, one end of the external component can be connected to the true value camera 140, and the second infrared positioning code can be attached to the other end of the external component, wherein the connection between the target camera 110 and the true value camera 140 and the external component is a rigid connection. By using external components, it can be ensured that the three infrared cameras 121, 122 and 123 can still collect infrared positioning code data when they are blocked above the target camera 110 and the true value camera 140.

[0078] In some other optional embodiments of the present application, the true value camera 140 is a multi-line laser radar. The positioning module 120 may include: a single-line laser radar and a structural member. Among them, the single-line laser radar is connected to the target camera 110 through a structural member, and the fourth external parameter between the single-line laser radar and the target camera 110 is accurately calibrated. And the single-line laser radar is configured to scan the surrounding environment to build a first environment map based on Simultaneous Localization and Mapping (SLAM), and determine the two-dimensional pose information of the target camera 110 based on the first environment map and the fourth external parameter. The single-line laser radar is connected to the true value camera 140 through a structural member, and the fifth external parameter between the single-line laser radar and the true value camera 140 is accurately calibrated. And the single-line laser radar is configured to scan the surrounding environment to build a second environment map based on SLAM, and determine the two-dimensional pose information of the true value camera 140 based on the second environment map and the fifth external parameter. Among them, the single-line laser radar and the target camera 110 are rigidly connected through the connection of structural parts, and the single-line laser radar and the true value camera 140 are rigidly connected through the connection of structural parts.

[0079] When performing three-dimensional reconstruction of the entire scene, the multi-line laser radar scans the entire scene to obtain point cloud information of the entire scene. The single-line laser radar obtains two-dimensional pose information corresponding to the single frame point cloud information when the multi-line laser radar obtains point cloud information of the entire scene. The stitching module 150 performs data stitching on the point cloud information of the entire scene obtained by the multi-line laser radar according to the two-dimensional pose information obtained by the single-line laser radar to complete the three-dimensional reconstruction of the entire scene and obtain the three-dimensional reconstruction data of the entire scene.

[0080] When obtaining the true value of the three-dimensional data of the target scene, the target camera 110 shoots the target scene to obtain the data of the target scene. The single-line laser radar obtains the two-dimensional pose information corresponding to the data of a single frame of the target scene when the target camera 110 obtains the data of the target scene. The cropping module 130 crops the true value of the three-dimensional data of the target scene from the three-dimensional reconstructed data of the entire scene according to the data of the target scene obtained by the target camera 110 and the two-dimensional pose information obtained by the single-line laser radar.

[0081] In some other optional embodiments of the present application, the true value camera 140 is a multi-line laser radar. The positioning module 120 may include: a multi-line laser radar and a structural member. Among them, the multi-line laser radar is connected to the target camera 110 through a structural member, and the sixth external parameter between the multi-line laser radar and the target camera 110 is accurately calibrated. And the multi-line laser radar is configured to scan the surrounding environment to build a third environment map based on SLAM, and determine the three-dimensional pose information of the target camera 110 based on the third environment map and the sixth external parameter. The multi-line laser radar is connected to the true value camera 140 through a structural member, and the seventh external parameter between the multi-line laser radar and the true value camera 140 is accurately calibrated. And the line laser radar is configured to scan the surrounding environment to build a fourth environment map based on SLAM, and determine the three-dimensional pose information of the true value camera 140 based on the fourth environment map and the seventh external parameter. Among them, the connection between the multi-line laser radar and the target camera 110 through the structural member is a rigid connection, and the connection between the multi-line laser radar and the true value camera 140 through the structural member is a rigid connection.

[0082] When performing three-dimensional reconstruction of the entire scene, the multi-line laser radar scans the entire scene to obtain point cloud information of the entire scene. The multi-line laser radar obtains three-dimensional pose information corresponding to the single-frame point cloud information when the multi-line laser radar obtains point cloud information of the entire scene. The stitching module 150 performs data stitching on the point cloud information of the entire scene obtained by the multi-line laser radar according to the three-dimensional pose information obtained by the multi-line laser radar, completes the three-dimensional reconstruction of the entire scene, and obtains the three-dimensional reconstruction data of the entire scene.

[0083] When obtaining the true value of the three-dimensional data of the target scene, the target camera 110 shoots the target scene to obtain the data of the target scene. When the target camera 110 obtains the data of the target scene, the multi-line laser radar obtains the three-dimensional pose information corresponding to the data of a single frame of the target scene. The cropping module 130 crops the true value of the three-dimensional data of the target scene from the three-dimensional reconstructed data of the entire scene according to the data of the target scene obtained by the target camera 110 and the three-dimensional pose information obtained by the multi-line laser radar.

[0084] The embodiment of the present application also provides a method 200 for obtaining a true value based on three-dimensional reconstruction.

[0085] Figure 4The flowchart of the method 200 for obtaining a true value based on three-dimensional reconstruction according to an embodiment of the present application is shown. The method 200 for obtaining a true value based on three-dimensional reconstruction according to an embodiment of the present application can be applied to the device 100 for obtaining a true value based on three-dimensional reconstruction according to any of the above embodiments. Figure 4 As shown, the true value acquisition method 200 based on three-dimensional reconstruction of the embodiment of the present application may include the following steps:

[0086] S210, collecting data of a target scene through a target camera, and collecting first position information corresponding to the target camera for a single frame of target scene data through a positioning module.

[0087] S220, performing data cropping on the 3D reconstructed data of the entire scene including the target scene based on the data of the target scene and the first pose information through a cropping module to obtain a true value of the 3D data of the target scene.

[0088] It should be understood that the steps shown in method 200 are not exclusive, and other steps may be performed before, after or between any of the steps shown. In addition, some of the steps shown may be performed simultaneously, or may be performed in a sequence different from that shown in FIG. 3.

[0089] Alternatively, if Figure 5 As shown, Figure 5 A schematic flow chart of a method 200 for obtaining a true value based on three-dimensional reconstruction according to another embodiment of the present application is shown. The method 200 for obtaining a true value based on three-dimensional reconstruction according to the embodiment of the present application may also include:

[0090] S230, collecting the true value of the three-dimensional data of the entire scene through the true value camera, and collecting the second posture information corresponding to the true value camera for the true value of the three-dimensional data of the single frame of the entire scene through the positioning module.

[0091] S240, performing data splicing on the true value of the three-dimensional data of the entire scene based on the second posture information through a splicing module to obtain three-dimensional reconstructed data of the entire scene.

[0092] Optionally, the three-dimensional reconstruction-based truth value acquisition method 200 of the embodiment of the present application may further include:

[0093] Generate a timestamp based on the reference time generated by the calculation unit by the positioning module and the target camera respectively;

[0094] Synchronize the positioning module with the target camera time based on timestamp alignment;

[0095] Generate a timestamp by the module and the true value camera based on the reference time generated by the calculation unit respectively;

[0096] The localization module is synchronized with the ground truth camera time based on timestamp alignment.

[0097] Optionally, the three-dimensional reconstruction-based truth value acquisition method 200 of the embodiment of the present application may further include:

[0098] The reference time is generated by a calculation unit in the positioning module.

[0099] Optionally, step S210 may further include:

[0100] The target scene data is collected by the target camera at a first collection frequency, and the first position information of the target camera is collected by the positioning module at a second collection frequency; wherein the second collection frequency is an integer multiple of the first collection frequency.

[0101] Step S230 may further include:

[0102] The three-dimensional data truth of the entire scene is collected by the truth camera at a third collection frequency, and the second posture information of the truth camera is collected by the positioning module at a fourth collection frequency; wherein the fourth collection frequency is an integer multiple of the third collection frequency.

[0103] Optionally, step S240 may further include:

[0104] The splicing module is used to splice the data that meets the target accuracy in the true value of the three-dimensional data of the whole scene based on the second pose information to obtain the three-dimensional reconstruction data of the whole scene.

[0105] Optionally, the data that meets the target accuracy in the true value of the three-dimensional data of the whole scene is spliced ​​by a splicing module based on the second pose information to obtain the three-dimensional reconstruction data of the whole scene, which may also include:

[0106] Determine the data interval in which the true value camera has the target accuracy;

[0107] The data that conforms to the data interval in the true value of the three-dimensional data of the whole scene is spliced ​​based on the second pose information to obtain the three-dimensional reconstructed data of the whole scene.

[0108] Optionally, the three-dimensional reconstruction-based truth value acquisition method 200 of the embodiment of the present application may further include:

[0109] Based on the data interval with target accuracy of the true value camera, the data in the true value of the three-dimensional data of the whole scene that does not conform to the data interval is set to zero, so as to obtain data that meets the target accuracy.

[0110] Optionally, step S230 may further include:

[0111] Data of a material with normal reflectivity attached to a surface with special reflectivity is collected as the three-dimensional data true value of the surface with special reflectivity.

[0112] Step S210 may further include:

[0113] Data of the surface with special reflectivity after removing materials with normal reflectivity is collected as data of the surface with special reflectivity.

[0114] Optionally, the three-dimensional reconstruction-based truth value acquisition method 200 of the embodiment of the present application may further include:

[0115] The data of surfaces with special reflectivity in the 3D reconstruction data are corrected through the stitching module.

[0116] Optionally, step S210 may further include:

[0117] The data of the first infrared positioning code is collected respectively by a plurality of infrared cameras; wherein the first infrared positioning code is attached to the upper surface of the target camera; and the plurality of infrared cameras are installed above the entire scene.

[0118] The three-dimensional position information of the target camera is determined based on the data of the first infrared positioning code, the first external parameter between the calibrated first infrared positioning code and the target camera, and the third external parameter between every two infrared cameras among the calibrated infrared cameras.

[0119] Step S230 may further include:

[0120] The data of the second infrared positioning code is collected respectively by a plurality of infrared cameras; wherein the second infrared positioning code is attached to the upper surface of the true value camera.

[0121] Based on the data of the second infrared positioning code, the second external parameter and the third external parameter between the calibrated second infrared positioning code and the true value camera, the three-dimensional position information of the true value camera is determined.

[0122] Optionally, step S210 may further include:

[0123] A first environment map is constructed by scanning the surrounding environment through a single-line laser radar, wherein the single-line laser radar is connected to the target camera through a structural member.

[0124] Based on the first environment map, the calibrated single-line laser radar and the fourth external parameter between the target camera, the two-dimensional position information of the target camera is determined.

[0125] Step S230 may further include:

[0126] A second environment map is constructed by scanning the surrounding environment through a single-line laser radar, wherein the single-line laser radar is connected to the true value camera through a structural member.

[0127] Based on the second environment map, the calibrated single-line laser radar and the fifth external parameter between the true value camera, the two-dimensional pose information of the true value camera is determined.

[0128] Alternatively, if Figure 4 As shown, step S210 may further include:

[0129] The third environment map is constructed by scanning the surrounding environment through a multi-line laser radar; wherein the multi-line laser radar is connected to the target camera through a structural member.

[0130] Based on the third environment map, the calibrated multi-line laser radar and the sixth external parameter between the target camera, the three-dimensional position information of the target camera is determined.

[0131] Step S230 may further include:

[0132] The fourth environment map is constructed by scanning the surrounding environment through a multi-line laser radar, wherein the multi-line laser radar is connected to the true value camera through a structural member.

[0133] Based on the fourth environment map, the calibrated multi-line laser radar and the seventh external parameter between the true value camera, the three-dimensional pose information of the true value camera is determined.

[0134] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A true value acquisition device based on three-dimensional reconstruction, characterized in that: include: a target camera, configured to collect data of a target scene; A positioning module, configured to collect the first position information corresponding to the target camera for a single frame of data of the target scene; as well as The cropping module is configured to perform data cropping on the three-dimensional reconstruction data of the full scene including the target scene based on the data of the target scene and the first posture information to obtain the true value of the three-dimensional data of the target scene.

2. The device according to claim 1, characterized in that Also includes: A true value camera is configured to collect the true value of three-dimensional data of the full scene; The positioning module is further configured to collect second posture information corresponding to the true value camera for the true value of the three-dimensional data of the single frame of the full scene; as well as The stitching module is configured to perform data stitching on the true value of the three-dimensional data of the full scene based on the second posture information to obtain the three-dimensional reconstruction data of the full scene.

3. The device according to claim 2, characterized in that The true value camera includes one of a laser radar, an active binocular structured light camera, a passive binocular camera, a monocular speckle structured light camera, an iTOF camera, a DTof camera, and a single-line structured light camera.

4. The device according to claim 2, characterized in that The positioning module comprises: A first infrared positioning code is attached to the upper surface of the target camera and calibrates a first external parameter between the first infrared positioning code and the target camera; A second infrared positioning code is attached to the upper surface of the true value camera and calibrates a second external parameter between the second infrared positioning code and the true value camera; and A plurality of infrared cameras are installed above the entire scene, and a third external parameter between every two infrared cameras among the plurality of infrared cameras is calibrated, and the plurality of infrared cameras are configured to respectively collect data of the first infrared positioning code, and determine the three-dimensional posture information of the target camera based on the data of the first infrared positioning code, the first external parameter and the third external parameter, and respectively collect data of the second infrared positioning code, and determine the three-dimensional posture information of the true value camera based on the data of the second infrared positioning code, the second external parameter and the third external parameter.

5. The device according to claim 4, characterized in that The positioning module also includes an external component; One end of the external component is connected to the target camera, and the first infrared positioning code is attached to the other end of the external component; One end of the external component is connected to the true value camera, and the second infrared positioning code is attached to the other end of the external component.

6. A method for obtaining true value based on three-dimensional reconstruction, characterized in that: The method applied to the device for obtaining a true value based on three-dimensional reconstruction according to any one of claims 1 to 5 comprises: Collecting data of a target scene through a target camera, and collecting first position information corresponding to the target camera for a single frame of data of the target scene through a positioning module; and The three-dimensional reconstruction data of the whole scene including the target scene is cropped based on the data of the target scene and the first posture information through a cropping module to obtain the true value of the three-dimensional data of the target scene.

7. The method according to claim 6, characterized in that Also includes: Collecting the true value of the three-dimensional data of the whole scene through a true value camera, and collecting the second posture information corresponding to the true value camera for the true value of the three-dimensional data of the whole scene in a single frame through the positioning module; as well as The three-dimensional data true value of the full scene is stitched based on the second posture information through a stitching module to obtain the three-dimensional reconstruction data of the full scene.

8. The method according to claim 7, characterized in that The first position information corresponding to the target camera is collected from the data of the single frame of the target scene by the positioning module, including: The data of the first infrared positioning code is collected respectively by a plurality of infrared cameras; wherein the first infrared positioning code is attached to the upper surface of the target camera; and the plurality of infrared cameras are installed above the entire scene; and Based on the data of the first infrared positioning code, the calibrated first external parameter between the first infrared positioning code and the target camera, and the calibrated third external parameter between every two infrared cameras among the plurality of infrared cameras, the three-dimensional pose information of the target camera is determined; and the second pose information corresponding to the true value camera is collected for the true value of the three-dimensional data of the single frame of the whole scene through the positioning module, including: The plurality of infrared cameras are used to collect data of the second infrared positioning code respectively; wherein the second infrared positioning code is attached to the upper surface of the true value camera; and Based on the data of the second infrared positioning code, the second external parameter between the calibrated second infrared positioning code and the true value camera, and the third external parameter, the three-dimensional posture information of the true value camera is determined.

9. The method according to claim 7, characterized in that: The first position information corresponding to the target camera is collected from the data of the single frame of the target scene by the positioning module, including: Scanning the surrounding environment by a single-line laser radar to construct a first environment map; wherein the single-line laser radar is connected to the target camera through a structural member; Determine two-dimensional pose information of the target camera based on the first environment map and a fourth external parameter between the calibrated single-line laser radar and the target camera; The second posture information corresponding to the true value camera is collected by the positioning module for the true value of the three-dimensional data of the whole scene in a single frame, including: The second environment map is constructed by scanning the surrounding environment by the single-line laser radar; wherein the single-line laser radar is connected to the true value camera through the structural member; Based on the second environment map, the calibrated fifth external parameter between the single-line laser radar and the true value camera, the two-dimensional pose information of the true value camera is determined.

10. The method according to claim 7, characterized in that The first position information corresponding to the target camera is collected from the data of the single frame of the target scene by the positioning module, including: Scanning the surrounding environment by a multi-line laser radar to construct a third environment map; wherein the multi-line laser radar is connected to the target camera through a structural member; Determine the three-dimensional pose information of the target camera based on the third environment map, the calibrated sixth external parameter between the multi-line laser radar and the target camera; The second posture information corresponding to the true value camera is collected by the positioning module for the true value of the three-dimensional data of the whole scene in a single frame, including: The fourth environment map is constructed by scanning the surrounding environment by the multi-line laser radar; wherein the multi-line laser radar is connected to the true value camera through the structural member; Based on the fourth environment map, the calibrated seventh external parameter between the multi-line laser radar and the true value camera, the three-dimensional pose information of the true value camera is determined.

Images