Data collection device, method, apparatus and storage medium

Abstract

The present invention relates to a data acquisition device, a method, a device, and a storage medium, which relate to the field of data processing technology, and the data acquisition device includes a depth camera, a laser light source, a scanning mirror, a laser camera, and a processor, wherein the depth camera is for acquiring an initial depth image of a measurement object, the laser light source is for emitting a laser beam to the scanning mirror, and the scanning mirror is for reflecting the laser beam to scan toward the measurement object, the laser camera is for acquiring a line laser image of the measurement object during the scanning, and the processor is for fusing the different line laser images based on the initial depth image to acquire a target depth image of the measurement object. By applying the technical solutions provided in the embodiments of the present application, the accuracy of the acquired depth image can be improved.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority from a Chinese patent application bearing application number 202211182006.6 and entitled "Data collection device, method, apparatus and storage medium," filed with the State Intellectual Property Office of the People's Republic of China on September 27, 2022, the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD This application relates to the field of data acquisition technology, and in particular to data acquisition devices, methods, apparatus and storage media. [Background technology]

[0003] To obtain three-dimensional data of an object such as a bolt, wrench, etc., a depth image of the object is obtained, and the depth image can provide three-dimensional data such as the length, width, height, etc. of the object.

[0004] In the related art, generally, a laser light source emits a laser beam to an object, a camera captures an image of the object, and then detects distortion information of the laser beam in the image, and a depth image of the object can be obtained based on the distortion information. Then, three-dimensional data of the object can be obtained based on the depth information in the depth image.

[0005] When the above technical solution is applied, a depth image of an object can be obtained, but due to reflection and refraction on the surface of the object, interference occurs with the laser beam irradiated onto the object, and when detecting distortion information of the laser beam in the image, the obtained distortion information may be inaccurate, and the obtained depth image of the object may be inaccurate. Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide a data collection device, method, apparatus and storage medium so as to improve the accuracy of the depth image of the object obtained.

[0007] In a first aspect, embodiments of the present application provide a data collection device including a depth camera, a laser light source, a scanning mirror, a laser camera, and a processor; the depth camera is for acquiring an initial depth image of a measurement object; the laser light source is for emitting a laser beam to the scanning mirror, and the scanning mirror is for reflecting the laser beam so as to scan the measurement object; the laser camera is for acquiring a line laser image of the measurement object during scanning; The processor is for fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object.

[0008] In one embodiment of the present application, the processor specifically: For each line laser image, determine a matching region in the initial depth image that corresponds to the line laser image; According to the distribution positions in the initial depth image of the matching areas corresponding to the different line laser images, the different line laser images are merged to obtain a target depth image of the measurement object.

[0009] In one embodiment of the present application, in the case of static scanning, the processor specifically: For different line laser images, the same initial depth image is used as a reference to fuse the line laser images to obtain a target depth image of the measurement object; Or, The depth camera is specifically for acquiring a plurality of frames of initial depth images of the measurement object; Specifically, the processor: Fusing the acquired initial depth images of the multiple frames to obtain a fused depth image; For each line laser image, determine a matching region in the fused depth image that corresponds to that line laser image; According to the distribution positions in the fused depth image of the matching areas corresponding to the different line laser images, the different line laser images are fused to obtain a target depth image of the measurement object.

[0010] In one embodiment of the present application, in the case of dynamic scanning, the depth camera is specifically for acquiring a plurality of frames of initial depth images of the measurement object in dynamic scanning, and the acquisition time of the initial depth image of each frame is the same as the acquisition time of the line laser image of each frame; Specifically, the processor: For each line laser image, determine a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; According to the distribution positions of the matching regions corresponding to the different line laser images in the different initial depth images, the different line laser images are fused to obtain the target depth image of the measurement object.

[0011] In one embodiment of the present application, the processor specifically: For each line laser image, determine a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; For different initial depth images, obtain differences between the different initial depth images, calculate position and orientation change information of the depth camera according to the differences between the different initial depth images, and perform position correction on matching areas corresponding to different line laser images in the different initial depth images according to the calculated position and orientation change information; According to the distribution positions of the position-corrected matching areas corresponding to the different line laser images in the different initial depth images, the different line laser images are fused to obtain a target depth image of the measurement object.

[0012] In one embodiment of the present application, the depth camera includes a time-of-flight (TOF) light source and a TOF camera, the TOF light source is for emitting a TOF light beam to the measurement object; The TOF camera is used to acquire an initial depth image of the measurement object using the time of flight of the TOF light beam, and the time of flight is the time from when the TOF light source emits the TOF light beam to when the TOF camera receives the TOF light beam reflected by the measurement object.

[0013] In one embodiment of the present application, the laser camera is a binocular camera; and / or The laser light source is a multi-line laser light source or a single-line laser light source.

[0014] In one embodiment of the present application, the scanning mirror includes a mirror and a movement mechanism that can move the mirror.

[0015] In a second aspect, an embodiment of the present application provides a data acquisition method adapted to a processor in a data acquisition device, the data acquisition device further comprising: a depth camera; a laser light source; a scanning mirror; and a laser camera; The data collection method includes: receiving an initial depth image of a measurement object acquired by the depth camera; receiving a line laser image of the measurement object acquired by the laser camera, the line laser image being acquired by the laser camera during a process in which the scanning mirror reflects a laser beam so as to scan the measurement object, and the laser beam being emitted from the laser light source to the scanning mirror; and fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object.

[0016] In one embodiment of the present application, fusing different line laser images based on the initial depth image to obtain the target depth image of the measurement object includes: For each line laser image, determining a matching region in the initial depth image corresponding to the line laser image; and fusing the different line laser images according to distribution positions in the initial depth image of matching areas corresponding to the different line laser images to obtain a target depth image of the measurement object.

[0017] In one embodiment of the present application, in the case of static scanning, fusing different line laser images based on the initial depth image to obtain the target depth image of the measurement object includes: For different line laser images, using the same initial depth image as a reference, fuse each line laser image to obtain a target depth image of the measurement object; Or, The method includes fusing the received multiple frames of initial depth images to obtain a fused depth image; for each line laser image, determining a matching area corresponding to the line laser image in the fused depth image; and fusing different line laser images according to the distribution positions in the fused depth image of the matching areas corresponding to different line laser images to obtain a target depth image of the measurement object.

[0018] In one embodiment of the present application, in the case of dynamic scanning, fusing different line laser images based on the initial depth image to obtain the target depth image of the measurement object includes: For each line laser image, determining a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; and fusing the different line laser images according to distribution positions in the different initial depth images of matching regions corresponding to the different line laser images to obtain a target depth image of the measurement object.

[0019] In one embodiment of the present application, fusing different line laser images according to distribution positions of matching areas in different initial depth images corresponding to different line laser images to obtain a target depth image of the measurement object includes: For different initial depth images, obtain the difference between the different initial depth images, calculate the position and orientation change information of the depth camera according to the difference between the different initial depth images, and perform position correction on the matching areas corresponding to the different line laser images in the different initial depth images according to the calculated position and orientation change information; and fusing the different line laser images according to the distribution positions in the different initial depth images of the position-corrected matching areas corresponding to the different line laser images to obtain a target depth image of the measurement object.

[0020] In a third aspect, an embodiment of the present application provides a data acquisition device installed in a processor in a data acquisition device, the data acquisition device further comprising a depth camera, a laser light source, a scanning mirror, and a laser camera; The data collection device a first image receiving module for receiving an initial depth image of the measurement object acquired by the depth camera; a second image receiving module that receives a line laser image of the measurement object acquired by the laser camera, the line laser image being acquired by the laser camera during a process in which the scanning mirror reflects a laser beam so as to scan the measurement object, and the laser beam being emitted from the laser light source to the scanning mirror; and and a target depth image determination module for fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object.

[0021] In one embodiment of the present application, the target depth image determination module specifically comprises: For each line laser image, determine a matching region in the initial depth image that corresponds to the line laser image; According to the distribution positions in the initial depth image of the matching areas corresponding to the different line laser images, the different line laser images are merged to obtain a target depth image of the measurement object.

[0022] In one embodiment of the present application, in the case of static scanning, the target depth image determination module specifically comprises: For different line laser images, each line laser image is merged based on the same initial depth image to obtain a target depth image of the measurement object; Or, The method fuses the received initial depth images of multiple frames to obtain a fused depth image, and for each line laser image, determines a matching area corresponding to the line laser image in the fused depth image, and fuses different line laser images according to the distribution positions in the fused depth image of the matching areas corresponding to different line laser images to obtain a target depth image of the measurement object.

[0023] In one embodiment of the present application, in the case of dynamic scanning, the target depth image determination module comprises: a region determination sub-module for determining, for each line laser image, a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; and a target depth image determination sub-module for fusing different line laser images according to the distribution positions of the matching areas corresponding to the different line laser images in different initial depth images to obtain a target depth image of the measurement object.

[0024] In one embodiment of the present application, the target depth image determination sub-module specifically includes: For different initial depth images, obtain differences between the different initial depth images, calculate position and orientation change information of the depth camera according to the differences between the different initial depth images, and perform position correction on matching areas corresponding to different line laser images in the different initial depth images according to the calculated position and orientation change information; According to the distribution positions of the position-corrected matching areas corresponding to the different line laser images in the different initial depth images, the different line laser images are fused to obtain a target depth image of the measurement object.

[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of any of the data collection methods described in the second aspect above.

[0026] In a fifth aspect, embodiments of the present application further provide a computer program product including instructions, which, when executed on a computer, cause the computer to perform any of the methods described in the second aspect.

[0027] Beneficial effects of the embodiments of the present application:

[0028] In a data acquisition technique provided in an embodiment of the present application, the data acquisition device includes a depth camera, a laser light source, a scanning mirror, the laser camera, and a processor. The depth camera is configured to acquire an initial depth image of the object to be measured, the laser light source is configured to emit a laser beam toward the scanning mirror, which is configured to reflect the laser beam toward the object to scan it, the laser camera is configured to capture a line laser image of the object during scanning, and the processor is configured to fuse the different line laser images based on the initial depth image to acquire a target depth image of the object. In this way, the depth camera acquires the initial depth image of the object to be measured, the laser light source is configured to emit a laser beam toward the scanning mirror, which is configured to reflect the laser beam toward the object to scan it, the laser camera is configured to capture a line laser image of the object during scanning, and the processor acquires the target depth image of the object based on both the initial depth image and the line laser image. Here, the laser line energy is concentrated, resulting in higher imaging quality of the line laser image. For example, the line laser image has better imaging quality when the surface is highly reflective, such as a black surface. The present embodiment can obtain an accurate target depth image based on a line laser image with high imaging quality, so that applying the technical solution provided in the present embodiment can improve the accuracy of the obtained target depth image of the object. [Brief explanation of the drawings]

[0029] The drawings described herein are provided to facilitate a better understanding of the present application and are intended to constitute a part of the present application. The exemplary embodiments and the description thereof are intended to be illustrative, not limiting, of the present application. [Figure 1] FIG. 1 is a schematic diagram of the configuration of a data collection device provided in an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram of another data collection device provided in an embodiment of the present invention. [Figure 3] FIG. 3 is a flow diagram of the data collection method provided in the present example. [Figure 4] FIG. 4 is a schematic diagram of the configuration of the data collection device provided in the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0030] In order to clarify the objectives, technical solutions and advantages of the embodiments of the present application, the embodiments of the present application will be described in more detail below by way of examples with reference to the drawings. Of course, the described embodiments are only a part of the embodiments of the present application, and are not all of the embodiments. All other embodiments that a person skilled in the art can obtain based on the embodiments of the present application are also included in the scope of protection of the present application.

[0031] To improve the accuracy of the depth image of the object obtained, the present embodiments provide data acquisition devices, methods, apparatus and storage media, each of which is described in detail below.

[0032] An embodiment of the present application provides a data collection device, the data collection device including a depth camera, a laser light source, a scanning mirror, a laser camera, and a processor; The depth camera is for acquiring an initial depth image of the measurement object; the laser light source is for emitting a laser beam to the scanning mirror, and the scanning mirror is for reflecting the laser beam so as to scan the laser beam onto the measurement object; The laser camera is for acquiring a line laser image of the measurement object during scanning; The processor is for fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object.

[0033] In this way, first, the depth camera acquires an initial depth image of the measurement object, then the laser light source emits a laser beam to the scanning mirror, the scanning mirror reflects the laser beam to scan the measurement object, the laser camera acquires a line laser image of the measurement object during scanning, and the processor acquires a target depth image of the measurement object according to both the initial depth image and the line laser image, so that by applying the technical solutions provided in the embodiments of the present application, the accuracy of the acquired target depth image of the object can be improved.

[0034] The following describes the data collection equipment in detail.

[0035] Referring to FIG. 1, FIG. 1 is a schematic diagram of the configuration of a data collection device provided in an embodiment of the present application, which includes a depth camera 101, a laser light source 102, a scanning mirror 103, a laser camera 104, and a processor 105.

[0036] The depth camera 101 is for acquiring an initial depth image of the measurement object.

[0037] The laser light source 102 is for emitting a laser beam to the scanning mirror 103, and the scanning mirror 103 is for reflecting the laser beam so as to scan the object to be measured.

[0038] The process of scanning the measurement object by the laser light source 102 emitting a laser beam to the scanning mirror 103 may be called scanning mirror scanning.

[0039] The laser camera 104 is used to acquire a line laser image of the measurement object during scanning.

[0040] The processor 105 is for fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object.

[0041] The laser light source 102 is a multi-line laser light source or a single-line laser light source.

[0042] The laser beam emitted from the laser light source 102 may be a linear beam, a planar beam, a dot matrix beam, etc. The laser camera 104 may be a monocular camera, a binocular camera, etc.

[0043] The depth camera 101 and the laser camera 104 are maintained in such a way that the relative position and orientation relationship does not change during the image acquisition process.

[0044] For example, the depth camera 101 and the laser camera 104 are fixed and connected by connecting members such as a connecting rod or a connecting plate, thereby maintaining the relative position and orientation relationship between the depth camera 101 and the laser camera 104 unchanged.

[0045] For example, the depth camera 101 and the laser camera 104 may be attached separately. In this case, when attaching these two types of cameras, the positions and orientations of these two types of cameras are calibrated to ensure that the relative position and orientation relationship between these two types of cameras is maintained unchanged during the process of acquiring images from these two types of cameras. If the relative position and orientation relationship between these two types of cameras can be set in advance, when attaching these two types of cameras, one type of camera is first attached, and then the position and orientation of the other type of camera is determined based on the position and orientation of the attached camera and the previously set relative position and orientation relationship. This allows the other type of camera to be attached according to the determined position and orientation.

[0046] The depth camera 101 and the laser camera 104 are each communicatively connected to the processor 105 via a wired or wireless connection, allowing the processor 105 to acquire images of the measurement object obtained from the depth camera 101 and the laser camera 104.

[0047] The measurement object may be a person, a bolt, a wrench, or the like.

[0048] The processor 105 can fuse different line laser images based on the camera parameters of the depth camera 101 and the laser camera 104. The camera parameters include the internal parameters of the depth camera 101, the internal parameters of the laser camera 104, and the position and orientation transformation relationship between the depth camera 101 and the laser camera 104. The internal parameters include the focal length, the center point, the deformation coefficient, etc. The camera parameters can be obtained by calibration in advance.

[0049] The laser light source 102 can emit a laser beam to the scanning mirror 103. The scanning mirror 103 reflects the laser beam to the measurement object. Then, the laser camera 104 captures a line laser image of the measurement object. Since the surface of the measurement object is usually not flat, the laser beam distributed on the surface of the measurement object in the line laser image is distorted. This distortion can reflect depth information of the surface of the measurement object. The laser camera 104 can transmit the line laser image to the processor 105.

[0050] The scanning mirror 103 may be a mechanical rotating polygon mirror or a MEMS (Micro-Electro-Mechanical System) rotating polygon mirror.

[0051] In one embodiment of the present application, the scanning mirror 103 includes a mirror and a movement mechanism for moving the mirror.

[0052] The processor 105 receives the above initial depth image and line laser image, and can determine the target depth image of the measurement object based on the initial depth image using the line laser image and the camera parameters of the depth camera 101 and the laser camera 104.

[0053] In the process of fusing different line laser images, the processor 105 may fuse line laser images based on initial depth images of the same frame for line laser images of different frames, or may fuse line laser images based on initial depth images of different frames.

[0054] For multiple embodiments in which the processor 105 fuses the line laser images, please refer to the examples below, and details will be omitted here.

[0055] The depth camera 101 is used to acquire an initial depth image of the object to be measured.

[0056] In one embodiment of the present application, the depth camera 101 includes a TOF (Time of Flight) camera and a TOF light source. The TOF light emitted from the TOF light source may be a point light, a linear light, a planar light, a dot matrix light, a spot light, etc. The TOF camera may be a monocular camera, a binocular camera, etc.

[0057] Specifically, the TOF camera may acquire depth images using iTOF (Indirect Time-of-Flight) technology or DTOF (Direct Time-of-Flight) technology, and can acquire initial depth images at the centimeter level.

[0058] The TOF camera is used to capture an initial depth image of the object by the time of flight of the TOF beam, where the time of flight is the time from when the TOF light source emits the TOF beam to when the TOF camera receives the TOF beam reflected by the object.

[0059] Specifically, the TOF light source can emit a TOF beam toward the measurement object. The measurement object can reflect the TOF beam. The TOF camera receives the TOF beam reflected from the measurement object, determines the emission time when the TOF light source emits the TOF beam and the reception time when the TOF camera receives the TOF beam reflected from the measurement object, calculates the time between the emission time and the reception time as the flight time of the TOF beam from the TOF light source to the measurement object and from the measurement object to the TOF camera, and calculates the distance between the TOF camera and the measurement object based on the flight time and the propagation speed of the TOF beam. Based on the distance, a depth image of the measurement object can be obtained as an initial depth image. The TOF camera can send the initial depth image to the processor 105. In this way, first, the depth camera 101 acquires an initial depth image of the measurement object, then the laser light source 102 emits a laser beam to the scanning mirror 103, the scanning mirror 103 reflects the laser beam to scan the measurement object, the laser camera 104 acquires a line laser image of the measurement object during scanning, and the processor 105 acquires a target depth image of the measurement object according to both the initial depth image and the line laser image. Therefore, by applying the technical solutions provided in the embodiments of the present application, the accuracy of the acquired target depth image of the object can be improved.

[0060] The following describes an embodiment in which the processor fuses line laser images.

[0061] In one embodiment of the present application, the processor 105 specifically: For each line laser image, determine a matching region in the initial depth image corresponding to the line laser image; According to the distribution positions in the initial depth image of the matching areas corresponding to the different line laser images, the different line laser images are fused to obtain the target depth image of the measurement object.

[0062] Specifically, the line laser image and the initial depth image are images of the same measurement object captured by different cameras, so both the line laser image and the initial depth image contain the same object region corresponding to the measurement object. Furthermore, the laser camera captures the image of the measurement object in the process of scanning the measurement object with a laser beam, so the region in the line laser image where the laser beam exists belongs to the object region corresponding to the measurement object. In view of this, the matching region in the initial depth image corresponding to the line laser image is understood to be the region in the initial depth image where the portion of the measurement object corresponding to the region in the line laser image where the laser beam exists corresponds.

[0063] For example, if the measurement object is a bolt, the line laser image may be an image of the bolt acquired by a laser camera when scanning the nut of the bolt with a laser beam, and the initial depth image is an image of the bolt acquired by a depth camera. In this case, the matching region in the initial depth image corresponding to the laser image is understood to be the region in the initial depth image corresponding to the nut of the bolt.

[0064] The time instants at which different line laser images are acquired are different, and the areas in the images where the laser beams exist are also different. For each line laser image, a matching area corresponding to the line laser image can be determined in the initial depth image.

[0065] In one embodiment of the present application, for each line laser image, image feature matching is performed between the line laser image and the initial depth image, and the area of ​​the same real scene corresponding to the line laser image in the initial depth image can be determined; that is, the image features of the area where the laser beam exists in the line laser image are extracted, and the image features of the initial depth image are extracted, and feature matching is performed on the image features extracted in these two images, and the features of the initial depth image that match the image features of the line laser image are determined, and the image area in the initial depth image represented by the matched features can be determined as the above-mentioned matching area.

[0066] The same real scene is understood as a target region of the measurement object, for example, if the measurement object is a bolt, the target region of the measurement object may be the region where the nut of the bolt is located. The region corresponding to the same real scene in the initial depth image is a matching region.

[0067] After determining the matching area corresponding to each line laser image, the area content of the area where the laser beam exists in each line laser image can be stitched according to the distribution position in the initial depth image of the matching area corresponding to each line laser image, so that the stitched image can be obtained as the target depth image.

[0068] In one embodiment of the present application, the line laser image may be fused based on the technical solutions in the related art, and the description thereof is omitted here.

[0069] Scanning of a measurement object with a laser beam can be divided into static scanning and dynamic scanning. In static scanning, the measurement object and the data acquisition device remain relatively stationary during scanning. In dynamic scanning, the measurement object and the data acquisition device move relatively during scanning.

[0070] The following describes specific embodiments in which the processor 105 fuses each line laser image for static and dynamic scanning.

[0071] For example, in the case of static scanning, the processor 105 may fuse each line laser image according to one of the following two embodiments.

[0072] In the first embodiment, the processor 105 specifically: For different line laser images, the same initial depth image is used as a reference to fuse the line laser images and obtain a target depth image of the measurement target.

[0073] In the second embodiment, the depth camera 101 is specifically for acquiring initial depth images of a plurality of frames of a measurement object, Specifically, the processor 105: fusing the acquired initial depth images of the plurality of frames to obtain a fused depth image; for each line laser image, determining a matching region in the fused depth image corresponding to the line laser image; According to the distribution positions in the fused depth image of the matching regions corresponding to the different line laser images, the different line laser images are fused to obtain a target depth image of the measurement object.

[0074] For example, in the case of dynamic scanning, the depth camera 101 is specifically for acquiring a plurality of frames of initial depth images of the measurement object in dynamic scanning, and the acquisition time of the initial depth image of each frame is the same as the acquisition time of the line laser image of each frame; Specifically, the processor 105: for each line laser image, determining a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; According to the distribution positions of the matching regions corresponding to the different line laser images in the different initial depth images, the different line laser images are fused to obtain the target depth image of the measurement object.

[0075] Since different line laser images are acquired at different times, the process of performing image feature matching based on line laser images and initial depth images acquired at different times may be referred to as feature matching in the time domain.

[0076] In one embodiment of the present application, the processor 105 specifically: for each line laser image, determining a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; For different initial depth images, obtain the difference between the different initial depth images, calculate the position and orientation change information of the depth camera 101 according to the difference between the different initial depth images, and perform position correction on the matching areas corresponding to the different line laser images in the different initial depth images according to the calculated position and orientation change information; According to the distribution positions of the position-corrected matching areas corresponding to the different line laser images in the different initial depth images, the different line laser images are fused to obtain a target depth image of the measurement object.

[0077] Referring to Figure 2, Figure 2 is a schematic diagram of another data collection device provided in an embodiment of the present application. The data collection device includes a TOF light source, a TOF camera, a laser light source, a laser camera, a processor, and a galvanometer mirror. The laser camera can be divided into a left camera and a right camera. The TOF light source, TOF camera, laser light source, laser camera, processor, and galvanometer mirror are fixedly attached to a connecting plate.

[0078] Here, the galvanometer mirror is similar to the scanning mirror, and both are used to reflect a laser beam so as to scan the laser beam onto the object to be measured.

[0079] The laser light source emits a laser beam to the galvanometer mirror, and the galvanometer mirror reflects the received laser beam onto the object to be measured, thereby realizing scanning of the object to be measured with the laser beam.

[0080] The laser camera is used to capture multiple frames of laser images of the object to be measured during scanning.

[0081] The TOF light source is for emitting a TOF light beam to the measurement object.

[0082] The TOF camera is used to acquire multiple frames of initial depth images of the measurement object by the time of flight of the TOF ray during scanning, and the acquisition time of each frame of the initial depth image is the same as the acquisition time of each frame of the laser image.

[0083] For each laser image, the processor acquires a reference depth image of the object to be measured using the laser image and camera parameters of the TOF camera and the laser camera based on an initial depth image acquired simultaneously with the laser image, and fuses the multiple reference depth images to acquire a target depth image of the object to be measured.

[0084] In the data acquisition technical solution provided in the above embodiment, the data acquisition device includes a time-of-flight TOF light source, a TOF camera, a laser light source, a laser camera, and a processor, where the TOF light source is for emitting a TOF beam toward the measurement object, and the TOF camera is for acquiring an initial depth image of the measurement object according to the time-of-flight of the TOF beam, where the time-of-flight is the time from when the TOF light source emits the TOF beam to when the TOF beam reflected by the measurement object is received. The laser light source is for emitting a laser beam toward the measurement object, and the laser camera is for acquiring a laser image of the measurement object, and the processor is for determining a target depth image of the measurement object based on the initial depth image according to the laser image and camera parameters of the TOF camera and the laser camera. In this way, the initial depth image of the measurement object is first acquired using the TOF beam, and then the initial depth image and the laser image are fused according to the camera parameters of the TOF camera and the laser camera, and a depth image of the measurement object can be acquired according to both the initial depth image and the laser image. Therefore, by applying the method provided in the above embodiment, the accuracy of the depth image of the object obtained can be improved.

[0085] Referring to Figure 3, Figure 3 is a flow diagram of a data acquisition method provided in an embodiment of the present application. The data acquisition method is applied to a processor in a data acquisition device, and the data acquisition device further includes a depth camera, a laser light source, a scanning mirror, and a laser camera. Specifically, the relative position and orientation relationship between the depth camera and the laser camera is maintained unchanged during the image acquisition process. For example, the depth camera is fixedly connected to the laser camera.

[0086] The data collection method includes the following steps.

[0087] S301, receiving an initial depth image of a measurement object acquired by a depth camera.

[0088] S302, a line laser image of the measurement object acquired by the laser camera is received.

[0089] The line laser image is captured by a laser camera while the scanning mirror reflects the laser beam so as to scan the measurement target. The laser beam is emitted from a laser light source to the scanning mirror.

[0090] S303, based on the initial depth image, different line laser images are fused to obtain a target depth image of the measurement object.

[0091] In addition, the present embodiment does not limit the order of execution of the above steps S301 and S302; step S301 may be executed first, then step S302, step S302 may be executed first, then step S301, or step S301 and step S302 may be executed simultaneously.

[0092] In one embodiment of the present application, fusing different line laser images based on the initial depth image to obtain the target depth image of the measurement object includes: For each line laser image, determining a matching region in the initial depth image corresponding to the line laser image; and fusing the different line laser images according to distribution positions in the initial depth image of matching areas corresponding to the different line laser images to obtain a target depth image of the measurement object.

[0093] By applying the data acquisition technology provided in the embodiments of the present application, the accuracy of the target depth image of the measurement object obtained can be improved.

[0094] In one embodiment of the present application, in the case of static scanning, fusing different line laser images based on the initial depth image to obtain the target depth image of the measurement object includes: For different line laser images, the method includes fusing each line laser image based on the same initial depth image to obtain a target depth image of the measurement object.

[0095] By applying the data acquisition technology provided in the embodiments of the present application, the accuracy of the target depth image of the measurement object obtained can be improved.

[0096] In one embodiment of the present application, in the case of static scanning, fusing different line laser images based on the initial depth image to obtain the target depth image of the measurement object includes: The method includes fusing the acquired initial depth images of multiple frames to obtain a fused depth image, determining for each line laser image a matching area corresponding to the line laser image in the fused depth image, and fusing different line laser images according to the distribution positions in the fused depth image of the matching areas corresponding to different line laser images to obtain a target depth image of the measurement object.

[0097] When data is acquired by applying the technical solution provided in the embodiment of the present application, the fused depth image is derived from multiple frames of initial depth images, so that the fused depth image contains more complete three-dimensional object information of the measurement target, and thus for each line laser image, the matching area corresponding to the line laser image can be accurately determined in the fused depth image, and different line laser images can be fused according to the distribution position of the more accurate matching area corresponding to each line laser image to obtain the target depth image, thereby improving the accuracy of the obtained target depth image.

[0098] In one embodiment of the present application, in the case of dynamic scanning, fusing different line laser images based on the initial depth image to obtain the target depth image of the measurement object includes: For each line laser image, determining a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; and fusing the different line laser images according to distribution positions in the different initial depth images of matching regions corresponding to the different line laser images to obtain a target depth image of the measurement object.

[0099] By applying the data acquisition technology provided in the embodiments of the present application, the accuracy of the target depth image of the measurement object obtained can be improved.

[0100] In one embodiment of the present application, fusing different line laser images according to distribution positions of matching areas in different initial depth images corresponding to different line laser images to obtain a target depth image of the measurement object includes: For different initial depth images, calculate position and orientation change information of the depth camera according to the difference between the different initial depth images, and perform position correction on matching areas corresponding to different line laser images in the different initial depth images according to the calculated position and orientation change information; and fusing the different line laser images according to the distribution positions in the different initial depth images of the position-corrected matching areas corresponding to the different line laser images to obtain a target depth image of the measurement object.

[0101] By applying the technical solutions provided in the embodiments of the present application to acquire data, position correction can be performed on the matching areas corresponding to different line laser images in different initial depth images, thereby improving the accuracy of the matching areas corresponding to different line laser images in different initial depth images, and thereby fusing different line laser images according to the distribution positions of the position-corrected matching areas to improve the accuracy of the obtained target depth image.

[0102] In addition to the beneficial effects mentioned above, each of the above method embodiments also has the following beneficial effects.

[0103] In each of the above method embodiments, a depth camera is used to obtain an initial depth image of the object to be measured, a laser light source is used to emit a laser beam to a scanning mirror, the scanning mirror reflects the laser beam to scan the object to be measured, the laser camera obtains a line laser image of the object to be measured during scanning, and a processor obtains a target depth image of the object according to both the initial depth image and the line laser image, thereby improving the accuracy of the target depth image of the object obtained by applying the technical solutions provided in the embodiments of the present application.

[0104] Referring to FIG. 4, FIG. 4 is a schematic diagram of a data collection device provided in an embodiment of the present application. The data collection device is installed in a processor in a data collection device. The data collection device further includes a depth camera, a laser light source, a scanning mirror, and a laser camera. Specifically, the relative position and orientation relationship between the depth camera and the laser camera is maintained unchanged during the image acquisition process. For example, the depth camera is fixedly connected to the laser camera.

[0105] The data collection device a first image receiving module 401 for receiving an initial depth image of the measurement object acquired by the depth camera; a second image receiving module 402 that receives a line laser image of the measurement object captured by the laser camera, the line laser image being captured by the laser camera during a process in which a scanning mirror reflects a laser beam so as to scan the measurement object, and the laser beam is emitted from a laser light source to the scanning mirror; and a target depth image determination module 403 for fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object.

[0106] When data is acquired by applying the technical solution provided in the embodiment of the present application, the laser line energy is concentrated and the imaging quality of the line laser image is high, so that the embodiment of the present application can acquire a more accurate target depth image based on the line laser image with high imaging quality, thereby improving the accuracy of the target depth image of the object obtained by applying the technical solution provided in the embodiment of the present application.

[0107] In one embodiment of the present application, the target depth image determination module 403 specifically includes: For each line laser image, determine a matching region in the initial depth image that corresponds to the line laser image; According to the distribution positions in the initial depth image of the matching areas corresponding to the different line laser images, the different line laser images are merged to obtain a target depth image of the measurement object.

[0108] By applying the data acquisition technology provided in the embodiments of the present application, the accuracy of the target depth image of the measurement object obtained can be improved.

[0109] In one embodiment of the present application, in the case of static scanning, the target depth image determination module 403 specifically includes: For different line laser images, the same initial depth image is used as a reference to fuse the line laser images and obtain a target depth image of the measurement object.

[0110] By applying the data acquisition technology provided in the embodiments of the present application, the accuracy of the target depth image of the measurement object obtained can be improved.

[0111] In one embodiment of the present application, in the case of static scanning, the target depth image determination module 403 specifically includes: The method fuses the acquired initial depth images of multiple frames to obtain a fused depth image, and for each line laser image, determines a matching area corresponding to the line laser image in the fused depth image, and fuses different line laser images according to the distribution positions in the fused depth image of the matching areas corresponding to different line laser images, to obtain a target depth image of the measurement object.

[0112] When data is acquired by applying the technical solution provided in the embodiment of the present application, the fused depth image is derived from multiple frames of initial depth images, so that the fused depth image contains more complete three-dimensional object information of the measurement target, and thus for each line laser image, the matching area corresponding to the line laser image can be accurately determined in the fused depth image, and different line laser images are fused according to the distribution position of the accurate matching area corresponding to each line laser image to obtain the target depth image, thereby improving the accuracy of the obtained target depth image.

[0113] In one embodiment of the present application, in the case of dynamic scanning, the target depth image determination module 403 comprises: a region determination sub-module for determining, for each line laser image, a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; and a target depth image determination sub-module for fusing different line laser images according to the distribution positions of the matching areas corresponding to the different line laser images in different initial depth images to obtain a target depth image of the measurement object.

[0114] By applying the data acquisition technology provided in the embodiments of the present application, the accuracy of the target depth image of the measurement object obtained can be improved.

[0115] In one embodiment of the present application, the target depth image determination sub-module specifically includes: For different initial depth images, calculate position and orientation change information of the depth camera according to the difference between the different initial depth images, and perform position correction on matching areas corresponding to different line laser images in the different initial depth images according to the calculated position and orientation change information; According to the distribution positions of the position-corrected matching areas corresponding to the different line laser images in the different initial depth images, the different line laser images are fused to obtain a target depth image of the measurement object.

[0116] By applying the technical solutions provided in the embodiments of the present application to obtain data, position correction can be performed on the matching areas corresponding to different line laser images in different initial depth images, thereby improving the accuracy of the matching areas corresponding to different line laser images in different initial depth images, and thereby fusing different line laser images according to the distribution positions of the position-corrected matching areas to improve the accuracy of the obtained target depth image.

[0117] In addition to the beneficial effects mentioned above, each of the above device embodiments also has the following beneficial effects.

[0118] In each of the above device embodiments, a depth camera is used to obtain an initial depth image of the object to be measured, a laser light source is used to emit a laser beam to a scanning mirror, the scanning mirror reflects the laser beam to scan the object to be measured, the laser camera obtains a line laser image of the object to be measured during scanning, and a processor obtains a target depth image of the object according to both the initial depth image and the line laser image, so that the accuracy of the target depth image of the object obtained can be improved by applying the technical solutions provided in the embodiments of the present application.

[0119] In a further embodiment provided herein, there is provided a computer-readable storage medium having a computer program recorded thereon, the computer program implementing any of the data collection methods described above when executed by a processor.

[0120] In a further embodiment provided herein, there is further provided a computer program product including commands, which when executed on a computer, cause the computer to implement any of the data acquisition methods in the above embodiments.

[0121] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer commands. When the computer program commands are loaded into a computer and executed, the process or function described in the embodiments of the present application can be implemented in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other rewritable device. The computer commands can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer commands can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can read and write, or it can be a data storage device, including a server, data center, etc., that integrates one or more available media. The available media may be magnetic media (eg, floppy disks, hard disks, tapes), optical media (eg, DVDs), and the like.

[0122] In the data acquisition technical solution provided in the above embodiment, the data acquisition device includes a depth camera, a laser light source, a scanning mirror, a laser camera, and a processor, wherein the depth camera is for acquiring an initial depth image of the measurement object, the laser light source is for emitting a laser beam to the scanning mirror, which is for reflecting the laser beam to scan the measurement object, the laser camera is for acquiring a line laser image of the measurement object during scanning, and the processor is for fusing different line laser images based on the initial depth image to acquire a target depth image of the measurement object. In this way, the depth camera first acquires an initial depth image of the measurement object, then the laser light source emits a laser beam to the scanning mirror, which reflects the laser beam to scan the measurement object, the laser camera acquires a line laser image of the measurement object during scanning, and the processor acquires a target depth image of the measurement object according to both the initial depth image and the line laser image. Therefore, applying the technical solution provided in the embodiment can improve the accuracy of the target depth image of the object obtained.

[0123] It should be noted that, in this context, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another and are not intended to require or imply any actual relationship or order between those entities or operations. Furthermore, the terms "comprise," "include," "includes," or any other variant thereof, mean a non-exclusive "comprise." Thus, a process, method, article, or device comprising a set of elements does not include only those elements, but also other elements not expressly listed, or elements inherent in such process, method, article, or device. Absent further limitations, an element limited by a phrase "comprising..." does not exclude the presence of other identical elements in a process, method, article, or device that includes the recited element.

[0124] The embodiments in this specification are described in a related manner, and identical or similar parts between the embodiments may be cross-referenced. The emphasis in each embodiment is on the differences from other embodiments. In particular, the apparatus embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, and therefore will be briefly described. For relevant parts, please refer to the description of the method embodiments.

[0125] The above description is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. 1. A data collection device comprising: a depth camera, a laser light source, a scanning mirror, a laser camera, and a processor; the depth camera is for acquiring an initial depth image of a measurement object; the laser light source is for emitting a laser beam to the scanning mirror, and the scanning mirror is for reflecting the laser beam so as to scan the measurement object; the laser camera is for acquiring a line laser image of the measurement object during scanning; The processor is configured to fuse different line laser images based on the initial depth image to obtain a target depth image of the measurement object. Data collection equipment.

2. The processor is for determining, for each line laser image, a matching area corresponding to the line laser image in the initial depth image, and fusing different line laser images according to distribution positions in the initial depth image of matching areas corresponding to different line laser images, to obtain a target depth image of the measurement object.

10. The data acquisition device of claim 1.

3. For static scanning, The processor is configured to fuse different line laser images based on the same initial depth image to obtain a target depth image of the measurement object; Or, The depth camera is for acquiring multiple frames of initial depth images of the measurement object, and the processor is for fusing the acquired multiple frames of initial depth images to acquire a fused depth image, and for each line laser image, determining a matching area corresponding to the line laser image in the fused depth image, and fusing different line laser images according to distribution positions in the fused depth image of matching areas corresponding to different line laser images, to acquire a target depth image of the measurement object.

10. The data acquisition device of claim 1.

4. For dynamic scanning, the depth camera is for acquiring a plurality of frames of initial depth images of the measurement object in dynamic scanning, and the acquisition time of the initial depth image of each frame is the same as the acquisition time of the line laser image of each frame; The processor is for determining, for each line laser image, a matching area corresponding to the line laser image in the initial depth image acquired at the same time, and fusing the different line laser images according to distribution positions of the matching areas corresponding to the different line laser images in the different initial depth images to obtain a target depth image of the measurement object.

10. The data acquisition device of claim 1.

5. The processor is characterized in that, for each line laser image, it determines a matching area corresponding to the line laser image in the initial depth image acquired at the same time, for different initial depth images, obtains a difference between the different initial depth images, calculates position and posture change information of the depth camera according to the difference between the different initial depth images, performs position correction on the matching areas corresponding to the different line laser images in the different initial depth images according to the calculated position and posture change information, and fuses the different line laser images according to the distribution positions in the different initial depth images of the position-corrected matching areas corresponding to the different line laser images, to obtain a target depth image of the measurement object.

5. The data acquisition device of claim 4.

6. The depth camera includes a time-of-flight (TOF) light source and a TOF camera, the TOF light source being configured to emit a TOF light beam toward the measurement object; The TOF camera is configured to acquire an initial depth image of the measurement object by a time of flight of the TOF light beam, and the time of flight is a time from when the TOF light source emits the TOF light beam to when the TOF camera receives the TOF light beam reflected by the measurement object. The data collection device according to any one of claims 1 to 5.

7. the laser camera is a binocular camera; and / or The laser light source is a multi-line laser light source or a single-line laser light source. The data collection device according to any one of claims 1 to 5.

8. The scanning mirror includes a mirror and an operating mechanism for moving the mirror. The data collection device according to any one of claims 1 to 5.

9. 1. A data acquisition method applied to a processor in a data acquisition device, the data acquisition device further comprising a depth camera, a laser light source, a scanning mirror, and a laser camera; The data collection method includes: receiving an initial depth image of a measurement object acquired by the depth camera; receiving a line laser image of the measurement object acquired by the laser camera, the line laser image being acquired by the laser camera during a process in which the scanning mirror reflects a laser beam so as to scan the measurement object, and the laser beam being emitted from the laser light source to the scanning mirror; and fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object. Data collection method.

10. Fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object, For each line laser image, determining a matching region in the initial depth image corresponding to the line laser image; According to the distribution positions of the matching areas corresponding to the different line laser images in the initial depth image, fusing the different line laser images to obtain a target depth image of the measurement object, The data collection method according to claim 9.

11. In the case of static scanning, fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object includes: For different line laser images, using the same initial depth image as a reference, fuse each line laser image to obtain a target depth image of the measurement object; Or, The method includes: fusing the received initial depth images of multiple frames to obtain a fused depth image; determining, for each line laser image, a matching area corresponding to the line laser image in the fused depth image; fusing different line laser images according to distribution positions in the fused depth image of matching areas corresponding to different line laser images, to obtain a target depth image of the measurement object. The data collection method according to claim 9.

12. In the case of dynamic scanning, fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object includes: For each line laser image, determining a matching region corresponding to the line laser image in the simultaneously acquired initial depth image; According to the distribution positions of the matching areas corresponding to the different line laser images in the different initial depth images, fusing the different line laser images to obtain a target depth image of the measurement object, The data collection method according to claim 9.

13. According to the distribution positions of the matching regions corresponding to the different line laser images in the different initial depth images, fusing the different line laser images to obtain the target depth image of the measurement object; For different initial depth images, obtain the difference between the different initial depth images, calculate the position and orientation change information of the depth camera according to the difference between the different initial depth images, and perform position correction on the matching areas corresponding to the different line laser images in the different initial depth images according to the calculated position and orientation change information; and fusing the different line laser images according to distribution positions of the position-corrected matching areas corresponding to the different line laser images in the different initial depth images to obtain a target depth image of the measurement object. The data collection method according to claim 12.

14. a data acquisition device installed in a processor in a data acquisition device, the data acquisition device further comprising a depth camera, a laser light source, a scanning mirror, and a laser camera; The data collection device a first image receiving module for receiving an initial depth image of a measurement object acquired by the depth camera; a second image receiving module that receives a line laser image of the measurement object acquired by the laser camera, the line laser image being acquired by the laser camera during a process in which the scanning mirror reflects a laser beam so as to scan the measurement object, and the laser beam being emitted from the laser light source to the scanning mirror; and a target depth image determination module for fusing different line laser images based on the initial depth image to obtain a target depth image of the measurement object; Data collection device.

15. The target depth image determination module is for determining, for each line laser image, a matching area in the initial depth image corresponding to the line laser image, and fusing different line laser images according to distribution positions in the initial depth image of matching areas corresponding to different line laser images, to obtain a target depth image of the measurement object; or In the case of static scanning, the target depth image determination module is for fusing different line laser images with the same initial depth image as a reference to obtain a target depth image of the measurement object, or for fusing the received multiple frames of initial depth images to obtain a fused depth image, for each line laser image, determining a matching area corresponding to the line laser image in the fused depth image, and fusing different line laser images according to the distribution positions in the fused depth image of the matching areas corresponding to different line laser images, to obtain a target depth image of the measurement object; or In the case of dynamic scanning, the target depth image determination module includes: an area determination sub-module for determining, for each line laser image, a matching area corresponding to the line laser image in the initial depth image acquired at the same time; and a target depth image determination sub-module for fusing different line laser images according to distribution positions of matching areas corresponding to different line laser images in different initial depth images to obtain a target depth image of the measurement object; or The target depth image determination submodule is for obtaining differences between different initial depth images for different initial depth images, calculating position and posture change information of the depth camera according to the differences between the different initial depth images, performing position correction on matching areas corresponding to different line laser images in the different initial depth images according to the calculated position and posture change information, and fusing the different line laser images according to the distribution positions in the different initial depth images of the position-corrected matching areas corresponding to the different line laser images, to obtain the target depth image of the measurement object.

15. The data acquisition device of claim 14.

16. A computer-readable storage medium having a computer program recorded thereon, the computer program being characterized in that, when executed by a processor, the computer program realizes the data collection method according to any one of claims 9 to 13. A computer-readable storage medium.

Images