Depth data measuring device and application method thereof

Abstract

The invention discloses a depth data measuring device and an application method thereof. The measuring device comprises a projection assembly which is used for projecting linear light to an imaging object with relative motion; the image sensor is used for imaging based on the reflected light of the linear light and sending an effective signal with a timestamp when the change of the light passing quantity received by each pixel in unit time exceeds a threshold value; and a processor for calculating surface depth information of the imaging object based on the pixel position and the timestamp that report the valid signal, and the relative motion information. According to the invention, active light imaging is carried out on an imaging object with relative motion by using linear light, imaging is carried out only aiming at the changed effective signals by combining with an image sensor which can independently generate the effective signals when the brightness of each pixel changes, and position determination of depth data is carried out by using a timestamp and relative motion information. Therefore, high-speed depth information acquisition and processing are realized through a simple projection mechanism and extremely low computing power requirements.

Description

Technical Field

[0001] The present disclosure relates to the field of three-dimensional data measurement, and in particular to a depth data measurement device and an application method thereof. Background Art

[0002] A depth data camera is a device that collects depth information of a target object. This type of camera is widely used in 3D scanning, 3D modeling and other fields. For example, more and more smartphones are now equipped with depth cameras for face recognition.

[0003] In the prior art, there is a tendency to use increasingly complex devices and projection patterns to achieve high-precision imaging. High-precision imaging usually requires high frame rates, which places abnormally high demands on the processor power and bandwidth of the depth data measurement device.

[0004] Therefore, a solution is needed to obtain more accurate depth information at a lower computational cost. Summary of the invention

[0005] A technical problem to be solved by the present disclosure is to provide a depth data measurement device and an application method thereof. The solution uses linear light to perform active light imaging on an imaging object with relative motion, and combines an image sensor in which each pixel can independently generate a valid signal when the brightness changes. It is possible to image only the valid signal that has changed, and use timestamps and relative motion information to determine the position of the depth data, thereby achieving high-speed depth information acquisition and processing with a simple projection mechanism and extremely low computing power requirements.

[0006] According to a first aspect of the present disclosure, there is provided a depth data measuring device, comprising: a projection component for projecting linear light onto an imaging object in relative motion; an image sensor for imaging the imaging object based on reflected light of the linear light, the image sensor comprising a plurality of pixels, each pixel sending a valid signal with a timestamp when a change in the amount of light received per unit time exceeds a threshold; and a processor for calculating the surface depth information of the imaging object based on the pixel position and timestamp of the reported valid signal, and the relative motion information.

[0007] Optionally, the first relative speed is the movement speed of the imaging object relative to the depth data measuring device, and includes: the movement speed of the imaging object when the depth data measuring device is stationary; the movement speed of the depth data measuring device when the imaging object is stationary; or the relative movement speed of the imaging object and the depth data measuring device when both the imaging object and the depth data measuring device are in motion.

[0008] Optionally, the depth data measuring device further includes: a speed information acquiring device, configured to acquire the first relative speed information.

[0009] Optionally, the image sensor includes: a first image sensor and a second image sensor arranged on both sides of the projection assembly, wherein the processor calculates the depth information of the imaging object using the pixel position and timestamp of the reported valid signals from the first image sensor and the second image sensor, and the first relative speed motion.

[0010] Optionally, the projection component projects linear light pulses at a predetermined frequency.

[0011] Optionally, the first relative speed measuring device as described in claim 1, wherein the linear light projection direction of the projection component remains unchanged with other components and a housing of the depth data measuring device.

[0012] According to a second aspect of the present disclosure, a method for measuring volume information of items on a conveyor belt is provided, the method being performed by the depth data measuring device described in the first aspect, and comprising: projecting linear light toward items on the conveyor belt; imaging the projected area of ​​the items based on reflected light of the linear light, and acquiring a valid signal with a timestamp; calculating the depth information of the items at the corresponding timestamp based on the pixel position of the reported valid signal; and acquiring movement information of the conveyor belt as movement information of the items; and calculating the volume information of the items based on the depth information of the items at multiple timestamps and the movement information of the items.

[0013] Optionally, the moving direction of the conveyor belt is a first direction, and the length direction of the linear light is a second direction perpendicular to the first direction.

[0014] Optionally, the projection direction of the linear light is perpendicular to the conveyor belt surface or a tangent to the conveyor belt surface.

[0015] According to a third aspect of the present disclosure, a road condition detection method is provided, which is performed by a depth data measuring device as described in the first aspect, and includes: projecting linear light toward a road surface; imaging the projected area of ​​the road surface based on reflected light of the linear light, and acquiring a valid signal with a timestamp; calculating the depth information of the road surface at a corresponding timestamp based on the pixel position of the reported valid signal; acquiring motion information of a motion device on which the depth data measuring device is installed; and determining the road surface condition of the road surface position corresponding to each timestamp based on the depth information of the road surface at multiple timestamps and the motion information.

[0016] Optionally, the road surface is a railway road surface with a predetermined depth distribution pattern, and based on the depth information of the road surface at multiple timestamps, determining the road surface state of the road surface position corresponding to each timestamp includes: comparing the depth information of the road surface at multiple timestamps with the depth information of the corresponding road surface position of the predetermined depth distribution pattern to determine the road surface state; and / or synthesizing an actual road surface depth distribution based on the depth information of the road surface at multiple timestamps and the road surface position corresponding to each timestamp, and determining the road surface state based on the actual road surface depth distribution.

[0017] According to a fourth aspect of the present disclosure, a method for detecting surface defects of an object is provided, the method being performed by the depth data measuring device as described in the first aspect, and comprising: projecting linear light toward the surface of the object; imaging the projected area of ​​the surface of the object based on the reflected light of the linear light, and acquiring a valid signal with a timestamp; calculating the depth information of the surface of the object at the corresponding timestamp based on the pixel position of the reported valid signal; acquiring motion information of a motion device on which the depth data measuring device is installed; and determining whether there is a defect at the surface position of the object corresponding to each timestamp based on the depth information of the surface of the object at multiple timestamps and the motion information.

[0018] Optionally, the surface of the object has a predetermined contour, and based on the depth information of the object surface at multiple timestamps, determining whether there is a defect at the object surface position corresponding to each timestamp includes: comparing the depth information of the object surface at multiple timestamps with the predetermined contour of the corresponding object surface position to determine the road surface condition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0020] Figure 1A -B shows an example of projecting time-coded stripe structured light.

[0021] Figure 2 A schematic diagram showing the composition of a depth data measuring device according to an embodiment of the present invention is shown.

[0022] Figure 3 A schematic flow chart of a method for measuring volume information of items on a conveyor belt according to an embodiment of the present invention is shown.

[0023] Figure 4 A diagram showing, in one embodiment, measuring information of items on a conveyor belt using the depth data measuring device of the present invention.

[0024] Figure 5A -B shows an example of measuring information of items on a conveyor belt using a depth data measuring device.

[0025] Figure 6 A schematic flow chart of a road condition detection method according to an embodiment of the present invention is shown.

[0026] Figure 7 A schematic flow chart of a method for detecting surface defects of an object according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0028] In the field of computer vision technology that actively projects light and performs imaging based on the reflection of the projected light from the object to obtain depth information (three-dimensional information) of the object, structured light with a specific distribution pattern in the two-dimensional direction is usually selected.

[0029] Structured light technology is a technology that actively projects a known pattern into an imaging area. The information about the deformation of the structured light pattern caused by the surface of the object in the imaging area can be used to calculate the surface depth information of the object in the visual system. Structured light needs to be encoded. For example, a widely used structured light is stripe-coded structured light.

[0030] Figure 1A -B shows an example of projecting time-coded stripe structured light. The figure briefly illustrates the coding principle of stripe structured light using two gray levels and five bits of binary time coding. The projection device can sequentially project the following Figure 1A The five patterns shown in the figure are divided into 32 areas by two grayscales, light and dark. The first pattern includes only two areas, dark on the left and light on the right. The second pattern includes four areas from left to right, dark-light-dark-light, and so on, until the fifth pattern includes 32 areas formed by alternating dark and light.

[0031] In each pattern, each area corresponds to its own projection angle, where it can be assumed that the bright area corresponds to the code "1" and the dark area corresponds to the code "0". The code values ​​of a point on the object in the projection space in the five code patterns are combined in the order of projection to obtain the regional code value of the point, thereby determining the area where the point is located and then decoding to obtain the scanning angle of the point. Figure 1BA schematic diagram of encoding according to five patterns is shown. As shown in the figure, each pixel will obtain a code of either "1" or "0" in the five images, so that each pixel can obtain a five-bit binary code value. In a monocular imaging system (i.e., only one image sensor is included to image the five patterns), the decoding process may include comparison and calculation with the reference image code value. In a binocular imaging system, the above decoding process can be simplified by directly matching the code values ​​of each point in the first and second image sensors.

[0032] Obtained by scanning the line light on and off Figure 1A In the case of the five patterns shown, assuming that the time for projecting a pattern is 32ms, then in the first 32ms, along with the change of the scanning position, the linear light is first off for 16ms, then on for 16ms; in the second 32ms, along with the change of the scanning position, the linear light is first off for 8ms, then on for 8ms, then off for 8ms, then on for 8ms; in the third 32ms, along with the change of the scanning position, the linear light is first off for 4ms, then on for 4ms, and this is repeated 4 times; in the fourth 32ms, along with the change of the scanning position, the linear light is first off for 2ms, then on for 2ms, and this is repeated 8 times; in the fifth 32ms, along with the change of the scanning position, the linear light is first off for 1ms, then on for 1ms, and this is repeated 16 times. Five scanning patterns obtained in these 5 32ms

[0033] It should be understood that in order to improve the matching accuracy, the number of projected patterns in the time coding can be increased, such as 6, 7 or even more. For example, in the application scenario of binocular imaging, 10 time-coded stripe patterns means that, for example, each pixel in each left and right image frame contains 10 area coding values ​​that are either 0 or 1, thereby achieving left and right image matching with higher accuracy (for example, pixel level). When the projection rate of the projection device remains unchanged, compared with Figure 1A The example of five coded patterns and ten patterns is equivalent to achieving higher-precision image matching at a higher time domain cost (doubled).

[0034] In order to project stripe-coded structured light, a reflective mechanism is required to scan and project the linear light within a predetermined round-trip angle, and combine the corresponding on-off mode of the linear light to form multiple corresponding stripe patterns. The solution is complex and requires a lot of computing power. In view of this, the present invention proposes a depth data measurement solution that achieves extremely high depth information acquisition speed and extremely low computing power requirements by simply projecting linear light and combining an image sensor in which each pixel can independently generate an effective signal when the brightness changes in an application scenario where the measuring device and the imaging object have a relative speed.

[0035] Figure 2A schematic diagram of the composition of a depth data measuring device according to an embodiment of the present invention is shown. The left side of the figure shows a diagram of a depth data measuring device 200 projecting linear light to an imaging object with a relative motion relationship therewith and forming an image. The right side of the figure shows a stereoscopic image of the imaging object being scanned by the linear light. As shown on the left side of the figure, the measuring device 200 includes a projection component 210, an image sensor 220, and a processor 230.

[0036] The projection component 210 is used to project linear light onto an imaging object that is in relative motion. In the illustrated example, the imaging object moves at a first speed in a first direction (illustrated as the x direction) at a constant speed. The length direction of the linear light is a second direction (illustrated as the y direction) that is perpendicular to the first direction. Assume that the z direction is a direction perpendicular to both the x and y directions, that is, the vertical direction of the left figure. In the illustrated example, the projection component 110 projects the linear light in the z direction, that is, vertically downward. In other embodiments, the second direction serving as the length direction of the linear light may not be perpendicular to the direction of relative motion (as long as it is at a predetermined angle, that is, not parallel). Similarly, the linear light may not be in the z direction (as long as it is at a predetermined angle, that is, not parallel). Subsequently, the linear light will be combined with Figure 5A and Figure 5B This is illustrated by examples.

[0037] Here, the linear light projected by the projection assembly 110 itself does not move relative to other components and the housing of the depth data measuring device, that is, the projection assembly 110 does not include a reflection mechanism that can rotate to allow the linear light to scan within a predetermined angle. The linear light projected by the projection assembly 110 can image an entire imaging object (e.g., items on a conveyor belt) because there is relative motion between the depth data measuring device and the imaging object.

[0038] In different application scenarios, the relative motion between the depth data measurement device and the imaging object can be that the depth data measurement device is stationary and the imaging object is in motion; the imaging object is stationary and the depth data measurement device is in motion; or both are in motion, but the motion of the two is not in the same direction and at the same speed, that is, there is relative motion.

[0039] It can be seen that the first relative speed is the speed of the imaging object relative to the depth data measuring device, and includes: the speed of the imaging object when the depth data measuring device is stationary; the speed of the depth data measuring device when the imaging object is stationary; or the relative speed of the imaging object and the depth data measuring device when both the imaging object and the depth data measuring device are in motion and stationary. The following will describe the implementation of the imaging object movement in conjunction with the conveyor belt item volume information detection scheme; the implementation of the depth data measuring device movement will be described in conjunction with the road surface state detection and object surface defect detection schemes.

[0040] exist Figure 2 For example, the imaging object is an object being transported on a conveyor belt. The object has an irregular rectangular shape (more like a shape obtained by stitching together two rectangles of the same length but different heights and widths). The relative movement of the measuring device 200 and the imaging object is caused by the horizontal movement of the conveyor belt. That is, the measuring device 200 can be the lower part of a cantilever disposed above the conveyor belt, and projects a linear light downward. The linear light projected onto the surface of the imaging object is Figure 2 The left figure shows a point (because the left figure is a plan view from the y direction), while the right stereogram shows it as a line. Since the linear light is irradiated to the upper surface of the imaging object, there is a height difference between the upper surface of the imaging object and the surface of the conveyor belt, and there is a fixed baseline distance between the projection component 210 and the image sensor 220, even if the projection component 210 projects the linear light vertically downward, the image sensor 220, which actually has a parallax with the projection angle, still displays a broken line. There is a height difference between the straight line reflected from the upper surface of the object and the straight lines reflected from the conveyor belt on both sides. At this time, the height (H) and width (W) of the current position of the imaging object can be determined according to the position of the lit pixels on the imaging image. Specifically, the height (H) of the current position of the imaging object can be obtained according to the height difference between the central straight line and the straight lines on both sides, and the pixel positions of the central straight line and the straight lines on both sides. The width (W) corresponding to the current position of the imaging object can be obtained according to the width of the central straight line and the pixel position of the central straight line.

[0041] Since linear light is projected, there are very few pixels in the imaged image that actually carry depth information. For example, the image sensor 220 is shown as an 8x9=72 pixel sensor. When imaging the reflected linear light, only 9 pixels are illuminated (i.e., the number of pixels in 1 row). In actual use scenarios, the image sensor 220 usually has a higher pixel configuration, such as 600x800 pixels, and the reflected linear light can occupy more rows (e.g., 3x800=2400 pixels are illuminated) or columns (depending on the imaging direction), but most of the pixels in the imaged image still do not carry any depth information at all. However, in the case of using a conventional image sensor, the processor, for example, still needs to fully process the 600x800 pixel image and then extract the desired depth information from it. Since only 3x800 pixels of the 600x800 pixels contain valid information, the conventional depth information processing method causes a lot of waste of the processor's computing power.

[0042] To this end, the present invention uses a new type of image sensor. Specifically, although the image sensor 220 can be used to image the imaging object based on the reflected light of the linear light like a conventional image sensor, each pixel in the multiple pixels included in the image sensor can send a valid signal with a time stamp when the change in the amount of light received per unit time exceeds a threshold. Figure 2 In the example shown, since the current time t i Only @4x1, @4x2, @2x3, @2x4, @2x5, @2x6, @2x7, @4x8, and @4x9 are projected and cause the change in the amount of light received per unit time to exceed the threshold, so these 9 pixels can send a valid signal with a current timestamp.

[0043] Here, the unit time can be a very small time, such as a time period of 1 μs, and each pixel in the same image sensor can preferably calculate the unit time according to the same start and end time. The threshold can be set according to the application scenario, but it needs to be within the upper and lower limits that the image sensor can achieve. When the image sensor 220 actually has a pixel array of, for example, 800x600, each of the 480,000 pixels can independently generate and send a valid signal with a timestamp.

[0044] Corresponding to the imaging principle of the image sensor 220, in one embodiment, the projected linear light may be a constantly on laser, and the pixels at the reflection positions of the leading edge (from nothing to something) and the trailing edge (from something to nothing) of the linear light generate effective signals. In one embodiment, the projected linear light may be a laser pulse. The flashing frequency and pulse duration of the laser pulse may be reasonably selected according to the first relative speed, the threshold value, and the relative distance of the imaging object. In one embodiment, the projected linear light may be a high-frequency pulse light that flashes 20,000 times per second, thereby supporting high-speed surface defect detection.

[0045] After acquiring a valid signal, the processor 230 may calculate the surface depth information of the imaging object based on the pixel position and timestamp of the reporting valid signal and the first relative speed motion. The surface depth information of the imaging object may be used to acquire the volume information of the imaging object, for example, acquiring the length, width and height information.

[0046] For example, in Figure 2 In the example, if the conveyor belt moves at a speed of 0.5 m / s and the depth measurement device continuously obtains valid signals from the time stamps of 0 seconds to 1 second, it can be determined that the length L of the imaging object is 0.5 meters. And as mentioned above, the height (H) and width (W) of the imaging object at the current position can be determined based on the pixel position of the valid signal corresponding to each time stamp. Figure 2 In the example shown in FIG. 1 , since the height (H) of the front and rear of the imaged object is different, the height difference between the middle pixel and the pixels on both sides that report valid information will be different.

[0047] When the depth data measuring device of the present invention performs depth measurement on the imaging object, it is usually also necessary to know the value of the first relative speed in order to realize, for example, volume calculation of the imaging object, or to make corresponding depth annotations for the specific imaging position of the imaging object. At this time, the depth data measuring device also needs to include: an information acquisition device for acquiring the value of the first relative speed. In one embodiment, the speed information acquisition device may be a communication component for acquiring the value of the first relative speed from the outside, for example, acquiring the current motor speed of the conveyor belt. In another embodiment, when applied to a scene where the imaging object is stationary and the depth data measuring device is in motion, the speed information acquisition device itself may be a measuring device, for example, an accelerometer, for measuring the acceleration of the position, and thereby acquiring the movement speed of the depth data measuring device itself.

[0048] exist Figure 2What is shown in the example is a monocular depth data measurement device, that is, the measurement device includes only one image sensor. In a monocular setting, an additional calibration operation of a reference image is required. In other embodiments, the image sensor includes: a first image sensor and a second image sensor arranged on both sides of the projection assembly, wherein the processor calculates the depth information of the imaging object using the pixel position and timestamp of the reported valid signals from the first image sensor and the second image sensor, and the first relative speed motion. The binocular scheme uses the parallax of the signals included in the first image sensor and the second image sensor to perform depth calculation, and no calibration operation for the reference image is required, so it has a wider range of applications.

[0049] The depth data measuring device of the present invention can be used in various scenarios where there is a relative displacement between the measuring device and the imaging object.

[0050] Figure 3 A schematic flow chart of a method for measuring volume information of items on a conveyor belt according to an embodiment of the present invention is shown. The method can be executed by the depth data measuring device of the present invention as described above.

[0051] In step S310, a linear light is projected toward an item on a conveyor belt. In step S320, the item is imaged based on the reflected light of the linear light, and a valid signal with a timestamp is obtained. In step S330, the depth information of the item at the corresponding timestamp is calculated based on the pixel position of the valid signal. In step S340, the motion information of the conveyor belt is obtained as the motion information of the item. In step S350, the volume information of the item is calculated based on the depth information of the item at multiple timestamps and the motion information of the item.

[0052] It should be understood that it takes a certain amount of time for an object to pass through the area projected by the linear light. For example, when the conveyor belt moves at 0.5 m / s, a 25 cm long package placed upright takes 0.5 seconds to complete the scanning of the linear light, that is, the time when the front edge of the package passes through the linear light differs from the time when the rear edge of the package passes through the linear light by 0.5 seconds. In other words, the depth data measurement device of the present invention requires at least the time for an object to pass through the linear light to measure the volume of the object.

[0053] For ease of understanding, Figure 4 FIG. 1 shows a diagram of measuring information of items on a conveyor belt using a depth data measuring device of the present invention in one embodiment. Figure 4As shown, the depth data measuring device 410 is mounted on a cantilever 440 and is used to image the items 430 on the conveyor belt 420. In the illustrated example, the depth data measuring device 410 projects linear light toward the z direction (i.e., the vertical direction), and the three items on the left (e.g., three express boxes) have completed the measurement, and the fourth item will start measuring later. Since no items pass through the current projection position, the projected linear light is displayed as a straight line. Correspondingly, the pixels that also report valid signals in the image sensor of the measuring device 410 should also be located in a specific row (or a column, depending on the placement direction). Only a specific row of pixels reporting valid signals can be identified as no measurement of the items on the conveyor belt. And only when the pixels reporting valid signals in the image sensor of the measuring device 410 are located in different rows, the length, width and height of the corresponding items are calculated.

[0054] Figure 4 In the preferred example shown, the moving direction of the conveyor belt is a first direction, and the length direction of the linear light is a second direction perpendicular to the first direction. The projection direction of the linear light is perpendicular to the surface of the conveyor belt or the tangent of the surface of the conveyor belt (when the conveyor belt has a certain curvature). In other examples, the length direction or projection direction of the linear light can also be other directions.

[0055] Figure 5A -B shows an example of using a depth data measuring device to measure information about items on a conveyor belt. Figure 5A In the example, the length direction of the linear light is still the second direction perpendicular to the first direction of the conveyor belt. However, the projection direction of the linear light is not perpendicular to the surface of the conveyor belt, but has a certain angle α with the z direction. At this time, the length, width and height of the object can be calculated by making corresponding transformations based on this angle. Similarly, in Figure 5B In the example, not only is the projection direction of the linear light not perpendicular to the conveyor belt surface (i.e., there is an angle α with the z direction), but the length direction of the linear light is also not perpendicular to the first direction of the conveyor belt movement, but there is an angle β with the y direction. At this time, when calculating the length, width and height of the item, it is necessary to perform a transformation based on the values ​​of α and β.

[0056] As described above, an implementation example in which the measuring device is fixed and the imaging object is moving is given. The measuring device of the present invention can also be applied to a scene in which the imaging object is fixed and the measuring device itself is moving.

[0057] Figure 6 A schematic flow chart of a road condition detection method according to an embodiment of the present invention is shown. The method can be executed by the depth data measurement device of the present invention as described above.

[0058] In step S610, a linear light is projected toward a road surface. In step S620, the road surface is imaged based on the reflected light of the linear light, and a valid signal with a timestamp is obtained. In step S630, the depth information of the road surface at the corresponding timestamp is calculated based on the pixel position of the reported valid signal. In step S640, motion information (e.g., acceleration information) of a motion device equipped with the depth data measuring device is obtained. Here, the motion information (e.g., acceleration information) can be used to obtain the road surface position corresponding to each timestamp. In step S650, the road surface state of the road surface position corresponding to each timestamp is determined based on the depth information and motion information of the road surface at multiple timestamps.

[0059] In this method, a depth data measuring device can be installed on the bottom of the test vehicle and measure the road surface condition during the test measurement driving process.

[0060] In one embodiment, the road surface can be a highway road surface, such as a highway road surface. At this time, the measuring device only needs to measure the flatness of the road surface. For example, under normal circumstances, the pixel positions reporting valid signals should be located in the same row, or in several adjacent rows. If there are several pixels reporting valid signals on higher or lower rows, it means that there are potholes or bumps on the road surface and they need to be repaired. At this time, the timestamp of the valid signal corresponding to the pothole or bump can be recorded, and the road surface position corresponding to the timestamp can be calculated based on the speed information (for example, the acceleration information measured by the accelerometer), and repairs can be performed later.

[0061] In one embodiment, the road surface is a railway road surface with a predetermined depth distribution pattern. The depth data measuring device of the present invention can be installed at the bottom of a train or at the bottom of a special rail test vehicle to detect the road surface condition. At this time, according to the depth information of the road surface at multiple timestamps, determining the road surface condition of the road surface position corresponding to each timestamp includes: comparing the depth information of the road surface at multiple timestamps with the depth information of the corresponding road surface position of the predetermined depth distribution pattern to determine the road surface condition; and / or synthesizing the actual road surface depth distribution according to the depth information of the road surface at multiple timestamps and the road surface position corresponding to each timestamp, and determining the road surface condition according to the actual road surface depth distribution. Since there are ups and downs of sleepers and road surfaces on railway roads, when detecting the railway road surface condition, it is necessary to detect whether the surface distribution synthesized based on the depth information conforms to the railway surface distribution.

[0062] The depth data measuring device of the present invention can also be used for detecting surface defects of other objects, especially large objects (such as large ships). Figure 7 A schematic flow chart of a method for detecting surface defects of an object according to an embodiment of the present invention is shown. The method can be executed by the depth data measuring device of the present invention as described above.

[0063] In step S710, linear light is projected toward the surface of the object. In step S720, the surface of the object is imaged based on the reflected light of the linear light, and a valid signal with a timestamp is obtained. In step S730, the depth information of the surface of the object at the corresponding timestamp is calculated based on the pixel position of the reported valid signal. In step S740, motion information of a motion device equipped with the depth data measuring device is obtained. Here, the motion information (such as acceleration information) can be used to obtain the position information of the object surface corresponding to each timestamp. In step S750, based on the depth information of the object surface at multiple timestamps, it is determined whether there is a defect on the surface of the object corresponding to each timestamp.

[0064] The surface of the object to be measured may have a predetermined profile. In this case, the depth information of the object surface at multiple time stamps may be compared with the predetermined profile of the corresponding object surface position to determine the road surface state. In other embodiments, if the measured object surface should have a smooth distribution, the presence of pits and protrusions may be determined by pixel position and defects may be detected.

[0065] A depth data measurement device and an application method thereof according to the present invention have been described in detail above with reference to the accompanying drawings. This scheme uses linear light to perform active light imaging on an imaging object with relative motion, and is combined with an image sensor in which each pixel can independently generate a valid signal when the brightness changes. It is possible to image only the valid signal that has changed, and use timestamps and relative motion information to determine the position of the depth data, thereby achieving high-speed depth information acquisition and processing with a simple projection mechanism and extremely low computing power requirements.

[0066] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A depth data measuring device, comprising: A projection component, used for projecting linear light onto an imaging object in relative motion; An image sensor, used for imaging the imaging object based on the reflected light of the linear light, and sending a valid signal with a time stamp when the change of the amount of light received by each pixel per unit time exceeds a threshold value; as well as A processor is used to calculate the surface depth information of the imaged object based on the pixel position and time stamp of the reported valid signal and the relative motion information.

2. The measuring device according to claim 1, wherein: The first relative speed is the speed of movement of the imaging object relative to the depth data measuring device, and includes: The moving speed of the imaging object when the depth data measuring device is stationary; The movement speed of the depth data measuring device when the imaging object is stationary; or When the imaging object and the depth data measuring device are both in motion, the relative motion speed of the imaging object and the depth data measuring device.

3. The measuring device according to claim 2, wherein: The depth data measuring device also includes: The speed information acquiring device is used to acquire the first relative speed information.

4. The measuring device according to claim 1, wherein: The image sensor comprises: A first image sensor and a second image sensor are arranged on both sides of the projection assembly, The processor calculates depth information of the imaged object using pixel positions and timestamps of reporting valid signals from the first image sensor and the second image sensor, and the first relative speed motion.

5. The measuring device according to claim 1, wherein: The projection component projects linear light pulses at a predetermined frequency.

6. The measuring device according to claim 1, wherein: The first relative speed The measuring device according to claim 1, wherein the linear light projection direction of the projection assembly remains unchanged with other components and the housing of the depth data measuring device.

7. A method for measuring volume information of items on a conveyor belt, the method being performed by the depth data measuring device according to any one of claims 1 to 6, and comprising: Project a line light towards the items on the conveyor belt; Imaging the object based on the reflected light of the linear light, and acquiring a valid signal with a time stamp; Calculate the depth information of the object at the corresponding timestamp based on the pixel position reporting the valid signal; as well as Acquiring movement information of the conveyor belt as movement information of the object; as well as The volume information of the object is calculated according to the depth information of the object at multiple time stamps and the motion information of the object.

8. The method of claim 7, wherein: The moving direction of the conveyor belt is a first direction, and the length direction of the linear light is a second direction perpendicular to the first direction.

9. The method of claim 7, wherein: The projection direction of the linear light is perpendicular to the surface of the conveyor belt or a tangent line of the surface of the conveyor belt.

10. A method for detecting road conditions, the method being performed by the depth data measuring device according to any one of claims 1 to 6, and comprising: Projecting linear light toward the road surface; Imaging the road surface based on the reflected light of the linear light, and acquiring a valid signal with a time stamp; Calculating the depth information of the road surface at the corresponding time stamp based on the pixel position reporting the valid signal; Acquiring motion information of the depth data measuring device; as well as The road surface state at the road surface position corresponding to each time stamp is determined based on the depth information of the road surface at the multiple time stamps and the motion information.

11. The method of claim 10, wherein: The road surface is a railway road surface having a predetermined depth distribution pattern, and according to the depth information of the road surface at multiple time stamps, determining the road surface state of the road surface position corresponding to each time stamp includes: Comparing the depth information of the road surface at multiple time stamps with the depth information of the corresponding road surface position of the predetermined depth distribution pattern to determine the road surface state; and / or An actual road surface depth distribution is synthesized based on the depth information of the road surface at multiple time stamps and the road surface position corresponding to each time stamp, and a road surface state is determined based on the actual road surface depth distribution.

12. A method for detecting surface defects of an object, the method being performed by the depth data measuring device according to any one of claims 1 to 6, and comprising: Project linear light toward the surface of an object; Imaging the surface of the object based on the reflected light of the linear light, and acquiring a valid signal with a time stamp; Calculating the depth information of the surface of the object at the corresponding timestamp based on the pixel position reporting the valid signal; Acquiring motion information of the depth data measuring device; as well as According to the depth information of the object surface at multiple time stamps and the motion information, it is determined whether there is a defect at the position of the object surface corresponding to each time stamp.

13. The method of claim 12, wherein: The object surface has a predetermined contour, and according to the depth information of the object surface at multiple time stamps, determining whether there is a defect at the position of the object surface corresponding to each time stamp includes: The depth information of the object surface at multiple time stamps is compared with a predetermined profile corresponding to the object surface position to determine the road surface state.

Images