Angle measurement method and device of iTOF camera, medium and equipment

Abstract

The invention relates to the technical field of laser measurement, in particular to an angle measurement method and device of an iTOF camera, a medium and equipment. The method comprises the following steps: arranging at least two retroreflection targets on the surface of a target object to be measured in a direction parallel to the horizontal direction; acquiring signal data reflected by each retro-reflection target by using an iTOF camera, and reconstructing according to the signal data to obtain point cloud data in each retro-reflection target area; calculating the center of mass of each retroreflection target according to the point cloud data in each retroreflection target area; and calculating the relative angle between the to-be-measured target object and the iTOF camera according to the mass centers of the two retroreflection targets. According to the invention, the retroreflection target is arranged on the surface of the to-be-measured target, and the retroreflection target has stronger reflected light in a specific direction than the surface of the to-be-measured target, so that the effective light energy collected by the iTOF camera is enhanced, the ratio of invalid light is reduced, and the accuracy of the measurement result of the iTOF camera is improved.

Description

Technical Field

[0001] The present invention relates to the field of laser measurement technology, and in particular to an angle measurement method, device, medium and equipment for an iTOF camera. Background Art

[0002] At present, iTOF (indirect Time-of-Flight) cameras have been widely used in many fields. However, the current iTOF (indirect Time-of-Flight) cameras generally have the following defects in angle measurement, distance measurement and other measurement applications:

[0003] (1) The iTOF camera is used with a Flash array light source. In order to ensure the luminous efficiency of the laser light source and meet the human eye safety indicators, a higher-power laser cannot be selected. In order to ensure the sampling frame rate and dynamic response efficiency of the device, a longer exposure time cannot be configured. This results in a low upper limit for iTOF ranging and limited application scenarios. Currently, the ranging capabilities of most iTOF products on the market are basically within 5m.

[0004] (2) The iTOF camera uses Flash for one-time exposure imaging, and the edge of the target object is prone to have large flying points on the point cloud, which affects the accuracy of the measurement results.

[0005] (3) When there are two or more targets with similar distance, surface reflectivity and material, it is difficult to distinguish them in the depth and amplitude diagrams of the iTOF camera, which can easily lead to confusion when selecting the measurement object, affecting the stability of the measurement results.

[0006] (4) iTOF cameras usually do not have the ability to withstand strong light. Currently, most iTOF products on the market have serious degradation in measurement performance in outdoor high-illuminance environments. Especially under long exposure times, the ambient light component increases sharply, and there are more ambient light noise points on the image after imaging than indoors. The overall number of effective pixels decreases, affecting the accuracy of the measurement results. Summary of the invention

[0007] In view of this, the present invention provides an angle measurement method, device, medium and equipment for an iTOF camera to solve the problem in the prior art that the accuracy of the iTOF camera measurement results is easily affected.

[0008] In a first aspect, the present invention provides an angle measurement method for an iTOF camera, the method comprising: setting at least two retroreflective targets on the surface of a target to be measured in a direction parallel to the horizontal direction; using an iTOF camera to obtain signal data reflected by each retroreflective target, and reconstructing point cloud data within each retroreflective target area based on the signal data; calculating the center of mass of each retroreflective target based on the point cloud data within each retroreflective target area; and calculating the relative angle between the target to be measured and the iTOF camera based on the center of mass of the two retroreflective targets.

[0009] In the present invention, a retroreflective target is set on the surface of the target to be measured. The retroreflective target reflects stronger light in a specific direction than the surface of the target to be measured, thereby enhancing the effective light energy collected by the iTOF camera, thereby reducing the proportion of invalid light and improving the accuracy of the iTOF camera measurement results. At the same time, by setting at least two retroreflective targets on the surface of the target to be measured in a direction parallel to the horizontal direction, and calculating the centroid of each retroreflective target based on the iTOF camera measurement data, the relative angle between the target to be measured and the iTOF camera can be calculated through the centroids of the two retroreflective targets.

[0010] In an optional embodiment, before using the iTOF camera to collect the light signal reflected by the retroreflective target, the method also includes: configuring the exposure time of the iTOF camera according to the distance between the camera and the target to be measured; and setting the confidence filtering parameters of the iTOF camera according to the difference between the reflected light intensity of the material in the retroreflective target area and the reflected light intensity of the material in the area outside the retroreflective target on the surface of the target to be measured.

[0011] In the present invention, by setting a retroreflective target on the surface of the target to be measured, the exposure time can be reconfigured, and the exposure time is reduced compared to when the retroreflective target is not set. At the same time, due to the difference between the reflection energy of the retroreflective target and the reflection energy of the diffuse reflection target to be measured, the confidence filter parameters can be further reset, and the confidence filter parameters are improved compared to when the retroreflective target is not set, thereby filtering out as much environmental noise as possible. Therefore, the present invention can not only ensure the detection of the target, but also reduce the proportion of ambient light in the sampling signal, and alleviate the interference of ambient light on the measurement process.

[0012] In an optional embodiment, an iTOF camera is used to obtain signal data reflected by a retroreflective target, including: using the iTOF camera to collect signal data reflected by the surface of the target to be measured based on the configured exposure time; filtering the signal data reflected by the surface of the target to be measured based on the confidence filter parameters to obtain filtered data; obtaining a corresponding grayscale image based on the filtered data; and obtaining signal data reflected by the retroreflective target according to segmentation of the grayscale image.

[0013] In the present invention, the signal data is collected and processed by setting the exposure time and confidence filtering parameters, thereby realizing accurate extraction of the retroreflective target area data.

[0014] In an optional implementation, the centroid is determined based on the sum of all point cloud data within the retroreflective target area; and the relative angle is determined based on the slope between the centroids of two retroreflective targets.

[0015] In an optional embodiment, when the target to be measured includes multiple surfaces that need to perform angle measurement, at least two retroreflective targets are set in a direction parallel to the horizontal direction on the surface of the target to be measured, including: at least two retroreflective targets are set in a direction parallel to the horizontal direction on each surface of the target to be measured, and the shapes of the retroreflective targets on different surfaces are different.

[0016] In the present invention, by arranging retroreflective targets of different shapes on different surfaces, a data basis is provided for identifying different surfaces.

[0017] In an optional embodiment, an iTOF camera is used to obtain signal data reflected by a retroreflective target, including: placing the iTOF camera at a preset distance in front of each surface to obtain signal data reflected by the retroreflective target on each surface; using a pre-trained machine learning model to identify the signal data corresponding to each surface, and determine the surface to which each signal data corresponds.

[0018] In the present invention, by using a pre-trained machine learning model to identify the signal data corresponding to each surface, accurate identification of retroreflective targets of different shapes can be achieved.

[0019] In an optional embodiment, when the target object to be measured is a semi-trailer, the multiple surfaces that need to be measured for angles include a front side, a first side side, and a second side side, and the relative angle between the target object to be measured and the iTOF camera is calculated according to the centroid of two retro-reflective targets, including: calculating the first relative angle between the front side and the iTOF camera, the second relative angle between the first side side and the iTOF camera, and the third relative angle between the second side side and the iTOF camera according to the centroid of the two retro-reflective targets on each surface; converting the second relative angle and the third relative angle according to the coordinate system corresponding to the first relative angle to obtain the converted fourth relative angle and fifth relative angle; and automatically guiding the semi-trailer to reverse according to the first relative angle, the fourth relative angle, and the fifth relative angle.

[0020] In the present invention, the relative angle between each surface and the camera is calculated, thereby providing accurate and safe reversing guidance for the automatic navigation system; at the same time, the consistency and stability of the output angle are improved through angle conversion.

[0021] In a second aspect, the present invention provides an angle measurement device for an iTOF camera, the device comprising: a target modification module, used to set at least two retroreflective targets on the surface of the target to be measured in a direction parallel to the horizontal direction; a signal processing module, used to use the iTOF camera to obtain signal data reflected by each retroreflective target, and reconstruct point cloud data within each retroreflective target area based on the signal data; a centroid calculation module, used to calculate the centroid of each retroreflective target based on the point cloud data within each retroreflective target area; an angle calculation module, used to calculate the relative angle between the target to be measured and the iTOF camera based on the centroids of the two retroreflective targets.

[0022] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the angle measurement method of the iTOF camera of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0023] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the angle measurement method of an iTOF camera of the first aspect or any corresponding embodiment thereof.

[0024] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the angle measurement method of an iTOF camera according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 is a flow chart of an angle measurement method of an iTOF camera according to an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of a diffuse reflection (Lambertian) target object photographed by a laser radar according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of a laser radar photographing a retroreflective target according to an embodiment of the present invention;

[0029] Figure 4is a schematic diagram of the detectable limit of diffuse reflection and retroreflection targets according to an embodiment of the present invention;

[0030] Figure 5 is a schematic diagram of ambient light captured by a laser radar according to an embodiment of the present invention;

[0031] Figure 6 is a diagram showing simulation results of laser energy reflected by targets of different reflective materials according to an embodiment of the present invention;

[0032] Figure 7 is a schematic diagram of a curve showing the relationship between exposure time and light intensity when an iTOF camera according to an embodiment of the present invention photographs a 90% diffuse reflection target and a retro-reflection target at a distance of 5 m;

[0033] Figure 8 is a schematic diagram of a specular reflection material according to an embodiment of the present invention;

[0034] Fig. 9 is a flow chart of an angle measurement method of another iTOF camera according to an embodiment of the present invention;

[0035] Fig.10 is a schematic diagram of setting a retroreflective target according to an embodiment of the present invention;

[0036] Fig.11 is a simulated grayscale image according to an embodiment of the present invention;

[0037] Fig.12 is a point cloud image of a simulated retroreflective target according to an embodiment of the present invention;

[0038] Fig.13 is a schematic flow chart of another angle measurement method of an iTOF camera according to an embodiment of the present invention;

[0039] Fig.14 is a schematic diagram of the iTOF camera setting position according to an embodiment of the present invention;

[0040] Fig.15 is a schematic diagram of retroreflective target images at different angles according to an embodiment of the present invention;

[0041] Fig.16 is a schematic diagram of two noodles being captured simultaneously by a camera during the reversing process according to an embodiment of the present invention;

[0042] Fig.17 is a schematic diagram of conversion of calculation results of the front and left side angles of a vehicle body according to an embodiment of the present invention;

[0043] Fig.18 is a structural block diagram of an angle measurement device of an iTOF camera according to an embodiment of the present invention;

[0044] Fig.19It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] According to an embodiment of the present invention, an embodiment of an angle measurement method of an iTOF camera is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0047] In this embodiment, a method for measuring the angle of an iTOF camera is provided. Figure 1 : is a flow chart of an angle measurement method of an iTOF camera according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0048] Step S101, at least two retro-reflective targets are set on the surface of the target object to be measured in a direction parallel to the horizontal direction. The target object to be measured is a diffuse reflection (Lambertian) target, that is, when a single beam of light is irradiated on the surface of the target object to be measured, a beam of light reflected in all directions will be formed, or a radiation solid angle Ω of 4π will be formed. r The retroreflective target is made of retroreflective material, which is a material that can produce specular reflection, that is, when a single beam of light is irradiated on the retroreflective material, the reflected light beam will follow the law of reflection and reflect in a specific direction, rather than radiating in all directions. Setting a retroreflective target in a direction parallel to the horizontal direction on the surface of the target to be measured means that the retroreflective target is set on the same horizontal line on the surface of the target to be measured.

[0049] Step S102: using an iTOF camera to acquire signal data reflected by each retroreflective target, and reconstructing point cloud data within each retroreflective target area based on the signal data.

[0050] Among them, the iTOF camera can be used with a laser light source such as an array light source. Specifically, the iTOF camera modulates the light waves emitted by the laser light source into a periodic signal of a certain frequency, such as a sine wave or a square wave, and measures the phase difference between the emitted signal and the signal when it is reflected by the target object to be measured and reaches the iTOF camera. The sensor collects the proportional relationship of energy values ​​in different time windows, analyzes the signal phase, and then indirectly calculates the flight time of light, and then converts the distance of the photographed scene based on information such as the speed of light to generate depth information.

[0051] In this embodiment, a planar light source is used to transmit a light signal modulated by an iTOF camera to the retroreflective target position of the target to be measured. The light signal is reflected by the retroreflective target, and the iTOF camera collects the reflected light signal for processing. When the iTOF camera processes the received signal, it can calculate the distance information of each position point of the reflected light signal according to the phase difference between the transmitted signal and the received signal, and obtain a depth map based on the distance information. Then, the camera's intrinsic parameters and depth map are used to convert each pixel point in the depth map into a corresponding three-dimensional coordinate through a projection equation. Then, the three-dimensional coordinates of the corresponding pixel points in the retroreflective target area are combined together to obtain point cloud data in the retroreflective target area.

[0052] Step S103, calculating the centroid of each retroreflective target based on the point cloud data in each retroreflective target area. Specifically, the mass of the retroreflective target can be ignored, so the centroid of each retroreflective target can be determined by the sum of all the point cloud data in the corresponding retroreflective target area. For example, the point cloud sets in two retroreflective target areas are {(x1, y1, z1), ..., (x n ,y n ,z n )} and {(x1,y1,z1),…,(x m ,y m ,z m )}, then the centroid of the two retroreflective targets (X1 centric ,Y1 centric ,Z1 centric ) and (X2 centric ,Y2 centric ,Z2 centric ) can be calculated using the following formulas:

[0053]

[0054] Step S104, calculate the relative angle between the target object to be measured and the iTOF camera according to the centroids of the two retroreflective targets. Specifically, when calculating the angle, the centroids of any two retroreflective targets can be selected first and calculated according to the following formula. The slope k between the two centroids can be calculated using the following formula:

[0055]

[0056] The slope is then converted into an angle or radian to obtain the relative angle between the target to be measured and the iTOF camera. It should be noted that the relative angle can be understood as the angle between the straight line formed by connecting the centroids of any two retroreflective targets and the X-axis in the plane where the centroids of the iTOF camera and any two retroreflective targets are located and the plane rectangular coordinate system is established with the iTOF camera as the origin and the direction parallel to the horizontal plane of the front surface of the camera.

[0057] In addition, in order to improve the accuracy of the calculation, when multiple retroreflective targets are set on the surface of the target to be measured, the relative angles corresponding to any two retroreflective targets can be calculated separately, and then the calculated multiple relative angles are averaged to obtain the relative angle between the target to be measured and the iTOF camera.

[0058] The angle measurement method of the iTOF camera provided in the embodiment of the present invention sets a retroreflective target on the surface of the target to be measured. The retroreflective target reflects stronger light in a specific direction than the surface of the target to be measured, thereby enhancing the effective light energy collected by the iTOF camera, thereby reducing the proportion of invalid light and improving the accuracy of the iTOF camera measurement results. At the same time, by setting at least two retroreflective targets on the surface of the target to be measured in a direction parallel to the horizontal direction, and calculating the centroid of each retroreflective target based on the iTOF camera measurement data, the relative angle between the target to be measured and the iTOF camera can be calculated through the centroids of the two retroreflective targets.

[0059] In addition, this embodiment further analyzes the reflected light energy of retro-reflective objects and diffuse reflective objects by means of theoretical calculation. Figure 2 The figure shows the optical system diagram of the laser radar photographing diffuse reflection targets. The system mainly consists of a laser emission system, an atmospheric medium, a diffuse reflection target (standard Lambertian body), and a laser receiving system. Taking a single-beam laser as an example, its propagation and energy change process in the system are as follows:

[0060] 1). After a single beam laser with an optical power of P is emitted from the transmitting system, it is collimated by the transmitting end optical system and projected onto the diffuse reflection target after a certain propagation distance R. Since a single beam laser has a very small divergence angle θ t , a diameter of approximately R·θ is formed on the diffuse reflection target. t Assume that the transmittance of the output optical system is η t , the atmospheric transmittance is η a , then the light energy projected by a single laser beam onto the diffuse reflection target during this process is:

[0061]

[0062] Among them, A receive is the illuminated area of ​​the diffuse reflection target, A emmiter is the area of ​​the single laser beam projected on the target. Assuming that the target is a plane with infinite area and the laser collimation is good, the illuminated area A of the diffusely reflected target in this system is receive Equal to the single beam laser projection area A emmiter , formula (1) is simplified to:

[0063] P1=P·η t ·η a (2)

[0064] 2) When the target is a standard Lambertian body, a single laser beam projected onto the diffuse reflective target will reflect the target at a specific reflectivity η. ref Reflects uniformly in all directions, forming a radiation solid angle Ω of 4π r , when the target plane is large enough, the solid angle Ω r =2π. Therefore, only part of the reflected light can propagate through the atmosphere to the laser receiving system again. In this process, the energy of the reflected light reaching the receiving end lens is:

[0065]

[0066] Among them, A lens is the area of ​​the lens in the receiving end optical system, and the diameter of the lens is D lens ; A reflection is the radiation area of ​​the reflected light from the Lambertian body at the laser receiving system. The solid angle Ω of the plane Lambertian body is known. r =2π, and the solid angle calculation formula is:

[0067]

[0068] A is the imaging area of ​​the light radiation at the receiving system, and R is the distance the light radiation propagates. The radiation area of ​​the Lambertian body at the laser receiving system is calculated as follows:

[0069] A reflection =Ω r ·R 2 =2π·R 2 (5)

[0070] In summary, formula (3) can be written as:

[0071]

[0072] 3). After the reflected light passes through the lens and filter, it is finally imaged on the receiving chip. After this process, the light energy becomes:

[0073]

[0074] Among them, η r is the transmittance of the receiving optical system. Assume Figure 1 The photosensitive chip in the lens is an inscribed square, so its length and width are Therefore, formula (7) can be written as:

[0075]

[0076] In summary, the laser radar system emits a single laser beam to illuminate the diffuse reflection target and the returned optical power P lambertian Can be written as:

[0077]

[0078] Where P is the single beam laser power, η t and η r is the transmittance of the emitting and receiving optical systems, η a is the atmospheric transmittance, η ref is the reflectivity of the diffuse reflection target, D lens is the diameter of the optical system lens, and R is the distance from the laser radar system to the target. From formula (9), it can be seen that the lower the target reflectivity, the lower the received light energy; the returned light energy is inversely proportional to the square of the distance to the target being measured. The farther the distance, the lower the received light energy. Therefore, the ability of using iTOF cameras to detect diffuse reflection targets is very limited.

[0079] Figure 3 The figure shows the optical system diagram of the laser radar photographing the retroreflective target. The system mainly consists of the laser emission system, the atmospheric medium, the retroreflective target, and the laser receiving system. The propagation and energy change process of a single beam laser in this system are as follows:

[0080] 1). After a single beam laser with an optical power of P is emitted from the transmitting system, it is collimated by the transmitting end optical system and projected onto the diffuse reflection target after a certain propagation distance R. Since a single beam laser has a very small divergence angle θ t , forming a diameter of approximately R·θ on the target t The light spot corresponds to a solid angle Ω t for:

[0081]

[0082] Assume that the transmittance of the output optical system is η t , the atmospheric transmittance is η a , then the light energy projected onto the target by a single laser beam during this process is:

[0083]

[0084] Among them, A receive is the illuminated area of ​​the target, A emmiter is the area of ​​a single laser beam projected on the target. When the target is a plane with infinite area, the illuminated area A of the target receive Equal to the single beam laser projection area A emmiter , formula (11) is simplified to:

[0085] P1=P·η t ·η a (12)

[0086] 2) Since the target is made of retro-reflective material, a single laser beam will not radiate in all directions after being projected onto it, but will be totally reflected in a specific direction according to the law of reflection (equivalent to η ref =1), its reflection solid angle Ω r Equal to the exit solid angle Ω t In this process, the energy of the reflected light reaching the lens of the receiving optical system is:

[0087]

[0088] Among them, A lens A is the area of ​​the lens in the optical system at the receiving end; reflection is the projection area of ​​the mirror-reflected beam at the laser receiving system:

[0089]

[0090] In summary, formula (14) can be written as:

[0091]

[0092] 3). After the reflected light passes through the lens and filter, it is finally imaged on the receiving chip. After this process, the light energy becomes:

[0093]

[0094] Among them, η r To receive the transmittance of the optical system, the length and width of the photosensitive chip are Therefore, formula (16) can be written as:

[0095]

[0096] In summary, the laser radar system emits a single laser beam to illuminate the retroreflective target and the returned optical power P mirror Can be written as:

[0097]

[0098] Where P is the single beam laser power, η t and η r is the transmittance of the emitting and receiving optical systems, η a is the atmospheric transmittance, D lens is the diameter of the receiving optical system lens, R is the distance from the laser radar system to the target, θ t is the single beam laser divergence angle.

[0099] Combining equations (9) and (18), we can compare and obtain the difference in light energy returned by the same laser beam after irradiating the diffuse reflection and retroreflection targets at the same distance:

[0100]

[0101] In an iTOF camera, the average single beam divergence angle is much less than 1° and the reflectivity of a diffuse reflective target is usually less than 1. Therefore, at the same distance, the energy returned by a retroreflective target is at least 8 times that of a diffuse reflective target. According to the inverse square rule of amplitude-distance, the detection limit of a retroreflective target captured by an iTOF camera can be improved by at least times, the effect difference is as follows Figure 4 shown.

[0102] Figure 2 and Figure 3 This is the case where the laser radar actively receives the signal. In the case of sufficient sunlight, there is also the following situation: Figure 5 The figure shows the passive reception of ambient light. The outdoor environment is mostly diffuse reflection objects. The propagation and energy change process of sunlight in this system is as follows:

[0103] 1). The light energy of sunlight shining on the diffuse reflection target object is mainly related to the atmospheric transmittance and the area of ​​the illuminated object:

[0104]

[0105] Among them, E sun is the sunlight illumination, 100kLUX illumination is about 1000W / m 2 .

[0106] 2) When the target is a standard Lambertian body, the ambient light will be projected onto the diffuse reflective target according to the specific reflectivity η. ref The reflected light is reflected uniformly in all directions, and the energy of the reflected light reaching the receiving end lens is:

[0107]

[0108] 3). After the reflected light passes through the lens and narrow-band filter, it is finally imaged on the receiving chip. In this process, the light energy is converted into:

[0109]

[0110] In summary, the optical power P passively received by the laser radar receiving system after sunlight irradiates the diffuse reflection target ambient Can be written as:

[0111]

[0112] After reducing the fraction, we get the following formula:

[0113]

[0114] Among them, E sun is the sunlight intensity, η t and η r is the transmittance of the emitting and receiving optical systems, η a is the atmospheric transmittance, η ref is the reflectivity of the diffuse reflection target, η filter is the transmittance of the narrowband filter relative to sunlight, D lens is the diameter of the lens of the receiving optical system. The ambient light energy passively received by the system has nothing to do with the distance between the radar and the target, and the sunlight illumination E sun , lens and its angle, and exposure time. The stronger the sunlight, the higher the ambient light energy received by the iTOF camera; the longer the exposure time, the higher the level of ambient light noise accumulated by the camera.

[0115] In this embodiment, a method for measuring the angle of an iTOF camera is provided, and the method comprises the following steps:

[0116] Step S201, at least two retro-reflective targets are set on the surface of the target object to be measured in a direction parallel to the horizontal direction. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0117] Step S202, configuring the exposure time of the iTOF camera according to the distance to the target to be measured; setting the confidence filter parameters of the iTOF camera according to the difference between the reflected light intensity of the material in the retroreflective target area and the reflected light intensity of the material in the area outside the retroreflective target on the surface of the target to be measured.

[0118] Specifically, according to the theoretical analysis results of the reflected light energy, if the relevant parameter P = 6W, η a =0.8,η t =η r =0.8,η filter =2e -3 ,θ t =0.3°,D lens= 10mm, the distance between the simulated target and the laser radar moves from 1 to 100m, and the simulation results of the received laser energy are as follows Figure 6 As shown. From the simulation results, it can be seen that there is a significant difference between the light intensity received by the chip for the retroreflective target and the 10%-90% diffuse reflection target. The energy received by the chip for the retroreflective target at the same distance is 99 times that of the 90% diffuse reflection target, and the detection limit has increased by 10 times year-on-year. When the sunlight illumination remains unchanged, the ambient light energy passively received by the chip is a fixed value. Most of the reflectivity in daily environments is distributed between 10% and 90%. Therefore, using retroreflective materials to transform targets can not only significantly improve the distance detection capability of the iTOF camera, but also ensure the differentiation of the target from the background environment and ambient light, which is convenient for the identification and positioning of the target and the measurement of physical information. This broadens the application scenarios of the iTOF camera and makes up for the shortcomings of the iTOF camera's weak ranging capability and poor outdoor test results.

[0119] In addition, for a certain iTOF camera, the relationship between exposure time and received light intensity when shooting a 90% diffuse reflection target and a reverse reflection target at 5m is shown in the figure below: Figure 7 As shown in the figure, assuming that the iTOF camera needs to collect the same 20 light intensity, it takes about 4000us exposure time to shoot a 90% diffuse reflection target, while it only takes about 40us exposure time to shoot a retroreflective target. Therefore, the exposure time can be proportionally reduced by using retroreflective materials to transform the target. At the same time, due to the difference between the reflected energy of the retroreflective target and the diffuse reflection target and the background environment, there is sufficient margin to increase the confidence filter threshold and filter out as much environmental noise as possible. It can not only ensure the detection of the target, but also reduce the proportion of ambient light in the sampled signal, alleviating the interference of ambient light on the measurement process.

[0120] Paste on 90% diffuse reflection board at 5m Figure 8 For the retroreflective material shown, the corresponding amplitude map can be obtained by opening the 100us exposure time. The position and outline of the retroreflective material can be clearly distinguished on the amplitude map. The average light intensity of the retroreflective material area is 420, while the average light intensity of the 90% diffuse reflection area is only 5. There is a light intensity difference of about 84 times between the two, which is consistent with the simulation example phenomenon. After further confidence filtering of the data, the corresponding point cloud is obtained. No significant flying points are found on the point cloud, providing good data quality for subsequent measurement.

[0121] Therefore, based on the above example analysis, in this embodiment, after setting a retroreflective target on the surface of the target to be measured, the exposure time of the iTOF camera can be configured according to the required detection distance. Among them, the required detection distance can be determined according to the actual situation. For example, if this embodiment is an application scenario for indoor obstacle avoidance, the exposure time can be configured according to the maximum measurement distance indoors. In addition, the light intensity detected at different detection distances is different, so the exposure time can be specifically determined based on the light intensity corresponding to different detection distances. It should be noted that the exposure time configured after setting the retroreflective target is less than the exposure time when measuring without setting the retroreflective target.

[0122] In addition, after setting the retroreflective target, the confidence filter parameters can also be reconfigured. Specifically, the confidence filter parameters include a confidence filter threshold. In the subsequent data processing process, the confidence filter threshold is used to delete the data below the confidence filter threshold, thereby deleting noise data, abnormal data, and low-precision data. In the present embodiment, after setting the retroreflective target, because the reflected light energy of the retroreflective target is quite different from the reflected light energy of the target to be measured (diffuse reflection target), the confidence filter threshold can be increased, thereby filtering out as much environmental noise as possible. It can not only ensure that the target is detected, but also reduce the proportion of ambient light in the sampled signal, alleviating the interference of ambient light on the measurement process.

[0123] Step S203: using an iTOF camera to acquire signal data reflected by each retroreflective target, and reconstructing point cloud data within each retroreflective target area based on the signal data.

[0124] Specifically, the above step S203 includes:

[0125] Step S2031: using an iTOF camera to collect signal data reflected from the surface of the target object to be measured based on the configured exposure time.

[0126] Step S2032: filtering the signal data reflected from the surface of the target object to be measured based on the confidence filtering parameters to obtain filtered data.

[0127] Step S2033, obtaining a corresponding grayscale image based on the filtered data;

[0128] Step S2034, obtaining signal data reflected by the retroreflective target according to the grayscale image segmentation.

[0129] Among them, data can be collected based on the exposure time configured in step S202, and the data can be processed based on the confidence filter threshold in the confidence filter parameter configured in step S202, and then an image processing algorithm such as histogram equalization can be used to obtain a grayscale image. Then, according to the difference in the amplitude of the retroreflective target position in the grayscale image and the amplitude of the reflected light of the surrounding target to be measured, the retroreflective target area is segmented to obtain signal data of the retroreflective target area.

[0130] Step S204, calculating the centroid of each retroreflective target based on the point cloud data within each retroreflective target area; see Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0131] Step S205, calculate the relative angle between the target object to be measured and the iTOF camera according to the centroid of the two retroreflective targets. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0132] In this embodiment, a method for measuring the angle of an iTOF camera is provided, and the method comprises the following steps:

[0133] Step S301, at least two retro-reflective targets are set in the direction parallel to the horizontal direction on the surface of the target to be measured. When the target to be measured is a fixed object, i.e., a non-moving object, usually only one surface of the target to be measured needs to be angle measured. When the target to be measured is a moving object, such as when navigating a moving semi-trailer by angle measurement, multiple surfaces of the target to be measured need to be angle measured. Therefore, the above step S301 includes:

[0134] Step S3011, at least two retroreflective targets are set on each surface of the target object to be measured in a direction parallel to the horizontal direction, and the retroreflective targets on different surfaces have different shapes. Specifically, by setting retroreflective targets of different shapes on different surfaces of the target object to be measured, it is convenient for the subsequent iTOF camera to distinguish the signal data obtained from each surface.

[0135] Step S302: using an iTOF camera to acquire signal data reflected by each retroreflective target, and reconstructing point cloud data within each retroreflective target area based on the signal data.

[0136] Specifically, the above step S302 includes:

[0137] Step S3021, placing the iTOF camera at a preset distance in front of each surface to obtain the signal data reflected by the retroreflective target on each surface. In order to obtain the reflected signal of the retroreflective target on each surface of the target to be measured, the iTOF camera can be placed at a preset distance in front of each surface to collect the data of the reflected signal.

[0138] Step S3022: Use a pre-trained machine learning model to identify the signal data corresponding to each surface, and determine the surface corresponding to each signal data.

[0139] The iTOF camera can be used to collect and obtain image data of the retroreflective target on each surface. In order to automatically distinguish each surface, a general machine learning model can be used to build an image recognition network in advance, and then the image of the corresponding shape of the retroreflective target is used as a training set to input into the image recognition network for training to obtain a trained model. Afterwards, when the image data of the retroreflective target on each surface is obtained, it is input into the trained model to determine its corresponding surface.

[0140] Step S303, calculating the centroid of each retroreflective target based on the point cloud data within each retroreflective target area; see Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0141] Step S304, calculating the relative angle between the target to be measured and the iTOF camera according to the centroid of the two retroreflective targets. Wherein, when the target to be measured is a semi-trailer, the multiple surfaces for which angle measurement is required include the front side, the first side side, and the second side side. Specifically, for a semi-trailer, retroreflective targets are set on various surfaces of the semi-trailer compartment. The semi-trailer compartment includes two side sides, the front side close to the front of the vehicle, and the back side away from the front of the vehicle. In this embodiment, retroreflective targets are set on the two side sides and the front side.

[0142] Specifically, the above step S304 includes:

[0143] Step S3041, calculate the first relative angle between the front side and the iTOF camera, the second relative angle between the first side and the iTOF camera, and the third relative angle between the second side and the iTOF camera according to the centroid of the two retroreflective targets on each surface; wherein, the calculation method of the first relative angle, the second relative angle and the third relative angle can refer to the calculation method of the relative angles in the above-mentioned step S104, which will not be repeated here.

[0144] Step S3042, converting the second relative angle and the third relative angle according to the coordinate system corresponding to the first relative angle to obtain the fourth relative angle and the fifth relative angle after conversion; specifically, according to step S3021, it can be known that the signal data corresponding to each surface is obtained by the iTOF camera at different positions, that is, the coordinate system used to calculate each relative angle is different. Therefore, in order to achieve the unification of the angle measurement results, it is converted by selecting a reference coordinate system. In this embodiment, the coordinate system corresponding to the first relative angle is used as the reference coordinate system, and the second relative angle and the third relative angle are converted. Among them, for the convenience of conversion, the coordinate system corresponding to the second relative angle and the coordinate system corresponding to the third relative angle differ by 90° from the coordinate system corresponding to the first relative angle. Therefore, the second relative angle and the third relative angle can be respectively subtracted from 90° to obtain the fourth relative angle and the fifth relative angle after conversion. For example, for the second relative angle θ corresponding to the left side side , which corresponds to the fourth relative angle θ after conversion front It can be expressed as θ front =θ side -90°.

[0145] Step S3043, automatic reversing guidance of the semi-trailer is performed according to the first relative angle, the fourth relative angle and the fifth relative angle. Specifically, after obtaining the first relative angle, the fourth relative angle and the fifth relative angle, they can be input into the automatic navigation system of the semi-trailer, so as to provide the automatic navigation system with guidance to the vehicle.

[0146] As a specific application example of the embodiment of the present invention, when the angle measurement method is applied to indoor navigation and obstacle avoidance, such as Fig. 9 As shown, the angle measurement method of the iTOF camera can be implemented according to the following process:

[0147] Step 1: Select appropriate retro-reflective materials (i.e., specular reflective materials) to transform the surface of the target object (i.e., obstacle, such as wall or pillar, etc.), such as Fig.10 As shown, in order to measure the angle information of the target to be measured, two retro-reflective targets, numbered 1 and 2, are pasted horizontally on the surface of the target to be measured.

[0148] Step 2: According to the actual detection distance, configure the appropriate exposure time to ensure that the reflected light intensity in the retroreflective target area is sufficient; increase the confidence filter parameter to filter the data outside the retroreflective target area to obtain Fig.11 Grayscale image shown.

[0149] Step 3: Based on Fig.11 The grayscale image shown is used for target detection, the data of the retroreflective target area is segmented and the point cloud is reconstructed using the camera internal parameters, as shown in Fig.12shown.

[0150] Step 4: Cluster the point cloud to obtain the point cloud set 1{(x1,y1,z1),…,(x n ,y n ,z n )} (n is the total length of the point cloud) and the point cloud set 2{(x1,y1,z1),…,(x n ,y n ,z n )}(n is the total length of the point cloud), and the centroid 1(X1 centric ,Y1 centric ,Z1 centric ) and centroid 2 (X2 centric ,Y2 centric ,Z2 centric ).

[0151] Where n is the total number of point cloud sets (25)

[0152] Step 5: Calculate the slope k of the target to be measured according to the calculated center of mass according to formula (26).

[0153]

[0154] Step 6: Convert the slope into radians and output the angle measurement result.

[0155] As a specific application example of the embodiment of the present invention, when the angle measurement method is applied to outdoor vehicle identification and reverse guidance, such as Fig.13 As shown, the angle measurement method of the iTOF camera can be implemented according to the following process:

[0156] 1. Connect the device and computer media.

[0157] After configuring the IP connection between the iTOF camera and the computer medium, set the appropriate camera exposure time for the iTOF camera.

[0158] 2. Set up equipment to collect data sets.

[0159] like Fig.14 As shown in the figure, the iTOF camera is set up at three straight-line distances relative to the front, left side, and right side of the vehicle during the actual reversing process to capture Fig.15 The images of the retroreflective target (retroreflective mark) at different viewing angles are shown, and then the above acquisition operation is repeated after being displaced forward and backward by a certain distance, thereby acquiring a number of retroreflective mark grayscale images at different distances and viewing angles.

[0160] 3. Data set classification and model training.

[0161] The grayscale images retained after screening are classified and labeled according to L, F, and R labels using a labeling tool to obtain a training set. Based on a general image recognition framework, the training set images are input into the model for iterative training until the accuracy of the validation set reaches the expected standard, and then the model training is terminated and the trained model is exported.

[0162] 4. Model application.

[0163] In actual angle measurement, the image is collected in real time and input into the trained model to identify the pixel coordinate interval of the retroreflective target on the image and output the corresponding vehicle hanging surface position information. Therefore, position recognition through the model can avoid Fig.16 As shown in the figure, when the camera simultaneously captures two hanging surfaces (F+L or F+R) during the reversing process, the output measurement angle jumps between the front and side, causing a sudden change in driving direction and causing safety hazards. If the front of the vehicle is identified, the retroreflective target point cloud is extracted to directly calculate the angle and output it to the industrial control platform. If the side of the vehicle is identified, it is necessary to execute after calculating the angle Fig.17 The coordinate system transformation shown above converts the left side angle into the angle of the front side relative to the camera coordinate system: θ front =θ Side -90° (the same applies to the right side), ultimately achieving the goal of quickly distinguishing the relative positions of the vehicle and the trailer and outputting angle measurement results with unified reference coordinates, providing accurate and safe reversing guidance for the automatic navigation system.

[0164] In this embodiment, an angle measurement device for an iTOF camera is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0165] This embodiment provides an angle measurement device of an iTOF camera, such as Fig.18 As shown, including:

[0166] The target transformation module 181 is used to set at least two retro-reflective targets on the surface of the target to be measured in a direction parallel to the horizontal direction;

[0167] A signal processing module 182 is used to obtain signal data reflected by each retroreflective target using an iTOF camera, and reconstruct point cloud data within each retroreflective target area based on the signal data;

[0168] A centroid calculation module 183, used to calculate the centroid of each retroreflective target based on the point cloud data within each retroreflective target area;

[0169] The angle calculation module 184 is used to calculate the relative angle between the target object to be measured and the iTOF camera according to the centroids of the two retroreflective targets.

[0170] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0171] The embodiment of the present invention also provides a computer device having the above Fig.18 The angle measurement setup of the iTOF camera is shown.

[0172] See also Fig.19 , Fig.19 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.19 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.19 A processor 10 is taken as an example.

[0173] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0174] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0175] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0177] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0178] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0179] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0180] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An angle measurement method of an iTOF camera, characterized in that: The method comprises: At least two retro-reflective targets are arranged on the surface of the target object to be measured in a direction parallel to the horizontal direction; An iTOF camera is used to obtain signal data reflected by each retroreflective target, and point cloud data within each retroreflective target area is reconstructed based on the signal data; Calculate the centroid of each retroreflective target based on the point cloud data within the area of ​​each retroreflective target; The relative angle between the target object and the iTOF camera is calculated based on the centroid of the two retroreflective targets.

2. The method according to claim 1, characterized in that Before using the iTOF camera to collect the light signal reflected by the retroreflective target, the method further includes: Configure the exposure time of the iTOF camera according to the distance to the target object; The confidence filter parameters of the iTOF camera are set according to the difference between the reflected light intensity of the material in the retroreflective target area and the reflected light intensity of the material in the area outside the retroreflective target on the surface of the target to be measured.

3. The method according to claim 2, characterized in that The iTOF camera is used to obtain the signal data reflected by the retroreflective target, including: Based on the configured exposure time, an iTOF camera is used to collect signal data reflected from the surface of the target to be measured; Filtering the signal data reflected from the surface of the target object to be measured based on the confidence filtering parameters to obtain filtered data; Obtain a corresponding grayscale image based on the filtered data; Signal data reflected by the retroreflective target is obtained according to the grayscale image segmentation.

4. The method according to claim 1, characterized in that: The centroid is determined based on the sum of all point cloud data within the retroreflective target area; The relative angle is determined based on the slope between the centroids of the two retroreflective targets.

5. The method according to claim 1, characterized in that When the target to be measured includes multiple surfaces that need to be measured in angle, at least two retro-reflective targets are arranged in a direction parallel to the horizontal direction on the surface of the target to be measured, including: At least two retroreflective targets are arranged on each surface of the target object to be measured in a direction parallel to the horizontal direction, and the retroreflective targets on different surfaces have different shapes.

6. The method according to claim 5, characterized in that The iTOF camera is used to obtain the signal data reflected by the retroreflective target, including: The iTOF camera is placed at a preset distance in front of each surface to obtain the signal data reflected by the retroreflective target on each surface; A pre-trained machine learning model is used to identify the signal data corresponding to each surface and determine the surface corresponding to each signal data.

7. The method according to claim 6, characterized in that When the target to be measured is a semi-trailer, the multiple surfaces that need to be angled include the front side, the first side side, and the second side side. The relative angle between the target to be measured and the iTOF camera is calculated according to the centroid of the two retroreflective targets, including: Calculate a first relative angle between the front side and the iTOF camera, a second relative angle between the first side and the iTOF camera, and a third relative angle between the second side and the iTOF camera according to the centroids of the two retroreflective targets on each surface; Convert the second relative angle and the third relative angle according to the coordinate system corresponding to the first relative angle to obtain a converted fourth relative angle and a fifth relative angle; Automatic reversing guidance of the semitrailer is performed according to the first relative angle, the fourth relative angle and the fifth relative angle.

8. An angle measurement device for an iTOF camera, characterized in that: The device comprises: A target modification module is used to set at least two retro-reflective targets on the surface of the target to be measured in a direction parallel to the horizontal direction; A signal processing module, used to obtain signal data reflected by each retroreflective target using an iTOF camera, and reconstruct point cloud data within each retroreflective target area based on the signal data; A centroid calculation module, used to calculate the centroid of each retroreflective target based on the point cloud data within the area of ​​each retroreflective target; The angle calculation module is used to calculate the relative angle between the target object to be measured and the iTOF camera according to the centroid of the two retroreflective targets.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the angle measurement method of the iTOF camera according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the angle measurement method of the iTOF camera according to any one of claims 1 to 7.

Images