Temperature drift detection method and depth camera
By adjusting the position of the depth camera and fitting the normal vector of the depth map, the problem of temperature drift detection of the depth camera was solved, ensuring the optimization of the optical structure and heat dissipation design of the depth camera, and improving accuracy and reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Hefei Xinming Intelligent Technology Co., Ltd.
- Filing Date
- 2023-03-29
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies lack effective methods for detecting temperature drift, making it difficult to determine whether the temperature drift of a depth camera is within acceptable limits, thus affecting its accuracy.
By adjusting the position of the depth camera to make the light-emitting surface parallel to the surface under test, the depth camera is controlled to project images onto the surface under test, and depth maps are obtained at the initial and preset times. The temperature drift angle is determined by fitting the normal vector, and the temperature drift of the depth camera is judged to be qualified.
It enables accurate detection of temperature drift in depth cameras, ensuring the optimization of the optical and heat dissipation structures of depth cameras, and improving the accuracy and reliability of depth cameras.
Smart Images
Figure CN116430366B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of depth camera technology, specifically relating to a temperature drift detection method and a depth camera. Background Technology
[0002] Depth cameras (Time of Flight, TOF) generate a significant amount of heat during operation. This heat causes deformation of the camera's optical structure, altering its optical performance and affecting the camera's accuracy. Therefore, it's necessary to know the magnitude of this deformation. Measuring temperature drift reflects the degree to which the depth camera is affected by heat, thus determining the product's design compliance. However, current methods for detecting temperature drift are lacking, making it difficult to determine whether a depth camera's temperature drift is within acceptable limits. Summary of the Invention
[0003] The purpose of this application is to provide a temperature drift detection method and a depth camera, which can perform temperature drift detection and thus determine whether the temperature drift of the depth camera is qualified.
[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.
[0005] According to one aspect of an embodiment of this application, this application provides a temperature drift detection method applied to a depth camera. The depth camera includes a light-emitting surface for emitting light and a photosensitive surface for receiving light, the light-emitting surface and the photosensitive surface being parallel. A test plate is disposed in the light-emitting direction of the depth camera, and the test plate has a test surface facing the depth camera. The temperature drift detection method includes:
[0006] Adjust the position of the depth camera so that the light-emitting surface of the depth camera is parallel to the surface to be measured;
[0007] Control the depth camera to project an image onto the surface to be measured;
[0008] At the initial moment when the depth camera is turned on, a first depth map of the image is acquired, and a first normal vector of the plane containing the first depth map is obtained.
[0009] After a preset time following the initial moment, a second depth map of the image is acquired, and a second normal vector of the plane containing the second depth map is obtained.
[0010] The temperature drift angle is determined based on the first normal vector and the second normal vector;
[0011] The temperature drift of the depth camera is checked against the temperature drift angle to determine whether the temperature drift is within acceptable limits.
[0012] In one aspect, the step of obtaining the first normal vector of the plane containing the first depth map includes:
[0013] A first plane is obtained by performing plane fitting on the first depth map, and a first normal vector is obtained based on the first plane;
[0014] The step of obtaining the second normal vector of the plane containing the second depth map includes:
[0015] A second plane is obtained by performing plane fitting on the second depth map, and a second normal vector is obtained based on the second plane.
[0016] In one aspect, the first depth map includes first point cloud data, and the second depth map includes second point cloud data;
[0017] The step of performing plane fitting on the first depth map to obtain a first plane includes:
[0018] The first point cloud data is preprocessed to remove invalid points and noise;
[0019] The first plane is obtained by fitting the preprocessed first point cloud data;
[0020] The step of performing plane fitting on the second depth map to obtain the second plane includes:
[0021] The second point cloud data is preprocessed to remove invalid points and noise;
[0022] The second plane is obtained by fitting the preprocessed second point cloud data.
[0023] In one aspect, the preprocessing steps for the first point cloud data include:
[0024] Remove invalid points and points with a depth value of zero from the first point cloud data to obtain the remaining first point cloud data;
[0025] The remaining first point cloud data is subjected to histogram distribution processing;
[0026] Remove first point cloud data with a percentage less than a first preset percentage, and remove first point cloud data with a percentage greater than a second preset percentage, and obtain the removed point cloud data as the first intermediate data;
[0027] The second step of preprocessing cloud data includes:
[0028] Remove invalid points and points with a depth value of zero from the second point cloud data to obtain the remaining second point cloud data;
[0029] The remaining second point cloud data is subjected to histogram distribution processing;
[0030] Remove second point cloud data with a percentage less than the first preset percentage, and remove second point cloud data with a percentage greater than the second preset percentage, and obtain the removed point cloud data as the second intermediate data.
[0031] In one aspect, the step of determining the temperature drift angle based on the first normal vector and the second normal vector includes:
[0032] Perform a matrix transformation on the first normal vector to obtain the first rotation matrix;
[0033] The first change angle of the first normal vector is obtained based on the first rotation matrix;
[0034] Perform a matrix transformation on the second normal vector to obtain the second rotation matrix;
[0035] The second change angle of the second normal vector is obtained based on the second rotation matrix;
[0036] The temperature drift angle is determined based on the first change angle and the second change angle.
[0037] In one aspect, the step of performing a matrix transformation on the first normal vector to obtain a first rotation matrix includes:
[0038] Perform a Rodrigues transformation on the first normal vector to obtain the first rotation matrix;
[0039] The step of performing a matrix transformation on the second normal vector to obtain the second rotation matrix includes:
[0040] Perform a Rodrigues transformation on the second normal vector to obtain the second rotation matrix.
[0041] In one aspect, the step of determining the temperature drift angle based on the first normal vector and the second normal vector includes, prior to:
[0042] A reference coordinate system is constructed using the light-emitting surface of the depth camera. The reference coordinate system includes a first coordinate axis extending horizontally along the light-emitting surface, a second coordinate axis extending vertically along the light-emitting surface, and a third coordinate axis perpendicular to the light-emitting surface.
[0043] The step of determining the temperature drift angle based on the first normal vector and the second normal vector further includes:
[0044] The first rotation matrix is transformed into Eulerian space to obtain a first rotation angle of the first normal vector relative to the first coordinate axis, a second rotation angle of the first normal vector relative to the second coordinate axis, and a third rotation angle of the first normal vector relative to the third coordinate axis, wherein the first rotation angle includes the first rotation angle, the second rotation angle, and the third rotation angle;
[0045] The second rotation matrix is transformed into Eulerian space to obtain the fourth rotation angle of the second normal vector relative to the first coordinate axis, the fifth rotation angle of the second normal vector relative to the second coordinate axis, and the sixth rotation angle of the second normal vector relative to the third coordinate axis, wherein the second rotation angle includes the fourth rotation angle, the fifth rotation angle, and the sixth rotation angle;
[0046] The first rotation angle and the fourth rotation angle are compared to obtain the first drift angle. The second rotation angle and the fifth rotation angle are compared to obtain the second drift angle. The third rotation angle and the sixth rotation angle are compared to obtain the third drift angle. The temperature drift angle includes the first drift angle, the second drift angle and the third drift angle.
[0047] In one aspect, the step of detecting whether the temperature drift of the depth camera is acceptable based on the temperature drift angle includes:
[0048] The first drift angle, the second drift angle, and the third drift angle are compared with the preset standard angle, respectively.
[0049] If the first drift angle, the second drift angle, and the third drift angle are all less than or equal to the preset standard angle, then the temperature drift of the depth camera is qualified.
[0050] If at least one of the first drift angle, the second drift angle, and the third drift angle is greater than the preset standard angle, then the temperature drift of the depth camera is unqualified.
[0051] In one aspect, the length of the surface to be measured is L, the width of the surface to be measured is W, and the distance between the light-emitting surface of the depth camera and the surface to be measured is D, then the following conditions are met:
[0052] Wherein, FOV_L is the field of view angle in the L direction, and FOV_W is the field of view angle in the W direction.
[0053] In addition, to solve the above problems, this application also provides a depth camera, which includes: a mounting plate, a light emitter and a light receiver, wherein the light emitter and the light receiver are disposed on the same surface of the mounting plate, the light emitter has a light emitting surface for emitting light, and the light receiver has a photosensitive surface for receiving light, and the depth camera uses the temperature drift detection method described above for detection.
[0054] In this application, a depth camera projects an image onto the surface to be measured. At the initial moment of camera startup, a set of depth images, known as the first depth map, is captured. After a preset time following the initial moment, another set of depth images, known as the second depth map, is captured. By capturing images after the preset time, the depth camera has already been operating for a certain period, and the heat generated has caused deformation of the camera's optical structure. The first normal vector is perpendicular to the plane containing the first depth map, and the second normal vector is perpendicular to the plane containing the second depth map. Due to the deformation of the optical structure caused by heat radiation, the planes formed by the first and second depth maps will also differ, and this difference can be reflected by the first and second normal vectors. Therefore, the magnitude of the temperature drift angle can be determined using the first and second normal vectors, thereby determining whether the temperature drift of the depth camera is acceptable.
[0055] It should be understood in this application that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description
[0056] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0057] Figure 1 The schematic diagram illustrates the flow steps of the temperature drift detection method in this application.
[0058] Figure 2 The schematic diagram illustrates steps S310 and S410 of the temperature drift detection method in this application.
[0059] Figure 3 The schematic diagram illustrates steps S311 and S312 of the temperature drift detection method in this application.
[0060] Figure 4The schematic diagram illustrates steps S411 and S412 of the temperature drift detection method in this application.
[0061] Figure 5 The schematic diagram illustrates steps S311a, S311b, and S311c of the temperature drift detection method in this application.
[0062] Figure 6 The schematic diagram illustrates steps S411a, S411b, and S411c of the temperature drift detection method in this application.
[0063] Figure 7 The schematic diagram illustrates the steps of transforming the first normal vector and the second normal vector in the temperature drift detection method of this application.
[0064] Figure 8 Schematic illustration Figure 7 A schematic diagram illustrating the steps of performing the Rodrigues transformation on the first and second normal vectors.
[0065] Figure 9 The schematic diagram illustrates the steps involved in constructing a reference coordinate system for the temperature drift detection method described in this application.
[0066] Figure 10 The schematic diagram illustrates the process steps of the temperature drift detection method in this application for determining temperature drift.
[0067] Figure 11 A schematic diagram of the depth camera structure in this application is shown.
[0068] The annotations in the attached figures are explained as follows:
[0069] 10. Mounting plate; 20. Light emitter; 30. Light receiver. Detailed Implementation
[0070] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.
[0071] See Figure 1As shown, this application provides a temperature drift detection method applied to a depth camera. The depth camera includes a light-emitting surface for emitting light and a light-receiving surface for receiving light. The light-emitting surface and the light-receiving surface are located in the same plane, or parallel to each other. The light-emitting surface and the light-receiving surface being in the same plane can also be understood as being parallel. Typically, the light emitted by a depth camera is infrared laser. Infrared light does not affect the user's vision, avoiding interference. Lasers have good directionality and are not easily affected by ambient light. Depth cameras generate heat during operation. For example, a depth camera includes an infrared laser emitter and a CMOS (Complementary Metal-Oxide-Semiconductor) sensor. The infrared laser emitter generates a large amount of heat when emitting infrared laser light, and similarly, the CMOS sensor also generates a large amount of heat when receiving infrared laser light.
[0072] The depth camera's light-emitting direction is aligned with the test board, which has a surface facing the depth camera. The light-emitting surface of the depth camera and the test surface are positioned opposite each other, or in other words, parallel to each other. However, in actual testing, the angle of placement may introduce some angular error. The test board is typically a white board with a flatness greater than 99% and a reflectivity greater than 70%.
[0073] Temperature drift detection methods include:
[0074] Step S10: Adjust the position of the depth camera so that its light-emitting surface is parallel to the surface under test. Adjusting the depth camera position includes adjusting the distance between the depth camera and the test plate, as well as adjusting the angle between the light-emitting surface and the test surface. This adjustment can be done using instruments or manually. Parallelism between the light-emitting surface and the test surface ensures that the light from the light-emitting surface strikes the test surface perpendicularly, and after reflection, accurately strikes the photosensitive surface.
[0075] Step S20: Control the depth camera to project an image onto the surface to be measured; start the depth camera to begin operation, emitting light rays to form an image on the surface to be measured. The image can be a pre-defined pattern, such as spaced horizontal stripes, vertical stripes, or a scatter plot.
[0076] Step S30: At the initial moment when the depth camera is turned on, acquire the first depth map of the image and obtain the first normal vector of the plane containing the first depth map. When the depth camera is turned on, the heat generated at this time will not significantly affect the deformation of the depth camera, meaning that there is essentially no temperature drift in the depth camera at this time. The initial moment can be understood as the moment when t equals 0. The first depth map is acquired through detection by the CMOS sensor, and the normal vector of the plane containing the first depth map is confirmed as the first normal vector.
[0077] Step S40: After a preset time from the initial moment, acquire the second depth map of the image and obtain the second normal vector of the plane containing the second depth map. The time between the preset times can be understood as time t1, during which the depth camera has been working for a certain period of time, generating considerable heat and causing some temperature drift. The preset time is typically around 30 minutes, but it can be set according to the type of depth camera, either less than 30 minutes or more than 30 minutes, such as 10 minutes, 20 minutes, 40 minutes, and 50 minutes, etc. The second depth map is acquired through detection using a CMOS sensor, and the normal vector of the plane containing the second depth map is confirmed as the second normal vector.
[0078] Step S50: Determine the temperature drift angle based on the first and second normal vectors. After obtaining the first and second normal vectors, the temperature drift angle can be determined. There are at least two methods for this determination. The first method is to determine the drift angle by comparing the first and second normal vectors, calculating the angle between them. The second method is to compare the first and second normal vectors with the same standard system, determining the angle by which the first normal vector and the second normal vector have drifted relative to the standard system. Then, the two angles are compared, and the difference is calculated to determine the temperature drift angle. The first method is simpler and has less computational burden. The second method, using a standard system unaffected by external environmental interference, has higher accuracy.
[0079] Step S60: Detect whether the temperature drift of the depth camera is within acceptable limits based on the temperature drift angle. After determining the temperature drift angle, the degree to which the depth camera is affected by heat can be judged based on the magnitude of the temperature drift angle, thereby detecting whether the temperature drift of the depth camera is within acceptable limits.
[0080] In this embodiment, a depth camera projects an image onto the surface to be measured. At the initial moment of camera startup, a set of depth images, known as the first depth map, is captured. After a preset time following the initial startup, another set of depth images, known as the second depth map, is captured. By capturing images after the preset time, the depth camera has already been operating for a certain period, and the generated heat has caused deformation of the camera's optical structure. The first normal vector is perpendicular to the plane containing the first depth map, and the second normal vector is perpendicular to the plane containing the second depth map. Due to the influence of heat radiation, the optical structure has deformed, resulting in a difference between the planes formed by the first and second depth maps. This difference is reflected by the first and second normal vectors. Therefore, the magnitude of the temperature drift angle can be determined using the first and second normal vectors, thereby determining whether the temperature drift of the depth camera is within acceptable limits.
[0081] If the temperature drift of the depth camera is found to be unacceptable, it indicates that the depth camera design is flawed and requires further optimization. Additionally, depth cameras typically incorporate a heat dissipation structure; if the temperature drift is unacceptable, it suggests that the heat dissipation effect of this structure is inadequate, necessitating further optimization of the heat dissipation design.
[0082] See Figure 2 As shown, the steps to obtain the first normal vector of the plane containing the first depth map include:
[0083] Step S310: Perform plane fitting on the first depth map to obtain a first plane, and obtain a first normal vector based on the first plane. Since there may be some errors in obtaining the first depth map, some image points may not be on a flat plane. A plane is formed by fitting data from the first depth map; this plane is the first plane. The first plane is generated through fitting and represents the plane where most image points of the first depth map are located.
[0084] The steps to obtain the second normal vector of the plane containing the second depth map include:
[0085] Step S410: Perform plane fitting on the second depth map to obtain a second plane, and obtain a second normal vector based on the second plane. A plane is formed by fitting data from the second depth map; this plane is the second plane. Similarly, the second plane is generated by fitting and represents the plane where most image points of the second depth map are located. The first plane and the second plane facilitate the obtaining of the first normal vector and the second normal vector.
[0086] The first depth map includes the first point cloud data, and the second depth map includes the second point cloud data. There are multiple sets of the first point cloud data and multiple sets of the second point cloud data.
[0087] See Figure 3 As shown, the steps for obtaining the first plane by performing plane fitting on the first depth map include:
[0088] Step S311: Preprocess the first point cloud data to remove invalid points and noise. During the acquisition of this first point cloud data, invalid points and noise may exist in the first point cloud data due to abnormal photosensitivity or external interference.
[0089] Step S312: Fit the first plane based on the preprocessed first point cloud data; through preprocessing, invalid points and noise are removed to ensure that the fitted first plane can more accurately represent the first depth map.
[0090] See Figure 4 As shown, the steps for obtaining the second plane by performing plane fitting on the second depth map include:
[0091] Step S411: Preprocess the second point cloud data to remove invalid points and noise; similarly, during the acquisition of the second point cloud data, due to abnormal photosensitivity or external interference, invalid points and noise may also exist in the second point cloud data.
[0092] Step S412: Fit the second plane based on the preprocessed second point cloud data. Preprocessing removes invalid points and noise, ensuring that the fitted second plane more accurately represents the first depth map.
[0093] See Figure 5 As shown, in order to further improve the accuracy of point cloud data, the preprocessing steps for the first point cloud data include:
[0094] Step S311a: Remove invalid points and points with a depth value of zero from the first point cloud data to obtain the remaining first point cloud data. Invalid points can be understood as points without a depth value. The depth value represents the distance between the light-emitting surface and the surface under test, and the distance between the two cannot be zero. Therefore, a depth value of zero is an obvious anomaly. Removing these invalid points and points with a depth value of zero ensures the accuracy of the first point cloud data.
[0095] Step S311b: Perform histogram distribution processing on the remaining first point cloud data; the first point cloud data represents the distance between the photosensitive surface and the surface to be measured. Histogram distribution processing can be understood as using the height of the rectangle to represent the frequency of the corresponding point cloud data and the width of the rectangle to represent the depth data range. Histogram distribution processing can clearly show the frequency distribution of the point cloud data.
[0096] Step S311c involves removing point cloud data with a percentage less than a first preset percentage and removing point cloud data with a percentage greater than a second preset percentage, obtaining the removed point cloud data as the first intermediate data. By removing point cloud data from both ends of the histogram with a smaller percentage, the concentration of the point cloud data is further improved. The first preset percentage is less than the second preset percentage; point cloud data with a percentage lower than the first preset percentage is small, and point cloud data with a percentage higher than the second preset percentage is also small. For example, the first preset percentage is 0.5%, and the second preset percentage is 99.5%. The percentages are based on the depth value; less than 0.5% indicates the depth value is too small, and greater than 99.5% indicates the depth value is too large. The first and second preset percentages are used to remove data with excessively large and small depth values. Of course, the first preset percentage is not limited to 0.5%; it can also be 0.1%, 0.2%, 0.3%, 0.4%, 0.6%, 0.7%, 0.8%, 0.9%, and 1.0%, etc. Similarly, the value of the second preset percentage is not limited to 99.5%, but can also be 99.0%, 99.1%, 99.2%, 99.3%, 99.4%, 99.6%, 99.7%, 99.8%, and 99.9%, etc. The point cloud data is filtered and denoised by histogram distribution processing in conjunction with the first and second preset percentages.
[0097] In addition to using histogram distribution to process point cloud data, the mean and standard deviation methods can also be used to filter the first intermediate data, which can also remove noisy points.
[0098] See Figure 6 As shown, the preprocessing steps for the second point cloud data include:
[0099] Step S411a: Remove invalid points and points with a depth value of zero from the second point cloud data to obtain the remaining second point cloud data;
[0100] Step S411b: Perform histogram distribution processing on the remaining second point cloud data;
[0101] Step S411c: Remove second point cloud data with a percentage less than the first preset percentage, and remove second point cloud data with a percentage greater than the second preset percentage, obtaining the removed point cloud data as the second intermediate data; the preprocessing steps for the second point cloud data can refer to the preprocessing steps for the first point cloud data. During the processing of the second point cloud data, filtering is also based on the first and second preset percentages to ensure that it has the same filtering standards as the first point cloud data.
[0102] In addition to using histogram distribution to process point cloud data, the mean and standard deviation methods can also be used to filter the second intermediate data to remove noise points.
[0103] See Figure 7 As shown, the steps for determining the temperature drift angle based on the first normal vector and the second normal vector include:
[0104] Step S510: Perform matrix transformation on the first normal vector to obtain the first rotation matrix;
[0105] Step S520: Obtain the first change angle of the first normal vector based on the first rotation matrix; complete the data transformation of the first normal vector to obtain the first change angle.
[0106] Step S530: Perform a matrix transformation on the second normal vector to obtain the second rotation matrix;
[0107] Step S540: Obtain the second change angle of the second normal vector based on the second rotation matrix; complete the data transformation of the first normal vector to obtain the first change angle.
[0108] Step S550: Determine the temperature drift angle based on the first and second change angles. Through the above data transformations, the first change angle of the first normal vector and the second change angle of the second normal vector can be obtained. The first and second change angles can be compared to each other to determine the temperature drift angle, or they can be compared with the same reference standard to determine the temperature drift angle.
[0109] See Figure 8 As shown, the further step of performing matrix transformation on the first normal vector to obtain the first rotation matrix includes:
[0110] Step S511: Perform a Rodrigues transformation on the first normal vector to obtain the first rotation matrix.
[0111] The following example illustrates the processing of the first normal vector:
[0112] The equation of the first plane is Ax + By + D = CZ. Taking its derivative and expressing it in matrix form is...
[0113] Where m is the number of data points for x, y, and z, and A, B, C, and D are constants, then the plane normal vector is...
[0114]
[0115] The Rodrigues transformation of the plane normal vector V, expressed in the form of a rotation matrix, can be represented as follows:
[0116]
[0117] Where c = cos(θ), s = sin(θ), a = 1 - cos(θ), and θ represents the angle between the normal vector V and the z-axis.
[0118] See Figure 9 As shown, the steps for performing a matrix transformation on the second normal vector to obtain the second rotation matrix include:
[0119] Step S531: Perform a Rodrigues transformation on the second normal vector to obtain the second rotation matrix. The transformation process for the second normal vector can be referenced from the transformation process for the first normal vector.
[0120] To ensure the accuracy of the temperature drift angle measurement, the step of determining the temperature drift angle based on the first normal vector and the second normal vector includes:
[0121] Step S01: Construct a reference coordinate system using the light-emitting surface of the depth camera. The reference coordinate system includes a first coordinate axis x extending horizontally along the light-emitting surface, a second coordinate axis y extending vertically along the light-emitting surface, and a third coordinate axis z perpendicular to the light-emitting surface. Construct a reference coordinate system that is not affected by the external environment.
[0122] The step of determining the temperature drift angle based on the first normal vector and the second normal vector also includes:
[0123] Step S521: Transform the first rotation matrix into Eulerian space to obtain the first rotation angle of the first normal vector relative to the first coordinate axis, the second rotation angle of the first normal vector relative to the second coordinate axis, and the third rotation angle of the first normal vector relative to the third coordinate axis, wherein the first rotation angle includes the first rotation angle, the second rotation angle, and the third rotation angle; compare the first normal vector with the reference coordinate system.
[0124] Transforming the first rotation matrix into Eulerian space, i.e., rotations around the axes in the order zyx, let Roll represent the magnitude of the rotation of the optical axis of the depth camera around the x-axis, Pitch represent the magnitude of the rotation of the optical axis around the y-axis, and Yaw represent the magnitude of the rotation of the optical axis around the z-axis. Then Roll, Pitch, and Yaw can be expressed as:
[0125] Roll = atan2(aVz*Vy+sVx,aVz) 2 +c)
[0126]
[0127] Yaw=atan2(aVy*Vx+sVz,aVx 2 +c)
[0128] It should be noted that atan2(y, x) is the arctangent function: use atan(y / x) when the absolute value of x is greater than the absolute value of y; otherwise, use atan(x / y).
[0129] Step S541: Transform the second rotation matrix to Eulerian space to obtain the fourth rotation angle of the second normal vector relative to the first coordinate axis, the fifth rotation angle of the second normal vector relative to the second coordinate axis, and the sixth rotation angle of the second normal vector relative to the third coordinate axis. The second rotation angle includes the fourth, fifth, and sixth rotation angles. Compare the second normal vector with the reference coordinates. Since the reference coordinate system is unaffected by external environmental interference, the established first, second, and third coordinate axes will not change. The transformation process for the second rotation matrix is the same as the transformation process for the first rotation matrix.
[0130] Step S551: Compare the first rotation angle and the fourth rotation angle to obtain the first drift angle; compare the second rotation angle and the fifth rotation angle to obtain the second drift angle; compare the third rotation angle and the sixth rotation angle to obtain the third drift angle. The temperature drift angle includes the first drift angle, the second drift angle, and the third drift angle.
[0131] The angles of change of the first normal vector relative to the reference coordinate system and the angles of change of the second normal vector relative to the reference coordinate system are obtained. The difference between these two angles is then calculated to obtain the first drift angle, the second drift angle and the third drift angle.
[0132] See Figure 10 As shown, the steps for determining whether the temperature drift of a depth camera is acceptable, based on the temperature drift angle, include:
[0133] Step S610: Compare the first drift angle, the second drift angle, and the third drift angle with the preset standard angle respectively; the standard angle can be set according to the model of the depth camera or the accuracy requirements.
[0134] Step S611: If the first drift angle, the second drift angle, and the third drift angle are all less than or equal to the preset standard angle, then the temperature drift of the depth camera is qualified. If the first drift angle, the second drift angle, and the third drift angle are all less than or equal to the preset standard angle, it means that the heat has little effect on the optical structure of the depth camera, and it also means that the heat dissipation design of the depth camera meets the requirements, thus confirming that the design of the depth camera is qualified.
[0135] In step S612, if at least one of the first drift angle, the second drift angle, and the third drift angle is greater than a preset standard angle, then the temperature drift of the depth camera is unqualified. If at least one of the first drift angle, the second drift angle, and the third drift angle is greater than a preset standard angle, it indicates that heat radiation has seriously affected the optical structure of the depth camera, and further redesign is required.
[0136] In addition, the criteria for determining the pass / fail standard can be as follows: if any temperature drift angle is less than or equal to the preset standard angle, it is considered pass / fail. For example, if two drift angles are greater than the preset standard angle and one is less than the preset standard angle, it is considered pass / fail. Or, if one drift angle is greater than the preset standard angle and two are less than the preset standard angle, it is considered pass / fail.
[0137] The distance between the depth camera and the test surface is typically between 0.5m and 2.0m. To ensure that the image emitted by the depth camera is projected onto the test surface of the test surface, with the length of the test surface being L, the width of the test surface being W, and the distance between the light-emitting surface of the depth camera and the test surface being D, the following conditions must be met:
[0138] Wherein, FOV_L is the field of view angle in the L direction, and FOV_W is the field of view angle in the W direction. The length of the surface to be measured is guaranteed to be greater than the horizontal field of view angle range. The width of the surface to be measured is guaranteed to be greater than the vertical field of view angle range.
[0139] See Figure 11 As shown, this application also provides a depth camera, which includes: a mounting plate 10, a light emitter 20, and a light receiver 30. The light emitter 20 and the light receiver 30 are disposed on the same surface of the mounting plate. The light emitter 20 has a light-emitting surface for emitting light, and the light receiver 30 has a photosensitive surface for receiving light. The depth camera uses the temperature drift detection method described above for detection. The light emitter can be understood as an infrared laser, and the light receiver can be understood as a CMOS sensor.
[0140] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.
[0141] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method for detecting temperature drift, characterized in that, The temperature drift detection method is applied to a depth camera, which includes a light-emitting surface for emitting light and a light-receiving surface for receiving light. The light-emitting surface and the light-receiving surface are parallel. A test plate is positioned in the light-emitting direction of the depth camera, and the test plate has a test surface facing the depth camera. The temperature drift detection method includes: Adjust the position of the depth camera so that the light-emitting surface of the depth camera is parallel to the surface to be measured; Control the depth camera to project an image onto the surface to be measured; At the initial moment when the depth camera is turned on, a first depth map of the image is acquired, and a first normal vector of the plane containing the first depth map is obtained. After a preset time following the initial moment, a second depth map of the image is acquired, and a second normal vector of the plane containing the second depth map is obtained. The temperature drift angle is determined based on the first normal vector and the second normal vector; specifically, this includes performing a Rodrigues transformation on the first normal vector to obtain a first rotation matrix. The first change angle of the first normal vector is obtained based on the first rotation matrix; Perform a Rodrigues transformation on the second normal vector to obtain the second rotation matrix; The second change angle of the second normal vector is obtained based on the second rotation matrix; The temperature drift angle is determined based on the first and second angles of change. The temperature drift of the depth camera is checked against the temperature drift angle to determine whether the temperature drift is within acceptable limits.
2. The temperature drift detection method according to claim 1, characterized in that, The step of obtaining the first normal vector of the plane containing the first depth map includes: A first plane is obtained by performing plane fitting on the first depth map, and a first normal vector is obtained based on the first plane; The step of obtaining the second normal vector of the plane containing the second depth map includes: A second plane is obtained by performing plane fitting on the second depth map, and a second normal vector is obtained based on the second plane.
3. The temperature drift detection method according to claim 2, characterized in that, The first depth map includes first point cloud data, and the second depth map includes second point cloud data; The step of performing plane fitting on the first depth map to obtain a first plane includes: The first point cloud data is preprocessed to remove invalid points and noise; The first plane is obtained by fitting the preprocessed first point cloud data; The step of performing plane fitting on the second depth map to obtain the second plane includes: The second point cloud data is preprocessed to remove invalid points and noise; The second plane is obtained by fitting the preprocessed second point cloud data.
4. The temperature drift detection method according to claim 3, characterized in that, The steps for preprocessing the first point cloud data include: Remove invalid points and points with a depth value of zero from the first point cloud data to obtain the remaining first point cloud data; The remaining first point cloud data is subjected to histogram distribution processing; Remove the first point cloud data with a percentage less than a first preset percentage, and remove the first point cloud data with a percentage greater than a second preset percentage, and obtain the point cloud data after removal as the first intermediate data; The second step of preprocessing cloud data includes: Remove invalid points and points with a depth value of zero from the second point cloud data to obtain the remaining second point cloud data; The remaining second point cloud data is subjected to histogram distribution processing; Remove second point cloud data with a percentage less than the first preset percentage, and remove second point cloud data with a percentage greater than the second preset percentage, and obtain the removed point cloud data as the second intermediate data.
5. The temperature drift detection method according to claim 1, characterized in that, The step of determining the temperature drift angle based on the first normal vector and the second normal vector includes, prior to: A reference coordinate system is constructed using the light-emitting surface of the depth camera. The reference coordinate system includes a first coordinate axis extending horizontally along the light-emitting surface, a second coordinate axis extending vertically along the light-emitting surface, and a third coordinate axis perpendicular to the light-emitting surface. The step of determining the temperature drift angle based on the first normal vector and the second normal vector further includes: The first rotation matrix is transformed into Eulerian space to obtain a first rotation angle of the first normal vector relative to the first coordinate axis, a second rotation angle of the first normal vector relative to the second coordinate axis, and a third rotation angle of the first normal vector relative to the third coordinate axis, wherein the first rotation angle includes the first rotation angle, the second rotation angle, and the third rotation angle; The second rotation matrix is transformed into Eulerian space to obtain the fourth rotation angle of the second normal vector relative to the first coordinate axis, the fifth rotation angle of the second normal vector relative to the second coordinate axis, and the sixth rotation angle of the second normal vector relative to the third coordinate axis, wherein the second rotation angle includes the fourth rotation angle, the fifth rotation angle, and the sixth rotation angle; The first rotation angle and the fourth rotation angle are compared to obtain the first drift angle. The second rotation angle and the fifth rotation angle are compared to obtain the second drift angle. The third rotation angle and the sixth rotation angle are compared to obtain the third drift angle. The temperature drift angle includes the first drift angle, the second drift angle and the third drift angle.
6. The temperature drift detection method according to claim 5, characterized in that, The step of detecting whether the temperature drift of the depth camera is qualified based on the temperature drift angle includes: The first drift angle, the second drift angle, and the third drift angle are compared with the preset standard angle, respectively. If the first drift angle, the second drift angle, and the third drift angle are all less than or equal to the preset standard angle, then the temperature drift of the depth camera is qualified. If at least one of the first drift angle, the second drift angle, and the third drift angle is greater than the preset standard angle, then the temperature drift of the depth camera is unqualified.
7. The temperature drift detection method according to claim 1, characterized in that, The length of the surface to be measured is L, the width of the surface to be measured is W, and the distance between the light-emitting surface of the depth camera and the surface to be measured is D. Then, the following conditions are met: Where FOV_L is the field of view in the L direction and FOV_W is the field of view in the W direction.
8. A depth camera, characterized in that, The depth camera includes a mounting plate, a light emitter, and a light receiver. The light emitter and the light receiver are disposed on the same surface of the mounting plate. The light emitter has a light-emitting surface for emitting light, and the light receiver has a photosensitive surface for receiving light. The depth camera performs detection using the temperature drift detection method as described in any one of claims 1 to 7.