Assembly error determination method and system, electronic equipment and readable storage medium
By taking speckle maps and calculating the speckle position relationship, the assembly error between VCSEL and DOE in structured light cameras is quickly and accurately determined, which solves the cost and unstable problem in the prior art and improves the depth recovery accuracy.
Patent Information
- Application Number
- CN202311112797.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-29
- Publication Date
- 2025-03-07
AI Technical Summary
The prior art is costly and unstable when determining the assembly error between VCSEL and DOE in a speckle projector, making it difficult to quickly and accurately obtain the real assembly error in actual use of a structured optical camera.
By taking a speckle map on the target test board, the location of each speckle is determined, and the first speckle and the second speckle are selected from the four target levels, and the assembly error, including the rotation angle and the inclination angle are calculated based on their positional relationship.
The real assembly error between VCSEL and DOE of the structured optical camera is achieved at a low cost and rapid basis, and the depth recovery accuracy of the structured optical camera is improved.
Smart Images
Figure CN119583788A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of machine vision technology, and in particular to an assembly error determination method, system, electronic device, and readable storage medium. Background Art
[0002] At present, the most dynamic technical branch in the field of machine vision is depth perception technology, and speckle structured light technology is an important part of depth perception technology. As the most common active stereo vision technology, speckle structured light technology has a wide range of applications in face recognition, autonomous driving, security monitoring and other fields. The speckle structured light camera is mainly composed of a speckle projector and an infrared lens. The speckle projector projects a speckle structured light pattern onto the target scene, and the infrared camera captures the projection of the speckle structured light pattern on the target scene, and then performs depth recovery to obtain the depth information of the target scene.
[0003] The speckle projector is composed of VCSEL (Vertical-Cavity Surface-Emitting Laser), collimator and DOE (Diffractive Optical Elements). Ideally, the relative position (bulk position) between the VCSEL and DOE of the same batch and specification of the speckle projector is fixed, but in actual production, due to problems with the assembly process, there will inevitably be assembly errors between the VCSEL and DOE. This assembly error is generally detected and calibrated by professional instruments when the speckle projector leaves the factory. This process requires high time and resource costs, and with the actual use of the structured light camera, the assembly error between the VCSEL and DOE may also change due to external forces or changes in the external environment, which has an adverse effect on the depth recovery effect of the structured light camera. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide an assembly error determination method, system, electronic device and readable storage medium, which can quickly and cheaply determine the actual assembly error between VCSEL and DOE, providing a basis for improving the depth recovery accuracy of structured light cameras.
[0005] To solve the above technical problems, an embodiment of the present application provides an assembly error determination method, comprising the following steps: determining the position of each speckle in an acquired speckle pattern, wherein the speckle pattern is obtained by photographing a target test plate by a structured light camera to be tested; selecting a first speckle and a second speckle at any order in the speckle pattern, and determining a speckle corresponding to the first speckle and a speckle corresponding to the second speckle in four target orders respectively; wherein a distance between the first speckle and the second speckle is greater than a first preset threshold, and a distance between a center of each of the target orders and a center of a zero order is the same; and determining an assembly error between a VCSEL and a DOE of the structured light camera to be tested according to a positional relationship between specks corresponding to each of the first specks and a positional relationship between specks corresponding to each of the second specks.
[0006] The embodiment of the present application further provides an assembly error determination system, comprising: a structured light camera to be tested, a target test board, an acquisition module, a speckle extraction module, a reference module, and an execution module; the structured light camera to be tested is used to photograph the target test board to obtain a speckle pattern; the acquisition module is used to acquire the speckle pattern; the speckle extraction module is used to determine the position of each speckle in the speckle pattern; the reference module is used to select a first speckle and a second speckle at any level in the speckle pattern, and respectively determine a speckle corresponding to the first speckle and a speckle corresponding to the second speckle in four target levels, wherein the distance between the first speckle and the second speckle is greater than a first preset threshold, and the distance between the center of each target level and the center of the zero level is the same; the execution module is used to determine the assembly error between the VCSEL and the DOE of the structured light camera to be tested according to the positional relationship between the speckles corresponding to each of the first speckles and the positional relationship between the speckles corresponding to each of the second speckles.
[0007] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned assembly error determination method.
[0008] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned assembly error determination method when executed by a processor.
[0009] The assembly error determination method, system, electronic device and readable storage medium provided by the embodiment of the present application first obtain a speckle pattern by photographing the target test board through the structured light camera to be tested, determine the position of each speckle in the acquired speckle pattern, then select the first speckle and the second speckle at any level in the speckle pattern, and respectively determine the speckle corresponding to the first speckle and the speckle corresponding to the second speckle in four target levels, the distance between the first speckle and the second speckle is greater than the first preset threshold, the distance between the center of each target level and the center of the zero level is the same, and finally, according to the positional relationship between the speckles corresponding to each first speckle and the positional relationship between the speckles corresponding to each second speckle, determine the assembly error between the VCSEL and the DOE of the structured light camera to be tested. Considering that in the industry, when determining the assembly error, it is usually calibrated on a dedicated calibration station when the speckle projector leaves the factory, the calibration process is costly, and the camera is unstable after actual use and is easily affected by the outside world. In the embodiments of the present application, the current real assembly error of the camera can be obtained at any time. It only needs to take a speckle image of the target test plate. The server finds the first speckle and the second speckle, and determines the positional relationship between the speckles corresponding to the two speckles at four levels. According to the difference between the positional relationship between the four speckles corresponding to the first speckle and the positional relationship between the four speckles corresponding to the second speckle, the real assembly error between the VCSEL and the DOE of the structured light camera to be tested can be determined quickly and at low cost, which provides a basis for improving the depth recovery accuracy of the structured light camera.
[0010] In addition, the determining the speckle corresponding to the first speckle and the speckle corresponding to the second speckle in four target levels respectively includes: determining the third speckle, the fourth speckle, the fifth speckle and the sixth speckle corresponding to the first speckle in the first target level, the second target level, the third target level and the fourth target level respectively; determining the seventh speckle, the eighth speckle, the ninth speckle and the tenth speckle corresponding to the second speckle in the first target level, the second target level, the third target level and the fourth target level respectively; the positional relationship between the speckles corresponding to the first speckles, The present invention comprises: the positional relationship between the third speckle and the fourth speckle, the positional relationship between the third speckle and the fifth speckle, the positional relationship between the fourth speckle and the sixth speckle, and the positional relationship between the fifth speckle and the sixth speckle; the positional relationship between the speckles corresponding to each of the second speckles comprises: the positional relationship between the seventh speckle and the eighth speckle, the positional relationship between the seventh speckle and the ninth speckle, the positional relationship between the eighth speckle and the tenth speckle, and the positional relationship between the ninth speckle and the tenth speckle. The positional relationship between the third speckle and the fourth speckle, and the positional relationship between the fifth speckle and the sixth speckle reflect the rotation of the speckle pattern relative to the y-axis of the image coordinate system, and the positional relationship between the third speckle and the fifth speckle, and the positional relationship between the fourth speckle and the sixth speckle reflect the rotation of the speckle pattern relative to the x-axis of the image coordinate system. According to the difference between the two rotations relative to the y-axis and the x-axis, the assembly error between the VCSEL and the DOE of the structured light camera to be tested can be accurately determined.
[0011] In addition, the assembly error includes a rotation angle, and the determining the assembly error between the VCSEL and the DOE of the structured light camera to be tested according to the positional relationship between the specks corresponding to each of the first specks and the positional relationship between the specks corresponding to each of the second specks includes: connecting the first midpoint and the second midpoint to obtain a first vector, and connecting the third midpoint and the fourth midpoint to obtain a second vector; wherein the first midpoint is the midpoint between the center point of the third speckle and the center point of the fourth speckle, the second midpoint is the midpoint between the center point of the fifth speckle and the center point of the sixth speckle, the third midpoint is the midpoint between the center point of the third speckle and the center point of the fifth speckle, and the fourth midpoint is the midpoint between the center point of the fourth speckle and the center point of the sixth speckle; connecting the fifth midpoint and the sixth midpoint to obtain a third vector, and connecting the seventh midpoint and the fourth midpoint to obtain a third vector. The method further comprises the steps of: obtaining a fourth vector by calculating eight midpoints of the seventh and eighth speckles; wherein the fifth midpoint is the midpoint between the center point of the seventh speckle and the center point of the eighth speckle, the sixth midpoint is the midpoint between the center point of the ninth speckle and the center point of the tenth speckle, the seventh midpoint is the midpoint between the center point of the seventh speckle and the center point of the ninth speckle, and the eighth midpoint is the midpoint between the center point of the eighth speckle and the center point of the tenth speckle; calculating a first difference between the first angle and the second angle; wherein the first angle is the angle between the second vector and the fourth vector, and the second angle is the angle between the first vector and the third vector; determining a rotation angle between the VCSEL and the DOE according to the first difference and a preset first corresponding relationship; wherein the first corresponding relationship is a corresponding relationship between the first difference and the rotation angle.
[0012] In addition, the assembly error also includes a first tilt angle and a second tilt angle, and the assembly error between the VCSEL and the DOE of the structured light camera to be tested is determined according to the positional relationship between the speckles corresponding to each of the first speckles and the positional relationship between the speckles corresponding to each of the second speckles, including: determining the first tilt angle between the VCSEL and the DOE according to the length of the first vector, the length of the third vector and a preset second corresponding relationship; wherein the second corresponding relationship is the corresponding relationship between the length of the first vector, the length of the third vector and the first tilt angle; determining the second tilt angle between the VCSEL and the DOE according to the length of the second vector, the length of the fourth vector and a preset third corresponding relationship; wherein the third corresponding relationship is the corresponding relationship between the length of the second vector, the length of the fourth vector and the second tilt angle.
[0013] In addition, the four target levels are: (-1, -1) level, (-1, 1) level, (1, -1) level and (1, 1) level.
[0014] In addition, after determining the assembly error between the VCSEL and DOE of the structured light camera to be tested, the method further includes: judging whether the second difference between the assembly error between the VCSEL and DOE of the structured light camera to be tested and the assembly error of the factory calibration is greater than a preset allowable threshold; if the second difference between the assembly error between the VCSEL and DOE of the structured light camera to be tested and the assembly error of the factory calibration is greater than the preset allowable threshold, then generating factory return repair information. If the difference between the determined actual assembly error and the factory calibration assembly error is too large, it means that the structured light camera has been severely damaged, the depth recovery accuracy cannot be guaranteed, and it needs to be returned to the factory for repair.
[0015] In addition, the determining the position of each speckle in the acquired speckle pattern includes: respectively determining the speckles corresponding to the rough speckles in the speckle pattern in the preset GT speckle pattern; selecting a first seed point in the rough speckle, and selecting a second seed point in the corresponding speckle in the GT speckle pattern, and determining the center point of the actual speckle corresponding to the rough speckle according to the first seed point and the point in the speckle pattern with the same coordinates as the second seed point; based on a preset search step, searching within a preset search range centered on the center point of the actual speckle to determine the length of the major and minor axes of the actual speckle; and determining the position of each speckle in the speckle pattern according to the center point of the actual speckle and the length of the major and minor axes of the actual speckle. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.
[0017] Figure 1 is a flow chart of an assembly error determination method provided by an embodiment of the present application;
[0018] Figure 2 is a schematic diagram of a target test board provided in one embodiment of the present application;
[0019] Figure 3 It is a schematic diagram of how many times a speckle pattern with 7×7 levels is divided, provided in one embodiment of the present application;
[0020] Figure 4 In one embodiment of the present application, a process for determining an assembly error between a VCSEL and a DOE of a structured light camera to be tested according to a positional relationship between specks corresponding to each first speckle and a positional relationship between specks corresponding to each second speckle Figure 1 ;
[0021] Figure 5is a schematic diagram of a first vector and a second vector provided in an embodiment of the present application;
[0022] Figure 6 is a schematic diagram of a first corresponding relationship provided in an embodiment of the present application;
[0023] Figure 7 In one embodiment of the present application, a process for determining an assembly error between a VCSEL and a DOE of a structured light camera to be tested according to a positional relationship between specks corresponding to each first speckle and a positional relationship between specks corresponding to each second speckle Figure 2 ;
[0024] Figure 8 is a flow chart for determining the position of each speckle in an acquired speckle pattern in an embodiment of the present application;
[0025] Fig. 9 is a schematic diagram of an assembly error determination system provided by another embodiment of the present application;
[0026] Fig.10 It is a schematic diagram of the structure of an electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION
[0027] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.
[0028] An embodiment of the present application relates to a method for determining an assembly error, which is applied to an electronic device, wherein the electronic device may be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described by taking a server as an example. The following is a specific description of the implementation details of the method for determining an assembly error of this embodiment. The following content is only provided for the convenience of understanding the implementation details and is not necessary for the implementation of this solution.
[0029] The specific process of the assembly error determination method of this embodiment can be as follows: Figure 1 As shown, including:
[0030] Step 101 : determining the position of each speckle in the acquired speckle pattern, where the speckle pattern is obtained by photographing a target test plate with a structured light camera to be tested.
[0031] In the specific implementation, the server controls the structured light camera to be tested to shoot at the target test board to obtain a speckle pattern. The server determines the assembly error between the VCSEL and DOE of the structured light camera to be tested based on the speckle pattern. The premise of this is that the server must perform speckle extraction on the speckle pattern, that is, determine the position of each speckle in the speckle pattern.
[0032] In some examples, the target test board can be Figure 2 As shown, the red block, green block, blue block and black block are rectangular and located at the four vertices of the test board. There are 4 black and white interval bands, which are located at the four sides of the test board. In the middle of the test board are gray blocks and white blocks of varying heights, which are arranged in a clockwise order of increasing height, that is, the height of the gray block in the upper left corner is d, the height of the white block in the upper right corner is d+d1, the height of the gray block in the lower right corner is d+2d1, and the height of the white block in the lower left corner is d+3d1.
[0033] In some examples, the server may perform operations such as connected domain detection, erosion and dilation, and morphological closing based on the brightness value (grayscale value) of the speckle image to determine the position of each speckle in the speckle image.
[0034] In some examples, the server may match each speckle in a preset GT speckle pattern on the speckle pattern based on a block matching algorithm to determine the position of each speckle in the speckle pattern.
[0035] Step 102 : selecting a first scattered spot and a second scattered spot at any level in the speckle pattern, and determining a scattered spot corresponding to the first scattered spot and a scattered spot corresponding to the second scattered spot in four target levels respectively.
[0036] Specifically, after the server determines the position of each speckle in the speckle pattern, it can select any level in the speckle pattern, select the first speckle and the second speckle in the selected level, and determine the speckle corresponding to the first speckle and the speckle corresponding to the second speckle in the four target levels respectively. The selection rule of the target level should ensure that the distance between the center of each target level and the center of the zero level is the same, and the selection rule of the first speckle and the second speckle should ensure that the distance between the first speckle and the second speckle is greater than a first preset threshold value, and the first preset threshold value can be set by a person skilled in the art according to actual needs.
[0037] In a specific implementation, the server determines the third speckle, the fourth speckle, the fifth speckle and the sixth speckle corresponding to the first speckle in the first target level, the second target level, the third target level and the fourth target level respectively, and then determines the seventh speckle, the eighth speckle, the ninth speckle and the tenth speckle corresponding to the second speckle in the first target level, the second target level, the third target level and the fourth target level respectively.
[0038] In some examples, the server may select the first speckle and the second speckle from the zero order, because the zero order is least affected by the pincushion distortion, and it is more scientific to select the first speckle and the second speckle from the zero order.
[0039] In some examples, since it is necessary to ensure that the distances between the centers of the target orders and the center of the zero order are the same, the selected target orders need to be axially symmetric in the speckle pattern. Figure 3 The speckle pattern with 7×7 levels is shown as an example (for the convenience of display, Figure 3 (The pincushion distortion effect is not shown), the server can select the four levels of (-1, -1), (-1, 1), (1, 1), (1, -1) as the target levels, or the four levels of (0, 1), (1, 0), (0, -1), (-1, 0) as the target levels, or even the four levels of (-2, -1), (-2, 1), (2, 1), (2, -1) as the target levels.
[0040] In some examples, the four target orders selected by the server are: (-1, -1) order, (-1, 1) order, (1, -1) order and (1, 1) order. These four orders are less affected by the pincushion distortion and are conducive to the determination of assembly errors.
[0041] Step 103 : determining an assembly error between the VCSEL and the DOE of the structured light camera to be tested according to the positional relationship between the scattered spots corresponding to the first scattered spots and the positional relationship between the scattered spots corresponding to the second scattered spots.
[0042] In some examples, after the server determines the third speckle, the fourth speckle, the fifth speckle and the sixth speckle corresponding to the first speckle in the first target level, the second target level, the third target level and the fourth target level respectively, and determines the seventh speckle, the eighth speckle, the ninth speckle and the tenth speckle corresponding to the second speckle in the first target level, the second target level, the third target level and the fourth target level respectively, the positional relationship between the third speckle and the fourth speckle, the positional relationship between the third speckle and the fifth speckle, the positional relationship between the fourth speckle and the sixth speckle, and the positional relationship between the seventh speckle and the eighth speckle, the positional relationship between the seventh speckle and the ninth speckle, the positional relationship between the eighth speckle and the tenth speckle, and the positional relationship between the ninth speckle and the tenth speckle, and determines the assembly error between the VCSEL and the DOE of the structured light camera to be tested according to these positional relationships.
[0043] Specifically, the positional relationship between the third speckle and the fourth speckle, and the positional relationship between the fifth speckle and the sixth speckle reflect the rotation of the speckle pattern relative to the y-axis of the image coordinate system (similarly, the positional relationship between the seventh speckle and the eighth speckle, and the positional relationship between the ninth speckle and the tenth speckle also reflect the rotation of the speckle pattern relative to the y-axis of the image coordinate system), the positional relationship between the third speckle and the fifth speckle, and the positional relationship between the fourth speckle and the sixth speckle reflect the rotation of the speckle pattern relative to the x-axis of the image coordinate system (similarly, the positional relationship between the seventh speckle and the ninth speckle, and the positional relationship between the eighth speckle and the tenth speckle also reflect the rotation of the speckle pattern relative to the x-axis of the image coordinate system). According to the difference between the two rotations relative to the y-axis and the x-axis, the assembly error between the VCSEL and the DOE of the structured light camera to be tested can be accurately determined.
[0044] In some examples, the server may connect the center point of the third speckle with the center point of the fourth speckle to obtain a first straight line, connect the center point of the fifth speckle with the center point of the sixth speckle to obtain a second straight line, and calculate a first slope difference between the first straight line and the second straight line; connect the center point of the seventh speckle with the center point of the eighth speckle to obtain a third straight line, connect the center point of the ninth speckle with the center point of the tenth speckle to obtain a fourth straight line, and calculate a second slope difference between the third straight line and the fourth straight line; connect the center point of the third speckle with the center point of the fifth speckle to obtain a fifth straight line, connect the center point of the fourth speckle with the center point of the sixth speckle to obtain a sixth straight line, and calculate a third slope difference between the fifth straight line and the sixth straight line; connect the center point of the seventh speckle with the center point of the ninth speckle to obtain a seventh straight line, connect the center point of the eighth speckle with the center point of the tenth speckle to obtain an eighth straight line, and calculate a fourth slope difference between the seventh straight line and the eighth straight line; and then determine the assembly error between the VCSEL and the DOE of the structured light camera to be tested according to the difference between the first slope difference and the second slope difference, and the difference between the third slope difference and the fourth slope difference.
[0045] In this embodiment, the server first obtains a speckle pattern by photographing the target test board through the structured light camera to be tested, determines the position of each scattered spot in the obtained speckle pattern, then selects the first scattered spot and the second scattered spot at any level in the speckle pattern, and respectively determines the scattered spot corresponding to the first scattered spot and the scattered spot corresponding to the second scattered spot in four target levels, the distance between the first scattered spot and the second scattered spot is greater than the first preset threshold, and the distance between the center of each target level and the center of the zero level is the same, and finally, according to the positional relationship between the scattered spots corresponding to each first scattered spot and the positional relationship between the scattered spots corresponding to each second scattered spot, the assembly error between the VCSEL and the DOE of the structured light camera to be tested is determined. Considering that in the industry, when determining the assembly error, it is usually calibrated on a dedicated calibration station when the speckle projector leaves the factory, the calibration process is costly, and the camera is unstable after actual use and is easily affected by the outside world. In the embodiments of the present application, the current real assembly error of the camera can be obtained at any time. It only needs to take a speckle pattern of the target test plate. The server finds the first scattered spot and the second scattered spot, and determines the positional relationship between the scattered spots corresponding to the two scattered spots at four levels. According to the difference between the positional relationship between the four scattered spots corresponding to the first scattered spot and the positional relationship between the four scattered spots corresponding to the second scattered spot, the real assembly error between the VCSEL and the DOE of the structured light camera to be tested can be determined quickly and at low cost, which provides a basis for improving the depth recovery accuracy of the structured light camera.
[0046] In some embodiments, the assembly error includes a rotation angle, that is, a rotation angle of the VCSEL of the structured light camera to be tested compared to the DOE. The server determines the assembly error between the VCSEL of the structured light camera to be tested and the DOE according to the positional relationship between the speckles corresponding to the first speckles and the positional relationship between the speckles corresponding to the second speckles. The assembly error can be determined by: Figure 4 The steps shown are implemented, specifically including:
[0047] Step 201, connecting the first midpoint and the second midpoint to obtain a first vector, and connecting the third midpoint and the fourth midpoint to obtain a second vector.
[0048] In a specific implementation, the first midpoint is the midpoint between the center point of the third speckle and the center point of the fourth speckle, the second midpoint is the midpoint between the center point of the fifth speckle and the center point of the sixth speckle, the third midpoint is the midpoint between the center point of the third speckle and the center point of the fifth speckle, and the fourth midpoint is the midpoint between the center point of the fourth speckle and the center point of the sixth speckle. The server connects the first midpoint and the second midpoint to obtain a first vector, and connects the third midpoint and the fourth midpoint to obtain a second vector. The first vector and the second vector can be as follows: Figure 5 shown.
[0049] Step 202, connecting the fifth midpoint and the sixth midpoint to obtain a third vector, and connecting the seventh midpoint and the eighth midpoint to obtain a fourth vector.
[0050] In a specific implementation, the fifth midpoint is the midpoint between the center point of the seventh speckle and the center point of the eighth speckle, the sixth midpoint is the midpoint between the center point of the ninth speckle and the center point of the tenth speckle, the seventh midpoint is the midpoint between the center point of the seventh speckle and the center point of the ninth speckle, the eighth midpoint is the midpoint between the center point of the eighth speckle and the center point of the tenth speckle, and the server connects the fifth midpoint and the sixth midpoint to obtain a third vector, and connects the seventh midpoint and the eighth midpoint to obtain a fourth vector.
[0051] Step 203: Calculate a first difference between the first angle and the second angle.
[0052] In a specific implementation, after the server obtains the first vector, the second vector, the third vector and the fourth vector, it can determine the first angle and the second angle, the first angle being the angle between the second vector and the fourth vector, and the second angle being the angle between the first vector and the third vector. After the server determines the first angle and the second angle, it can further calculate the first difference between the first angle and the second angle.
[0053] In some examples, the first angle may be recorded as ry, the second angle may be recorded as rx, and the first difference may be recorded as ry-rx.
[0054] Step 204: Determine a rotation angle between the VCSEL and the DOE according to the first difference and a preset first corresponding relationship.
[0055] In a specific implementation, after determining the first difference, the server can determine the rotation angle between the VCSEL and the DOE according to the first difference and a preset first corresponding relationship. The preset first corresponding relationship is the corresponding relationship between the first difference and the rotation angle. The first corresponding relationship can be pre-fitted, that is, the VCSEL and DOE of a structured light camera are pre-installed at different rotation angles, and the ry-rx values at different rotation angles are recorded and linear fitting is performed to obtain the first corresponding relationship. The first corresponding relationship is obtained through a large number of experiments and is scientific and reliable.
[0056] In some examples, the first correspondence may be as follows Figure 6 As shown, ry-rx is the first difference, is the rotation angle between VCSEL and DOE.
[0057] In some embodiments, the assembly error further includes a first tilt angle and a second tilt angle, the first tilt angle being the tilt angle of the VCSEL of the structured light camera to be tested compared to the DOE in the y-axis direction, and the second tilt angle being the tilt angle of the VCSEL of the structured light camera to be tested compared to the DOE in the x-axis direction. The server determines the assembly error between the VCSEL and the DOE of the structured light camera to be tested according to the positional relationship between the speckles corresponding to the first speckles and the positional relationship between the speckles corresponding to the second speckles, which can be determined by the following method: Figure 7 The steps shown are implemented, specifically including:
[0058] Step 301 : determining a first tilt angle between the VCSEL and the DOE according to the length of the first vector, the length of the third vector and a preset second corresponding relationship.
[0059] Step 302: Determine a second tilt angle between the VCSEL and the DOE according to the length of the second vector, the length of the fourth vector and a preset third corresponding relationship.
[0060] In a specific implementation, after the server determines the first vector, the second vector, the third vector, and the fourth vector, the length of each of the first vector, the second vector, the third vector, and the fourth vector can be further determined. The server can determine the first tilt angle between the VCSEL and the DOE according to the length of the first vector, the length of the third vector, and the preset second corresponding relationship, and then determine the second tilt angle between the VCSEL and the DOE according to the length of the second vector, the length of the fourth vector, and the preset third corresponding relationship.
[0061] In some examples, the server records the length of the first vector as h1, the length of the second vector as w1, the length of the third vector as h2, and the length of the fourth vector as w2. The first tilt angle is tilty, and the second tilt angle is tiltx. The preset second correspondence is the correspondence between the length of the first vector, the length of the third vector, and the first tilt angle, specifically, tilty and The preset third corresponding relationship is the corresponding relationship between the length of the second vector, the length of the fourth vector and the second tilt angle, specifically, the corresponding relationship between tilt and The second and third corresponding relationships can be pre-fitted, that is, a VCSEL of a structured light camera is pre-installed with a DOE at different first and second tilt angles, and the corresponding relationships under different first and second tilt angles are recorded. The value and The second and third corresponding relationships are obtained through a large number of experiments and are scientific and reliable.
[0062] In some examples, the tilt angle can be expressed by a plane equation. Assume that the plane equation of the VCSEL of the structured light camera to be tested is z=Ax+By+C. Using the back focal length C of the camera to be tested, as well as The plane equation of the VCSEL of the structured light camera to be tested can be solved.
[0063] In one embodiment, after determining the assembly error between the VCSEL and DOE of the structured light camera to be tested, the server may also determine whether the second difference between the assembly error between the VCSEL and DOE of the structured light camera to be tested and the assembly error of the factory calibration is greater than a preset allowable threshold, and if the second difference between the assembly error between the VCSEL and DOE of the structured light camera to be tested and the assembly error of the factory calibration is greater than the preset allowable threshold, then the return to factory repair information is generated. Considering that if the difference between the determined actual assembly error and the factory calibration assembly error is too large, it means that the structured light camera has been severely damaged, the depth recovery accuracy cannot be guaranteed, and it needs to be returned to the factory for repair.
[0064] In one embodiment, the server determines the position of each speckle in the acquired speckle image by: Figure 8 The steps shown are implemented, specifically including:
[0065] Step 401 : determining the speckles corresponding to the rough speckles in the speckle pattern in the preset GT speckle pattern respectively.
[0066] In some examples, the position of the speckle can be roughly determined directly from the speckle map according to the brightness, which can be called rough speckle. The server traverses the rough speckles in the speckle map, obtains the coordinates of the center point of the circumscribed rectangle of the current rough speckle, and finds the point with the same coordinates in the GT speckle map, and takes the speckle with the same coordinates in the GT speckle map as the speckle corresponding to the current rough speckle in the GT speckle map.
[0067] Step 402, select a first seed point in the rough speckle, select a second seed point in the corresponding speckle in the GT speckle map, and determine the center point of the actual speckle corresponding to the rough speckle according to the first seed point and the point in the speckle map with the same coordinates as the second seed point.
[0068] Specifically, after determining the speckle corresponding to the rough speckle in the GT speckle map, the server can first select a first seed point in the rough speckle, select a second seed point from the speckle corresponding to the rough speckle in the GT speckle map, and continue to find a point with the same coordinates as the second seed point in the speckle map, and finally determine the center point of the actual speckle corresponding to the rough speckle according to the first seed point and the point with the same coordinates as the second seed point in the speckle map, that is, realize the basic positioning of the actual speckle corresponding to the rough speckle.
[0069] In some examples, the server may select the centroid of the rough speckle as the first seed point, or the center point of the circumscribed rectangle of the rough speckle as the first seed point, or the center point of the inscribed rectangle of the rough speckle as the first seed point.
[0070] In some examples, the first seed point is the centroid point of the rough speckle, and the second seed point is the centroid point of the speckle corresponding to the rough speckle in the GT speckle map. The server can connect the centroid point of the rough speckle and the point in the speckle map with the same coordinates as the centroid point of the speckle corresponding to the rough speckle in the GT speckle map, take the midpoint of the line between the two as the positioning reference point, and perform two-dimensional matching within a preset positioning search range centered on the positioning reference point according to a preset matching algorithm and the centroid point of the speckle corresponding to the rough speckle in the GT speckle map, and use the point with the greatest matching similarity with the centroid point of the speckle corresponding to the rough speckle in the GT speckle map within the positioning search range as the center point of the actual speckle corresponding to the rough speckle.
[0071] In some examples, the server may determine the centroid point of the rough speckle according to the pixel value of each pixel point in the rough speckle.
[0072] Step 403: Based on a preset search step length, a search is performed within a preset search range centered on the center point of the actual speckle to determine the lengths of the major and minor axes of the actual speckle.
[0073] In a specific implementation, after the server determines the center point of the actual speckle corresponding to the rough speckle, the reference positioning of the actual speckle is completed. Then, the speckle extraction can be performed based on the center point of the actual speckle. The server can search within a preset search range centered on the center point of the actual speckle based on a preset search step size to determine the lengths of the major and minor axes of the actual speckle.
[0074] In some cases, both the rough speckle and the actual speckle are elliptical due to the diffraction characteristics of the DOE.
[0075] In some examples, the server may perform ellipse fitting on the speckle corresponding to the rough speckle in the GT speckle image to obtain the ellipse corresponding to the speckle corresponding to the rough speckle in the GT speckle image, and calculate the sum of the pixel values of each pixel point in the ellipse corresponding to the speckle corresponding to the rough speckle in the GT speckle image, and then based on a preset search step, establish a number of ellipses with the center point of the actual speckle as the center in the speckle image, and calculate the sum of the pixel values of each pixel point in these ellipses, the server selects the ellipse whose sum of the pixel values of each pixel point in these ellipses is closest to the sum of the pixel values of each pixel point in the ellipse corresponding to the speckle corresponding to the rough speckle in the GT speckle image, and uses the major and minor axis lengths of the selected ellipse as the major and minor axis lengths of the actual speckle corresponding to the first speckle.
[0076] Step 404: determine the position of each speckle in the speckle pattern according to the center point of the actual speckle and the lengths of the major and minor axes of the actual speckle.
[0077] Specifically, after determining the center point of the actual speckle and the lengths of the major and minor axes of the actual speckle, the server can extract the actual speckle corresponding to the rough speckle according to the center point of the actual speckle and the lengths of the major and minor axes of the actual speckle.
[0078] In one example, after determining the center point of the actual speckle and the lengths of the major and minor axes of the actual speckle, the server can construct an ellipse with the center point of the actual speckle as the center and according to the lengths of the major and minor axes of the actual speckle. The ellipse is the actual speckle corresponding to the rough speckle.
[0079] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.
[0080] Another embodiment of the present application relates to an assembly error determination system. The implementation details of the assembly error determination system of this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for the implementation of this solution. The schematic diagram of the assembly error determination system of this embodiment can be as follows Fig. 9 As shown, it includes a structured light camera to be tested 501 , a target test board 502 , an acquisition module 503 , a speckle extraction module 504 , a reference module 505 and an execution module 506 .
[0081] The structured light camera 501 to be tested is used to photograph the target test board 502 to obtain a speckle pattern.
[0082] The acquisition module 503 is used to acquire a speckle pattern from the structured light camera 501 to be tested.
[0083] The speckle extraction module 504 is used to determine the position of each speckle in the speckle pattern.
[0084] The reference module 505 is used to select the first speckle and the second speckle at any level in the speckle pattern, and respectively determine the speckle corresponding to the first speckle and the speckle corresponding to the second speckle in four target levels, the distance between the first speckle and the second speckle is greater than a first preset threshold, and the distance between the center of each target level and the center of the zero level is the same.
[0085] The execution module 506 is used to determine the assembly error between the VCSEL and the DOE of the structured light camera 501 to be tested according to the positional relationship between the speckles corresponding to the first speckles and the positional relationship between the speckles corresponding to the second speckles.
[0086] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.
[0087] Another embodiment of the present application relates to an electronic device, such as Fig.10 As shown, it includes: at least one processor 601; and a memory 602 that is communicatively connected to the at least one processor 601; wherein the memory 602 stores instructions that can be executed by the at least one processor 601, and the instructions are executed by the at least one processor 601 so that the at least one processor 601 can execute the assembly error determination method in the above-mentioned embodiments.
[0088] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.
[0089] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.
[0090] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.
[0091] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.
[0092] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.
Claims
1. A method for determining an assembly error, characterized in that: include: Determine the position of each speckle in the acquired speckle pattern, wherein the speckle pattern is obtained by photographing a target test plate by a structured light camera to be tested; A first speckle and a second speckle are selected at any level in the speckle pattern, and a speckle corresponding to the first speckle and a speckle corresponding to the second speckle are determined in four target levels respectively; wherein a distance between the first speckle and the second speckle is greater than a first preset threshold, and a distance between a center of each of the target levels and a center of a zero level is the same; An assembly error between the VCSEL and the DOE of the structured light camera to be tested is determined according to a positional relationship between the speckles corresponding to each of the first speckles and a positional relationship between the speckles corresponding to each of the second speckles.
2. The assembly error determination method according to claim 1, characterized in that: The determining, in four target levels, a speckle corresponding to the first speckle and a speckle corresponding to the second speckle, respectively, comprises: determining a third speckle, a fourth speckle, a fifth speckle, and a sixth speckle corresponding to the first speckle in a first target level, a second target level, a third target level, and a fourth target level, respectively; determining a seventh speckle, an eighth speckle, a ninth speckle, and a tenth speckle corresponding to the second speckle in the first target level, the second target level, the third target level, and the fourth target level, respectively; The positional relationship between the speckles corresponding to the first speckles includes: a positional relationship between the third speckle and the fourth speckle, a positional relationship between the third speckle and the fifth speckle, a positional relationship between the fourth speckle and the sixth speckle, and a positional relationship between the fifth speckle and the sixth speckle; The positional relationship between the speckles corresponding to each of the second speckles includes: The positional relationship between the seventh speckle and the eighth speckle, the positional relationship between the seventh speckle and the ninth speckle, the positional relationship between the eighth speckle and the tenth speckle, and the positional relationship between the ninth speckle and the tenth speckle.
3. The assembly error determination method according to claim 2, characterized in that: The assembly error includes a rotation angle, and determining the assembly error between the VCSEL and the DOE of the structured light camera to be tested according to the positional relationship between the speckles corresponding to the first speckles and the positional relationship between the speckles corresponding to the second speckles includes: The first midpoint and the second midpoint are connected to obtain a first vector, and the third midpoint and the fourth midpoint are connected to obtain a second vector; wherein the first midpoint is the midpoint between the center point of the third speckle and the center point of the fourth speckle, the second midpoint is the midpoint between the center point of the fifth speckle and the center point of the sixth speckle, the third midpoint is the midpoint between the center point of the third speckle and the center point of the fifth speckle, and the fourth midpoint is the midpoint between the center point of the fourth speckle and the center point of the sixth speckle; The third vector is obtained by connecting the fifth midpoint and the sixth midpoint, and the fourth vector is obtained by connecting the seventh midpoint and the eighth midpoint; wherein the fifth midpoint is the midpoint between the center point of the seventh speckle and the center point of the eighth speckle, the sixth midpoint is the midpoint between the center point of the ninth speckle and the center point of the tenth speckle, the seventh midpoint is the midpoint between the center point of the seventh speckle and the center point of the ninth speckle, and the eighth midpoint is the midpoint between the center point of the eighth speckle and the center point of the tenth speckle; Calculate a first difference between the first angle and the second angle; wherein the first angle is the angle between the second vector and the fourth vector, and the second angle is the angle between the first vector and the third vector; According to the first difference and a preset first corresponding relationship, a rotation angle between the VCSEL and the DOE is determined; wherein the first corresponding relationship is a corresponding relationship between the first difference and the rotation angle.
4. The assembly error determination method according to claim 3, characterized in that: The assembly error further includes a first tilt angle and a second tilt angle, and determining the assembly error between the VCSEL and the DOE of the structured light camera to be tested according to the positional relationship between the speckles corresponding to the first speckles and the positional relationship between the speckles corresponding to the second speckles includes: Determine a first tilt angle between the VCSEL and the DOE according to the length of the first vector, the length of the third vector and a preset second corresponding relationship; wherein the second corresponding relationship is a corresponding relationship between the length of the first vector, the length of the third vector and the first tilt angle; The second tilt angle between the VCSEL and the DOE is determined according to the length of the second vector, the length of the fourth vector and a preset third corresponding relationship; wherein the third corresponding relationship is the corresponding relationship between the length of the second vector, the length of the fourth vector and the second tilt angle.
5. The assembly error determination method according to any one of claims 1 to 4, characterized in that: The four target levels are: (-1, -1) level, (-1, 1) level, (1, -1) level and (1, 1) level.
6. The assembly error determination method according to any one of claims 1 to 4, characterized in that: After determining the assembly error between the VCSEL and the DOE of the structured light camera to be tested, the method further includes: Determine whether a second difference between an assembly error between a VCSEL and a DOE of the structured light camera to be tested and an assembly error of an out-of-box calibration is greater than a preset allowable threshold; If the second difference between the assembly error between the VCSEL and the DOE of the structured light camera to be tested and the factory calibration assembly error is greater than the preset allowable threshold, then the return to factory repair information is generated.
7. The assembly error determination method according to any one of claims 1 to 4, characterized in that: The step of determining the position of each speckle in the acquired speckle pattern comprises: respectively determining the speckles corresponding to the rough speckles in the speckle pattern in the preset GT speckle pattern; Selecting a first seed point in the rough speckle, and selecting a second seed point in the corresponding speckle in the GT speckle map, and determining a center point of an actual speckle corresponding to the rough speckle according to the first seed point and a point in the speckle map having the same coordinates as the second seed point; Based on a preset search step, searching within a preset search range centered on the center point of the actual speckle to determine the lengths of the major and minor axes of the actual speckle; The position of each speckle in the speckle pattern is determined according to the center point of the actual speckle and the lengths of the major and minor axes of the actual speckle.
8. An assembly error determination system, characterized in that: include: The structured light camera to be tested, the target test board, the acquisition module, the speckle extraction module, the reference module, and the execution module; The structured light camera to be tested is used to photograph the target test plate to obtain a speckle pattern; The acquisition module is used to acquire the speckle pattern; The speckle extraction module is used to determine the position of each speckle in the speckle pattern; The reference module is used to select a first speckle and a second speckle at any level in the speckle pattern, and determine a speckle corresponding to the first speckle and a speckle corresponding to the second speckle in four target levels respectively, wherein the distance between the first speckle and the second speckle is greater than a first preset threshold, and the distance between the center of each target level and the center of the zero level is the same; The execution module is used to determine an assembly error between the VCSEL and the DOE of the structured light camera to be tested according to a positional relationship between the speckles corresponding to the first speckles and a positional relationship between the speckles corresponding to the second speckles.
9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the assembly error determination method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the assembly error determination method according to any one of claims 1 to 7 is implemented.