Method, device and storage medium for determining parameters of phase height conversion mapping model
By setting multiple calibration blocks of different heights on the calibration plate, positioning the calibration blocks using shadow areas, combining pixel height and phase values, the rapid determination of the phase height conversion mapping model parameters is achieved, and the low efficiency of parameter determination in the prior art is solved. The problem of high computing resource utilization improves efficiency and accuracy.
Patent Information
- Application Number
- CN202011511827.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-18
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-12-18
AI Technical Summary
When determining the parameters of the phase height conversion mapping model, the data volume is large and the computing resource occupies high, resulting in low parameter determination efficiency.
By receiving the calibration plate image information sent by the imaging device, the shadow area of the calibration block and its corresponding area are determined, and the parameters of the phase height conversion mapping model are determined using pixel height, coordinates and phase values, so that parameter calibration can be completed by a single scan. Simplify the process and save computing resources.
It improves the efficiency and accuracy of parameter determination, reduces the amount of data, reduces the system's computing resource occupation, and enhances the robustness of region determination.
Smart Images

Figure CN114648589B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to image processing technology, and in particular to a method, device and storage medium for determining parameters of a phase height conversion mapping model. Background Art
[0002] At present, the processing industry generally adopts 3D Automatic Optic Inspection (3DAOI) system, which uses the principle of structured light imaging to obtain 3D point cloud data of target objects in the target area, so as to detect the target area. Specifically, the sinusoidal fringe phase value generated by the phase shift method is first used as the encoding information to mark the object. The phase value is projected onto the object by the projector in the form of light intensity. After being modulated by the object, it is collected by the corresponding pixel of the camera in the form of grayscale value. After decoding, the corresponding sinusoidal phase can be parsed. After obtaining the sinusoidal phase, the height information of the corresponding position is generated through the phase height conversion mapping model, and combined with the coordinates of the pixel, the three-dimensional data of the target object can be obtained.
[0003] In the prior art, when determining the parameters of the phase-height conversion mapping model, a standard calibration block with a known height is usually used for calibration, and the phase maps corresponding to different calibration blocks are collected one by one during calibration. Then, the least squares method is used to solve the phase maps to obtain the parameters of the phase-height conversion mapping model.
[0004] However, when the parameters of the phase height conversion mapping model are determined by the above method to complete the calibration of the phase height conversion mapping model, due to the large amount of collected data, it will not only occupy more computing resources but also make the parameter determination efficiency low. Summary of the Invention
[0005] To solve the problems existing in the prior art, the present application provides a method, device and storage medium for determining parameters of a phase height conversion mapping model, which can reduce the occupied computing resources.
[0006] In a first aspect, an embodiment of the present application provides a method for determining parameters of a phase height conversion mapping model, comprising:
[0007] Receive image information of the calibration plate sent by the camera device;
[0008] Determining a plurality of first areas in the image information, where the first areas are shadow areas corresponding to the calibration blocks included in the calibration plate;
[0009] Determine a second area corresponding to the first area, where the second area is an area where the calibration block is located, and pixels included in at least two second areas have target areas with different heights;
[0010] Determine the parameters of the phase-to-height conversion mapping model based on the height of the target area corresponding to the pixels in the second area, the pixel coordinates, and the phase values corresponding to the pixels. The phase-to-height conversion mapping model is used to determine the height of the target area corresponding to each pixel in the object to be detected.
[0011] In the embodiment of the present application, since the pixels included in at least two second regions correspond to target regions with different heights, only a single scan of the calibration plate is required to determine the parameters of the phase-height conversion mapping model. This not only simplifies the parameter determination process and improves its efficiency, but also reduces the amount of data and conserves system computing resources. Furthermore, since the shadow area corresponding to the calibration block can be used to accurately locate the region where the calibration block is located to complete the determination of the phase-height conversion mapping model parameters, the accuracy and robustness of region determination can be improved.
[0012] In a second aspect, an embodiment of the present application provides a device for determining phase height conversion mapping model parameters, comprising:
[0013] A receiving module, used to receive image information of the calibration plate sent by the camera device;
[0014] a determining module, configured to determine a plurality of first areas in the image information, wherein the first areas are shadow areas corresponding to the calibration blocks included in the calibration plate;
[0015] The determining module is further configured to determine a second area corresponding to the first area, where the second area is an area where the calibration block is located, and where pixels included in at least two first areas correspond to target areas with different heights;
[0016] The determination module is further used to determine the parameters of the phase-height conversion mapping model based on the height of the target area corresponding to the pixels in the second area, the pixel coordinates and the phase values corresponding to the pixels. The phase-height conversion mapping model is used to determine the height of the target area corresponding to each pixel in the object to be detected.
[0017] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0018] processor;
[0019] Memory; and
[0020] computer programs;
[0021] The computer program is stored in the memory and configured to be executed by the processor, and the computer program includes instructions for executing the method according to the first aspect.
[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a server to execute the method described in the first aspect.
[0023] The present application provides a method, apparatus and storage device for determining parameters of a phase height conversion mapping model, which receives image information of a calibration plate sent by an imaging device, determines multiple first areas in the image information, the first area being a shadow area corresponding to a calibration block included in the calibration plate, and determines a second area corresponding to the first area, the second area being the area where the calibration block is located, and the heights of target areas corresponding to pixels in at least two second areas are different, and then determines the parameters of the phase height conversion mapping model based on the heights of the target areas corresponding to the pixels in the second area, the pixel coordinates and the phase values corresponding to the pixels, so as to calibrate the phase height conversion mapping model, wherein the phase height conversion mapping model is used to determine the height of the target area corresponding to each pixel in the object to be detected. Since the heights of the target areas corresponding to the pixels in at least two second areas are different, only a single scan of the calibration plate is required to determine the parameters of the phase height conversion mapping model, thereby not only simplifying the parameter determination process and improving the efficiency of parameter determination, but also reducing the amount of data and saving computing resources of the system. Furthermore, since the area where the calibration block is located can be accurately located through the shadow area corresponding to the calibration block to complete the determination of the phase height conversion mapping model parameters, the accuracy and robustness of area determination can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0025] Figure 1a It is a front view of the calibration block in the prior art;
[0026] Figure 1b A top view of a calibration block in the prior art;
[0027] Figure 2 A schematic diagram of the system architecture of a method for determining parameters of a phase height conversion mapping model provided in an embodiment of the present application;
[0028] Figure 3 A flow chart of a method for determining parameters of a phase height conversion mapping model provided in an embodiment of the present application;
[0029] Figure 4 A schematic diagram of processing image information;
[0030] Figure 5 A flowchart of another method for determining phase height conversion mapping model parameters provided in an embodiment of the present application;
[0031] Figure 6 A schematic diagram of a device for determining parameters of a phase height conversion mapping model provided in an embodiment of the present application;
[0032] Figure 7 A schematic structural diagram of an electronic device embodiment provided in an embodiment of the present application. DETAILED DESCRIPTION
[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0034] The terms "first," "second," "third," and "fourth," etc. (if any) in the description and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this way are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products, or apparatus.
[0035] The method for determining the phase height conversion mapping model parameters provided in the embodiment of the present application is applied to the detection scenario of the target area or target object in the processing industry. For example, it can be applied to the scenario of high-speed, high-precision quantitative defect detection on the printed circuit board (PCB) production line.
[0036] At present, 3D AOI systems are usually used to mark objects using the sinusoidal fringe phase value generated based on the phase shift method as coding information. After the projection device projects the sinusoidal fringe pattern on the calibration plate in the form of light intensity, the camera device captures the sinusoidal fringe pattern and decodes it to parse the corresponding sinusoidal phase. After obtaining the sinusoidal phase, the height information of the corresponding position is generated through the phase height conversion mapping model shown in the following formula (1). Combined with the pixel coordinates, the three-dimensional data of the target object can be obtained:
[0037]
[0038] Wherein, Z is the height of the target area corresponding to the target pixel in the image containing the target object captured by the camera device. The target pixel can be any pixel in the image. The target area corresponding to the target pixel can be understood as the area corresponding to the target pixel in the target object. Therefore, the area is the area within the target object. a1, a2…a 20 is the parameter of the phase height conversion mapping model, (u, v) is the coordinate of the target pixel, is the sinusoidal phase corresponding to the target pixel. As can be seen from formula (1), the height of the target area corresponding to each pixel has a one-to-one correspondence with its corresponding phase, which satisfies the polynomial model shown in formula (1). This polynomial model is the phase-to-height conversion mapping model.
[0039] In the prior art, in order to obtain the parameters a1, a2...a of the phase height conversion mapping model, 20 , usually a standard calibration block with a known height is used for calibration. During calibration, the phase map corresponding to each calibration block needs to be collected, and the parameters of the phase-height conversion mapping model can be solved using the least squares method. Figure 1a is a front view of the calibration block in the prior art, Figure 1b is a top view of the calibration block in the prior art, such as Figure 1a-Figure 1b As shown, the standard calibration blocks with known heights include calibration block A, calibration block B, calibration block C and calibration block D. The camera equipment needs to take pictures of calibration block A, calibration block B, calibration block C and calibration block D respectively to collect the phase map corresponding to each calibration block, and use the least squares method to solve the parameters of the phase-to-height conversion mapping model based on the collected phase map.
[0040] However, the aforementioned method for determining the parameters of the phase-height conversion mapping model requires four calibration blocks, requiring four field-of-view alignments and image acquisitions. This makes the entire process very cumbersome and results in inefficient parameter determination. Furthermore, the need for four image acquisitions results in a large amount of collected and processed data, placing high demands on memory and video memory, and consuming significant computing resources.
[0041] In an embodiment of the present application, in order to simplify the calibration process, a plurality of calibration blocks of different heights and separated from each other can be set on a calibration plate, and all of them are within the field of view of the camera device. In this way, during calibration, only a single scan of the calibration plate is required to complete the acquisition of calibration blocks of multiple heights, instead of requiring four scans to complete the calibration as in the prior art. In this embodiment, each calibration block has its own corresponding height, and each calibration block of the calibration plate has a different height. The calibration blocks of different heights are separated from each other by a certain distance, so that the shadows produced by the projected calibration blocks will not be affected by the adjacent calibration blocks. When using the above-mentioned calibration plate, since a calibration plate includes a plurality of calibration blocks of different heights and separated from each other, it is very important to effectively distinguish the different height areas in the calibration plate. Those skilled in the art will understand that in a 3D AOI system, the camera device usually captures images directly above the calibration plate, while the projection device projects on the side and above the calibration plate. Then, calibration blocks of different heights will produce corresponding shadow areas due to occlusion. Therefore, in the embodiment of the present application, the positioning of calibration blocks of different heights can be completed based on the above-mentioned shadow areas to effectively distinguish different calibration blocks. After distinguishing different calibration blocks, the parameters of the phase height conversion mapping model can be determined based on the height, pixel coordinates and phase values corresponding to the pixels in each calibration block according to the target area. Using the above method, on the one hand, since only one image data needs to be collected, the operation process can be simplified and the efficiency of parameter determination can be improved. On the other hand, the amount of data can be greatly reduced, saving a lot of computing resources. On the other hand, since the calibration block can be positioned through the shadow area, the accuracy of positioning can be improved.
[0042] Before describing the method for determining the parameters of the phase height conversion mapping model of the present application, Figure 2 Let's understand the system architecture to which this application applies.
[0043] Figure 2 A schematic diagram of the system architecture of the method for determining the phase height conversion mapping model parameters provided in the embodiment of the present application is shown as follows: Figure 2 As shown, the system includes a projection device 201, a camera device 202 and a translation stage 203. The optical axis of the camera device 202 is perpendicular to the horizontally placed translation stage 203. The optical axis of the projection device 201 forms an angle with the translation stage 203, and the optical axis of the camera device 202 and the optical axis of the projection device 201 are controlled to be in the same plane.
[0044] Among them, the projection device 201 can be a digital projector, and the digital projector can be a digital liquid crystal projection device (Liquid Crystal Display, LCD) projector, a digital micromirror projection device (DMD projector) or a silicon substrate liquid crystal projection device (Liquid Crystal on Silicon, LCOS projector). A computer image processing system can be used to conveniently generate a grayscale stripe pattern and write it into the digital projection device.
[0045] The imaging device 202 may be a camera, which may be a charge coupled device, a liquid crystal device, a spatial light modulation device, a complementary metal oxide semiconductor (CMOS) device, or a digital camera.
[0046] In addition, the system also includes an electronic device (not shown) with a processor, which can communicate with the projection device 201 and the camera device 202 via wired or wireless means to control the projection device 201 to project a grayscale stripe pattern onto the calibration plate, or receive image information of the calibration plate sent by the camera device 202, thereby processing the image information to determine the parameters of the phase height conversion mapping model.
[0047] When the system is used to detect the three-dimensional information of an object, the calibration plate 204 is placed on the translation stage 203, which is placed horizontally and can move up and down. The translation stage 203 can be an electrically controlled translation stage, an electric stepper device with a computer-controlled stepping distance, or a translation stage with manually controlled movement.
[0048] In addition, the calibration plate 204 includes multiple calibration blocks of different heights and spaced apart from each other. In the embodiment of the present application, eight calibration blocks of different heights are used as an example for illustration. The calibration plate 204 is placed on the translation stage 203, and a grayscale stripe pattern can be projected onto the calibration plate 204 by controlling the projection device 201. In addition, by controlling the movement of the translation stage 203, all calibration blocks of the calibration plate 204 are within the field of view of the camera device 202. In this way, during calibration, only a single scan of the calibration plate 204 by the camera device 202 is required to complete the acquisition of calibration blocks of multiple heights, thereby simplifying the operation, reducing the amount of data, and saving computing resources.
[0049] After understanding the system architecture of this application, combined with Figure 3 The scheme of the method for determining the phase height conversion mapping model parameters of the present application is described in detail.
[0050] Figure 3This is a flow chart of a method for determining parameters of a phase height conversion mapping model provided in an embodiment of the present application. This embodiment of the present application provides a method for determining parameters of a phase height conversion mapping model. This method can be executed by any device that executes a method for determining parameters of a phase height conversion mapping model. The device can be implemented by software and / or hardware. In this embodiment, the device can be integrated into an electronic device. Figure 3 As shown, the method for determining the phase height conversion mapping model parameters provided in the embodiment of the present application includes the following steps:
[0051] Step 301: Receive image information of a calibration plate sent by a camera device.
[0052] In this step, if Figure 2 As shown, the camera device 202 can collect image information of the calibration plate 204 placed on the translation stage 203, and can send the collected image information to the electronic device.
[0053] Step 302: Determine a plurality of first regions in the image information, where the first regions are shadow regions corresponding to the calibration blocks included in the calibration plate.
[0054] In this step, because the calibration plate includes multiple, spaced-apart calibration blocks of varying heights, and because the optical axis of projection device 201 forms an angle with translation stage 203, when projection device 201 projects the grayscale stripe pattern onto calibration plate 204, which is positioned on translation stage 203, shadow regions corresponding to the respective calibration blocks are inevitably generated on calibration plate 204. After acquiring image information of the calibration plate, the electronic device can determine the shadow region corresponding to each calibration block based on this image information.
[0055] In one possible implementation, when determining multiple first regions in image information, the image information can be binarized to obtain a binary map, and then a connected component labeling algorithm (also known as connected component analysis / labeling) is used to determine multiple first regions in the binary map.
[0056] Specifically, Figure 4 A schematic diagram of image information processing is shown in FIG. Figure 4 As shown, since the shooting scene and background lighting conditions are fixed, a fixed threshold value can be used to perform binarization processing on the obtained image information, thereby obtaining a corresponding binary image.
[0057] For example, in order to improve the accuracy of the calibration block positioning, the binary image can be subjected to denoising to obtain a denoised binary image, so that a plurality of first regions can be determined in the denoised binary image. In this embodiment of the present application, a morphological opening operation can be used to perform denoising on the binary image.
[0058] After obtaining the binary image, a connected component labeling algorithm is used to label the connected regions of the binary image. In addition, to avoid interference between multiple regions that need to be calibrated, a segmentation-first-then-detection strategy is adopted. That is, after labeling the connected regions of the binary image, the outline rectangles of the regions of multiple calibration blocks are separated.
[0059] In order to improve the accuracy of positioning the calibration block area, the outline rectangular frame can be expanded outward by the offset value mask_offset according to a preset offset value mask_offset, wherein the specific value of the offset value mask_offset can be set based on experience or actual conditions, for example, it can be set to 100 pixels. Furthermore, the expanded rectangular frame can be cropped in the binary image, so as to obtain multiple images including the shadow area of the calibration block, that is, to obtain multiple images including the first area.
[0060] In this embodiment, after binarizing the image information to obtain a binary image, a connected component labeling algorithm is used to determine multiple first areas in the binary image. This not only makes it easier to determine the shadow area of the calibration block, but also makes the determined shadow area more accurate.
[0061] Step 303: Determine a second area corresponding to the first area, where the second area is the area where the calibration block is located, and pixels included in at least two second areas have different heights corresponding to the target area.
[0062] In this step, after determining the shadow area of each calibration block in the image information, the area where each calibration block is located, that is, the second area corresponding to the first area, can be determined based on the shadow area.
[0063] In addition, since the calibration plate includes at least two calibration blocks of different heights, the target areas corresponding to the pixels included in the second region where the two calibration blocks of different heights are located have different heights. It is understandable that the target areas corresponding to the pixels included in the same second region have the same height. Exemplarily, in order to improve the accuracy of the parameters of the phase-height conversion mapping model, the heights of all calibration blocks in the calibration plate can be set to be different. In this case, the target areas corresponding to the pixels included in the second region where all calibration blocks are located have different heights.
[0064] For example, if Figure 4 As shown, the calibration plate includes 8 calibration blocks with different heights. Therefore, the height of the target area corresponding to the pixels included in the second area where each calibration block is located is different from the height of the target area corresponding to the pixels included in the second area where other calibration blocks are located.
[0065] It can be understood that, since the calibration blocks have different heights, the area of the shadow region corresponding to each calibration block is also different.
[0066] In one possible implementation, when determining the second area corresponding to the first area, a connected area marking algorithm can be used to determine multiple outline rectangular boxes in the image information, wherein each outline rectangular box includes a first area and a second area corresponding to the first area. Then, straight line detection is performed on the image within each outline rectangular box to determine multiple valid straight lines within the outline rectangular box, where the number of white pixels on the valid straight line is greater than a preset threshold. Then, based on the multiple valid straight lines, the second area corresponding to the first area is determined within the outline rectangular box.
[0067] For details, please refer to Figure 4 As shown, as described in the above embodiments, the electronic device can separate the outline rectangular frames of multiple calibration block areas, and the outline rectangular frame includes a first area and a second area corresponding to the first area, that is, the outline rectangular frame includes a shadow area of the calibration block and an area where the calibration block corresponding to the shadow area is located. The electronic device performs edge detection and Hough line detection on the image within each outline rectangular frame to obtain multiple straight lines, and then uses the prior direction information to determine a valid straight line from the multiple straight lines, wherein the number of white pixels on the valid straight line is greater than a preset threshold. For example, Figure 4 The lines a, b, c, and d in the figure are the determined valid lines. The preset threshold can be set according to actual conditions or experience, for example, 200 pixels. The specific value of the preset threshold is not limited in the present embodiment.
[0068] For example, in an embodiment of the present application, a first coordinate system is constructed with the upper left corner of the image information transmitted by the camera device as the origin, the horizontal direction as the horizontal axis, and the vertical direction as the vertical axis. Because the electronic device can determine the coordinate values of the four vertices of each outlined rectangular box when using the connected component labeling algorithm to determine each outlined rectangular box, the electronic device can extract the first coordinate of the upper left corner of each outlined rectangular box in the first coordinate system.
[0069] In addition, for each outline rectangular frame, after determining multiple valid straight lines within the outline rectangular frame, the electronic device can determine the first straight line with the smallest horizontal coordinate among the valid straight lines perpendicular to the horizontal axis of the first coordinate system, and the second straight line with the smallest vertical coordinate among the valid straight lines perpendicular to the vertical axis of the first coordinate system, and determine the intersection of the first straight line and the second straight line.
[0070] It can be understood that the rectangular area formed by the line connecting the upper left corner of the outline rectangular frame and the above-mentioned intersection as the diagonal line is the second area within the outline rectangular frame.
[0071] The second coordinate of the intersection point of the first straight line and the second straight line can be determined as follows:
[0072] For each outline rectangular frame, a second coordinate system can be constructed with the upper left corner of the outline rectangular frame as the origin, the horizontal direction as the horizontal axis, and the vertical direction as the vertical axis. The electronic device uses the Hough line detection algorithm to determine the length of each valid straight line while determining the multiple valid straight lines within the outline rectangular frame. Based on the length of each valid straight line, the coordinates of the intersection of the first straight line and the second straight line in the second coordinate system can be determined. Furthermore, it is necessary to convert the coordinates in the second coordinate system into coordinates in the first coordinate system. Specifically, the coordinates of the intersection of the first straight line and the second straight line in the second coordinate system can be added to the first coordinates of the upper left corner of the outline rectangular frame in the first coordinate system to obtain the coordinates of the intersection of the first straight line and the second straight line in the second coordinate system.
[0073] It should be understood that the rectangular area constructed with the first coordinate and the second coordinate as the vertex coordinates of the diagonal line is determined as the second area within the outline rectangular frame.
[0074] Specifically, such as Figure 4 As shown in FIG, a first coordinate system is constructed with the upper left corner of the original image as the origin, the horizontal direction as the horizontal axis, and the vertical direction as the vertical axis. When the electronic device uses the connected component labeling algorithm to determine each outline rectangular frame, it can determine the coordinate values of the four vertices of each outline rectangular frame. Therefore, the first coordinate of the upper left corner of the outline rectangular frame in the first coordinate system can be extracted, such as P left_top In addition, the first straight line with the smallest horizontal coordinate among the valid straight lines perpendicular to the horizontal axis of the first coordinate system within the outline rectangle and the second straight line with the smallest vertical coordinate among the valid straight lines perpendicular to the vertical axis of the first coordinate system can also be determined, and the intersection point of the first straight line and the second straight line can be determined, such as P right_bottom .
[0075] Taking a contour rectangle segmented from a binary image after connected area detection as an example, a second coordinate system is constructed with the upper left corner of the contour rectangle as the origin, the horizontal direction as the horizontal axis, and the vertical direction as the vertical axis. The electronic device can determine P based on the length of each valid straight line. right_bottom The coordinates in the second coordinate system. Further, P right_bottom The coordinates in the second coordinate system are the same as P left_top Add the first coordinates in the first coordinate system to get P right_bottom Coordinates in the second coordinate system.
[0076] P left_top and P right_bottom The rectangular area constructed as the vertex of the diagonal line is determined as the second area within the outline rectangular frame, that is, the area where the calibration block is located within the outline rectangular frame. left_top and P right_bottom The coordinates of the second area can be determined in the original image, thereby completing the positioning of the calibration block area.
[0077] In this embodiment, after roughly locating the area using the shadow area generated by the unique imaging characteristics of the system, the calibration block area is precisely located by combining algorithms such as connected area marking and Hough line detection. This can achieve automatic positioning of the calibration block area while also improving robustness and accuracy.
[0078] Exemplarily, considering that noise is easily generated at the edge of the area of the determined calibration block, the edge of the determined rectangular area may be retracted by a preset number of pixels to update the second area.
[0079] Specifically, such as Figure 4 As shown, after the rectangular area is determined, in order to eliminate the noise generated at the edge of the area, the P left_top and P right_bottom The edge of the determined rectangular area is indented by a preset number of pixels, so that the indented rectangular area is determined as the second area. The preset number can be set according to actual conditions or experience, for example, 70 pixels.
[0080] Step 304: Determine parameters of a phase-to-height conversion mapping model according to the height of the target area corresponding to the pixels in the second area, the pixel coordinates, and the phase values corresponding to the pixels.
[0081] The phase-height conversion mapping model is used to determine the height of the target area corresponding to each pixel in the object to be detected.
[0082] In this step, since the heights of the calibration blocks included in the calibration plate are preset, for the second area corresponding to each calibration block, the electronic device can obtain the height of the target area corresponding to each pixel in the second area.
[0083] It should be understood that after the electronic device acquires the image information, the image information will be stored in the form of a pixel matrix. When processing the pixel matrix, it is usually in the aforementioned first coordinate system to determine the coordinates of each pixel point. Therefore, after the electronic device determines the second area, it traverses all pixels in the second area in the first coordinate system to obtain the coordinates of each pixel in the second area. For example, if P left_top The coordinates are (10, 20), P right_bottom The coordinates of P are (50, 60), then by traversing all the pixels in the second area, the coordinates of other pixels can be obtained, such as P left_top The coordinates of the next pixel in the horizontal direction are (11, 20), the coordinates of the next pixel in the vertical direction are (10, 21), and so on.
[0084] Furthermore, the electronic device can also determine the phase value corresponding to each pixel in the second area based on the phase shift method. After obtaining the height of the target area corresponding to each pixel in each second area, the pixel coordinates, and the phase value corresponding to the pixel, the parameters of the phase-to-height conversion mapping model can be determined according to formula (1), thereby completing the calibration of the phase-to-height conversion mapping model.
[0085] It should be noted that since the calibration plate includes multiple calibration blocks with different heights, the accuracy of the parameters can be improved by determining the parameters of the phase-height conversion mapping model based on the height, pixel coordinates and phase values corresponding to the target area corresponding to the pixels in the second area where each calibration block is located.
[0086] The method for determining the parameters of the phase height conversion mapping model provided in the embodiment of the present application is to receive image information of a calibration plate sent by an imaging device, determine multiple first areas in the image information, the first area being the shadow area corresponding to the calibration block included in the calibration plate, and determine a second area corresponding to the first area, the second area being the area where the calibration block is located, and there are at least two second areas where the target areas corresponding to the pixels are different in height, and then determine the parameters of the phase height conversion mapping model based on the height of the target areas corresponding to the pixels in the second area, the pixel coordinates and the phase values corresponding to the pixels, thereby completing the calibration of the phase height conversion mapping model, wherein the phase height conversion mapping model is used to determine the height of the target area corresponding to each pixel in the object to be detected. Since the heights of the target areas corresponding to the pixels included in at least two second areas are different, only a single scan of the calibration plate is required to determine the parameters of the phase height conversion mapping model, thereby not only simplifying the parameter determination process and improving the efficiency of parameter determination, but also reducing the amount of data and saving the computing resources of the system. Furthermore, since the area where the calibration block is located can be accurately located through the shadow area corresponding to the calibration block to complete the determination of the phase height conversion mapping model parameters, the accuracy and robustness of area determination can be improved.
[0087] Figure 5 A flow chart of another method for determining parameters of a phase height conversion mapping model provided in an embodiment of the present application is provided. Figure 3 Based on the embodiment shown, the process of determining the three-dimensional data of the object to be detected after determining the parameters of the phase height conversion mapping model is described in detail. Figure 5 As shown, the method includes:
[0088] Step 501: Receive a target image containing an object to be detected sent by a camera device.
[0089] In this step, as shown in FIG1 , the object to be detected can be placed on a translation stage 203 , a target image of the object to be detected can be captured by a camera 202 , and the camera 202 can send the target image to an electronic device.
[0090] Step 502: Determine the height of the target area corresponding to each pixel according to the coordinates of each pixel of the target image, the phase corresponding to each pixel, and the parameters of the phase-to-height conversion mapping model.
[0091] In this step, after the electronic device receives the target image, the target image will be stored in the form of a pixel matrix. When processing the pixel matrix, a coordinate system is usually constructed with the upper left corner of the target image as the origin, the horizontal direction as the horizontal axis, and the vertical direction as the vertical axis. The coordinates of each pixel point are determined in this coordinate system. Therefore, after the electronic device determines the target image and constructs the coordinate system in the above manner, it will traverse all pixels in the target image, thereby obtaining the coordinates of each pixel in the target image and determining the phase corresponding to each pixel through the phase shift method. Among them, the parameters of the phase height conversion mapping model are based on Figure 3 Parameters determined in the illustrated embodiment. After acquiring these parameters, the electronic device can determine the height of the target area corresponding to each pixel according to formula (1).
[0092] Step 503: Determine the three-dimensional data of the object to be detected according to the height of the target area corresponding to each pixel.
[0093] In this step, after determining the height of the target area corresponding to each pixel, the electronic device can determine the three-dimensional point cloud data of the object to be detected, thereby completing the detection or test of the object to be detected.
[0094] The method for determining the parameters of the phase-height conversion mapping model provided in an embodiment of the present application receives a target image containing an object to be detected sent by a camera device, and determines the height of the target area corresponding to each pixel based on the coordinates of each pixel of the target image, the phase corresponding to each pixel, and the parameters of the phase-height conversion mapping model. Then, based on the height of the target area corresponding to each pixel, the three-dimensional data of the object to be detected is determined. Since the parameters of the phase-height conversion mapping model can be quickly determined according to the aforementioned embodiment, the efficiency of determining the three-dimensional data can be improved.
[0095] Figure 6 Schematic diagram of a device for determining phase height conversion mapping model parameters provided in an embodiment of the present application, such as Figure 6 As shown, the phase height conversion mapping model parameter determination device 60 includes:
[0096] Receiving module 601, used to receive image information of the calibration plate sent by the camera device;
[0097] A determination module 602 is configured to determine a plurality of first regions in the image information, where the first regions are shadow regions corresponding to the calibration blocks included in the calibration plate;
[0098] The determining module 602 is further configured to determine a second area corresponding to the first area, where the second area is an area where the calibration block is located, and where at least two pixels included in the first area correspond to target areas with different heights;
[0099] The determination module 602 is further used to determine the parameters of the phase-height conversion mapping model based on the height of the target area corresponding to the pixels in the second area, the pixel coordinates and the phase value corresponding to the pixels. The phase-height conversion mapping model is used to determine the height of the target area corresponding to each pixel in the object to be detected.
[0100] Optionally, the determining module 602 is specifically configured to:
[0101] performing binarization processing on the image information to obtain a binary image;
[0102] A connected component labeling algorithm is used to determine the multiple first regions in the binary image.
[0103] Optionally, the determining module 602 is specifically configured to:
[0104] Using the connected component labeling algorithm, a plurality of outline rectangular frames are determined in the image information, each of the outline rectangular frames including a first region and a second region corresponding to the first region;
[0105] Performing line detection on the image within each outline rectangular frame to determine a plurality of valid straight lines within the outline rectangular frame, wherein the number of white pixels on the valid straight lines is greater than a preset threshold;
[0106] A second area corresponding to the first area is determined within the outline rectangular frame according to the multiple valid straight lines.
[0107] Optionally, the determining module 602 is specifically configured to:
[0108] Constructing a coordinate system with the upper left corner of the image information as the origin, the horizontal direction as the horizontal axis, and the vertical direction as the vertical axis, and determining a first coordinate of the upper left corner of the outline rectangular frame in the coordinate system;
[0109] In the coordinate system, determining the second coordinate of the intersection of a first straight line with the smallest horizontal coordinate among the valid straight lines perpendicular to the horizontal axis and a second straight line with the smallest vertical coordinate among the valid straight lines perpendicular to the vertical axis;
[0110] A rectangular area constructed with the first coordinates and the second coordinates as vertex coordinates of a diagonal line is determined as the second area within the outline rectangular frame.
[0111] Optionally, the apparatus further includes: an updating module 603, configured to indent the edge of the rectangular area by a preset number of pixels to update the second area.
[0112] Optionally, the determining module 602 is specifically configured to:
[0113] Performing denoising on the binary image to obtain a denoised binary image;
[0114] A connected component labeling algorithm is used to determine the multiple first regions in the denoised binary image.
[0115] Optionally, the receiving module 601 is further configured to receive a target image containing an object to be detected sent by a camera device;
[0116] The determining module 602 is further configured to determine the height of the target area corresponding to each pixel according to the coordinates of each pixel of the target image, the phase corresponding to each pixel, and the parameters of the phase-to-height conversion mapping model;
[0117] The determination module 602 is further configured to determine the three-dimensional data of the object to be detected according to the height of the target area corresponding to each pixel.
[0118] The device 60 for determining the phase height conversion mapping model parameters provided in the embodiment of the present application can execute the technical solution of the method for determining the phase height conversion mapping model parameters in any of the above-mentioned embodiments. Its implementation principle and beneficial effects are similar to the implementation principle and beneficial effects of the method for determining the phase height conversion mapping model parameters. Please refer to the implementation principle and beneficial effects of the method for determining the phase height conversion mapping model parameters, and no further details will be given here.
[0119] It should be noted that the division of the modules of the above devices is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity or physically separated. Furthermore, these modules can all be implemented in the form of software called by a processing element; or all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the determination module can be a separate processing element, or it can be integrated into a chip of the above device. In addition, it can be stored in the memory of the above device in the form of program code, and called by a processing element of the above device to perform the functions of the above processing module. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element or instructions in the form of software.
[0120] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code on a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0121] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)).
[0122] Figure 7 This is a schematic diagram of the structure of an electronic device embodiment provided in the present application. Figure 7As shown, the electronic device may include: a processor 701, a memory 702, a communication interface 703 and a system bus 704, wherein the memory 702 and the communication interface 703 are connected to the processor 701 through the system bus 704 and communicate with each other, the memory 702 is used to store computer execution instructions, the communication interface 703 is used to communicate with other devices, and when the processor 701 executes the computer program, it implements the technical solution of the method for determining the phase height conversion mapping model parameters as shown in the above embodiment.
[0123] In this Figure 7 In the embodiment, the processor 701 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0124] The memory 702 may include a random access memory (RAM), a read-only memory (RAM), or a non-volatile memory (non-volatile memory), such as at least one disk storage.
[0125] The communication interface 703 is used to implement communication between the database access apparatus and other devices (such as a client, a read-write library, and a read-only library).
[0126] The system bus 704 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. A system bus can be categorized as an address bus, a data bus, a control bus, and the like. For ease of illustration, the figure shows only one thick line, but this does not imply that there is only one bus or only one type of bus.
[0127] Optionally, an embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are run on a computer, the computer executes the technical solution of the method for determining the phase height conversion mapping model parameters as shown in the above embodiment.
[0128] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the applications disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.
[0129] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A method for determining parameters of a phase height conversion mapping model, It is characterized in that include: Receiving image information of the calibration plate sent by the camera device; Determine a plurality of first areas in the image information, where the first areas are shadow areas corresponding to the calibration blocks included in the calibration plate; Determine a second area corresponding to the first area, the second area being an area where the calibration block is located, and pixels included in at least two second areas correspond to target areas with different heights; Determine the parameters of a phase height conversion mapping model according to the height of the target area corresponding to the pixels in the second area, the pixel coordinates and the phase value corresponding to the pixels, wherein the phase height conversion mapping model is used to determine the height of the target area corresponding to each pixel in the object to be detected; The determining a second area corresponding to the first area includes: Using a connected region labeling algorithm, a plurality of outline rectangular frames are determined in the image information, each of the outline rectangular frames includes a first region and a second region corresponding to the first region; Performing straight line detection on the image within each outline rectangular frame to determine a plurality of valid straight lines within the outline rectangular frame, wherein the number of white pixel points on the valid straight lines is greater than a preset threshold; A second area corresponding to the first area is determined within the outline rectangular frame according to the plurality of valid straight lines.
2. The method according to claim 1, It is characterized in that The determining of a plurality of first areas in the image information comprises: Binarizing the image information to obtain a binary image; The multiple first regions are determined in the binary image by using a connected region labeling algorithm.
3. The method according to claim 2, It is characterized in that The determining, based on the plurality of valid straight lines, a second area corresponding to the first area within the outline rectangular frame comprises: A coordinate system is constructed with the upper left corner of the image information as the origin, the horizontal direction as the horizontal axis, and the vertical direction as the vertical axis, and a first coordinate of the upper left corner of the outline rectangular frame in the coordinate system is determined; In the coordinate system, determine the second coordinate of the intersection of a first straight line with the smallest horizontal coordinate among the valid straight lines perpendicular to the horizontal axis and a second straight line with the smallest vertical coordinate among the valid straight lines perpendicular to the vertical axis; A rectangular area constructed with the first coordinates and the second coordinates as vertex coordinates of a diagonal line is determined as the second area within the outline rectangular frame.
4. The method according to claim 3, It is characterized in that The method further comprises: The edge of the rectangular area is indented by a preset number of pixels to update the second area.
5. The method according to claim 2, It is characterized in that The method of using a connected region labeling algorithm to determine the plurality of first regions in the binary graph includes: Performing denoising on the binary image to obtain a denoised binary image; A connected region labeling algorithm is used to determine the multiple first regions in the denoised binary image.
6. The method according to any one of claims 1 to 5, It is characterized in that After determining the parameters of the phase height conversion mapping model according to the height of the target area corresponding to the pixels in the second area, the pixel coordinates and the phase values corresponding to the pixels, the method further includes: Receiving a target image including an object to be detected sent by a camera device; Determine the height of the target area corresponding to each pixel according to the coordinates of each pixel of the target image, the phase corresponding to each pixel and the parameters of the phase height conversion mapping model; The three-dimensional data of the object to be detected is determined according to the height of the target area corresponding to each pixel.
7. A device for determining parameters of a phase height conversion mapping model, It is characterized in that include: A receiving module, used for receiving image information of the calibration plate sent by the camera device; A determination module, configured to determine a plurality of first areas in the image information, wherein the first areas are shadow areas corresponding to the calibration blocks included in the calibration plate; The determination module is further used to determine a second area corresponding to the first area, the second area being an area where the calibration block is located, and there are at least two target areas whose pixels included in the first area correspond to different heights; The determination module is further used to determine the parameters of the phase height conversion mapping model according to the height of the target area corresponding to the pixels in the second area, the pixel coordinates and the phase value corresponding to the pixels, and the phase height conversion mapping model is used to determine the height of the target area corresponding to each pixel in the object to be detected; The determination module is specifically used for: Using a connected region labeling algorithm, a plurality of outline rectangular frames are determined in the image information, each of the outline rectangular frames includes a first region and a second region corresponding to the first region; Performing straight line detection on the image within each outline rectangular frame to determine a plurality of valid straight lines within the outline rectangular frame, wherein the number of white pixel points on the valid straight lines is greater than a preset threshold; A second area corresponding to the first area is determined within the outline rectangular frame according to the plurality of valid straight lines.
8. An electronic device, It is characterized in that include: processor; Memory; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor, wherein the computer program includes instructions for executing the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and the computer program enables the server to execute the method according to any one of claims 1 to 6.
