A method, device and system for extracting three-dimensional coordinates of the center line of a line laser stripe
By combining the Zynq-7000 series hardware platform and CMOS monocular camera with FPGA technology, rapid and accurate three-dimensional coordinate extraction of the center line of line laser stripes was achieved, solving the problems of insufficient speed, accuracy and noise resistance of existing devices and improving the convenience of industrial measurement.
Patent Information
- Application Number
- CN202210830487.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-15
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-07-15
AI Technical Summary
Existing 3D coordinate acquisition devices are inadequate in terms of extraction speed, accuracy, noise resistance, and convenience, especially in high-end industrial measurement fields where they fail to meet the requirements.
A monocular camera based on the Zynq-7000 series hardware platform was used to build a device. Combining a CMOS monocular camera and FPGA technology, the device achieved rapid and accurate three-dimensional coordinate extraction of the center line of the line laser stripe through image preprocessing and grayscale centroid method.
It improves the speed and accuracy of 3D coordinate extraction, enhances noise immunity, reduces the size and power consumption of the hardware platform, and meets the operational needs of high-end industrial measurement.
Smart Images

Figure CN115115604B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method, a device and a system for extracting three-dimensional coordinates of a center line of a linear laser stripe, and belongs to the technical field of three-dimensional visual measurement. Background Art
[0002] In the field of three-dimensional measurement of precision regular workpieces, non-contact industrial measurement has been widely used in large-scale engineering fields since the mid-20th century, such as rapid flaw detection of rails and dimensional measurement of high-precision precision workpieces in ship and aerospace construction. At present, most industrial measurement systems use non-contact visual measurement systems based on the principle of laser triangulation measurement. This type of measurement system has the advantages of fast speed, high accuracy, and good stability. However, with the continuous improvement of measurement requirements, the shortcomings of this type of measurement system are gradually exposed. Not only is the ability to resist noise weak, but also the requirements for environmental factors are high. Moreover, since most of these industrial measurement systems are used in high-end fields, the requirements for accuracy and speed are more prominent. General hardware platforms can no longer meet existing operating requirements.
[0003] The existing acquisition of three-dimensional coordinates of workpieces all have different defects. Therefore, it is necessary to propose a solution to improve speed, accuracy, noise resistance and convenience, and to solve the problems of slow speed, low accuracy, poor noise resistance and poor portability of centerline coordinates and three-dimensional coordinate extraction of such devices. Summary of the invention
[0004] The purpose of the present invention is to be able to quickly and accurately extract the three-dimensional coordinates corresponding to the center line coordinates of the laser stripes. The present invention uses a monocular camera to build a device structure framework according to the laser triangulation method in the optical measurement principle, and the overall architecture of the system is based on the Zynq-7000 series hardware platform of Xilinx Company, and a structural scheme is achieved to obtain the three-dimensional coordinates of the center line of the linear laser stripes. The Zynq-7000 series chip used integrates ARM and FPGA in terms of function, which not only makes the design of the embedded system more flexible and convenient, but also reduces the volume to a certain extent, and is compatible with the advantages and characteristics of FPGA, so that FPGA can be combined with the embedded system, better applied to related fields, and effectively expand the scope of use of FPGA. FPGA has good parallel processing and computing capabilities. Therefore, the Zynq-7000 series hardware platform can well improve the operating speed and calculation accuracy of the entire measuring device by relying on this feature.
[0005] By improving the algorithm, the requirements for speed and accuracy in extracting the centerline coordinates and three-dimensional coordinates of the laser stripes can be met; a variety of filtering schemes are used to improve the image and enhance the ability to resist noise; a highly integrated hardware design scheme is adopted to improve product convenience.
[0006] The basic solution adopted by the present invention is mainly divided into three parts: the first image acquisition module part uses a CMOS monocular camera to collect image data; the second image preprocessing part removes noise points, improves image quality, and confirms the effective light stripe measurement area through line caching, mean filtering, median filtering and threshold comparison of the image data; the third laser stripe center line coordinate extraction and three-dimensional coordinate conversion part uses the grayscale centroid method to calculate the image effective pixel data and coordinate values, and finally extracts the laser stripe center line coordinates, and generates three-dimensional coordinates according to the conversion relationship between coordinate systems.
[0007] In a first aspect, a method for extracting three-dimensional coordinates of a center line of a line laser stripe is provided, comprising:
[0008] Acquiring camera calibration parameters, light plane calibration parameters, and raw image data collected under the camera calibration parameters and light plane calibration parameters;
[0009] Verify, cache and filter the original image data to obtain valid image data;
[0010] Based on a preset image threshold, the effective image data is screened for image grayscale data values that can be used to calculate the center line coordinates using the grayscale centroid method, and the center line coordinate data corresponding to the laser stripes are obtained;
[0011] Based on the camera calibration parameters and the light plane calibration parameters, according to the conversion relationship between the center line coordinate data and the coordinate system, the three-dimensional coordinates corresponding to the center line coordinates of the line laser in the image are generated, that is, the three-dimensional coordinates of the center line of the line laser stripe.
[0012] In some embodiments, verifying the original image data includes:
[0013] The serial data of the original image data is converted into parallel data, and the parallel data is rearranged according to a certain rule to perform synchronous verification on the image gray value data.
[0014] In some embodiments, the row cache process includes:
[0015] The image data is cached in rows, and the cache is completed according to the flag signal and the template requirements. In order to facilitate the subsequent image preprocessing operations, the image row data is cached according to the flag signal. The number of rows of cached image data is kept in correspondence with the size of the filter processing template used subsequently.
[0016] First of all, the purpose and basic requirements of filtering processing are clear. The purpose is mainly divided into two points: first, to highlight the characteristic points of the image; second, to ensure that the image data is suitable for processing and to remove the noise points in the image; the basic requirements of image processing are: to minimize the damage to the edge contour of the image and ensure the integrity of the image; to ensure the clarity of the image as much as possible.
[0017] In some embodiments, filtering is determined by a flag signal to use mean filtering or median filtering;
[0018] Mean filtering includes: setting the template, the gray value of the center point is replaced by the average gray value of all pixels
[0019]
[0020] g center (x center ,y center ) is the gray value of the center coordinate calculated according to the template, f(x,y) represents the gray value of the image, and the template size M is: x×y;
[0021] The median filter includes: sorting the gray values of the M pixels in the template to obtain the middle value, and setting the middle value as the gray value of the center coordinate;
[0022] g center (x center ,y center ) = median[f(x,y)]
[0023] By adjusting the size of the template, the image gray value data is preprocessed to remove noise points and environmental interference on the image.
[0024] Furthermore, in some specific embodiments, the template size used for filtering is 5x1.
[0025] According to the intrinsic and extrinsic parameter values obtained after the calibration, the conversion of 3D point cloud coordinates is realized; there are mainly two parts: camera calibration and light plane calibration; the purpose of camera calibration is to obtain the values of the camera's intrinsic and extrinsic parameter matrices; the goal of light plane calibration is to complete 3D reconstruction; camera calibration mainly converts the center line coordinates through the image pixel coordinate system (O_uv), image coordinate system (O_xy), camera coordinate system (O c _X c Y c Z c ) and the world coordinate system (O w _X w Y w Z w )The conversion relationship between the four is used to calculate the intrinsic and extrinsic parameters of the camera.
[0026] In some embodiments, according to the conversion relationship between the center line coordinate data and the coordinate system, the three-dimensional coordinates corresponding to the center line coordinates of the laser in the image are generated, including: the relationship expression between the pixel coordinate system and the image coordinate system:
[0027]
[0028] Where (u, v) is the center line coordinate of the line laser stripe, (x, y) is the pixel coordinate system coordinate, the center point of the pixel coordinate is (u0, v0), and the center point of the pixel coordinate has a certain offset relationship with the sensor; the size of a single pixel of the sensor is: dx×dy;
[0029] The relationship between the image coordinate system and the camera coordinate system is expressed as follows:
[0030]
[0031] (u,v) is the center line coordinate of the line laser stripe, (x c ,y c ,z c ) is the camera coordinate system coordinate, f x 、f y They respectively represent the relationship between the focal length f and the size of a single pixel;
[0032] The relationship expression between the camera coordinate system and the world coordinate system is:
[0033]
[0034] Where (x c ,y c ,z c ) is the camera coordinate system coordinate, (x w ,y w ,z w ) represents the coordinates of the world coordinate system, R and T represent the rotation matrix and translation matrix involved when converting from the camera coordinate system to the world coordinate system; where f x ,f y ,u0,v0 and R,T are obtained through camera calibration parameters;
[0035] Light plane calibration is done in the camera coordinate system, adding a c The constraint equation of the value is expressed as:
[0036] ax c +by c +cz c +d=0
[0037] Among them, a, b, c, d represent the constant terms of the general equation of the light plane, which are obtained by calibrating the light plane parameters;
[0038] According to the above relationship, the center line coordinate data is converted into the world coordinate system coordinates to obtain the three-dimensional coordinates of the center line of the line laser stripe (x w ,y w 、z w ).
[0039] In a second aspect, the present invention provides a device for extracting three-dimensional coordinates of a center line of a line laser stripe, including a processor and a storage medium;
[0040] The storage medium is used to store instructions;
[0041] The processor is used to operate according to the instructions to execute the steps of the method according to the first aspect.
[0042] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.
[0043] In a fourth aspect, a system for extracting three-dimensional coordinates of a center line of a linear laser stripe comprises the device for extracting three-dimensional coordinates of a center line of a linear laser stripe according to the second aspect.
[0044] A system for extracting three-dimensional coordinates of the center line of a linear laser stripe, characterized by comprising the device for extracting three-dimensional coordinates of the center line of a linear laser stripe according to claim 7.
[0045] The linear laser stripe center line three-dimensional coordinate extraction system further includes:
[0046] A line laser is configured to provide a line laser to a workpiece to be inspected on the optical platform;
[0047] An image sensor is configured to collect image data and upload it to the three-dimensional coordinate extraction device of the center line of the linear laser stripe;
[0048] The image sensor adopts a CMOS image sensor, and the linear laser stripe center line three-dimensional coordinate extraction device adopts a Zynq-7000 series chip.
[0049] Beneficial effects: The present invention discloses a method, device and system for extracting three-dimensional coordinates of the center line of a linear laser stripe. The method is based on a three-dimensional coordinate extraction system built on a Zynq-7000 series hardware platform in combination with a monocular CMOS camera sensor; first, pixel data of an image is collected, and the collected differential serial image data is quickly converted into parallel image data; then the parallel image data is preprocessed by a variety of filtering methods such as mean filtering and median filtering to improve image quality and reduce the impact of noise on the image; finally, the grayscale centroid method is used to extract the center line coordinates of the laser stripe, and the center line coordinates are converted into three-dimensional coordinates through a conversion relationship between coordinate systems; when preprocessing the collected image data, a reasonable filtering method can be set through a flag bit for noise generated in different environments, thereby improving image quality, enhancing the system's anti-interference ability to environmental noise, and facilitating subsequent extraction of center line coordinates and conversion of three-dimensional coordinates. The former image preprocessing method, as well as the latter grayscale centroid method for extracting centerline coordinates and the conversion relationship for obtaining three-dimensional coordinates, are both implemented through algorithm mapping in the PL (Processing Logic) part of Zynq-7000, which can effectively improve the extraction speed and accuracy of laser stripe center coordinates and three-dimensional coordinates.
[0050] Based on the single-line structured light three-dimensional coordinate extraction device, the hardware part of the device uses a chip in the Zynq-7000 series, which integrates FPGA and ARM functional architecture. From the hardware analysis, the overall system power consumption is low and the main frequency is high. It meets the project's requirements for speed, accuracy and convenience, and effectively reduces the size of the hardware control platform. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 : Schematic diagram of a system for extracting three-dimensional coordinates of a center line of a linear laser stripe according to an embodiment of the present invention;
[0052] Figure 2 : System block diagram of the three-dimensional coordinate extraction method according to an embodiment of the present invention;
[0053] Figure 3 : Basic flow chart of system function modules of the embodiment of the present invention;
[0054] Figure 4 : Basic flow chart of the image preprocessing program according to the embodiment of the present invention;
[0055] Figure 5 : Block diagram of a 5x1 filter template according to an embodiment of the present invention;
[0056] Figure 6 : Basic block diagram of the embodiment of the present invention for transmitting PL data to the IP core inside the PS;
[0057] Figure 7: Functional module diagram of converting centerline coordinate data into world coordinates in an embodiment of the present invention;
[0058] Figure 8 : Distribution diagram of coordinate values extracted from the stripe centerline according to an embodiment of the present invention;
[0059] Fig. 9 : Comparison chart of world coordinate values extracted by different software in embodiments of the present invention. DETAILED DESCRIPTION
[0060] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific methods.
[0061] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0062] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0063] Example 1
[0064] A method for extracting three-dimensional coordinates of a center line of a line laser stripe, comprising:
[0065] Acquiring camera calibration parameters, light plane calibration parameters, and raw image data collected under the camera calibration parameters and light plane calibration parameters;
[0066] Verify, cache and filter the original image data to obtain valid image data;
[0067] Based on a preset image threshold, the effective image data is screened for image grayscale data values that can be used to calculate the center line coordinates using the grayscale centroid method, and the center line coordinate data corresponding to the laser stripes are obtained;
[0068] Based on the camera calibration parameters and the light plane calibration parameters, according to the conversion relationship between the center line coordinate data and the coordinate system, the three-dimensional coordinates corresponding to the center line coordinates of the line laser in the image are generated, that is, the three-dimensional coordinates of the center line of the line laser stripe.
[0069] In some embodiments, Figure 2 As shown: The construction and composition of hardware and structure are the basic part of the whole system framework. Other functional modules are image preprocessing, laser stripe centerline extraction and three-dimensional coordinate conversion.
[0070] Figure 3 The flow chart of the functional module is Figure 2 For further explanation of the content, the details are described as follows: First, a CMOS image sensor is used to collect image raw data and synchronization signals.
[0071] The original image data and synchronization signal are output by differential signals, and the image data and synchronization information data stream are de-differentiated. Therefore, the IBUFDS primitive provided by Xilinx's development tool is used here to convert the serial differential data signal into a single-ended serial signal, and then the IDELAY and ISERDES IP cores are used to complete the serial-to-parallel data stream conversion task and align the data stream.
[0072] The parallel image data and the data of the synchronous decoding channel are synchronized and CRC checked to ensure the correctness of the generated image data.
[0073] After the data is verified, the non-adjacent pixel data is rearranged into the required data stream state according to the requirements of the image sensor core.
[0074] According to the requirements of the flag signal and template size, the Shift_ram IP core is used to cache the image data in rows.
[0075] The gray value data is cached line by line (5 lines are cached) as follows Figure 5 As shown, the important functional module of image preprocessing - filtering is then executed.
[0076] The main purpose of filtering an image is to highlight the feature points of the image, ensure that the image data is suitable for processing, and remove the noise points in the image. The basic requirements of filtering are to minimize the damage to the edge contour of the image and ensure the integrity of the image; to ensure the clarity of the image as much as possible.
[0077] The data stream after the row buffer can be Figure 4As shown in the figure, the flag signal determines the use of mean filtering and median filtering to enhance the diversity of filtering. The mean filter uses division, rounding and saturation processing during calculation to ensure the accuracy of the calculated grayscale value. The median filter will perform reasonable sorting operations to avoid the defect that values of the same size cannot be sorted;
[0078] Set the image segmentation threshold. If the gray value is greater than the threshold, the relevant parameters of the image data will be cached and participate in the subsequent solution of the center line coordinates. Otherwise, it will not participate in the solution process.
[0079] Before the centerline is extracted from the image data, a ping-pong operation is performed on the image data stream to improve the efficiency of data processing and avoid confusion in the continuous calculation of the centerline coordinate values under continuous transmission.
[0080] After the centerline coordinate data is calibrated by the camera and the light plane, the three-dimensional coordinates are obtained. The IP core of Vid-in-AXI-stream is used to transmit the data to VDMA, and finally the data is stored in DDR for storage and upload.
[0081] like Figure 6 As shown, the extracted centerline coordinate data and three-dimensional coordinate data are cached, transmitted, and stored.
[0082] The image preprocessing method described: The basic principles of filtering are as follows:
[0083] The principle of mean filtering is as follows: the present invention sets the initial template to be 5x1, and the gray value of the center point is replaced by the gray value average of all pixel points.
[0084]
[0085] g center (x center ,y center ) is the gray value of the center coordinate calculated according to the template, f(x, y) represents the gray value of the image, and the template size M in this embodiment is: 5×1. Figure 5 As shown, the Shift-ram IP core is used to implement the line cache operation according to the filter template size and complete the image preprocessing.
[0086] Median filtering principle: Use a 5x1 template, sort the image grayscale values of the five pixels corresponding to the same clock in the template to obtain the middle value, and set the middle value as the value of the center coordinate.
[0087] g center (x center ,y center ) = median[f(x,y)]
[0088] Through the analysis of basic principles, the operation of the filtering module refers to Figure 5 Adjust the size of the template to ensure image preprocessing of image gray value data, remove noise points, reduce noise interference to the image, and improve image quality.
[0089] FPGA is used to implement the mapping of grayscale centroid method, and the center line coordinates are extracted from the preprocessed image. Then, the center line coordinate data is calibrated by camera and light plane to complete the conversion of three-dimensional coordinates.
[0090] The purpose of camera calibration is to obtain the values of the camera's intrinsic and extrinsic matrix;
[0091] The goal of light plane calibration is to complete three-dimensional reconstruction;
[0092] Camera calibration mainly involves converting the extracted center line coordinates through the image pixel coordinate system (O_uv), image coordinate system (O_xy), and camera coordinate system (O c _X c Y c Z c ) and the world coordinate system (O w _X w Y w Z w ) The conversion relationship between the four is used to calculate the camera's internal and external parameters; the relationship between the pixel coordinate system and the image coordinate system is expressed as:
[0093]
[0094] Where (u, v) is the coordinate of the center line of the line laser stripe, (x, y) is the coordinate of the pixel coordinate system, the center point of the pixel coordinate is (u0, v0), and the center point of the pixel coordinate will produce a certain offset due to the manufacturing process of the sensor itself; the size of a single pixel of the sensor is: dx×dy;
[0095] The relationship between the image coordinate system and the camera coordinate system is expressed as follows:
[0096]
[0097] (u,v) is the center line coordinate of the line laser stripe, (x c ,y c ,z c ) is the camera coordinate system coordinate, f x 、f y They respectively represent the relationship between the focal length f and the size of a single pixel;
[0098] The relationship expression between the camera coordinate system and the world coordinate system is:
[0099]
[0100] Where (x c ,y c ,z c ) is the camera coordinate system coordinate, (x w ,y w ,z w ) represents the coordinates of the world coordinate system, R and T represent the rotation matrix and translation matrix involved in converting from the camera coordinates to the world coordinate system respectively; the corresponding internal parameters f are obtained according to the conversion relationship between the coordinate systems x ,f y ,u0,v0 and the corresponding values of the external parameter matrix R,T;
[0101] Light plane calibration is done in the camera coordinate system by adding a c The constraint equation of the value, the plane equation expression is:
[0102] ax c +by c +cz c +d=0
[0103] Among them, a, b, c, d represent the constant terms of the general equation of the light plane. According to the above two calibration methods, Figure 7 As shown, the conversion method from the pixel coordinate system to the world coordinate system is determined to realize the conversion of the two-dimensional center line pixel coordinates to the three-dimensional coordinates;
[0104] The extracted centerline coordinates and 3D coordinates are stored in the DDR3 storage unit through the Vid-in-AXI-stream IP core to the VDMA, and finally the data is transmitted to the host computer for subsequent point cloud processing.
[0105] Combination Figure 8 It can be seen that the distribution of the extracted center line coordinate values is as follows: when the laser does not measure the workpiece, the graph is a basic straight line; when the laser line detects the workpiece, the line changes and appears as a broken discrete line. The difference between the bottom line and the broken point represents the height information of the object. Fig. 9 The figure shows the comparison results of outputting the world coordinate system values in three different ways: C++, MATLAB, and FPGA.
[0106] Example 2
[0107] In a second aspect, the present embodiment provides a device for extracting three-dimensional coordinates of a center line of a line laser stripe, including a processor and a storage medium;
[0108] The storage medium is used to store instructions;
[0109] The processor is used to operate according to the instructions to execute the steps of the method according to embodiment 1.
[0110] Example 3
[0111] In a third aspect, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.
[0112] Example 4
[0113] A system for extracting three-dimensional coordinates of the center line of a line laser stripe, comprising the device for extracting three-dimensional coordinates of the center line of a line laser stripe, and further comprising:
[0114] A line laser is configured to provide a line laser to a workpiece to be inspected on the optical platform;
[0115] An image sensor is configured to collect image data and upload it to the three-dimensional coordinate extraction device of the center line of the linear laser stripe;
[0116] The image sensor adopts a CMOS image sensor, and the linear laser stripe center line three-dimensional coordinate extraction device adopts a Zynq-7000 series chip.
[0117] In some embodiments, Figure 1 Shown is a schematic diagram of the system.
[0118] The framework consists of a CMOS image sensor, a Zynq-7000 series hardware system, a laser, an optical platform, and a PC, where the PC includes a processor and storage media;
[0119] The storage medium is used to store instructions;
[0120] The processor is used to operate according to the instructions to execute the steps of the method according to embodiment 1.
[0121] The line laser stripe center line three-dimensional coordinate extraction system includes: a line laser, an image sensor and its control module, a main control module based on the Zynq-7000 series hardware platform, and an optical fixed platform.
[0122] 1. The three-dimensional coordinate acquisition method comprises the following steps:
[0123] 1) Obtain the installation parameters of the device according to the field of view size and accuracy requirements;
[0124] 2) using the image sensor to collect raw grayscale image data;
[0125] 3) According to the camera image quality and noise analysis, the original image data is verified, line buffered, filtered, and other operations are performed to obtain valid image data;
[0126] 4) According to the acquired valid image data, the image threshold is set, the image grayscale data value that can be used to calculate the center line coordinates by the grayscale centroid method is screened, and the center line coordinate values corresponding to the laser stripes are obtained;
[0127] 5) Generate the three-dimensional coordinates corresponding to the center line coordinates of the laser center line in the image according to the conversion relationship between the calculated center line coordinate data and the coordinate system;
[0128] 6) The three-dimensional coordinate data is converted into a stream data format through the AXI bus protocol and sent to the host computer part of the PC to display the calculated three-dimensional coordinate value of the center line of the line laser stripe.
[0129] The hardware system includes two parts: image acquisition control and system control. The image acquisition part uses a CMOS image sensor in the present invention. The system control part adopts Xilinx's Zynq-7000 series chip, model xc7z030ffg617-1, including PL (Processing Logic) and PS (Processing System). The PL part contains rich resources such as LUT, which significantly improves the complex computing ability of data. The AXI protocol port is used between PL and PS to complete bidirectional control and data transmission.
[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.
[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0134] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for extracting the three-dimensional coordinates of the center line of a line laser stripe, characterized in that: include: Acquiring camera calibration parameters, light plane calibration parameters, and raw image data collected under the camera calibration parameters and light plane calibration parameters; Verify, cache and filter the original image data to obtain valid image data; Based on a preset image threshold, the effective image data is screened for image grayscale data values that can be used to calculate the center line coordinates using the grayscale centroid method, and the center line coordinate data corresponding to the laser stripes are obtained; Based on the camera calibration parameters and the light plane calibration parameters, according to the conversion relationship between the center line coordinate data and the coordinate system, the three-dimensional coordinates corresponding to the center line coordinates of the line laser in the image are generated, that is, the three-dimensional coordinates of the center line of the line laser stripe; According to the conversion relationship between the center line coordinate data and the coordinate system, the three-dimensional coordinates corresponding to the center line coordinates of the laser in the image are generated, including: The relationship between the pixel coordinate system and the image coordinate system is expressed as follows: Where (u, v) is the center line coordinate of the line laser stripe, (x, y) is the pixel coordinate system coordinate, the center point of the pixel coordinate is (u0, v0), and the center point of the pixel coordinate has a certain offset relationship with the sensor; the size of a single pixel of the sensor is: dx×dy; The relationship between the image coordinate system and the camera coordinate system is expressed as follows: (u,v) is the center line coordinate of the line laser stripe, (x c ,y c ,z c ) is the camera coordinate system coordinate, f x 、f y They respectively represent the relationship between the focal length f and the size of a single pixel; The relationship expression between the camera coordinate system and the world coordinate system is: Where (x c ,y c ,z c ) is the camera coordinate system coordinate, (x w ,y w ,z w ) represents the coordinates of the world coordinate system, R and T represent the rotation matrix and translation matrix involved in converting from the camera coordinate system to the world coordinate system respectively; Light plane calibration is done in the camera coordinate system, adding a c The constraint equation of the value is expressed as: ax c +by c +cz c +d=0 Where a, b, c, d represent the constant terms of the general equation of the light plane; According to the above relationship, the center line coordinate data is converted into the world coordinate system coordinates to obtain the three-dimensional coordinates of the center line of the line laser stripe (x w ,y w 、z w ).
2. The method for extracting three-dimensional coordinates of the center line of a line laser stripe according to claim 1, characterized in that: Verify the original image data, including: The serial data of the original image data is converted into parallel data, and the parallel data is rearranged according to a certain rule to perform synchronous verification on the image gray value data.
3. The method for extracting three-dimensional coordinates of the center line of a line laser stripe according to claim 1, characterized in that: Row cache processing, including: The image data is line cached and the cache is completed according to the flag signal and template requirements.
4. The method for extracting three-dimensional coordinates of the center line of a line laser stripe according to claim 1, characterized in that: The filtering is determined by the flag signal to use mean filtering or median filtering; Mean filtering includes: setting the template, the gray value of the center point is replaced by the average gray value of all pixels g center (x center ,y center ) is the gray value of the center coordinate calculated according to the template, f(x,y) represents the gray value of the image, and the template size M is: x×y; The median filter includes: sorting the gray values of the M pixels in the template to obtain the middle value, and setting the middle value as the gray value of the center coordinate; g center (x center ,y center )=median[f(x,y)] By adjusting the size of the template, the image gray value data is preprocessed to remove noise points and environmental interference on the image.
5. The method for extracting three-dimensional coordinates of the center line of a linear laser stripe according to claim 4, characterized in that: The template size used for filtering is 5x1.
6. A device for extracting three-dimensional coordinates of the center line of a line laser stripe, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 5.
7. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.
8. A system for extracting three-dimensional coordinates of the center line of a line laser stripe, characterized in that: It includes the three-dimensional coordinate extraction device of the center line of the linear laser stripe as described in claim 6.
9. The system for extracting three-dimensional coordinates of the center line of a line laser stripe according to claim 8, characterized in that: Also includes: A line laser is configured to provide a line laser to a workpiece to be inspected on the optical platform; An image sensor is configured to collect image data and upload it to the three-dimensional coordinate extraction device of the center line of the linear laser stripe; The image sensor adopts a CMOS image sensor, and the linear laser stripe center line three-dimensional coordinate extraction device adopts a Zynq-7000 series chip.
Citation Information
Cited By
Sensing, storing and computing integrated chip and unmanned aerial vehicle line laser three-dimensional reconstruction method and system
CN122176178A
