Camera calibration method, cropping data generation method, device, equipment and medium

Abstract

The present application discloses a camera calibration method, device, cutting data generation method for a cutting table, device, electronic device, and readable storage medium. The camera calibration method includes obtaining a calibration image of a calibration plate captured by a camera to be calibrated; the calibration plate is smaller than the shooting range of the camera to be calibrated and is placed within the shooting range of the camera to be calibrated; extracting ideal image coordinate information of multiple calibration points from the calibration image; and calculating calibration parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset due to lens distortion, and the world coordinate information of each calibration point in the world coordinate system of the calibration plate. This method can effectively improve the camera calibration accuracy, facilitate the generation of high-precision cutting data, and further improve the cutting accuracy of the cutting table.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a camera calibration method and device, a cutting data generation method and device for a cutting table, an electronic device, and a readable storage medium. Background Art

[0002] Camera calibration, or camera calibration, is the process of converting from a world coordinate system to a camera coordinate system, and then from the camera coordinate system to an image coordinate system. Related technologies typically perform image processing on a calibration object of known shape and size. Then, a series of mathematical transformations and calculations are used to calculate the intrinsic and extrinsic parameters of the camera model. These calculated intrinsic and extrinsic parameters are used as calibration parameters to complete camera calibration.

[0003] However, the camera calibration parameters generated when converting image coordinates and world coordinates in related technologies are fitting parameters for images covering the entire camera shooting range. Since the pixel surface to be fitted is large, the fitting accuracy is not high, resulting in low accuracy of the camera calibration parameters, that is, poor camera calibration accuracy. In industrial production, such as the process of cutting data on a cutting table, there are high precision requirements for converting the image coordinate system to the cutting table coordinate system, that is, the mapping of pixels to the actual coordinates of the cutting table. The internal and external parameters of the camera model and the mapping matrix between the world coordinate system and the cutting table coordinate system calculated using related technologies are not accurate. In addition, since the image coordinate system and the cutting table coordinate system are not planar mappings of the same order of magnitude, the cutting data accuracy is further low and cannot meet the actual accuracy requirements.

[0004] In view of this, how to improve the camera calibration accuracy, and thus help improve the accuracy of cropping data generation, is a technical problem that technical personnel in the field need to solve. Summary of the Invention

[0005] The present application provides a camera calibration method, device, cutting data generation method, device, electronic device and readable storage medium for a cutting table, which improve the camera calibration accuracy and are conducive to improving the accuracy of cutting data generation.

[0006] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0007] An embodiment of the present invention provides a camera calibration method, including:

[0008] Acquire a calibration image of a calibration plate captured by the camera to be calibrated; the calibration plate is smaller than the shooting range of the camera to be calibrated, and the calibration plate is placed within the shooting range of the camera to be calibrated;

[0009] extracting ideal image coordinate information of a plurality of calibration points from the calibration image;

[0010] The calibration parameters of the camera to be calibrated are calculated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset by lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located.

[0011] Optionally, the step of calculating the calibration parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset due to lens distortion, and the world coordinate information of each calibration point in the world coordinate system of the calibration plate includes:

[0012] The linear parameters of the camera to be calibrated, the rotation matrix from the world coordinate system to the camera coordinate system, and the translation vector are calculated based on the ideal image coordinate information of each calibration point and the world coordinate information in the world coordinate system where the calibration plate is located;

[0013] The nonlinear distortion parameters of the camera to be calibrated are calculated based on the ideal image coordinate information of each calibration point and the actual image coordinates that are offset due to lens distortion.

[0014] Optionally, after calculating the calibration parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset due to lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located, the method further includes:

[0015] The calibration parameters are optimized using the Levenberg-Marquardt algorithm.

[0016] Optionally, before extracting the ideal image coordinate information of the plurality of calibration points from the calibration image, the method further includes:

[0017] grayscale the calibration image to obtain a grayscale image;

[0018] Performing Gaussian filtering on the grayscale image based on filtering parameters determined by the grayscale image resolution and noise level to obtain a smoothed image;

[0019] Processing the smoothed image using a Laplace algorithm and extracting an edge image of the processed image;

[0020] Based on the edge image, pixel values of the grayscale image are modified using a preset threshold and the filtering parameter to obtain a candidate image that contains both gradient and grayscale and has separability characteristics;

[0021] Extracting a contour image after segmenting the candidate image by image binarization threshold;

[0022] Processing the contour image based on the shape of the calibration plate to obtain outer contour information of the calibration plate;

[0023] A target calibration image is determined based on the candidate image and the outer contour information, so as to determine calibration points in the target calibration image.

[0024] Optionally, extracting ideal image coordinate information of a plurality of calibration points from the calibration image includes:

[0025] Binarizing the target calibration image according to a preset grayscale value range to obtain multiple binary images;

[0026] For each binary image, determine the coordinate information of each calibration object within the preset calibration point selection range;

[0027] Determine the ideal image coordinates of each calibration object according to the coordinate values of the same calibration object in each binary image;

[0028] Select calibration points from each calibration object according to the preset calibration point selection rules.

[0029] Another aspect of the present invention provides a camera calibration device, including:

[0030] An image acquisition module is used to acquire a calibration image of a calibration plate captured by the camera to be calibrated; the calibration plate is smaller than the shooting range of the camera to be calibrated, and the calibration plate is placed within the shooting range of the camera to be calibrated;

[0031] A marking point determination module, configured to extract ideal image coordinate information of a plurality of calibration points from the calibration image;

[0032] The calibration parameter calculation module is used to calculate the calibration parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset by lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located.

[0033] An embodiment of the present invention provides a method for generating cutting data for a cutting table, wherein the method calibrates a cutting table camera using any one of the camera calibration methods described above, including:

[0034] Determine the mapping relationship between the world coordinate system and the cutting table coordinate system in advance based on the calibration parameters of the calibrated cutting table camera;

[0035] Based on the calibration parameters, the image coordinates of each image pixel point of the image to be cropped are converted into world coordinates of a world coordinate system;

[0036] According to the mapping relationship, each image pixel of the image to be cut is converted from the world coordinate to the cutting table coordinate of the cutting table coordinate system to generate cutting data.

[0037] Another embodiment of the present invention provides a cutting data generating device for a cutting table, comprising:

[0038] A calibration module, configured to calibrate the cutting table camera using any of the camera calibration methods described above;

[0039] A mapping relationship determination module is used to determine the mapping relationship between the world coordinate system and the cutting table coordinate system in advance based on the calibration parameters of the calibrated cutting table camera;

[0040] A coordinate conversion module, configured to convert the image coordinates of each pixel point of the image to be cropped into world coordinates of a world coordinate system based on the calibration parameters;

[0041] The cutting data generating module is used to convert each image pixel of the image to be cut from the world coordinate to the cutting coordinate of the cutting table coordinate system according to the mapping relationship, so as to generate cutting data.

[0042] An embodiment of the present invention also provides an electronic device, including a processor and a memory, wherein the processor is used to implement the steps of the cutting data generation method and / or camera calibration method of the cutting table as described in any of the preceding items when executing a computer program stored in the memory.

[0043] Finally, an embodiment of the present invention further provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cutting data generation method and / or camera calibration method of the cutting table as described in any of the preceding items are implemented.

[0044] The advantage of the camera calibration technical solution provided by the present application is that an image taken with a calibration plate is used as the calibration image, and the size of the calibration plate is only a part of the camera shooting range. In this way, the local image of the shooting range is involved in the camera calibration, and the final fitting parameters are the optimal fitting parameters belonging to this local range; the pixel surface to be fitted corresponding to the fitting parameters becomes smaller, so the fitting accuracy is higher than the entire image including the entire camera shooting range, and the accuracy of the obtained camera calibration parameters is also higher, thereby effectively improving the camera calibration accuracy.

[0045] The advantage of the cutting data generation technical solution provided by this application is that the cutting table camera is calibrated before generating the cutting data. Based on the camera calibration parameters, the image coordinate system is first converted into the world coordinate of the corresponding calibration plate, and then the world coordinate is subjected to a two-plane perspective transformation with the cutting table coordinate system. In this way, the perspective transformation is calculated at the same unit level, which can improve the accuracy and is conducive to generating high-precision cutting data, thereby improving the cutting accuracy of the cutting table.

[0046] In addition, the embodiment of the present invention also provides a cutting data generation method for a cutting table and its corresponding implementation devices and readable storage media for the camera calibration method, further making the method more practical. The cutting data generation method for a cutting table and its corresponding devices and readable storage media have corresponding advantages.

[0047] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of a flow chart of a camera calibration method provided by an embodiment of the present invention;

[0050] Figure 2 A structural diagram of a specific implementation of a camera calibration device provided by an embodiment of the present invention;

[0051] Figure 3 A schematic flow chart of another method for generating cutting data for a cutting table provided in an embodiment of the present invention;

[0052] Figure 4 A structural diagram of a specific implementation of a cutting data generating device for a cutting table provided in an embodiment of the present invention;

[0053] Figure 5 A structural diagram of a specific implementation of an electronic device provided by an embodiment of the present invention;

[0054] Figure 6 A schematic diagram of a calibration plate provided as an illustrative example in accordance with an embodiment of the present invention;

[0055] Figure 7 An image obtained by performing a schematic image processing on a calibration image provided by an embodiment of the present invention;

[0056] Figure 8 This is an image obtained after performing another schematic image processing on the calibration image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0058] In the specification, claims, and drawings of this application, the terms "first," "second," "third," "fourth," and so on are used to distinguish between different items, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0059] After introducing the technical solutions of the embodiments of the present invention, various non-limiting implementation methods of the present application are described in detail below.

[0060] See first Figure 1 , Figure 1 A schematic flow chart of a method for generating cutting data for a cutting table provided in an embodiment of the present invention may include the following contents:

[0061] S101: Acquire a calibration image of a calibration plate captured by a camera to be calibrated.

[0062] The calibration plate in this step must be smaller than the camera's field of view. The calibration image is captured by the camera when the plate is placed within the camera's field of view. This captures a partial image of the camera's field of view, rather than the entire field of view. Alternatively, the calibration plate can be placed in the center of the camera's field of view.

[0063] S102: Extracting ideal image coordinate information of multiple calibration points from the calibration image.

[0064] The calibration points in this step are the pixel coordinates of multiple pixels extracted from the calibration image, measured in pixels. The ideal image coordinates here refer to the coordinates of the pixels calculated from the captured image without considering external factors such as lens distortion. The number of calibration points selected by those skilled in the art can be flexibly determined based on the requirements of the subsequent step S103.

[0065] S103: Calculate the calibration parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point, the actual image coordinate offset due to lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located.

[0066] In this embodiment, the ideal image coordinate information and actual image coordinates of the calibration points are in the image coordinate system or the camera coordinate system. The coordinates of the calibration points in the image corresponding to the actual physical points on the calibration plate are called world coordinate information and are in the coordinate system of the external world where the calibration plate is located, i.e., the world coordinate system. The coordinate information of the cutting data of the cutting table is in the cutting table coordinate system. The calibration parameters of the camera to be calibrated include camera intrinsic parameters and camera extrinsic parameters. Camera intrinsic parameters may include, but are not limited to, linear parameters, radial distortion parameters, and tangential distortion parameters. Camera extrinsic parameters may include the rotation matrix and translation vector from the world coordinate system to the camera coordinate system. In this step, the linear parameters, rotation matrix and translation vector from the world coordinate system to the camera coordinate system of the camera to be calibrated are first calculated based on the ideal image coordinate information of each calibration point and the world coordinate information in the world coordinate system where the calibration plate is located. Then, the nonlinear distortion parameters of the camera to be calibrated are calculated based on the ideal image coordinate information of each calibration point and the actual image coordinates that are offset due to lens distortion. The nonlinear distortion function may include, but is not limited to, radial distortion parameters and tangential distortion parameters.

[0067] In the technical solution provided by the embodiment of the present invention, an image captured by a calibration plate is used as the calibration image, and the size of the calibration plate is only a part of the camera's shooting range. In this way, the local image of the shooting range is used for camera calibration, and the final fitting parameters are the optimal fitting parameters belonging to this local range. The pixel surface to be fitted corresponding to the fitting parameters becomes smaller, so the fitting accuracy is higher than that of the entire image containing the entire camera's shooting range, and the obtained camera calibration parameters are more accurate, thereby effectively improving the camera calibration accuracy.

[0068] In order to improve the accuracy of the camera's calibration parameters and make the image coordinate information of the calibration points on the calibration plate in the pixel coordinate system and the corresponding world coordinate position of the calibration plate in the world coordinate system more accurate, the Levenberg-Marquardt algorithm can be used to optimize the calibration parameters.

[0069] In order to improve the calculation accuracy of the ideal image coordinate information and thus improve the subsequent camera calibration accuracy, the calibration image may be processed before extracting the ideal image coordinate information. The image processing may include the following:

[0070] The calibration image is grayscaled to obtain a grayscale image.

[0071] The grayscale image is subjected to Gaussian filtering based on the filtering parameters determined by the grayscale image resolution and noise level to obtain a smoothed image.

[0072] The smoothed image is processed using the Laplace algorithm, and the edge image of the processed image is extracted.

[0073] Based on the edge image, the pixel values of the grayscale image are modified using preset thresholds and filtering parameters to obtain candidate images that contain both gradients and grayscale and exhibit separable properties. The preset thresholds and filtering parameters can be flexibly selected based on actual needs, and by modifying the pixel values, the image quality of the calibration image can be further improved.

[0074] Extract the contour image after the candidate image is segmented by image binarization threshold.

[0075] The contour image is processed based on the shape of the calibration plate to obtain the outer contour information of the calibration plate.

[0076] A target calibration image is determined based on the candidate image and the outer contour information, so as to determine calibration points in the target calibration image.

[0077] The above embodiment does not limit step S102. As an optional implementation, the present application further provides a method for selecting calibration points, which may include:

[0078] The target calibration image is binarized according to a preset grayscale value range to obtain multiple binary images.

[0079] For each binary image, the coordinate information of each calibration object within the preset calibration point selection range is determined.

[0080] The ideal image coordinates of each calibration object are determined according to the coordinate values of the same calibration object in each binary image.

[0081] Select calibration points from each calibration object according to the preset calibration point selection rules.

[0082] Among them, the preset grayscale value range and the preset calibration point selection range can be flexibly selected according to the actual application scenario. The calibration object is an entity in the calibration plate that can play a calibration role, and the calibration point is a part selected from the calibration object. In other words, the calibration point is a part of the calibration object. In order to distinguish the two, this embodiment uses different nouns to distinguish them. For each calibration object, the number of binary images is the same as the number of ideal image coordinates of the calibration object. The preset calibration point selection rule is to select which calibration objects from multiple calibration objects for subsequent calculation of calibration parameters, that is, the rules for calibration points in S102. Technical personnel in the relevant field can flexibly select according to actual needs, which is conducive to improving the calculation accuracy of the ideal image coordinates of each calibration point.

[0083] In this embodiment, because the edges of the pictures taken by the camera are blurred and the image pixel coordinates are integers, the center of the calibration plate is taken as the average of the centers of the circles under multiple threshold binarization, which is similar to obtaining the sub-pixel value of the center of the circle in the image, so that the calibration parameters of the camera calculated corresponding to the world coordinates of the actual calibration plate center are more accurate.

[0084] The embodiments of the present invention also provide a corresponding apparatus for the camera calibration method, further enhancing the practicality of the method. The apparatus can be described from the perspective of functional modules and hardware. The following describes the camera calibration apparatus provided by the embodiments of the present invention. The camera calibration apparatus described below can be used in conjunction with the camera calibration method described above.

[0085] From the perspective of functional modules, see Figure 2 , Figure 2 This is a structural diagram of a camera calibration device provided in an embodiment of the present invention in a specific implementation manner. The device may include:

[0086] The image acquisition module 201 is used to acquire a calibration image of a calibration plate captured by the camera to be calibrated; the calibration plate is smaller than the shooting range of the camera to be calibrated, and the calibration plate is placed within the shooting range of the camera to be calibrated.

[0087] The marking point determination module 202 is configured to extract ideal image coordinate information of a plurality of calibration points from the calibration image.

[0088] The calibration parameter calculation module 203 is used to calculate the calibration parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset by lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located.

[0089] As an optional implementation, the calibration parameter calculation module 203 can be further used to: calculate the linear parameters of the camera to be calibrated, the rotation matrix from the world coordinate system to the camera coordinate system, and the translation vector based on the ideal image coordinate information of each calibration point and the world coordinate information in the world coordinate system where the calibration plate is located; and calculate the nonlinear distortion parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point and the actual image coordinates that are offset due to lens distortion.

[0090] As another optional implementation, the apparatus may further include an optimization module configured to optimize the calibration parameters using a Levenberg-Marquardt algorithm after obtaining the calibration parameters of the camera to be calibrated.

[0091] As another optional embodiment, the above-mentioned device may also include a calibration image processing module, which is used to grayscale the calibration image to obtain a grayscale image; perform Gaussian filtering on the grayscale image based on filtering parameters determined by the grayscale image resolution and noise level to obtain a smoothed image; use the Laplace algorithm to process the smoothed image, and extract the edge image of the processed image; based on the edge image, use a preset threshold and filtering parameters to modify the pixel value of the grayscale image to obtain a candidate image that contains both gradient and grayscale and has separability characteristics; extract a contour image after the candidate image is segmented by the image binarization threshold; process the contour image based on the shape of the calibration plate to obtain the outer contour information of the calibration plate; determine the target calibration image based on the candidate image and the outer contour information to determine the calibration point in the target calibration image.

[0092] As an optional implementation of this embodiment, the above-mentioned marking point determination module 202 can be further used to: binarize the target calibration image according to a preset grayscale value range to obtain multiple binary images; for each binary image, determine the coordinate information of each calibration object within the preset calibration point selection range; determine the ideal image coordinates of each calibration object based on the coordinate value of the same calibration object in each binarized image; and select calibration points from each calibration object according to the preset calibration point selection rules.

[0093] It can be seen from the above that the embodiments of the present invention can effectively improve the camera calibration accuracy, which is conducive to improving the accuracy of cropping data generation.

[0094] Based on the relevant embodiments of the above-mentioned camera calibration method, this application provides a method for generating cutting data for a cutting table, see Figure 3 , Figure 3 This is a flow chart of a method for generating cutting data for a cutting table provided in an embodiment of the present invention. The embodiment of the present invention can be applied to a GLS fully automatic multi-layer cutting system, for example. Specifically, the method may include the following:

[0095] S301: Pre-calibrate the cutting machine camera.

[0096] In this step, the camera calibration method described in any of the above embodiments can be used to calibrate the camera of the cutting table to obtain the calibration parameters of the cutting table camera. The calibration parameters include camera intrinsic parameters and camera extrinsic parameters. The camera intrinsic parameters may include, but are not limited to, linear parameters, radial distortion parameters, and tangential distortion parameters. The camera extrinsic parameters may include the rotation matrix and translation vector from the world coordinate system to the camera coordinate system.

[0097] S302: Determine in advance the mapping relationship between the world coordinate system and the cutting table coordinate system based on the calibration parameters of the calibrated cutting table camera.

[0098] In this step, multiple calibration points can be pre-selected and their image coordinate values obtained. For each image pixel, the image pixel coordinate system can be converted to a coordinate value in the image's corresponding world coordinate system based on the camera calibration parameters obtained in the above embodiment. The mapping relationship between the world coordinate system and the cutting table coordinate system is calculated based on the world coordinate information of these calibration points and the coordinate information in the cutting table coordinate system. Because the coordinates in the world coordinate information and the cutting table coordinate system are in the same order of magnitude, for example, both can be in microns, which is different from the pixel units in the image coordinate system, planar mapping based on the same order of magnitude does not result in a significant loss of precision, which helps to improve the accuracy of the final cutting data generation.

[0099] S303: Based on the calibration parameters, the image coordinates of each image pixel of the image to be cropped are converted into world coordinates of the world coordinate system.

[0100] S304: According to the mapping relationship, each pixel point of the image to be cut is converted from the world coordinate to the cutting table coordinate of the cutting table coordinate system to generate cutting data.

[0101] In this step, after each image pixel point of the image to be cut is converted into the cutting table coordinates in the cutting table coordinate system, the image to be cut is mapped onto the cutting table, and cutting data is generated according to the cutting requirements and the cutting table coordinate information corresponding to each image pixel point, and the cutting is performed using the cutting table based on the generated cutting data.

[0102] In the technical solution provided by the embodiment of the present invention, the cutting table camera is calibrated before generating the cutting data. Based on the camera calibration parameters, the image coordinate system is first converted into the world coordinate of the corresponding calibration plate, and then the world coordinate is transformed with the cutting table coordinate system in two planes. In this way, the perspective transformation is calculated at the same unit level, which can improve the accuracy and is conducive to generating high-precision cutting data, thereby improving the cutting accuracy of the cutting table.

[0103] It should be noted that there is no strict order in which the steps in this application are performed. As long as they comply with the logical order, these steps can be performed simultaneously or in a predetermined order. Figure 1 and Figure 3 This is just a schematic and does not mean that this is the only execution order.

[0104] The embodiments of the present invention also provide a corresponding device for the method for generating cutting data for a cutting table, further enhancing the practicality of the method. The device can be described from the perspective of functional modules and hardware. The following describes the device for generating cutting data for a cutting table provided in an embodiment of the present invention. The device for generating cutting data for a cutting table described below can be used in conjunction with the method for generating cutting data for a cutting table described above.

[0105] From the perspective of functional modules, see Figure 4 , Figure 4 A structural diagram of a cutting data generating device for a cutting table provided in an embodiment of the present invention in a specific implementation manner, the device may include:

[0106] The calibration module 401 is used to calibrate the cutting table camera using the camera calibration method in any of the above embodiments.

[0107] The mapping relationship determination module 402 is used to determine the mapping relationship between the world coordinate system and the cutting table coordinate system in advance according to the calibration parameters of the calibrated cutting table camera.

[0108] The coordinate conversion module 403 is configured to convert the image coordinates of each pixel point of the image to be cropped into the world coordinates of the world coordinate system based on the calibration parameters.

[0109] The cutting data generating module 404 is configured to convert each pixel of the image to be cut from the world coordinate to the cutting coordinate of the cutting table coordinate system according to the mapping relationship, so as to generate cutting data.

[0110] It can be seen from the above that the embodiments of the present invention can effectively improve the accuracy of cropping data generation.

[0111] The cutting data generating device and camera calibration device of the cutting table mentioned above are described from the perspective of functional modules. Furthermore, the present application also provides an electronic device, which is described from the perspective of hardware. Figure 5 This is a structural diagram of another electronic device provided in an embodiment of the present application. Figure 5 As shown, the device includes a memory 50 for storing computer programs; a processor 51 for implementing the steps of the camera calibration method and / or the cutting data generation method for the cutting table mentioned in any of the above embodiments when executing the computer program.

[0112] Among them, the processor 51 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 51 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 51 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 51 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 51 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0113] The memory 50 may include one or more computer-readable storage media, which may be non-transitory. The memory 50 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 50 is at least used to store the following computer program 501, wherein, after the computer program is loaded and executed by the processor 51, it can implement the relevant steps of the camera calibration method and / or the cutting data generation method of the cutting table disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 50 may also include an operating system 502 and data 503, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 502 may include Windows, Unix, Linux, etc. The data 503 may include but is not limited to data corresponding to the camera calibration method and / or the cutting data generation result of the cutting table, etc.

[0114] In some embodiments, the electronic device may further include a display screen 52, an input / output interface 53, a communication interface 54 or a network interface, a power supply 55 and a communication bus 56. Among them, the display screen 52 and the input / output interface 53, such as a keyboard, are user interfaces, and the optional user interface may also include a standard wired interface, a wireless interface, etc. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface. The communication interface 54 may optionally include a wired interface and / or a wireless interface, such as a WI-FI interface, a Bluetooth interface, etc., which is generally used to establish a communication connection between the electronic device and other electronic devices. The communication bus 56 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0115] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, for example, it may also include a sensor 57 to implement various functions.

[0116] The functions of the functional modules of the electronic device described in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment and will not be repeated here.

[0117] It can be seen from the above that the embodiments of the present invention can effectively improve the camera calibration accuracy, which is conducive to improving the accuracy of cropping data generation.

[0118] It is understandable that if the camera calibration method and / or the cutting data generation method of the cutting table in the above-mentioned embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program codes.

[0119] Based on this, an embodiment of the present invention further provides a readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the camera calibration method and / or the cutting data generation method of the cutting table as described in any of the above embodiments.

[0120] The functions of the functional modules of the readable storage medium described in the embodiment of the present invention can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description of the above method embodiment and will not be repeated here.

[0121] In order to make the technical solution of the present application more clearly understood by those skilled in the art, the present application uses an illustrative example to illustrate the process of generating cutting data of a cutting table, which may include the following contents:

[0122] A1: Pre-made Figure 6 The calibration target is each dot on the calibration plate shown. The calibration plate's parameters are: 33 rows by 22 columns of dots, a 15mm center-to-center distance between dots, and a 5mm diameter. The calibration plate's dimensions are three-fifths the length and width of the camera's field of view, representing the area within which the calibration plate can be placed. The calibration plate is placed in the center of the camera's field of view, and the calibration image is captured using a D5600 camera, also known as the cutting table camera. The film size is 23.5 by 15.6 inches, the aspect ratio of the calibration image is 3:2, and the image size is 6000 by 4000 pixels. The height from the camera head to the table is 600mm.

[0123] A2: Yes Figure 6 Grayscale the image, converting the three channels into a single channel. Apply a Gaussian filter with a radius of r to the grayscale image to suppress Gaussian noise and smooth the image. r can be an odd number such as 3, 5, 7, 9, or 11. The value of r depends on the image resolution and noise level.

[0124] A3: Apply the Laplace algorithm with a radius of r to the Gaussian filtered image, perform isotropic differentiation, and extract edges.

[0125] A4: Using the image data generated in step A3 as the reference template data, modify the pixel values of the grayscale image generated in step A3 using a threshold value, so that the modified image contains both gradient and grayscale separability characteristics. The threshold value can be adjusted based on the radius in steps A2 and A3.

[0126] A5: Perform image binarization and threshold segmentation using the OTSU (maximum between-class variance) algorithm on the image generated in step A4 to obtain a higher quality image.

[0127] A6: Find the A4 contour and filter the contours to satisfy the outer contour of the contour containing 33*22 circular sub-contours. Use polygons to approximate the outer contour and retain the outer contour with only five vertices, such as Figure 7 shown.

[0128] A7: Cut out the original image of the white area in step A6 from the grayscale image generated by A3, and get the following Figure 8 The image shown.

[0129] A8: Yes Figure 8 Perform binarization with a grayscale value from startThresh (such as 140) to endThresh (such as 200) with a step size of 10, and for each binarized image, find the center coordinates of a circle with a diameter in the range of MinDiamMarks (such as 20) to MaxDiamMarks (such as 90).

[0130] A9: Take the average of each circle center coordinate generated in the above steps to obtain 33*22 circle center coordinates.

[0131] A10: Use the camera nonlinear model relationship to calculate the camera calibration parameters. The camera nonlinear model relationship can be expressed as:

[0132]

[0133] Where (u, v) is the image coordinate of the pixel calibration point p on the u and v axes of the image coordinate system, (X C , Y C , Z C ) is the coordinate of the corresponding point P in the camera coordinate system corresponding to p, (X W , Y W , Z W ) is the coordinate in the world coordinate system corresponding to P, s is the scale factor, a x , a yare the scale factors on the u and v axes respectively; (u0, v0) is the position of the principal point on the image plane, R, t are the rotation matrix and translation vector from the world coordinate system to the camera coordinate system, M1 is the linear camera internal parameter, and M2 is the camera external parameter.

[0134] Considering radial distortion and tangential distortion at the same time, the radial distortion and tangential distortion are calculated using the distortion parameter calculation formula. The distortion parameter calculation formula is:

[0135]

[0136] Where u d , v d is the coordinate in the image pixel coordinate system, x=X C / Z C , y=Y C / Z C , r 2 =x 2 +y 2 , r is the intermediate parameter, a x 、a y , u0, v0 are linear parameters in the camera internal parameters of the camera calibration parameters, k1, k2, p1, p2 are nonlinear distortion parameters in the camera internal parameters of the camera calibration parameters, k1 and k2 are radial distortion parameters, and p1 and p2 are tangential distortion parameters.

[0137] A11: Based on the calculation in step A10, the calibration parameters of the cutting table camera are obtained and optimized using the LM algorithm - Levenberg-Marquardt algorithm.

[0138] A12: Select four calibration points within the calibration range. For each image pixel, convert the four points in the image pixel coordinate system into the four points in the image corresponding world coordinate system according to Formula 1. That is, first convert the image coordinates into world coordinates, i.e., the coordinates of the calibration plate, in units of um. Then, based on the correspondence between the four points in the image corresponding world coordinate system and the cutting table coordinate system, generate the homography matrix map of the two planes and obtain the mapping function of the data points in the two coordinate systems. That is, determine the mapping parameters of the image and cutting table. The mapping function of the data points in the two coordinate systems can be calculated according to the following formula:

[0139]

[0140]

[0141] In the formula, (x a 、y a ) is the coordinate of the calibration point of the image corresponding to the world coordinate system, (x b0 、y b0 ) is the coordinate of the corresponding point in the cutting table coordinate system. map 11 、map12 、map 13 、map 31 、map 32 、map 33 、map 21 、map 22 、map 23 It is the element value of the homography matrix map.

[0142] A13: For each pixel coordinate point of the cropped image, first call the internal parameter to calculate the world coordinate, and then call the mapping matrix generated in step A12 to obtain its cutting table coordinate, so that the maximum error between the cutting table coordinate converted from the image point coordinate and the actual cutting table point coordinate can be less than the world coordinate measurement corresponding to 1 pixel.

[0143] As can be seen from the above, the embodiment of the present invention uses a calibration plate to take a picture for calibration, so that the camera linear parameters, radial parameter distortion parameters, and tangential distortion parameters are the optimal function fitting parameters of the current calibration range. When the image plane and the cutting table plane are plane-mapped, the image pixels are first converted into the actual world coordinate range of the calibration plate taken, and then the two images are perspectively transformed between the actual world coordinate range of the calibration plate and the cutting table coordinate system. The perspective transformation is calculated at the same unit level, which can effectively improve the accuracy of the cutting data and avoid the difference in the accuracy level between the direct image coordinates (i.e., integers) and the cutting table coordinates (based on mm floating points) when the direct perspective transformation is performed. Generally, the number of rows and columns of the image is four digits, while the number of cutting table coordinates accurate to the um level is five digits.

[0144] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0145] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0146] The above is a detailed introduction to a camera calibration method, device, cutting data generation method, device, electronic device and readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

Claims

1. A camera calibration method, characterized in that: include: Obtain a calibration image of the calibration plate captured by the camera to be calibrated; The calibration plate is smaller than the shooting range of the camera to be calibrated, and the calibration plate is placed within the shooting range of the camera to be calibrated, and the calibration image is a partial image of the shooting range of the camera to be calibrated; grayscale the calibration image to obtain a grayscale image; and Gaussian filter the grayscale image using filter parameters determined based on the resolution and noise level of the grayscale image to obtain a smoothed image. Processing the smoothed image using a Laplace algorithm, extracting an edge image from the processed image, and modifying pixel values of the grayscale image using a preset threshold and the filter parameter based on the edge image to obtain a candidate image that contains both gradient and grayscale and has separability characteristics; Extracting a contour image after segmenting the candidate image by image binarization threshold, and processing the contour image based on the shape of the calibration plate to obtain outer contour information of the calibration plate; determining a target calibration image based on the candidate image and the outer contour information, so as to determine calibration points in the target calibration image; Binarizing the target calibration image according to a preset grayscale value range to obtain multiple binary images, and determining the coordinate information of each calibration object within the preset calibration point selection range for each binary image; Determining the ideal image coordinates of each calibration object based on the coordinate values of the same calibration object in each binary image, and selecting calibration points from each calibration object according to a preset calibration point selection rule; the ideal image coordinate information is the coordinate information of the pixel points calculated from the image captured without considering lens distortion; The calibration parameters of the camera to be calibrated are calculated based on the ideal image coordinate information of each calibration point, the actual image coordinates that are offset due to lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located; wherein the ideal image coordinate information and the actual image coordinates of each calibration point are in the image coordinate system or the camera coordinate system.

2. The camera calibration method according to claim 1, wherein: The calibration parameters of the camera to be calibrated are calculated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset by lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located, including: The linear parameters of the camera to be calibrated, the rotation matrix from the world coordinate system to the camera coordinate system, and the translation vector are calculated based on the ideal image coordinate information of each calibration point and the world coordinate information in the world coordinate system where the calibration plate is located; The nonlinear distortion parameters of the camera to be calibrated are calculated based on the ideal image coordinate information of each calibration point and the actual image coordinates that are offset due to lens distortion.

3. The camera calibration method according to claim 2, wherein: After calculating the calibration parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset due to lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located, the method further includes: The calibration parameters are optimized using the Levenberg-Marquardt algorithm.

4. A camera calibration device, characterized in that: include: An image acquisition module is used to acquire a calibration image of the calibration plate captured by the camera to be calibrated; The calibration plate is smaller than the shooting range of the camera to be calibrated, and the calibration plate is placed within the shooting range of the camera to be calibrated; the calibration image is a partial image of the shooting range of the camera to be calibrated; a calibration image processing module, configured to grayscale the calibration image to obtain a grayscale image; perform Gaussian filtering on the grayscale image based on filtering parameters determined by the resolution and noise level of the grayscale image to obtain a smoothed image; process the smoothed image using a Laplace algorithm, extract an edge image of the processed image, and based on the edge image, modify the pixel values of the grayscale image using a preset threshold and the filtering parameters to obtain a candidate image that contains both gradient and grayscale and has separability characteristics; extract a contour image after segmenting the candidate image using an image binarization threshold, and process the contour image based on the shape of the calibration plate to obtain outer contour information of the calibration plate; determining a target calibration image based on the candidate image and the outer contour information, so as to determine calibration points in the target calibration image; A marking point determination module is used to perform binarization processing on the target calibration image according to a preset grayscale value range to obtain multiple binary images, and for each binary image, determine the coordinate information of each calibration object within the preset calibration point selection range; Determining the ideal image coordinates of each calibration object based on the coordinate values of the same calibration object in each binary image, and selecting calibration points from each calibration object according to a preset calibration point selection rule; the ideal image coordinate information is the coordinate information of the pixel points calculated from the image captured without considering lens distortion; The calibration parameter calculation module is used to calculate the calibration parameters of the camera to be calibrated based on the ideal image coordinate information of each calibration point, the actual image coordinates offset by lens distortion, and the world coordinate information of each calibration point in the world coordinate system where the calibration plate is located.

5. A method for generating cutting data for a cutting table, characterized in that: The camera calibration method according to any one of claims 1 to 3 is used to calibrate a cutting table camera, comprising: Determine the mapping relationship between the world coordinate system and the cutting table coordinate system in advance based on the calibration parameters of the calibrated cutting table camera; Based on the calibration parameters, the image coordinates of each image pixel point of the image to be cropped are converted into world coordinates of a world coordinate system; According to the mapping relationship, each image pixel of the image to be cut is converted from the world coordinate to the cutting table coordinate of the cutting table coordinate system to generate cutting data.

6. A cutting data generating device for a cutting table, characterized in that: include: A calibration module, configured to calibrate a cutting table camera using the camera calibration method according to any one of claims 1 to 3; A mapping relationship determination module is used to determine the mapping relationship between the world coordinate system and the cutting table coordinate system in advance based on the calibration parameters of the calibrated cutting table camera; A coordinate conversion module, configured to convert the image coordinates of each pixel point of the image to be cropped into world coordinates of a world coordinate system based on the calibration parameters; The cutting data generating module is used to convert each image pixel of the image to be cut from the world coordinate to the cutting coordinate of the cutting table coordinate system according to the mapping relationship, so as to generate cutting data.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is configured to implement the camera calibration method according to any one of claims 1 to 3 and / or the step of generating cutting data for a cutting table according to claim 5 when executing a computer program stored in the memory.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the camera calibration method according to any one of claims 1 to 3 and / or the step of generating cutting data for the cutting table according to claim 5.

Images