Camera calibration method in high-pressure die-casting environment and related equipment

Abstract

The invention discloses a camera calibration method and related equipment in a high-pressure die-casting environment, and relates to the field of camera calibration, and the method comprises the steps: obtaining a two-dimensional image of a die in the high-pressure die-casting environment through an infrared camera; processing the two-dimensional image to obtain a geometric feature region of the mold; performing iterative optimization on a first camera parameter of the infrared camera through a first algorithm, enabling a second pixel coordinate of the same pixel point in the geometric feature region to gradually approach to the first pixel coordinate, obtaining a second camera parameter after the iterative optimization of the first algorithm, and continuing to perform iterative optimization on the second camera parameter through a second algorithm to obtain a second camera parameter after the iterative optimization of the second algorithm. Obtaining a third camera parameter after iterative optimization of a second algorithm, wherein the second algorithm is an iterative optimization algorithm depending on the initial value; and taking the third camera parameter as a camera parameter of the infrared camera. According to the method, the camera parameters are iteratively optimized through the first algorithm which does not depend on the initial value, so that the second algorithm which depends on the initial value has a good initial value for iterative optimization, and the accuracy of camera calibration is improved.

Description

Technical Field

[0001] The present application relates to the field of camera calibration technology, and in particular to a camera calibration method and related equipment in a high-pressure die-casting environment. Background Art

[0002] Camera calibration is a key technology in the field of computer vision and image processing. Its purpose is to determine the camera parameters of the camera, which are used to map and transform the two-dimensional pixel coordinates of the image to the three-dimensional world coordinates. Since the image captured by the infrared temperature camera is an infrared temperature map, the camera calibration of the infrared temperature camera is a special camera calibration.

[0003] In the early days, the calibration of infrared temperature cameras relied on manual calibration, but with the development of technology, the calibration of infrared temperature cameras began to adopt automatic calibration methods based on computer software algorithms. The automatic calibration method analyzes the image data captured by the infrared temperature camera and adjusts the camera parameters of the infrared temperature camera in real time to improve the accuracy of the temperature in the infrared temperature map.

[0004] However, in the current high-pressure die-casting environment, the camera parameters of infrared temperature cameras obtained by automatic calibration methods based on computer software algorithms are often inaccurate. Summary of the invention

[0005] In view of the above problems, the present application provides a camera calibration method and related equipment in a high-pressure die-casting environment to achieve the purpose of improving the camera calibration accuracy of infrared cameras. The specific scheme is as follows:

[0006] A first aspect of the present application provides a camera calibration method in a high-pressure die-casting environment, the camera calibration method in a high-pressure die-casting environment comprising:

[0007] Acquire a 2D image of the mold in a high-pressure die casting environment using an infrared camera;

[0008] Processing the two-dimensional image to obtain a plurality of geometric feature areas of the mold;

[0009] Iteratively optimizing the first camera parameters of the infrared camera by using a first algorithm so that the second pixel coordinates of the same pixel point in the geometric feature area gradually approach the first pixel coordinates, and obtaining the second camera parameters after iterative optimization by the first algorithm, wherein the first pixel coordinates are: the two-dimensional pixel coordinates of the pixel point in the two-dimensional image, and the second pixel coordinates are: the two-dimensional pixel coordinates obtained after mapping the three-dimensional world coordinates of the pixel point in the geometric feature area according to the first camera parameters, the first camera parameters are the assumed camera parameters of the infrared camera, and the first algorithm is an iterative optimization algorithm that does not depend on the initial value;

[0010] Continuing to iteratively optimize the second camera parameters by a second algorithm to obtain third camera parameters after iterative optimization by the second algorithm, wherein the second algorithm is an iterative optimization algorithm that depends on an initial value;

[0011] The third camera parameter is used as the camera parameter of the infrared camera.

[0012] In a possible implementation, iteratively optimizing the first camera parameters of the infrared camera by using a first algorithm to obtain the second camera parameters iteratively optimized by the first algorithm includes:

[0013] Constructing an objective function of the difference between the first pixel coordinate and the second pixel coordinate, wherein the design variables of the objective function are camera parameters;

[0014] Inputting the objective function into the first algorithm for iterative optimization, and outputting the camera parameters used when the value of the objective function is minimized;

[0015] The camera parameters used when the value of the objective function is minimized are used as the second camera parameters.

[0016] In a possible implementation, inputting the objective function into the first algorithm for iterative optimization and outputting the camera parameters used when the value of the objective function is minimized includes:

[0017] Determine the current objective function;

[0018] Calculating the minimum value of the current objective function and obtaining the camera parameters corresponding to the minimum value of the current objective function;

[0019] Determine the relationship between the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter and a first preset threshold;

[0020] If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is not less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is used as the current camera parameter, and the three-dimensional world coordinates of the pixel points in the geometric feature area are remapped by the current camera parameter to obtain the second pixel coordinates, and the step of determining the current objective function is returned to be executed;

[0021] If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is output.

[0022] In a possible implementation, the further iteratively optimizing the second camera parameters by using the second algorithm to obtain third camera parameters iteratively optimized by the second algorithm includes:

[0023] Determine the second camera parameter as the current camera parameter;

[0024] Optimizing the current camera parameters by using a second algorithm to obtain optimized camera parameters;

[0025] Determine the relationship between the difference between the current camera parameter and the optimized camera parameter and a second preset threshold;

[0026] If the difference between the current camera parameters and the optimized camera parameters is not less than a second preset threshold, the optimized camera parameters are used as the current camera parameters, and the step of optimizing the current camera parameters to obtain the optimized camera parameters is returned;

[0027] If the difference between the current camera parameter and the optimized camera parameter is less than a second preset threshold, the optimized camera parameter is output as the third camera parameter.

[0028] In a possible implementation, the two-dimensional image is a grayscale image, and the processing of the two-dimensional image to obtain a plurality of geometric feature regions of the mold includes:

[0029] The two-dimensional image is processed by using a deep image processing technology, and a plurality of regions with high distinguishability in the two-dimensional image are extracted as a plurality of geometric feature regions of the mold.

[0030] A second aspect of the present application provides a camera calibration device in a high-pressure die-casting environment, the camera calibration device in a high-pressure die-casting environment comprising:

[0031] An image acquisition unit, used for acquiring a two-dimensional image of the mold in a high-pressure die-casting environment through an infrared camera;

[0032] An image processing unit, used for processing the two-dimensional image to obtain a plurality of geometric feature areas of the mold;

[0033] A first optimization unit is used to iteratively optimize the first camera parameters of the infrared camera through a first algorithm, so that the second pixel coordinates of the same pixel point in the geometric feature area gradually approach the first pixel coordinates, and obtain the second camera parameters after iterative optimization by the first algorithm, wherein the first pixel coordinates are: the two-dimensional pixel coordinates of the pixel point in the two-dimensional image, the second pixel coordinates are: the two-dimensional pixel coordinates obtained after mapping the three-dimensional world coordinates of the pixel point in the geometric feature area according to the first camera parameters, the first camera parameters are the assumed camera parameters of the infrared camera, and the first algorithm is an iterative optimization algorithm that does not depend on the initial value;

[0034] A second optimization unit, configured to continue to iteratively optimize the second camera parameters by using a second algorithm to obtain third camera parameters after iterative optimization by the second algorithm, wherein the second algorithm is an iterative optimization algorithm that depends on an initial value;

[0035] A parameter acquisition unit is used to use the third camera parameter as the camera parameter of the infrared camera.

[0036] In a possible implementation, the first optimization unit includes:

[0037] a function construction subunit, configured to construct an objective function of the difference between the first pixel coordinate and the second pixel coordinate, wherein the design variables of the objective function are camera parameters;

[0038] A function optimization subunit, configured to input the objective function into the first algorithm for iterative optimization, and output the camera parameters used when the value of the objective function is minimized;

[0039] The result acquisition subunit is used to use the camera parameters used when the value of the objective function is minimized as the second camera parameters.

[0040] In a possible implementation, the function optimization subunit is specifically configured as follows:

[0041] Determine the current objective function;

[0042] Calculating the minimum value of the current objective function and obtaining the camera parameters corresponding to the minimum value of the current objective function;

[0043] Determine the relationship between the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter and a first preset threshold;

[0044] If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is not less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is used as the current camera parameter, and the three-dimensional world coordinates of the pixel points in the geometric feature area are remapped by the current camera parameter to obtain the second pixel coordinates, and the step of determining the current objective function is returned to be executed;

[0045] If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is output.

[0046] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0047] The memory is used to store computer programs;

[0048] The processor is used to execute the computer program so that the electronic device can implement the camera calibration method in a high-pressure die-casting environment of the above-mentioned first aspect or any implementation method of the first aspect.

[0049] A fourth aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the camera calibration method in a high-pressure die-casting environment of the above-mentioned first aspect or any implementation method of the first aspect.

[0050] By means of the above technical solution, the present application provides a camera calibration method and related equipment in a high-pressure die-casting environment. This method obtains the initial data for camera calibration through the image pixel coordinates of the mold and the pixel coordinates mapped by the three-dimensional world coordinates, and first processes the initial data through a first algorithm that does not rely on the initial value, and iteratively optimizes the camera parameters, so that in a high-pressure die-casting environment, the second algorithm that relies on the initial value can have a good initial value for iterative optimization, thereby improving the accuracy of camera calibration. Furthermore, the image pixel coordinates used in this method are the pixel coordinates of the geometric area of ​​the mold. Since the geometric features of the mold are not easily affected by the high-pressure die-casting environment, the geometric features of the mold are not easily changed, which further reduces the influence of the high-pressure die-casting environment on the infrared temperature camera calibration, and improves the accuracy of camera calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0052] Figure 1 A schematic flow chart of a camera calibration method in a high-pressure die-casting environment provided in this application;

[0053] Figure 2 A schematic diagram of the structure of a camera calibration device in a high-pressure die-casting environment provided by the present application;

[0054] Figure 3 A hardware structure block diagram of an electronic device provided in this application. DETAILED DESCRIPTION

[0055] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0056] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0057] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0058] Camera calibration is mainly used to determine the camera parameters (including external parameters and internal parameters), such as camera position, focus and other parameters. Since the image captured by the infrared temperature camera is an infrared temperature map, the camera calibration of the infrared temperature camera is a special camera calibration. At present, the camera calibration of infrared temperature cameras is mainly carried out through the automatic calibration method based on computer software algorithms, and the automatic calibration method can be divided into two categories: calibration algorithm based on calibration objects and self-calibration algorithm based on initial values.

[0059] However, in the high-pressure die-casting scenario, the on-site implementation scenario is complex and dangerous. For the calibration algorithm based on the calibration object, it is full of difficulties to place the calibration object in the scene. As for the self-calibration algorithm based on the initial value, the inventor of this application has found through research that in the iterative optimization process of the self-calibration algorithm based on the initial value, the accuracy of the initial value of the camera parameter will significantly affect the accuracy of the camera parameters obtained by the iterative optimization algorithm that depends on the initial value. In the prior art, the initial value of the camera parameter can be set manually or by the self-calibration algorithm. However, in this case, factors such as humidity in the high-pressure die-casting scenario are also likely to affect the self-calibration algorithm, thereby affecting the initial value of the self-calibration algorithm. Therefore, in the high-pressure die-casting scenario, both the calibration algorithm based on the calibration object and the self-calibration algorithm based on the initial value have low accuracy, which in turn affects the accuracy of the camera calibration results.

[0060] In order to solve the above problems, the embodiment of the present application provides a camera calibration method in a high-pressure die-casting environment. The camera calibration method in a high-pressure die-casting environment of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0061] Reference Figure 1 , Figure 1A schematic diagram of a camera calibration method in a high-pressure die-casting environment provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, a camera calibration method in a high-pressure die-casting environment provided in an embodiment of the present application may include steps S10 to S14, and these steps are described in detail below.

[0062] S10, obtaining a two-dimensional image of the mold in the high-pressure die-casting environment by using an infrared camera;

[0063] S11, processing the two-dimensional image to obtain multiple geometric feature areas of the mold.

[0064] Among them, the infrared camera in this embodiment is specifically an infrared temperature camera, of course, it can also be specifically other types of infrared cameras. The mold is a tool often used in high-pressure die-casting, which is used to solidify the workpiece or product to give it a fixed shape. Since the mold needs to withstand extremely high injection pressure and the impact force of the molten metal during the high-pressure die-casting process, the material and structure of the mold have sufficient strength and rigidity. Therefore, it is the high strength of the mold that makes the mold not easily affected by the high-pressure die-casting scene. In this embodiment, the mold in the high-pressure die-casting environment can be selected as the basis for the camera calibration of the infrared camera. In this embodiment, a two-dimensional image of the mold can be captured by an infrared temperature camera for subsequent camera calibration.

[0065] Among them, the two-dimensional image can be a two-dimensional temperature image (displaying the temperature distribution of the object), or it can be a grayscale image or a black-and-white image obtained after processing the two-dimensional temperature image (the black-and-white image can be obtained by further processing the grayscale image). Specifically, when processing the two-dimensional temperature image, specific information in the two-dimensional temperature image can be highlighted by filtering, enhancement, edge detection, etc., then the processed two-dimensional temperature image no longer directly reflects the temperature distribution of the object, but can provide other information, such as the edge and shape of the object. The two-dimensional image taken in this embodiment is a grayscale image or a black-and-white image obtained after processing the two-dimensional temperature image. Of course, since the temperature change in the high-pressure die-casting scenario has a certain predictability, the temperature change in the geometric feature area is more obvious and regular, then the two-dimensional temperature image of the mold can also be directly used to extract the geometric feature area.

[0066] The geometric feature area of ​​the mold is the area where the mold has a distinct geometric shape. For example, the broken line of the edge of the mold, the concave gap of the mold, and the raised edge of the mold. In this embodiment, multiple geometric feature areas of the mold can be obtained by processing a two-dimensional image of the mold. Specifically, when the two-dimensional image is a grayscale image or a black and white image, this embodiment can process the two-dimensional image through a deep image processing technology, and extract multiple areas with high discrimination in the two-dimensional image as multiple geometric feature areas of the mold. Among them, the discrimination can refer to: the degree of difference in grayscale levels between different objects or areas in the image. The greater the difference, the higher the discrimination; conversely, the smaller the difference, the lower the discrimination.

[0067] This embodiment uses an infrared camera to capture a two-dimensional image of the mold, and processes the two-dimensional image through deep image processing technology to obtain multiple geometric feature areas of the mold in the two-dimensional image. In addition, during the shooting process of the infrared camera, if the mold is embedded in a die-casting machine, the infrared camera can capture the mold and the die-casting machine at the same time, and then process the partial image containing the mold separately. This embodiment uses geometric feature areas for camera calibration. Due to the material and structure of the mold, the geometric features of the mold are not easy to change in a high-pressure die-casting environment, which reduces the impact of the high-pressure die-casting environment on the calibration of the infrared temperature camera and can improve the reliability of the camera calibration.

[0068] S12, iteratively optimizing the first camera parameters of the infrared camera by using the first algorithm, so that the second pixel coordinates of the same pixel point in the geometric feature area gradually approach the first pixel coordinates, and obtaining the second camera parameters after iterative optimization by the first algorithm, wherein the first pixel coordinates are: the two-dimensional pixel coordinates of the pixel point in the two-dimensional image, the second pixel coordinates are: the two-dimensional pixel coordinates obtained after mapping the three-dimensional world coordinates of the pixel point in the geometric feature area according to the first camera parameters, the first camera parameters are the assumed camera parameters of the infrared camera, and the first algorithm is an iterative optimization algorithm that does not depend on the initial value;

[0069] S13, continuing to iteratively optimize the second camera parameters by the second algorithm to obtain third camera parameters after iterative optimization by the second algorithm, where the second algorithm is an iterative optimization algorithm that depends on the initial value;

[0070] S14. Using the third camera parameters as camera parameters of the infrared camera.

[0071] Among them, the first algorithm is an algorithm that does not depend on the initial value. In this embodiment, the first algorithm is the CMA-ES (Covariance Matrix Adaptation Evolution Strategy) algorithm. The second algorithm is an algorithm that depends on the initial value. In this embodiment, the second algorithm is the Levenberg-Marquardt algorithm, which is a numerical optimization algorithm for nonlinear minimization. The first algorithm does not depend on the initial value, so the first camera parameter (the assumed camera parameter) can be iteratively optimized, and a good initial value of the camera parameter can be provided for the second algorithm that depends on the initial value, so that the second algorithm can be further iteratively optimized.

[0072] The first pixel coordinates are the two-dimensional pixel coordinates of the pixel point in the geometric feature area of ​​the two-dimensional image, and the coordinate origin of the pixel coordinate system in the two-dimensional image is located at the upper left corner of the two-dimensional image. The first pixel coordinates of the pixel point can be directly obtained when the deep image processing technology processes the two-dimensional image to divide the geometric feature area.

[0073] The second pixel coordinate is the two-dimensional pixel coordinate of the pixel point obtained after the three-dimensional world coordinate is mapped by the camera parameters. Among them, the three-dimensional world coordinate is a coordinate system used to determine the position of a point, a vector or an object in three-dimensional space (real world). The coordinate system can be defined by the user independently. The origin of the coordinates and three mutually perpendicular axes are defined: the X axis, the Y axis and the Z axis. The three-dimensional world coordinates of the mold are known quantities in this embodiment. In this embodiment, the three-dimensional world coordinates of the mold can be directly obtained through a variety of technologies, such as depth cameras, structured light and other technologies, or directly obtained from existing three-dimensional digital models of the mold.

[0074] Specifically, the mapping process of the three-dimensional world coordinates can be divided into two conversion parts: world coordinate system to camera coordinate system, camera coordinate system to pixel coordinate system. In this embodiment, the world coordinate system can be realized from the camera coordinate system through the external parameters in the camera parameters, and the camera coordinate system can be realized from the pixel coordinate system through the internal parameters of the camera parameters. Of course, in this embodiment, the above two conversion parts can also be refined into three conversion parts: world coordinate system to camera coordinate system, camera coordinate system to image coordinate system, and image coordinate system to pixel coordinate system. At this time, in this embodiment, the world coordinate system can be realized from the camera coordinate system through the external parameters in the camera parameters, and the image coordinate system can be realized from the pixel coordinate system through the focal length (internal parameters) and the pinhole camera model in the camera parameters. Among them, the pinhole camera model is a camera imaging geometric model constructed for the infrared camera in this embodiment. The geometric model is based on the principle of pinhole imaging and projects objects in the three-dimensional world onto a two-dimensional image plane.

[0075] Since the camera parameters of the infrared camera (the actual camera parameters of the infrared camera) are the unknown quantities required by this embodiment, this embodiment needs to first assume the camera parameters of the infrared camera as the initial camera parameters of the infrared camera, which are the first camera parameters in this embodiment. The first pixel coordinates are the two-dimensional pixel coordinates in the two-dimensional image taken by the infrared camera, which are obtained by mapping the camera parameters of the infrared camera to the three-dimensional world coordinates of the mold. For the same three-dimensional world coordinates of the mold, the second pixel coordinates are the camera parameters set in this embodiment, and the three-dimensional world coordinates of the mold are mapped to obtain. Therefore, the first pixel coordinates can be used as a reference standard for camera calibration in this embodiment. When the second pixel coordinates of the same pixel point gradually approach the first pixel coordinates, it can be considered that the camera parameters set in this embodiment are gradually approaching the camera parameters of the infrared camera. Then this embodiment can iteratively optimize the first camera parameters through the first pixel coordinates, the second pixel coordinates and the first algorithm to obtain the second camera parameters as the initial values ​​of the second algorithm. The specific process can be shown in steps one to three as follows:

[0076] Step 1: construct an objective function of the difference between the first pixel coordinate and the second pixel coordinate, where the design variables of the objective function are camera parameters;

[0077] Step 2: Input the objective function into the first algorithm for iterative optimization, and output the camera parameters used when the value of the objective function is minimized;

[0078] Step 3: The camera parameters used when the value of the objective function is minimized are used as the second camera parameters.

[0079] Among them, although the mapping process of three-dimensional world coordinates is divided into two conversion processes, the mapping relationship involved in the conversion process can be synthesized into a mapping function relationship , then for any set of camera parameters , each corresponding to a mapping function relationship , which can map 3D world coordinates to 2D pixel coordinates. Among them, a set of camera parameters The expression can be: , is the camera position, is the focal coordinate, This is just an example to illustrate a set of camera parameters. Representation form.

[0080] In this embodiment, in order to improve the accuracy of camera calibration, the first pixel coordinates and the second pixel coordinates of multiple pixel points in the geometric feature area can be collected. Then the representation form of the objective function can be as follows:

[0081]

[0082] in, Can represent pixels; It can represent the three-dimensional world coordinates of the pixel point. ; Can represent camera parameters The corresponding mapping function relationship is A second pixel coordinate that can represent a pixel point; It can represent the first pixel coordinate of the pixel point. .

[0083] The above objective function is input into the first algorithm for iterative optimization, and a set of camera parameters corresponding to the minimum value of the objective function is output, and used as the initial value of the second algorithm. The iterative optimization process of the first algorithm can be as follows:

[0084] Determine the current objective function; calculate the minimum value of the current objective function, and obtain the camera parameters corresponding to the minimum value of the current objective function; determine the relationship between the difference between the camera parameters corresponding to the minimum value of the current objective function and the current camera parameters and the first preset threshold;

[0085] If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is not less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is used as the current camera parameter, and the three-dimensional world coordinates of the pixel point in the geometric feature area are remapped by the current camera parameter to obtain the second pixel coordinate, and the step of determining the current objective function is returned to be executed;

[0086] If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is less than a first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is output.

[0087] Among them, each round of iteration of the first algorithm can obtain a current camera parameter, and the value of the objective function decreases with the increase of the number of iterations. Wheel camera parameters With Current camera parameters of the wheel When the difference between the two is less than the first preset threshold, the first algorithm is stopped and a set of camera parameters is output. (second camera parameter) is used as the input of the second algorithm. The first preset threshold is a very small value, and the iteration stop condition of the first algorithm can be expressed as: , is the first preset threshold.

[0088] After the second algorithm obtains the iterative result of the first algorithm, the iterative result of the first algorithm is used as the initial value for further iterative optimization to obtain the third camera parameter as the camera parameter of the infrared camera. The iterative optimization process of the second algorithm can be as follows:

[0089] Determine the second camera parameter as the current camera parameter; optimize the current camera parameter by a second algorithm to obtain the optimized camera parameter; determine the relationship between the difference between the current camera parameter and the optimized camera parameter and a second preset threshold;

[0090] If the difference between the current camera parameters and the optimized camera parameters is not less than the second preset threshold, the optimized camera parameters are used as the current camera parameters, and the process returns to the step of optimizing the current camera parameters to obtain the optimized camera parameters;

[0091] If the difference between the current camera parameters and the optimized camera parameters is less than the second preset threshold, the optimized camera parameters are output as the third camera parameters.

[0092] The iterative optimization process of the second algorithm is similar to that of the first algorithm. Optimized camera parameters for the wheel With Current camera parameters of the wheel When the difference between the two is less than the second preset threshold, the second algorithm is stopped and the first Wheel camera parameters , then the camera parameters (The third camera parameter) may be a set of camera parameters as camera parameters of an infrared camera. Wherein, the second preset threshold is a very small value, and the second preset threshold is less than the first preset threshold, and the iteration stop condition of the second algorithm may be expressed as: , is the second preset threshold.

[0093] The embodiment of the present application provides a camera calibration method in a high-pressure die-casting environment, which obtains the initial data for camera calibration through the first pixel coordinates of the two-dimensional image of the mold and the second pixel coordinates mapped to the three-dimensional world coordinates, and first processes the initial data through a first algorithm that does not rely on the initial value, and iteratively optimizes the camera parameters, so that in the high-pressure die-casting environment, the second algorithm that relies on the initial value can have a good initial value for iterative optimization, thereby improving the accuracy of camera calibration. Furthermore, the image pixel coordinates used in this method are the pixel coordinates of the geometric area of ​​the mold. Since the geometric features of the mold are not easily affected by the high-pressure die-casting environment, the geometric features of the mold are not easily changed, which further reduces the impact of the high-pressure die-casting environment on the infrared temperature camera calibration, and improves the accuracy of camera calibration. In addition, in the implementation process of this embodiment, all are autonomously operated calibrations, which reduces the impact of human factors on camera calibration, and also reduces the cost of use, providing a strong guarantee for the processing after high-pressure die-casting.

[0094] The above introduces a camera calibration method in a high-pressure die-casting environment provided by an embodiment of the present application. The following will introduce a device for executing the camera calibration method in a high-pressure die-casting environment.

[0095] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a camera calibration device in a high-pressure die-casting environment provided by an embodiment of the present application. Figure 2 As shown, the camera calibration device in the high pressure die casting environment may include:

[0096] An image acquisition unit 100 is used to acquire a two-dimensional image of a mold in a high-pressure die-casting environment through an infrared camera;

[0097] An image processing unit 110 is used to process the two-dimensional image to obtain multiple geometric feature areas of the mold;

[0098] The first optimization unit 120 is used to iteratively optimize the first camera parameters of the infrared camera through the first algorithm, so that the second pixel coordinates of the same pixel point in the geometric feature area gradually approach the first pixel coordinates, and obtain the second camera parameters after iterative optimization by the first algorithm, the first pixel coordinates are: the two-dimensional pixel coordinates of the pixel point in the two-dimensional image, the second pixel coordinates are: the two-dimensional pixel coordinates obtained after mapping the three-dimensional world coordinates of the pixel point in the geometric feature area according to the first camera parameters, the first camera parameters are assumed camera parameters of the infrared camera, and the first algorithm is an iterative optimization algorithm that does not depend on the initial value;

[0099] A second optimization unit 130 is used to continue iteratively optimizing the second camera parameters by using a second algorithm to obtain third camera parameters after iterative optimization by the second algorithm, where the second algorithm is an iterative optimization algorithm that depends on an initial value;

[0100] The parameter acquisition unit 140 is configured to use the third camera parameter as the camera parameter of the infrared camera.

[0101] In a possible implementation, the first optimization unit 120 may include:

[0102] A function construction subunit, used to construct an objective function of the difference between the first pixel coordinate and the second pixel coordinate, wherein the design variables of the objective function are camera parameters;

[0103] A function optimization subunit, used for inputting the objective function into the first algorithm for iterative optimization, and outputting the camera parameters used when the value of the objective function is minimized;

[0104] The result acquisition subunit is used to use the camera parameters used when the value of the objective function is minimized as the second camera parameters.

[0105] In a possible implementation, the above function optimization subunit may be specifically configured as follows:

[0106] Determine the current objective function;

[0107] Calculate the minimum value of the current objective function and obtain the camera parameters corresponding to the minimum value of the current objective function;

[0108] Determine the relationship between the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter and the first preset threshold;

[0109] If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is not less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is used as the current camera parameter, and the three-dimensional world coordinates of the pixel point in the geometric feature area are remapped by the current camera parameter to obtain the second pixel coordinate, and the step of determining the current objective function is returned to be executed;

[0110] If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is less than a first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is output.

[0111] In a possible implementation, the second optimization unit 130 may be specifically configured as follows:

[0112] Determine the second camera parameters as current camera parameters;

[0113] Optimize the current camera parameters by using the second algorithm to obtain the optimized camera parameters;

[0114] Determine the relationship between the difference between the current camera parameters and the optimized camera parameters and a second preset threshold;

[0115] If the difference between the current camera parameters and the optimized camera parameters is not less than the second preset threshold, the optimized camera parameters are used as the current camera parameters, and the process returns to the step of optimizing the current camera parameters to obtain the optimized camera parameters;

[0116] If the difference between the current camera parameters and the optimized camera parameters is less than the second preset threshold, the optimized camera parameters are output as the third camera parameters.

[0117] In a possible implementation, the two-dimensional image may be a grayscale image, and the image processing unit 110 may be specifically configured as follows:

[0118] The two-dimensional image is processed by deep image processing technology, and multiple areas with high distinction in the two-dimensional image are extracted as multiple geometric feature areas of the mold.

[0119] The present application also provides an electronic device in an embodiment. Figure 3 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0120] like Figure 3 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0121] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a memory card, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0122] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any camera calibration method in a high-pressure die-casting environment provided in the embodiment of the present application.

[0123] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any camera calibration method in a high-pressure die-casting environment provided in an embodiment of the present application.

[0124] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0125] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0126] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0127] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD), etc.).

[0128] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0129] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A camera calibration method in a high pressure die casting environment, characterized in that: The camera calibration method in the high pressure die casting environment comprises: Acquire a 2D image of the mold in a high-pressure die casting environment using an infrared camera; Processing the two-dimensional image to obtain a plurality of geometric feature areas of the mold; Iteratively optimizing the first camera parameters of the infrared camera by using a first algorithm so that the second pixel coordinates of the same pixel point in the geometric feature area gradually approach the first pixel coordinates, and obtaining the second camera parameters after iterative optimization by the first algorithm, wherein the first pixel coordinates are: the two-dimensional pixel coordinates of the pixel point in the two-dimensional image, and the second pixel coordinates are: the two-dimensional pixel coordinates obtained after mapping the three-dimensional world coordinates of the pixel point in the geometric feature area according to the first camera parameters, the first camera parameters are the assumed camera parameters of the infrared camera, and the first algorithm is an iterative optimization algorithm that does not depend on the initial value; Continuing to iteratively optimize the second camera parameters by a second algorithm to obtain third camera parameters after iterative optimization by the second algorithm, wherein the second algorithm is an iterative optimization algorithm that depends on an initial value; The third camera parameter is used as the camera parameter of the infrared camera.

2. The camera calibration method in a high pressure die casting environment according to claim 1, characterized in that: Iteratively optimizing the first camera parameters of the infrared camera by using a first algorithm to obtain second camera parameters iteratively optimized by the first algorithm includes: Constructing an objective function of the difference between the first pixel coordinate and the second pixel coordinate, wherein the design variables of the objective function are camera parameters; Inputting the objective function into the first algorithm for iterative optimization, and outputting the camera parameters used when the value of the objective function is minimized; The camera parameters used when the value of the objective function is minimized are used as the second camera parameters.

3. The camera calibration method in a high pressure die casting environment according to claim 2, characterized in that: The step of inputting the objective function into the first algorithm for iterative optimization and outputting the camera parameters used when the value of the objective function is minimized includes: Determine the current objective function; Calculating the minimum value of the current objective function and obtaining the camera parameters corresponding to the minimum value of the current objective function; Determine the relationship between the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter and a first preset threshold; If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is not less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is used as the current camera parameter, and the three-dimensional world coordinates of the pixel points in the geometric feature area are remapped by the current camera parameter to obtain the second pixel coordinates, and the step of determining the current objective function is returned to be executed; If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is output.

4. The camera calibration method in a high pressure die casting environment according to claim 1, characterized in that: The step of continuing to iteratively optimize the second camera parameters by using the second algorithm to obtain third camera parameters iteratively optimized by the second algorithm includes: Determine the second camera parameter as the current camera parameter; Optimizing the current camera parameters by using a second algorithm to obtain optimized camera parameters; Determine the relationship between the difference between the current camera parameter and the optimized camera parameter and a second preset threshold; If the difference between the current camera parameters and the optimized camera parameters is not less than a second preset threshold, the optimized camera parameters are used as the current camera parameters, and the step of optimizing the current camera parameters to obtain the optimized camera parameters is returned; If the difference between the current camera parameter and the optimized camera parameter is less than a second preset threshold, the optimized camera parameter is output as the third camera parameter.

5. The camera calibration method in a high pressure die casting environment according to claim 1, characterized in that: The two-dimensional image is a grayscale image, and the processing of the two-dimensional image to obtain a plurality of geometric feature areas of the mold includes: The two-dimensional image is processed by using a deep image processing technology, and a plurality of regions with high distinguishability in the two-dimensional image are extracted as a plurality of geometric feature regions of the mold.

6. A camera calibration device in a high-pressure die-casting environment, characterized in that: The camera calibration device in the high-pressure die-casting environment comprises: An image acquisition unit, used for acquiring a two-dimensional image of the mold in a high-pressure die-casting environment through an infrared camera; An image processing unit, used for processing the two-dimensional image to obtain a plurality of geometric feature areas of the mold; A first optimization unit is used to iteratively optimize the first camera parameters of the infrared camera through a first algorithm, so that the second pixel coordinates of the same pixel point in the geometric feature area gradually approach the first pixel coordinates, and obtain the second camera parameters after iterative optimization by the first algorithm, wherein the first pixel coordinates are: the two-dimensional pixel coordinates of the pixel point in the two-dimensional image, the second pixel coordinates are: the two-dimensional pixel coordinates obtained after mapping the three-dimensional world coordinates of the pixel point in the geometric feature area according to the first camera parameters, the first camera parameters are the assumed camera parameters of the infrared camera, and the first algorithm is an iterative optimization algorithm that does not depend on the initial value; A second optimization unit, configured to continue to iteratively optimize the second camera parameters by using a second algorithm to obtain third camera parameters after iterative optimization by the second algorithm, wherein the second algorithm is an iterative optimization algorithm that depends on an initial value; A parameter acquisition unit is used to use the third camera parameter as the camera parameter of the infrared camera.

7. The camera calibration device in a high pressure die casting environment according to claim 6, characterized in that: The first optimization unit comprises: a function construction subunit, configured to construct an objective function of the difference between the first pixel coordinate and the second pixel coordinate, wherein the design variables of the objective function are camera parameters; A function optimization subunit, configured to input the objective function into the first algorithm for iterative optimization, and output the camera parameters used when the value of the objective function is minimized; The result acquisition subunit is used to use the camera parameters used when the value of the objective function is minimized as the second camera parameters.

8. The camera calibration method in a high pressure die casting environment according to claim 7, characterized in that: The function optimization subunit is specifically configured as follows: Determine the current objective function; Calculating the minimum value of the current objective function and obtaining the camera parameters corresponding to the minimum value of the current objective function; Determine the relationship between the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter and a first preset threshold; If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is not less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is used as the current camera parameter, and the three-dimensional world coordinates of the pixel points in the geometric feature area are remapped by the current camera parameter to obtain the second pixel coordinates, and the step of determining the current objective function is returned to be executed; If the difference between the camera parameter corresponding to the minimum value of the current objective function and the current camera parameter is less than the first preset threshold, the camera parameter corresponding to the minimum value of the current objective function is output.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the camera calibration method in a high-pressure die-casting environment as described in any one of claims 1 to 5.

10. A computer program product, characterized in that It comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the camera calibration method in a high-pressure die-casting environment as described in any one of claims 1 to 5.

Images