Calibration method of linear array camera, electronic equipment and storage medium

Abstract

The invention provides a calibration method of a linear array camera, electronic equipment and a storage medium. The method comprises the following steps: establishing a time sequence image tensor space, wherein the time sequence image tensor space is a three-dimensional data space formed by continuously acquiring images of a plane target in a scanning time sequence of acquiring the images of the plane target by a same-lens area-array camera and a line-array camera; extracting a two-dimensional data subspace formed by middle rows of each image from the three-dimensional data of the time sequence image tensor space to obtain a time sequence scanning image space; according to a perspective projection relation between the space coordinate system and an area array image coordinate system of the same-lens area array camera, establishing a mixed relation equation between the space coordinate system and a scanning image coordinate system in a time sequence scanning image space; and based on the coordinates of the target point of the plane target at the initial moment in the space coordinate system and the coordinates of the target point in the scanning image coordinate system, solving the target function so as to determine a parameter transformation matrix of the linear array camera. According to the scheme, the complete imaging parameters of the line-scan digital camera can be quantitatively calibrated.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and more particularly to a calibration method for a line array camera, an electronic device, and a storage medium. Background Art

[0002] The calibration of camera parameters is a very crucial step, and the accuracy of its calibration results and the stability of the calibration algorithm directly affect the accuracy of the final results of the relevant system. Therefore, camera calibration is one of the most basic and important technologies in photogrammetry and computer vision technologies. A line array camera has the characteristics of a wide field of view, a high sampling frequency, and a high resolution. It has unparalleled advantages in processing one-dimensional image signals and is widely used in high-precision measurements. When using a line array camera for tasks such as visual measurement, it is first necessary to calibrate the internal and external parameters of the line array camera.

[0003] Similarly, the calibration of a area array camera generally calibrates the internal and external parameters of the camera based on a planar target with calibration points (generally a checkerboard or a dot array) and the target image captured by the camera. A linear equation system for solving the parameter transformation matrix is generally used to obtain the internal and external parameters of the camera. Due to the difference in the imaging principles between the line array camera and the area array camera, it is not suitable to use the same method to calibrate the parameter transformation matrix and the internal and external parameters of the line array camera.

[0004] In the related art, the line array camera often qualitatively calibrates the roll angle of the camera by adjusting the angle of the line array camera so that the scanning line coincides with the target, but cannot quantitatively calibrate the complete imaging parameters.

[0005] In view of this, the present invention is specifically proposed. Summary of the Invention

[0006] In view of the above problems, the present invention is proposed. According to one aspect of the present invention, there is provided a calibration method for a line array camera, which calibrates the line array camera using a planar target; the method includes: Establish a temporal image tensor space, which is a three-dimensional data space formed by continuously capturing images of the planar target by a same-lens area array camera within the scanning time sequence of the line array camera capturing images of the planar target; wherein, the same-lens area array camera is a hypothetical area array camera with the same lens as the line array camera, and it is assumed that it uses an area array sensor with the same pixel size and array width as the sensor of the line array camera, and the middle row of its sensor array is equivalent to the sensor array of the line array camera; the scanning time sequence includes a time series composed of the scanning times of each row when the line array camera captures images of the planar target; Extract a two-dimensional data subspace composed of the middle rows of each image from the three-dimensional data of the temporal image tensor space, that is, a temporal scanning image space; According to the space coordinate systemx - y The perspective projection relationship between the planar image coordinate system of the same-shot area array camera u - v to establish the spatial coordinate system within the time-sequential scanned image space x - y and the scanned image coordinate system u - t a hybrid relationship equation; Based on the coordinates of the target points of the planar target at the initial moment in the spatial coordinate system and the coordinates of the target points of the planar target in the scanned image coordinate system, solve the objective function to determine the parameter transformation matrix of the line array camera, where the objective function is: ; wherein X is m the x coordinates x 0 ,x 1 , ~ x m-1 of the Y is m the y coordinates y 0 ,y 1 , ~ y m-1 of the U is m the image u coordinates u 0 ,u 1 , ~ u m-1 of the T is m the image t coordinates t 0 ,t 1 , ~ t m-1 ; where , are the predicted image coordinate values obtained by calculating the coordinate transformation formula of each target point based on the hybrid relationship equation.

[0007] Exemplarily, the parameter transformation matrix is: ; The hybrid relationship equation is: ; wherein s represents the scale factor; represents the moving speed of the planar target; The coordinate transformation formula is: .

[0008] Exemplarily, the method further includes: According to the parameter values in the parameter transformation matrix, the scanning line of the linear array camera and the spatial coordinate system are calculated. x The angle between the axes.

[0009] Exemplarily, the calculation of the scan line of the line array camera and the spatial coordinate system according to the parameter value in the parameter transformation matrix is as follows: x The angle between the axes, including determining the angle by the following formula: ; in, is the angle.

[0010] For example, in the spatial coordinate system x - y The image coordinate system of the same-lens area array camera u - v The perspective projection relationship between them is used to establish the spatial coordinate system in the time-series scanning image space. x - y Scanned image coordinate system u - t Before the mixed relationship equation between the two, the method also includes: Establish the area array image coordinate system u - v With the space coordinate system x - y The perspective projection relationship between the two, wherein the spatial coordinate system is based on the plane where the planar target is located. x - y flat.

[0011] Exemplarily, the parameter transformation matrix is: ; The parameters in the parameter transformation matrix satisfy the nonlinear constraint condition: ; And the boundary conditions: h 11 ≠0.

[0012] Exemplarily, solving the objective function includes: The objective function is transformed into a convex optimization problem under the following nonlinear constraints: ; Solve the convex optimization problem by using the Lagrange multiplier method or a numerical optimization algorithm.

[0013] Exemplarily, before solving the objective function, the method further includes: Obtain the coordinates of the target points of the planar target at the initial moment in the space coordinate system; Obtain the coordinates of the target points of the planar target in the scanning coordinate system, where the image of the planar target is acquired by the line array camera, and the planar target moves uniformly relative to the line array camera along y a direction.

[0014] According to another aspect of the present invention, there is provided an electronic device, including a processor and a memory, where a computer program is stored in the memory, and the processor is configured to execute the computer program to implement the method as described above.

[0015] According to still another aspect of the present invention, there is provided a computer-readable storage medium storing a computer program / instructions, and when the computer program / instructions are executed by a processor, the method as described above is implemented.

[0016] In the above technical solution, by establishing a subset relationship between the area array and the line array imaging spaces, and defining the temporal image tensor space during the scanning process, a perspective + temporal imaging model of the line array camera image (represented by a hybrid relationship equation in this article) is established. Thus, the problem that the line array camera image does not fully conform to perspective projection and the planar target calibration method based on perspective projection cannot be applied can be solved. In short, this solution can quantitatively calibrate the complete imaging parameters of the line array camera by using the planar target, thereby helping to improve the measurement accuracy of information such as the size, position, and moving speed of objects in the images acquired by the line array camera.

[0017] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically exemplified below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present invention in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present invention will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1Schematic flowchart showing a calibration method for a line array camera according to an embodiment of the present invention; Figure 2 Schematic diagram showing a space coordinate system according to an embodiment of the present invention; Figure 3 Schematic diagram showing a planar array image coordinate system according to an embodiment of the present invention; Figure 4 Schematic diagram showing a temporal image tensor space according to an embodiment of the present invention; Figure 5 Schematic block diagram showing an electronic device according to an embodiment of the present invention. Detailed implementation manners

[0020] In order to make the objectives, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments of the present invention. It should be understood that the present invention is not limited by the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

[0021] Line array cameras are often used to scan and image moving objects along the velocity direction. A strip-shaped target is usually adopted, and the roll angle of the camera is qualitatively calibrated by adjusting the angle of the line array camera so that the scanning line coincides with the target, but the complete imaging parameters cannot be quantitatively calibrated. Planar array cameras generally use planar targets to calibrate the complete imaging parameters. However, since the images of line array cameras do not fully conform to perspective projection, the calibration method based on planar target of perspective projection cannot be directly applied. In view of this, the present invention provides a calibration method, an electronic device and a storage medium for a line array camera. This method can quantitatively calibrate the complete imaging parameters of the line array camera by using a planar target, thereby helping to improve the measurement accuracy of information such as the size, position, and moving speed of objects in the images collected by the line array camera.

[0022] It can be understood that the "incomplete" herein is relative to the perspective projection of the planar array camera. The images captured by the planar array camera conform to the perspective projection relationship both in the image row direction and the image column direction, which is a "complete" perspective imaging. While the line array camera only conforms to the perspective projection relationship in the scanning row direction, so it is "incomplete" perspective imaging. The inventor found through research that it also conforms to the temporal relationship in the scanning speed direction. Therefore, the present invention is proposed based on the discovered hybrid imaging relationship of perspective + time sequence. The specific content of this solution will be described below.

[0023] According to one aspect of an embodiment of the present invention, a calibration method for a line array camera is provided. This method uses a planar target to calibrate the line array camera to obtain the complete imaging parameters of the line array camera. In this article, the complete imaging parameters are represented by the parameter transformation matrix of the line array camera.

[0024] It can be understood that a planar target is a two-dimensional pattern used in fields such as camera calibration, machine vision, and photogrammetry. It usually contains a series of marked points (or called feature points) with known sizes and arrangements. These marked points can be round dots, corner points, or other shapes, which are not limited in this article.

[0025] Figure 1 The schematic flowchart showing the calibration method of the line array camera according to an embodiment of the present invention is as follows. As Figure 1 shown, the method may include the following steps S110, step S120, step S130, and step S140.

[0026] In step S110, a temporal image tensor space is established. The temporal image tensor space is a three-dimensional data space formed by a same-lens area array camera continuously acquiring images of the planar target within the scanning time sequence of the line array camera acquiring images of the planar target. Among them, the same-lens area array camera is an assumed area array camera with the same lens as the line array camera, and it is assumed that it uses a area array sensor with the same pixel size and array width as the sensor of the line array camera, and the middle row of its sensor array is equivalent to the sensor array of the line array camera; the scanning time sequence includes the time series composed of the scanning times of each row when the line array camera acquires images of the planar target.

[0027] It can be understood that the line array camera can move relative to the target to be imaged, and continuous scanning imaging is performed using a strip-shaped target (i.e., the scanning line of the line array camera) during this movement. During the scanning process, it can be scanned in a way that the position of the line array camera is fixed and the target to be imaged is translated relative to the line array camera, or in a way that the position of the target to be imaged is fixed and the line array camera is translated relative to the target to be imaged. In this article, it is described on the premise that the position of the line array camera is fixed and the target to be imaged (i.e., the planar target) is translated. It can be understood that the situation where the position of the line array camera is fixed and the target to be imaged is translated relative to the line array camera is similar to the process of calibrating the line array camera in the above situation and will not be elaborated.

[0028] During the process of the planar target moving relative to the line array camera, it passes through the scanning line of the line array camera within a certain period of time. The time sequence of this period can be expressed as t =0, 1 ~ n -1. That is, the scanning time of the first row of the line array camera image is taken as the initial time t =0, and the scanning times of the subsequent rows are successively t=1, 2 ~ n -1。

[0029] In the solution of this example, when establishing the temporal image tensor space, it is assumed that a planar array camera with the same lens and sensor array plane width continuously captures images within the time sequence t =0, 1 ~ n -1, and these images are stacked along the third dimension t direction to obtain the temporal image tensor space abcd ( t ). Since this space is constructed by simulating the images continuously captured by the planar array camera, the coordinates within this space fully conform to perspective projection.

[0030] In the solution of this example, the spatial coordinate system can be constructed based on the plane where the planar target is located. Figure 2 Shows a schematic diagram of the spatial coordinate system according to an embodiment of the present invention. Figure 3 Shows a schematic diagram of the planar array image coordinate system according to an embodiment of the present invention. As Figure 2 shown, in this embodiment, the plane where the planar target is located is used as the x - y plane, and the translation speed direction of the planar target is used as the y direction to establish the spatial coordinate system. The spatial coordinate system x -y is the physical coordinate system of the planar target and the scanned plane where it is located, where the planar target moves uniformly relative to the line array camera along the y direction. The planar array image coordinate system u - v is the pixel coordinate system of the image captured by the same-lens planar array camera.

[0031] In Figure 2 , ABCD can be regarded as the corresponding imaging range of the same-lens planar array camera on the x - y plane. On this basis, as Figure 2-3 shown, the planar array image coordinate system of the same-lens planar array camera can be defined as u - v (the origin of coordinates is the leftmost pixel in the middle row); based on the above coordinate system, the sensor array range of the line array camera ef in the planar array image coordinate system u - v is v =0, and the corresponding imaging range on the spatial coordinate system x - y is EF .

[0032] As described above, the temporal image tensor space is a space formed by continuously acquiring images of a planar target by a same-lens area camera within the scanning time sequence of the images of the planar target acquired by a line camera on the acquisition plane of the line camera. Figure 4 FIG. shows a schematic diagram of the temporal image tensor space according to an embodiment of the present invention. As Figure 4 shown, the images respectively acquired by the same-lens area camera at t = 0, 1 to n -1 are stacked along the time dimension, and thus the temporal image tensor space is obtained.

[0033] In step S120, a two-dimensional data subspace formed by extracting the middle rows of each image from the three-dimensional data of the temporal image tensor space is obtained to obtain the temporal scanning image space.

[0034] In the solution of this example, in the area image coordinate system, v the axis direction is the velocity direction of the planar target. As described above, the middle row of the sensor array surface of the same-lens area camera is equivalent to the sensor array surface of the line camera. Taking Figure 4 as an example for illustration, in Figure 4 , a subspace with the coordinate v being a constant value v = 0 ef ( t ) can be extracted, and thus the temporal scanning image space is obtained. This temporal scanning imaging space can represent the scanning imaging process of the planar target passing through the scanning line t = 0 to n . The coordinate system of this temporal scanning imaging space is the scanning image coordinate system. This scanning image coordinate system is the pixel coordinate system of the scanning image of the line camera. EF

[0035] In step S130, according to the perspective projection relationship between the space coordinate system x - y and the area image coordinate system u - v of the same-lens area camera, a mixed relationship equation between the space coordinate system x - y and the scanning image coordinate system u - t is established within the temporal scanning image space.

[0036] Since the area image coordinate system is defined based on the imaging range of the same-lens area camera, the area image coordinate system and the space coordinate system must conform to the perspective projection relationship. In the solution of this example, the temporal scanning image space ef ( t ) is the temporal image tensor space abcd ( t ​The two-dimensional slice of u and t retains two-dimensional variables, namely ef ( t ) where there is a scanning image coordinate system u - t . The images in the temporal image tensor space are all acquired by the area array camera of the same lens. Therefore, the perspective projection relationship must be satisfied in this space, and then the temporal scanning image space also satisfies the perspective projection relationship. On this basis, substituting the speed value corresponding to the temporal scanning image space and the moving speed of the planar target into the perspective projection relationship between the area array image coordinate system and the space coordinate system, the space coordinate system x - y and the scanning image coordinate system u - t can obtain the mixed relationship equation.

[0037] In step S140, based on the coordinates of the target points of the planar target at the initial moment in the space coordinate system and the coordinates of the target points of the planar target in the scanning image coordinate system, solve the objective function to determine the parameter transformation matrix of the linear array camera, where the objective function is: ; where X is the m coordinates of the x target points x 0 ,x 1 , ~ x m-1 , Y is the m coordinates of the y target points y 0 ,y 1 , ~ y m-1 , U is the m image u coordinates of the u 0 ,u 1 , ~ u m-1 , T is the m image t coordinates of the t 0 ,t 1 , ~ t m-1 ; where and are the predicted image coordinate values obtained by calculating the coordinate transformation formula of each target point based on the mixed relationship equation;.

[0038] In the solution of this example, a parameter transformation matrix of the line array camera is determined by using a hypothetical area array camera. The objective function aims to obtain a parameter transformation matrix that can minimize the difference between the predicted value and the actual value of the image coordinates. By solving using the above objective function, the difference between the predicted value and the true value can be minimized, thereby improving the result accuracy.

[0039] The above technical solution establishes a subset relationship between the area array and the line array imaging spaces, and at the same time defines the temporal image tensor space during the scanning process, thereby establishing a perspective + temporal imaging model for the line array camera images (represented by a mixed relationship equation in this article). Thus, the problem that the line array camera images do not fully conform to perspective projection and the planar target calibration method based on perspective projection cannot be applied can be solved. In short, this solution can quantitatively calibrate the complete imaging parameters of the line array camera using a planar target, which helps to improve the measurement accuracy of information such as the size, position, and moving speed of objects in the images collected by the line array camera.

[0040] Exemplarily, the method further includes: calculating the angle between the scanning line of the line array camera and the x axis of the space coordinate system according to the parameter values in the parameter transformation matrix. The parameters of the parameter transformation matrix provide the mapping relationship between the scanning image coordinate system of the line array camera and the space coordinate system. The line array camera captures images by scanning line by line, and the parameter transformation matrix is used in computer vision to describe the conversion relationship between the image and the world coordinate system, and is an important intermediate parameter in camera calibration. When it comes to calibrating the line array camera, the main task is to solve the angle between the scanning lines of the line array camera and the coordinate axes of the space coordinate system. Since there is a trigonometric function relationship between the transformation matrix as an intermediate parameter and the angle of the scanning line, the angle between the scanning line of the line array camera and the x axis of the space coordinate system can be accurately determined by solving the parameter transformation matrix, that is, the calibration work of the line array camera is completed.

[0041] Exemplarily, the parameter transformation matrix is: ; The mixed relationship equation is: ; Wherein, s represents the scale factor; represents the moving speed of the planar target; Based on the mixed relationship equation, decomposition and derivation are carried out: ;

[0042] The coordinate transformation formula required for calculating the objective function can be obtained: 。

[0043] In this text, the x coordinates, y coordinates, u coordinates, and t coordinates all conform to the above coordinate transformation formula, and this fraction can be applied to substitute into the objective function to calculate one by one m the predicted values of the scanned image coordinates of the 、 。

[0044] In the solution of this example, the parameter transformation matrix is represented by a homography matrix. This method is a common way to describe the perspective projection transformation relationship within the target plane and will not be elaborated.

[0045] In some embodiments, before establishing the hybrid relationship equation between the space coordinate system x - y and the area array image coordinate system of the same-lens area array camera u - v in the time-sequential scanned image space, the method further includes: establishing the perspective projection relationship between the area array image coordinate system x - y and the space coordinate system u - t wherein the space coordinate system takes the plane where the planar target is located as the u - v plane. The establishment of this perspective projection relationship can provide an accurate basis for determining the hybrid relationship equation in the subsequent steps. x - y plane. The establishment of this perspective projection relationship can provide an accurate basis for determining the hybrid relationship equation in the subsequent steps. x - y plane. The establishment of this perspective projection relationship can provide an accurate basis for determining the hybrid relationship equation in the subsequent steps.

[0046] In some embodiments, the perspective projection relationship between the image coordinate system and the space coordinate system can be expressed as: 。

[0047] Exemplarily, according to the parameter values in the parameter transformation matrix, calculate the angle between the scan line of the line array camera and the x axis of the space coordinate system, including determining the angle through the following formula: ; wherein, is the angle.

[0048] In this example, the inventor adopted the following derivation process during the research: The internal parameter matrix of the same-lens area array camera: ; External parameter matrix of the linear array camera: ; Where: They are respectively the right-handed rotation matrices of the camera rotating around the Zc axis of the local coordinate system in roll and yaw, is the offset matrix of the camera coordinate origin relative to the world coordinate origin:

[0049]

[0050] ; , normalized to obtain the parameter transformation matrix , and the matrix where each item is: h 11 = g 11 / g 34 , h 12 = g 12 / g 34 , h 13 = g 14 / g 34 , h 21 = g 21 / g 34 , h 22 = g 22 / g 34 , h 23 = g 24 / g 34 , h 31 = g 31 / g 34 , h 32 = g 32 / g 34 , substitutef, u 0 、 、 Obtained: h 11 = -( f ∙ cos + u 0 ∙ sin ) ∙ cos / ( c x ∙ sin ∙ cos + c y ∙ sin ∙ sin + c z ∙ cos ); h 12 = -( f ∙ cos + u 0 ∙ sin ) ∙ sin / ( c x ∙ sin ∙ cos + c y ∙ sin ∙ sin + c z ∙ cos ); h 21 = f ∙ sin / ( c x ∙ sin ∙ cos + c y ∙ sin ∙ sin + c z ∙ cos ); h 22 = - f ∙ cos / ( c x ∙ sin ∙ cos + c y ∙ sin ∙ sin + c z ∙ cos ); h 31 = -sin ∙ cos / ( c x ∙ sin ∙ cos + c y ∙ sin ∙ sin + c z ∙ cos ); h 32 = -sin ∙ sin / ( c x ∙ sin ∙ cos + c y ∙ sin ∙ sin + c z ∙ cos ); Derivation Conclusion 1: h 12 / h 11 = tan ; - h 21 / h 22 = tan ; h 32 / h 31 = tan ; Derivation Conclusion 2: h 11 ∙ h 21 + h 12 ∙ h 22 = 0; h31 ∙ h 21 + h 32 ∙ h 22 = 0; h 31 ∙ h 12 - h 32 ∙ h 11 = 0。

[0051] According to the above derivation conclusion one, it can be seen that there is a functional relationship between the included angle x between the scanning line of the line array camera and the axis of the space coordinate system and the parameter values in the parameter transformation matrix. In this embodiment, h 12 and h 11 are used to determine this included angle. In other implementation solutions not disclosed in this article, h 21 and h 22 , or h 31 and h 32 can also be used to determine this included angle, which will not be elaborated.

[0052] The formula of the above solution can accurately determine the included angle x between the scanning line of the line array camera and the axis of the space coordinate system, thereby providing an accurate basis for the precise measurement and motion analysis of the target to be imaged by the line array camera.

[0053] In some embodiments, according to the above derivation conclusion two, the parameter transformation matrix in the mixed relationship equation: ; satisfies the non - linear constraint condition: ; According to the above derivation conclusion one, the first term of the parameter transformation matrix satisfies the boundary condition: h 11 ≠0.

[0054] By setting non - linear constraint conditions and boundary conditions for the parameters in the parameter transformation matrix in the above solution, the amount of calculation can be reduced, the solution process can be simplified, and the uniqueness of the solution can be ensured, improving the result accuracy.

[0055] Exemplarily, solving the objective function includes: transforming the objective function into a convex optimization problem under the following non-linear constraint conditions to solve the objective function: ; Solving the above convex optimization problem using the Lagrange multiplier method or a numerical optimization algorithm.

[0056] Optionally, solving the convex optimization problem using the Lagrange multiplier method includes solving through the following formula: 3; where, , , are the Lagrange coefficients of the constraint conditions h 1 、h 2 、h 3 respectively.

[0057] Optionally, when solving the convex optimization problem using a numerical optimization algorithm, the numerical optimization algorithm includes but is not limited to SLSQP (Sequential Least Squares Quadratic Programming), COBYLA (Constrained Optimization BY Linear Approximations) algorithm, Trust-Region Constrained method, etc. In some embodiments, a global optimization strategy can also be called on the basis of the numerical optimization algorithm to solve the above convex optimization problem, including but not limited to the Basin Hopping method, Dual Annealing method, etc.

[0058] The above solution can ensure that the obtained solution is the global optimal solution by transforming the solution of the objective function into a convex optimization problem, improving the accuracy and calculation efficiency of the determined parameter transformation matrix.

[0059] Exemplarily, before solving the objective function, the method further includes: obtaining the coordinates of the target points of the planar target at the initial moment in the space coordinate system; obtaining the coordinates of the target points of the planar target in the scanning image coordinate system, where the image of the planar target is collected by a line array camera, and the planar target moves uniformly along the y direction relative to the line array camera.

[0060] In the solution of this example, after establishing the space coordinate system, the coordinates of each target point on the planar target at the initial moment t =0 can be directly determined according to the preset size, shape, and initial position of the planar target as ( x 0 ,y 0), ( x 1 ,y1) to ( x m-1 ,y m-1 ). The coordinates of the target points in the scanning image coordinate system ( u 0 ,t 0), ([[]]END]] u 1 ,t 1) to ( u m-1 ,t m-1 ) are the pixel coordinate values that can be obtained from the scanned image. The method for obtaining the pixel coordinate values adopts the specific implementation manner of extracting the feature point coordinates that can be understood by those skilled in the art, and will not be elaborated herein.

[0061] By pre-determining the coordinates of the target points in the scanning image coordinate system and the space coordinate system, the above solution can provide a relatively accurate basis for determining the parameter transformation matrix in the subsequent steps.

[0062] According to another aspect of the embodiments of the present invention, an electronic device is also provided. Figure 5 The schematic block diagram showing an electronic device according to an embodiment of the present invention. As Figure 5 shown, the electronic device 500 includes: a processor 510 and a memory 520. A computer program is stored in the memory 520, and the processor 510 is configured to execute the computer program to implement the above method.

[0063] According to still another aspect of the embodiments of the present invention, a computer-readable storage medium is also provided. A computer program / instructions is stored in the storage medium, and when the computer program / instructions is executed by a processor, the above method is implemented. The storage medium may include, for example, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0064] Those of ordinary skill in the art can easily understand the implementation structure, working principle, and beneficial effects of the electronic device and the computer-readable storage medium by reading the above method. For the sake of brevity, it will not be elaborated herein.

[0065] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Those of ordinary skill in the art can make various changes and modifications therein without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as claimed in the appended claims.

[0066] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0067] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0068] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and technologies are not shown in detail so as not to obscure the understanding of this specification.

[0069] Similarly, it should be understood that, in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be construed as reflecting the intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected by the corresponding claims, the inventive point lies in that the corresponding technical problem can be solved by features fewer than all the features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, where each claim itself serves as a separate embodiment of the present invention.

[0070] Those skilled in the art can understand that, except for features being mutually exclusive, any combination can be adopted for all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0071] In addition, those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments is meant to be within the scope of the present invention and forms different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0072] Each component embodiment of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some of the modules in the electronic device according to the embodiments of the present invention. The present invention can also be implemented as a device program (such as a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0073] It should be noted that the above embodiments illustrate rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices can be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names.

[0074] As described above, this is only the specific implementation manner of the present invention or the description of the specific implementation manner. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of them should be covered by the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A calibration method for a linear array camera, characterized in that Calibrate the linear array camera using a planar target; the method includes: Establish a temporal image tensor space, which is a three-dimensional data space formed by a same-lens area array camera continuously collecting images of the planar target within the scanning time sequence of the linear array camera collecting images of the planar target; wherein, the same-lens area array camera is a hypothetical area array camera with the same lens as that of the linear array camera, and it is assumed that it uses an area array sensor with the same pixel size and array width as the sensor of the linear array camera, and the middle row of its sensor array is equivalent to the sensor array of the linear array camera; the scanning time sequence includes the time series formed by the scanning moments of each row when the linear array camera collects images of the planar target. Extract a two-dimensional data subspace composed of the middle rows of each image from the three-dimensional data of the temporal image tensor space to obtain a temporal scanning image space. According to the spatial coordinate system x - y and the perspective projection relationship with the area image coordinate system of the same-shot area array camera u - v Establish the spatial coordinate system within the temporal scanning image space x - y and the hybrid relationship equation with the scanning image coordinate system u - t ; Based on the coordinates of the target points of the planar target at the initial moment in the space coordinate system and the coordinates of the target points of the planar target in the scanning image coordinate system, solve the objective function to determine the parameter transformation matrix of the linear array camera, where the objective function is: ； Among them, X is the known m coordinates of x a target point x 0 ,x 1 , to x m-1 , Y is the known m coordinates of y a target point y 0 ,y 1 , to y m-1 , U is m the image u coordinates of u a target point ,u 0 , to u m-1 , T is m the image t coordinates of t a target point ,t 0 , to t m-1 ; , are the predicted image coordinates obtained by calculating the coordinate transformation formula of each target point based on the mixed relationship equation.

2. The method according to claim 1, characterized in that, The parameter transformation matrix is: ； The hybrid relationship equation is: ； Among them, s represents a scale factor; represents the moving speed of the planar target; The coordinate transformation formula is: 。 3. The method according to claim 2, wherein The method further includes: According to the parameter values in the parameter transformation matrix, calculate the angle between the scanning line of the linear array camera and the x axis of the space coordinate system.

4. The method according to claim 3, characterized in that Calculating the angle between the scanning line of the linear array camera and the x axis of the space coordinate system according to the parameter values in the parameter transformation matrix, including determining the angle through the following formula: ； Wherein, is the included angle.

5. The method according to claim 1, characterized in that In the space coordinate system according to x - y and the planar image coordinate system of the same lens planar array camera u - v Before establishing the perspective projection relationship, and establishing the space coordinate system x - y and the scanning image coordinate system u - t in the time-sequential scanning image space, the method further includes: Establish the planar array image coordinate system u - v and the perspective projection relationship with the spatial coordinate system x - y wherein the spatial coordinate system takes the plane where the planar target is located as x - y plane.

6. The method according to any one of claims 1-4, characterized in that, The parameter transformation matrix is: ； Wherein, the parameters in the parameter transformation matrix satisfy the non-linear constraint conditions: ； and boundary conditions: h 11 ≠ 0.

7. The method according to any one of claims 1-4, characterized in that, The solving of the objective function includes: Transform the objective function into a convex optimization problem under the following non-linear constraint conditions: ； Use the Lagrange multiplier method or a numerical optimization algorithm to solve the convex optimization problem.

8. The method according to any one of claims 1-4, characterized in that, Before solving the objective function, the method further includes: Obtain the coordinates of the target points of the planar target at the initial moment in the space coordinate system. Obtain the coordinates of the target points of the planar target in the coordinate system of the scanned image, where the image of the planar target is acquired by the line array camera, and the planar target moves uniformly along the y direction.

9. An electronic device, characterized in that, It includes a processor and a memory, and a computer program is stored in the memory, and the processor is used to execute the computer program to implement the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, A computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1-8 is implemented.

Images