Camera pitch angle determination method, device, apparatus and computer storage medium

Abstract

The application discloses a camera pitch angle determination method, device, equipment and computer storage medium. Height information of a camera, a camera image containing parallel lines and a preset virtual projection matrix are acquired. Coordinate points corresponding to the parallel lines are determined in the camera image, and the coordinate points are converted into virtual coordinate points based on the virtual projection matrix, and a first straight line equation and a second straight line equation corresponding to the virtual coordinate points are determined. According to the height information, the first straight line equation and the second straight line equation, the pitch angle of the camera is determined. According to the embodiment of the application, the pitch angle of the camera can be determined. The pitch angle is determined by a group of parallel lines in an image collected by the camera, intrinsic parameters of the camera and other data, so that the pitch angle of the camera can be determined only by using a group of parallel lines. It can be seen that the pitch angle of the camera can be more accurately and stably determined by using the above method.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of photogrammetry, and particularly relates to a camera pitch angle determination method, device, equipment and computer storage medium. BACKGROUND

[0002] Generally, the pitch angle of a camera configured on an automatic driving vehicle relative to the ground is an important parameter in an automatic driving system. However, the pitch angle of the camera may change due to the shaking of the automatic driving vehicle during driving. Therefore, the pitch angle of the camera needs to be measured at any time.

[0003] In the prior art, the pitch angle of the camera is determined by using a vanishing point estimation method. That is, the pitch angle of the camera is determined by two sets of mutually perpendicular parallel lines in an image collected by the camera.

[0004] However, in general, only one set of lane lines can be determined in the image collected by the camera, and it is difficult to determine two sets of mutually perpendicular parallel lines. Therefore, it is difficult to measure the pitch angle of the camera in some scenes by using the above method. SUMMARY

[0005] The embodiments of the application provide a camera pitch angle determination method, device, equipment and computer storage medium, which can more accurately and stably determine the pitch angle of the camera.

[0006] In one aspect, the embodiments of the application provide a camera pitch angle determination method, which comprises the following steps.

[0007] Obtaining height information of the camera, a camera image containing parallel lines, and a preset virtual projection matrix, wherein the virtual projection matrix is a projection matrix of a coordinate point in an image coordinate system of an image collected by the camera and a coordinate point in a ground coordinate system when the pitch angle of the camera is zero;

[0008] Determining a coordinate point corresponding to the parallel lines in the camera image;

[0009] Converting the coordinate point in the image coordinate system into a virtual coordinate point based on the virtual projection matrix, and determining a virtual plane on which the virtual coordinate point is located;

[0010] Determining a first straight line equation and a second straight line equation corresponding to the virtual coordinate point in the virtual plane;

[0011] Determining the pitch angle of the camera according to the height information, the first straight line equation and the second straight line equation.

[0012] Optionally, the preset virtual projection matrix is obtained, and the method specifically comprises the following steps.

[0013] obtaining an intrinsic parameter of the camera, and a coordinate conversion matrix between a ground coordinate system and a camera coordinate system of the camera when the pitch angle is zero;

[0014] multiplying an inverse matrix of the product of the intrinsic parameter and the coordinate conversion matrix to determine a virtual projection matrix.

[0015] Optionally, determining a coordinate point corresponding to the parallel line in the camera image specifically includes:

[0016] establishing an image coordinate system in the camera image;

[0017] obtaining an intrinsic parameter of the camera;

[0018] determining a target point corresponding to a coordinate origin of the camera coordinate system in the image coordinate system according to the intrinsic parameter;

[0019] when determining that a distance between the parallel line and the target point is less than a preset distance threshold, determining a coordinate point corresponding to the parallel line in the image coordinate system.

[0020] Optionally, after determining the target point corresponding to the coordinate origin of the camera coordinate system in the image coordinate system, the method further includes:

[0021] when determining that the distance between the parallel line and the target point is greater than or equal to the preset distance threshold, returning to the step of obtaining the camera image containing the parallel line until the distance between the parallel line and the target point is less than the distance threshold.

[0022] Optionally, in the virtual plane, determining a first straight line equation and a second straight line equation corresponding to the virtual coordinate point specifically includes:

[0023] in the virtual plane, determining a first parallel line segment and a second parallel line segment corresponding to the virtual coordinate point;

[0024] determining a parallel line segment direction according to the first parallel line segment and the second parallel line segment;

[0025] creating a plane coordinate system by taking the parallel line segment direction as a longitudinal coordinate axis;

[0026] determining a first straight line equation of the first parallel line segment and a second straight line equation of the second parallel straight line,

[0027] wherein the first straight line equation is:

[0028]

[0029] the second straight line equation is:

[0030]

[0031] is a horizontal coordinate of any point in the first parallel line segment, is a vertical coordinate of the point, is a slope of the first parallel line segment, is a constant, is a horizontal coordinate of any point in the second parallel line segment, is a vertical coordinate of the point, is a slope of the second parallel line segment, is a constant.

[0032] Optionally, the pitch angle of the camera is determined according to the height information, the first straight line equation and the second straight line equation, and specifically includes:

[0033] The pitch angle of the camera is determined according to the height information, the first equation and the second equation by the following formula:

[0034]

[0035] wherein, is the pitch angle of the camera, is the slope of the first parallel line segment, is the slope of the second parallel line segment, and is a constant, and h is the height information.

[0036] Optionally, the direction of the parallel line segment is determined according to the first parallel line segment and the second parallel line segment, and specifically includes:

[0037] The first parallel line segment and the second parallel line segment are extended until the first parallel line segment and the second parallel line segment intersect;

[0038] An angle bisector of an included angle between the first parallel line segment and the second parallel line segment is determined, and a direction along the angle bisector and away from the included angle is determined as the direction of the parallel line segment.

[0039] In another aspect, the embodiment of the present application provides a pitch angle determination device of a camera, and the device includes:

[0040] An acquisition unit is configured to acquire height information of the camera, a camera image containing parallel lines and a preset virtual projection matrix, wherein the virtual projection matrix is a projection matrix of a coordinate point in an image coordinate system of an image collected by the camera when the pitch angle of the camera is zero and a coordinate point in a ground coordinate system.

[0041] a coordinate unit configured to determine a coordinate point corresponding to the pair of parallel lines in the camera image;

[0042] a conversion unit configured to convert the coordinate point in the image coordinate system into a virtual coordinate point based on the virtual projection matrix, and determine a virtual plane on which the virtual coordinate point is located;

[0043] a fitting unit configured to determine a first straight line equation and a second straight line equation corresponding to the virtual coordinate point in the virtual plane;

[0044] a determination unit configured to determine the pitch angle of the camera according to the height information, the first straight line equation and the second straight line equation.

[0045] In another aspect, an embodiment of the present application provides a device for determining a pitch angle of a camera, the device comprising:

[0046] a processor and a memory storing computer program instructions;

[0047] the processor implements the method for determining a pitch angle of a camera as described in any one of the above aspects when executing the computer program instructions.

[0048] In another aspect, an embodiment of the present application provides a computer readable storage medium storing computer program instructions, the computer program instructions being executed by a processor to implement the method for determining a pitch angle of a camera as described in any one of the above aspects.

[0049] In another aspect, an embodiment of the present application provides a computer program product, instructions in the computer program product being executed by a processor of an electronic device to cause the electronic device to implement the method for determining a pitch angle of a camera as described in any one of the above aspects.

[0050] The method, device, apparatus and computer readable storage medium for determining a pitch angle of a camera according to the embodiments of the present application can determine the pitch angle of the camera. The pitch angle is determined by a set of parallel lines in an image collected by the camera and intrinsic parameters of the camera, and thus the pitch angle of the camera can be determined only by a set of parallel lines. It can be seen that the pitch angle of the camera can be determined more accurately and stably by using the above method. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced as follows, and other drawings can be obtained by those of ordinary skill in the art without creative labor on the basis of these drawings.

[0052] Figure 1is a flowchart of a method for determining a pitch angle of a camera according to an embodiment of the present application;

[0053] Figure 2 is a schematic diagram of parallel line segments according to an embodiment of the present application;

[0054] Figure 3 is a schematic diagram of a virtual plane according to an embodiment of the present application;

[0055] Figure 4 is a comparison diagram of planes according to an embodiment of the present application;

[0056] Figure 5 is a structural schematic diagram of a device for determining a pitch angle of a camera according to an embodiment of the present application;

[0057] Figure 6 is a structural schematic diagram of a device for determining a pitch angle of a camera according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] The features and exemplary embodiments of the various aspects of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application. The present application can be implemented without some of the specific details described below. The following description of the embodiments is merely intended to provide a better understanding of the present application through showing examples of the present application.

[0059] It should be noted that, in this document, the terms such as first and second are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Also, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0060] Camera intrinsic parameter: an intrinsic parameter of a camera, a coordinate conversion matrix for converting corresponding coordinates in a camera coordinate system and an image coordinate system.

[0061] Camera coordinate system: the coordinate origin is the optical center position of the camera, the X axis and the Y axis are parallel to the X axis and the Y axis of the image coordinate system respectively, and the Z axis is the optical axis of the camera, thereby forming a coordinate system.

[0062] Ground coordinate system: the ground coordinate system is a coordinate system fixed on the surface of the earth, the X axis points to an arbitrary direction of the ground plane, the Z axis is vertically upward, and the Y axis is perpendicular to the plane formed by the Z axis, thereby forming a right-handed coordinate system. When the rotation of the earth and the curved motion of the earth center are ignored, the ground coordinate system can be regarded as an inertial coordinate system. In one or more embodiments of the present application, the XOY plane of the ground coordinate system coincides with the ground where the vehicle is located. The XOY plane is a plane composed of the X axis, the coordinate origin, and the Y axis of the corresponding coordinate system.

[0063] To solve the problems in the prior art, the present application provides a camera pitch angle determination method, device, equipment and computer storage medium. First, the camera pitch angle determination method provided by the embodiments of the present application is introduced.

[0064] Figure 1 The flowchart of the camera pitch angle determination method provided by an embodiment of the present application is shown. As shown in Figure 1 The camera pitch angle determination method provided by the embodiments of the present application includes the following steps: S101 to S105.

[0065] S101: obtaining height information of the camera, a camera image containing parallel lines, and a preset virtual projection matrix, wherein the virtual projection matrix is a projection matrix of a coordinate point in an image coordinate system of an image collected by the camera when the pitch angle of the camera is zero and a coordinate point in a ground coordinate system

[0066] Generally, due to the angle of view, when a user observes a group of actual parallel lines along the extension direction of the line segments, the observed line segments are often not parallel. As shown in Figure 2 Figure 2 The present application provides a schematic diagram of observing parallel line segments. The line segment 301, the line segment 302, the line segment 303, the line segment 304, the line segment 305, and the line segment 306 are mutually parallel line segments, a user observes along the extension direction of the line segment 303 at the point 300, and obtains the observation image 307. Obviously, in the observation image 307, the line segment 301, the line segment 302, the line segment 303, the line segment 304, the line segment 305, and the line segment 306 are not parallel to each other.

[0067] Therefore, in one or more embodiments of the present application, the pitch angle of the camera can be determined based on the parallel lines in the image collected by the camera.

[0068] ​In one or more embodiments of the present application, the camera tilt angle determination method can be determined by an electronic device. Of course, the electronic device can be a vehicle control center, a mobile phone, a tablet computer, a server, or the like. The specific type of electronic device is not limited in the present application and can be set as needed.

[0069] Specifically, the electronic device can obtain height information of the camera, a camera image containing parallel lines, and a preset virtual projection matrix.

[0070] The height information can be preset or obtained by the electronic device in response to a user input. The specific manner in which the electronic device obtains the height information is not limited in the present application and can be set as needed. The camera image can be an image obtained by the electronic device in response to a user operation or determined by the electronic device from an image captured by the camera based on an Inception ResNet V2 model, a Dual Path Networks model, or the like. The specific manner in which the camera image is determined is not limited in the present application and can be set as needed.

[0071] For the virtual conversion matrix, first, the electronic device can obtain intrinsic parameters of the camera and a coordinate conversion matrix between a ground coordinate system and a camera coordinate system when the tilt angle of the camera is zero. The intrinsic parameters and the coordinate conversion matrix are both preset. The coordinate conversion matrix can convert a coordinate point of any object in the ground coordinate system into a coordinate point of the object in the camera coordinate system.

[0072] In one or more embodiments of the present application, the coordinate conversion matrix includes a rotation matrix and a translation matrix. As shown in the following formula:

[0073] 1

[0074] wherein (x, y) is a coordinate point of an object in the ground coordinate system, (u, v) is a coordinate point of the object in the camera coordinate system. , , , , is the rotation matrix, is the translation matrix.

[0075] ​​Since the pitch angle determination method of the camera is usually applied to a vehicle, the parallel line segments in the camera image captured by the camera when the vehicle is running are usually lane marking lines, ground marking lines and other line segments on the ground. Thus, in one or more embodiments of the present application, the electronic device can determine that the parallel lines in the camera image are in the XOY plane in the ground coordinate system. That is, the value of the third column of the rotation matrix is 0. Thus, in one or more embodiments of the present application, since the product of 0 and any number is 0, the value of the third column of the rotation matrix is meaningless. Thus, the electronic device can simplify formula 1 to obtain the following formula:

[0076]

[0077] wherein (X, Y) is a coordinate point of an object in the ground coordinate system, (u, v) is a coordinate point of the object in the camera coordinate system, and R is the coordinate conversion matrix. is the coordinate conversion matrix.

[0078] Secondly, since the coordinate conversion matrix is a coordinate conversion matrix for converting a coordinate point in the ground coordinate system into a coordinate point in the camera coordinate system, and the intrinsic parameter of the camera is a coordinate point in the camera coordinate system, which is converted into a coordinate point in the image coordinate system. Thus, the electronic device can determine the product of the intrinsic parameter and the coordinate conversion matrix as a ground image projection matrix. The ground image projection matrix can convert a coordinate point in the ground coordinate system into a coordinate point in the image coordinate system. The inverse matrix of the ground image projection matrix is determined to obtain a virtual projection matrix.

[0079] The above process is shown in the following formula:

[0080] 3

[0081] wherein, is the virtual projection matrix, is the ground image projection matrix, is the coordinate conversion matrix, is the intrinsic parameter. The virtual projection matrix is a projection matrix for converting a coordinate point in the image coordinate system into a coordinate point in the ground coordinate system when the pitch angle of the camera is zero.

[0082] ​​​​​​However, since the pitch angle of the camera is not determined and the pitch angle is not necessarily zero, the electronic device can determine the virtual projection matrix as a projection matrix for converting a coordinate point in an image coordinate system to a coordinate point in a virtual coordinate system, and determine an XOY plane in the virtual coordinate system as a virtual plane.

[0083] In this way, the electronic device can determine the virtual projection matrix, so as to determine the pitch angle of the camera.

[0084] S102: Determine a coordinate point corresponding to the parallel line in the camera image.

[0085] In one or more embodiments of the present application, after the electronic device determines the virtual projection matrix and the camera image containing the parallel line, the electronic device can determine a coordinate point corresponding to the parallel line in the camera image in the image coordinate system.

[0086] Specifically, the electronic device can establish an image coordinate system in the camera image, and then obtain the intrinsic parameter of the camera. According to the intrinsic parameter, a coordinate point corresponding to the coordinate origin of the camera coordinate system in the image coordinate system is determined, and the coordinate point is recorded as a target point.

[0087] Secondly, it is determined whether there is a parallel line in the camera image with a distance less than a preset distance threshold from the target point. If yes, a coordinate point corresponding to the parallel line with a distance less than the preset distance threshold from the target point in the camera coordinate system of the camera image is determined. If no, step S101 is returned until a parallel line with a distance less than the distance threshold from the target point is determined in the camera image.

[0088] Since the parallel line is a line segment, the number of coordinate points can be multiple. When determining the distance between the target point and the parallel line, the distance between the point closest to the target point in the parallel line and the target point can be determined as the distance between the parallel line and the target point. Of course, other ways can also be used to determine the distance between the parallel line and the target point, and the present application does not limit the specific way, which can be set as needed.

[0089] In this way, the electronic device can determine the coordinate point representing the parallel line of the camera image in the image coordinate system, so as to determine the coordinate point of the parallel line in the virtual coordinate system.

[0090] S103: Convert the coordinate point in the image coordinate system to a virtual coordinate point based on the virtual projection matrix, and determine a plane where the virtual coordinate point is located as a virtual plane.

[0091] In one or more embodiments of the present application, after determining the virtual projection matrix and the coordinate points corresponding to the parallel lines of the camera image in the image coordinate system, the electronic device can determine the coordinate points corresponding to the parallel lines of the camera image in the virtual coordinate system through the virtual projection matrix.

[0092] Specifically, the electronic device can convert the coordinate points corresponding to the parallel lines in the image coordinate system into virtual coordinate points in the virtual coordinate system based on the virtual projection matrix. Then, the plane where the virtual coordinate points are located is determined as the virtual plane.

[0093] In this way, the electronic device can represent the coordinate points of the parallel lines of the camera image in the virtual coordinate system.

[0094] S104: Determine a first straight line equation and a second straight line equation corresponding to the virtual coordinate points in the virtual plane.

[0095] To determine the pitch angle of the camera, the electronic device can determine the angle between the virtual plane and the XOY plane of the ground coordinate system. Since the virtual plane is obtained based on the coordinate points in the image coordinate system in the camera image and the virtual projection matrix when the coordinate points in the image coordinate system are converted into coordinate points in the ground coordinate system when the pitch angle of the camera is zero, and the parallel lines are actually located in the XOY plane in the ground coordinate system, the virtual plane where the parallel lines are located in the virtual coordinate system is the plane where the objects located in the XOY plane in the ground coordinate system in the image captured by the camera when the pitch angle is zero are converted into coordinate points in the ground coordinate system, and further, the angle between the virtual plane and the XOY plane in the ground coordinate system is the pitch angle of the camera.

[0096] Therefore, in one or more embodiments of the present application, to determine the angle between the virtual plane and the XOY plane in the ground coordinate system, the electronic device can determine a first straight line equation and a second straight line equation corresponding to the parallel lines in the virtual plane.

[0097] Specifically, first, the electronic device can determine the plane equation of the virtual plane according to the virtual coordinate points. Since the technology of determining a unique plane according to a plurality of three-dimensional coordinate points in the same plane is relatively mature, how to determine the plane equation of the virtual plane is not described herein. It should be noted that the number of virtual coordinate points is greater than 3, and the virtual coordinate points are all located in the same plane.

[0098] Secondly, the electronic device can determine a planar projection matrix of a coordinate point in the virtual coordinate system projected into the virtual plane according to the virtual coordinate system and the virtual plane. Since the determination of the projection matrix of a coordinate point in a coordinate system in a plane is already mature, how to determine the planar projection matrix will not be described here.

[0099] In the process of determining the planar coordinate system of the virtual plane, the first parallel line segment and the second parallel line segment can be respectively extended until the first parallel line segment intersects with the second parallel line segment. An angle bisector of the included angle of the first parallel line segment and the second parallel line segment is determined as the Y-axis of the planar coordinate system, and a direction away from the intersection point along the angle bisector is determined as the direction of the Y-axis. That is, a direction away from the included angle along the angle bisector is determined as the direction of the parallel line segment, and the direction of the Y-axis is determined to be the same as the direction of the parallel line segment. A point on the Y-axis is randomly determined as the coordinate origin of the planar coordinate system, and a straight line passing through the coordinate origin and perpendicular to the Y-axis is determined as the X-axis of the planar coordinate system. The X-axis, the coordinate origin, and the Y-axis are determined as the planar coordinate system.

[0100] Thirdly, the electronic device can determine the planar coordinate of the virtual coordinate point in the virtual plane according to the planar projection matrix.

[0101] Finally, the electronic device can determine a first straight line equation and a second straight line equation according to the planar coordinate. The first straight line equation is an equation representing the first parallel line segment in the parallel line, and the second straight line equation is an equation representing the second parallel line segment in the parallel line.

[0102] The first straight line equation is:

[0103] 4

[0104] The second straight line equation is:

[0105] 5

[0106] The abscissa of any point in the first parallel line segment is: The ordinate of the point is: The slope of the first parallel line segment is: It is a constant, The abscissa of any point in the second parallel line segment is: The ordinate of the point is: The slope of the second parallel line segment is: It is a constant.

[0107] As Figure 3 shown, Figure 3 a virtual plane diagram provided by the present application, wherein the X coordinate axis 201 and the Y coordinate axis 202 constitute a virtual plane coordinate system in the virtual plane 200, the first parallel line segment 203 in the virtual plane corresponds to the first parallel line segment 206 in the ground coordinate system, and the second parallel line segment 204 in the virtual plane corresponds to the second parallel line segment 205 in the ground coordinate system.

[0108] S105: determining the pitch angle of the camera according to the height information, the first straight line equation, and the second straight line equation.

[0109] As Figure 4 shown, Figure 4 a plane contrast diagram provided by the present application. The coordinate system composed of the X coordinate axis 403, the Y coordinate axis 404, and the Z coordinate axis 405 is the ground coordinate system, and the coordinate system composed of the X coordinate axis 409, the Y coordinate axis 410, and the Z coordinate axis 411 is the virtual coordinate system. The camera and the coordinate origin 414 of the virtual coordinate system are in the same position. The first parallel line segment 401 in the ground coordinate system is parallel to the second parallel line segment 402 in the ground coordinate system, the first parallel line segment 408 in the virtual coordinate system is parallel to the second parallel line segment 407 in the virtual coordinate system, and the angle 413 between the XOY plane 400 of the ground coordinate system and the virtual plane 406 is . The angle 412 between the second parallel line segment 407 in the virtual coordinate system and the second parallel line segment 402 in the ground coordinate system is .

[0110] Since the lengths of the first parallel line segment and the second parallel line segment are both fixed values, when the camera observes the first parallel line segment at different pitch angles in the same position, there is a projection relationship between the two images observed by the camera. As Figure 4 shown, the first auxiliary line 417 from the camera to the endpoint of the first parallel line segment of the ground coordinate system passes through the endpoint of the first parallel line segment of the virtual coordinate system. The second auxiliary line 416 from the camera to the endpoint of the second parallel line segment of the ground coordinate system passes through the endpoint of the second parallel line segment of the virtual coordinate system.

[0111] As Figure 4 shown, according to the projection relationship, the following formula can be derived:

[0112]

[0113] wherein, is the horizontal coordinate of any point in the first parallel line segment 408 in the virtual plane 406, is the vertical coordinate of the point, Here are the vertical coordinates of the point. This represents the x-coordinate of the target point projected from this point in the ground coordinate system. This represents the ordinate of the target point projected from this point in the ground coordinate system. Similarly, Let x be the x-coordinate of any point within the second parallel line segment 407 within the virtual plane 406. Let be the ordinate of the point. Here are the vertical coordinates of the point. This represents the x-coordinate of the target point projected from this point in the ground coordinate system. This is the ordinate of the target point projected onto the ground coordinate system.

[0114] like Figure 4 As shown, the third auxiliary line 418 connects the first auxiliary line 417 and the second auxiliary line 416, and the fourth auxiliary line 419 connects the first auxiliary line 417 and the second auxiliary line 416. The portion of the first auxiliary line 417 between the origin 414 of the virtual coordinate system and the fourth auxiliary line 419 is the first sub-auxiliary line. The portion of the second auxiliary line 416 between the origin 414 of the virtual coordinate system and the fourth auxiliary line 419 is the second sub-auxiliary line. h is the distance 415 between the camera and the ground, i.e., the height information. is the angle between the first parallel line segment of the virtual coordinate system and the first parallel line segment of the ground coordinate system, i.e., the angle between the XOY plane of the virtual coordinate system and the ground coordinate system, denoted as . The ordinates of the first endpoint 420 of the first parallel line segment and the second endpoint 421 of the second parallel line segment in this virtual coordinate system are both z. Therefore, when the coordinate of the first endpoint 420 is ( , When z), the coordinates of the corresponding point in the ground coordinate system are ( , ,0), when the coordinates of the second endpoint 421 are ( , When z), the coordinates of the corresponding point in the ground coordinate system are ( , (0). It is evident that the first triangle formed by the first, second, and third auxiliary lines, and the second triangle formed by the first, second, and fourth auxiliary lines, are similar triangles. Therefore, according to the theorem of similar triangles, the following formula holds:

[0115]

[0116] Based on formulas 6 and 7, the following formula can be obtained:

[0117]

[0118] According to the formula 4, the formula 5 and the formula 8, the following formula can be derived:

[0119]

[0120] When The formula 10 is obtained:

[0121]

[0122] The formula 11 is obtained by subtracting the formula 9 from the formula 10:

[0123]

[0124] According to the formula 11, the formula 12 can be proved:

[0125]

[0126] According to the formula 12, the formula 13 can be proved:

[0127]

[0128] Therefore, in one or more embodiments of the present application, specifically, the electronic device can calculate the angle between the XOY plane in the virtual coordinate system and the XOY plane of the ground coordinate system, i.e., the pitch angle of the camera, according to the first straight line equation, the second straight line equation and the height information, by the formula 12.

[0129] In the above manner, the electronic device can determine the pitch angle of the camera according to the first straight line equation, the second straight line equation and the height information of the camera.

[0130] The above is a specific implementation of the method for determining the pitch angle of the camera provided by the embodiments of the present application. As can be seen from the above embodiments, the pitch angle of the camera can be determined. The pitch angle is determined by a set of parallel lines in the image collected by the camera and the intrinsic parameters of the camera, which realizes that only a set of parallel lines is needed to determine the pitch angle of the camera. As can be seen, the pitch angle of the camera can be determined more accurately and stably by using the above manner.

[0131] In addition, the acquisition, storage, use, processing and the like of data in the technical solutions of the present application comply with the relevant provisions of national laws and regulations.

[0132] Based on the method for determining the pitch angle of the camera provided by the above embodiments, correspondingly, the present application also provides a specific implementation of a device for determining the pitch angle of the camera. Please refer to the following embodiments.

[0133] First, refer to Figure 5The camera pitch angle determination apparatus provided by the embodiment of the present application comprises the following units:

[0134] An acquisition unit 801 is configured to acquire height information of the camera, a camera image containing parallel lines, and a preset virtual projection matrix, wherein the virtual projection matrix is a projection matrix of a coordinate point in an image coordinate system of an image collected by the camera when a pitch angle of the camera is zero and a coordinate point in a ground coordinate system.

[0135] A coordinate unit 802 is configured to determine a coordinate point corresponding to the parallel lines in the camera image.

[0136] A conversion unit 803 is configured to convert the coordinate point in the image coordinate system into a virtual coordinate point based on the virtual projection matrix, and determine a virtual plane on which the virtual coordinate point is located.

[0137] A fitting unit 804 is configured to determine a first straight line equation and a second straight line equation corresponding to the virtual coordinate point in the virtual plane.

[0138] A determination unit 805 is configured to determine the pitch angle of the camera according to the height information, the first straight line equation, and the second straight line equation.

[0139] According to the above embodiment, the determination unit 805 can determine the pitch angle of the camera. The pitch angle is determined by a set of parallel lines in an image collected by the camera, intrinsic parameters of the camera, and other data acquired by the acquisition unit 801, which realizes that only a set of parallel lines is needed to determine the pitch angle of the camera. It can be seen that the pitch angle of the camera can be determined more accurately and stably in the above manner.

[0140] As another implementation manner of the present application, in order to determine the pitch angle of the camera more accurately and stably, the above apparatus can further comprise an acquisition sub-unit 8011.

[0141] The acquisition sub-unit 8011 is configured to acquire intrinsic parameters of the camera, a coordinate conversion matrix of a ground coordinate system and a camera coordinate system when the pitch angle of the camera is zero, and multiply an inverse matrix of the product of the intrinsic parameters and the coordinate conversion matrix to determine a virtual projection matrix.

[0142] As another implementation manner of the present application, in order to determine the pitch angle of the camera more accurately and stably, the above apparatus can further comprise a coordinate sub-unit 8021.

[0143] The coordinate subunit 8021 is configured to establish an image coordinate system in the camera image, acquire an intrinsic parameter of the camera, determine a coordinate origin of the camera coordinate system in a target point corresponding to the image coordinate system according to the intrinsic parameter, and determine a coordinate point corresponding to the parallel line in the image coordinate system when a distance between the parallel line and the target point is less than a preset distance threshold.

[0144] As another implementation form of the present application, in order to more accurately and stably determine the pitch angle of the camera, the above device can further specifically include an acquisition subunit 8012.

[0145] The acquisition subunit 8012 is configured to return the acquisition of the camera image containing the parallel line when the distance between the parallel line and the target point is greater than or equal to the preset distance threshold, until the distance between the parallel line and the target point is less than the distance threshold.

[0146] As another implementation form of the present application, in order to more accurately and stably determine the pitch angle of the camera, the above device can further specifically include a fitting subunit 8041.

[0147] The fitting subunit 8041 is configured to determine a first parallel line segment and a second parallel line segment corresponding to the virtual coordinate point in the virtual plane, determine a parallel line segment direction according to the first parallel line segment and the second parallel line segment, create a plane coordinate system by taking the parallel line segment direction as a longitudinal coordinate axis, determine a first straight line equation of the first parallel line segment and a second straight line equation of the second parallel straight line, wherein the first straight line equation is:

[0148]

[0149] The second straight line equation is:

[0150]

[0151] A horizontal coordinate of any point in the first parallel line segment is x, A vertical coordinate of the point is y, A slope of the first parallel line segment is m1, A constant is c1, A horizontal coordinate of any point in the second parallel line segment is x, A vertical coordinate of the point is y, A slope of the second parallel line segment is m2, A constant is c2.

[0152] As another implementation form of the present application, in order to more accurately and stably determine the pitch angle of the camera, the apparatus can further include a fitting sub-unit 8042.

[0153] The fitting sub-unit 8042 is configured to determine the pitch angle of the camera according to the height information, the first equation and the second equation by the following formula:

[0154]

[0155] wherein, is the pitch angle of the camera, is the slope of the first parallel line segment, is the slope of the second parallel line segment, and is a constant, and h is the height information.

[0156] As another implementation form of the present application, in order to more accurately and stably determine the pitch angle of the camera, the apparatus can further include a fitting sub-unit 8043.

[0157] The fitting sub-unit 8043 is configured to extend the first parallel line segment and the second parallel line segment until the first parallel line segment intersects with the second parallel line segment, determine an angle bisector of an included angle between the first parallel line segment and the second parallel line segment, and determine a direction along the angle bisector and away from the included angle as a parallel line segment direction.

[0158] Figure 6 A hardware structure schematic diagram of a camera pitch angle determination apparatus is shown.

[0159] The camera pitch angle determination apparatus can include a processor 901 and a memory 902 storing computer program instructions.

[0160] Specifically, the processor 901 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application.

[0161] The memory 902 can include mass storage for data or instructions. As an example and not by way of limitation, the memory 902 can include a Hard Disk Drive (HDD), a floppy disk drive, a flash memory, a Compact Disc (CD) or other optical disk, a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 902 can include removable or non-removable (or fixed) media, where appropriate. Where appropriate, the memory 902 can be internal or external to the integrated gateway disaster recovery appliance. In particular embodiments, the memory 902 is non-volatile, solid-state memory.

[0162] In particular embodiments, the memory 902 can include read-only memory (ROM), random access memory (RAM), magnetic disk storage mediums, optical storage mediums, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software that, when executed (by one or more processors), is operable to

[0163] The processor 901 implements the camera pitch angle determination method of any of the above embodiments by reading and executing computer program instructions stored in the memory 902.

[0164] In one example, the camera pitch angle determination device can further include a communication interface 903 and a bus 910. Wherein, as shown in the figure, the processor 901, the memory 902, the communication interface 903 are connected through the bus 910 and complete the communication between each other. Figure 6

[0165] The communication interface 903 is mainly used to realize the communication between each module, device, unit and / or equipment in the embodiments of the application.

[0166] ​Bus 910 includes hardware, software, or both, to couple components of the camera's pitch determination device to each other in communication. While Figure 9 illustrates a bus for the camera's pitch determination device, the camera's pitch determination device can include any interconnection scheme that allows communication between various components of the camera's pitch determination device. For example, without limitation, the bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or interconnect in addition to or instead of the buses described herein. The bus 910 can include one or more buses, where appropriate. Although this application describes and illustrates a particular bus, this application contemplates any suitable bus or interconnect.

[0167] The camera's pitch determination device can perform the camera's pitch determination method in the embodiments of the application, thereby realizing the camera's pitch determination method and device described in combination Figure 1 and Figure 5 with the above embodiments.

[0168] In addition, in combination with the camera's pitch determination method in the above embodiments, the embodiments of the application can provide a computer storage medium to realize. The computer storage medium has computer program instructions stored thereon; the computer program instructions are executed by a processor to realize any of the camera's pitch determination methods in the above embodiments.

[0169] It needs to be clear that the application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the application.

[0170] The functions shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, functional cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried on a carrier wave in a transmission medium or communication link. The "machine-readable medium" can include any medium that can store or transfer information. Examples of the machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic medium, radio frequency (RF) links, and the like. The code segments can be downloaded via a computer network such as the Internet, intranet, or the like.

[0171] It is also important to note that the examples described herein can be implemented in a variety of systems, including and not limited to a digital electronic circuit, an analog electronic circuit, a computer hardware, firmware, software, or in combinations of them.

[0172] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0173] The above only describes specific implementation of the present application. For the convenience and brevity of description, the specific working processes of the above-described system, modules and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein. It should be understood that the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A method for determining the pitch angle of a camera, characterized in that, include: Acquire the camera's height information, a camera image containing parallel lines, and a preset virtual projection matrix, wherein the virtual projection matrix is ​​the projection matrix of coordinate points in the image coordinate system and coordinate points in the ground coordinate system of the image acquired by the camera when the camera's pitch angle is zero. Determine the coordinate points corresponding to the parallel lines in the camera image; Based on the virtual projection matrix, the coordinate points in the image coordinate system are converted into virtual coordinate points, and the plane where the virtual coordinate points are located is determined to be a virtual plane; Within the virtual plane, determine the equations of the first and second lines corresponding to the virtual coordinate points; The camera's pitch angle is determined based on the altitude information, the first line equation, and the second line equation.

2. The method according to claim 1, characterized in that, Obtaining the preset virtual projection matrix specifically includes: Obtain the intrinsic parameters of the camera and the coordinate transformation matrix between the ground coordinate system and the camera coordinate system when the camera's pitch angle is zero; The inverse of the product of the intrinsics and the coordinate transformation matrix is ​​determined as the virtual projection matrix.

3. The method according to claim 1, characterized in that, Determining the coordinates of the parallel lines in the camera image specifically includes: In the camera image, establish an image coordinate system; Obtain the intrinsic parameters of the camera; Based on the intrinsic parameters, determine the target point corresponding to the origin of the camera coordinate system in the image coordinate system; When it is determined that the distance between the parallel line and the target point is less than a preset distance threshold, the coordinate point corresponding to the parallel line is determined in the image coordinate system.

4. The method according to claim 3, characterized in that, After determining the target point corresponding to the origin of the camera coordinate system in the image coordinate system, the method further includes: When it is determined that the distance between the parallel line and the target point is greater than or equal to a preset distance threshold, the process returns to the step of acquiring a camera image containing the parallel line until the distance between the parallel line and the target point is less than the distance threshold.

5. The method according to claim 1, characterized in that, Within the virtual plane, determining the equations of the first and second lines corresponding to the virtual coordinate points specifically includes: Within the virtual plane, determine the first parallel line segment and the second parallel line segment corresponding to the virtual coordinate point; The direction of the parallel line segments is determined based on the first parallel line segment and the second parallel line segment; A planar coordinate system is created using the direction of the parallel line segments as the vertical axis. Determine the equation of the first straight line of the first parallel line segment and the equation of the second straight line of the second parallel line segment. The equation of the first straight line is: The equation of the second straight line is: Let x be the x-coordinate of any point within the first parallel line segment. Let be the ordinate of the point. Let be the slope of the first parallel line segment. It is a constant. Let x be the x-coordinate of any point within the second parallel line segment. Let be the ordinate of the point. Let be the slope of the second parallel line segment. It is a constant.

6. The method according to claim 5, characterized in that, Based on the altitude information, the first straight line equation, and the second straight line equation, the camera's pitch angle is determined, specifically including: Based on the altitude information, the first line equation, and the second line equation, the camera's pitch angle is determined using the following formula: in, Let be the pitch angle of the camera. Let be the slope of the first parallel line segment. Let be the slope of the second parallel line segment. as well as is a constant, and h is the height information.

7. The method according to claim 5, characterized in that, Determining the direction of the parallel line segments based on the first and second parallel line segments specifically includes: Extend the first parallel line segment and the second parallel line segment until they intersect; The angle bisector of the angle between the first parallel line segment and the second parallel line segment is determined, and the direction along the angle bisector and away from the angle is determined as the direction of the parallel line segment.

8. A device for determining the pitch angle of a camera, characterized in that, The device includes: The acquisition unit is used to acquire the camera's height information, a camera image containing parallel lines, and a preset virtual projection matrix, wherein the virtual projection matrix is ​​the projection matrix of the coordinate points in the image coordinate system and the coordinate points in the ground coordinate system of the image acquired by the camera when the camera's pitch angle is zero. A coordinate unit is used to determine the coordinate point corresponding to the parallel line in the camera image; The transformation unit is used to convert coordinate points in the image coordinate system into virtual coordinate points based on the virtual projection matrix, and to determine the plane where the virtual coordinate points are located as a virtual plane; The fitting unit is used to determine the first line equation and the second line equation corresponding to the virtual coordinate point in the virtual plane. The determining unit is used to determine the pitch angle of the camera based on the altitude information, the first straight line equation, and the second straight line equation.

9. A device for determining the pitch angle of a camera, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the camera pitch angle determination method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the camera pitch angle determination method as described in any one of claims 1-7.

11. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the camera pitch angle determination method as described in any one of claims 1-7.

Images