Projection area determination method and apparatus, and electronic device

Abstract

The invention provides a projection area determination method and apparatus, and an electronic device, and belongs to the technical field of computer vision. The method comprises the steps that equipment parameters of imaging equipment of the aircraft are acquired, and the equipment parameters comprise an attitude angle parameter and a field angle parameter; according to the field angle parameter, a direction vector corresponding to a visual cone of the imaging equipment is determined, the direction vector is used for representing a boundary ray of the visual cone, and the direction vector corresponds to a local coordinate system of the imaging equipment; constructing a three-dimensional rotation matrix according to the attitude angle parameters, and converting the direction vector from a local coordinate system to a world coordinate system based on the three-dimensional rotation matrix to obtain a target direction vector; and determining position information of a projection intersection point of the boundary ray and the ground according to the vector value of the target direction vector, and determining a ground projection area of the imaging equipment according to the position information of the projection intersection point. The method can flexibly adapt to the three-dimensional attitude of the imaging equipment, and improves the calculation accuracy of the ground projection area.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer vision, and in particular, to a projection area determination method and device, and an electronic device. BACKGROUND

[0002] The projection area refers to a range of an area formed by the intersection of a viewing pyramid of an imaging device carried by an aircraft and a ground plane under a specific position and attitude.

[0003] In the related art, in order to determine the ground projection area of the imaging device of the aircraft, a simplified model is usually used to improve the calculation efficiency. For example, in the related art, it is assumed that the pitch angle and roll angle of the imaging device are close to 0°, and the yaw angle changes slowly, so that a three-dimensional space problem is simplified into an approximate two-dimensional geometric model (such as orthographic projection). In the ground processing stage, the geometric error is corrected based on a high-precision image registration algorithm (such as SIFT, SURF, etc.). That is, the related art determines the ground projection area of the imaging device by combining a coarse model with fine registration.

[0004] However, in the above-mentioned manner, it is not flexible to adapt to various three-dimensional attitudes of the imaging device, so that when the imaging device has dynamic changes in the three-dimensional attitude, the calculation of the ground projection area is not accurate. SUMMARY

[0005] Therefore, the present disclosure provides a projection area determination method and device, and an electronic device, to solve the problem of inaccurate projection deformation calculation caused by dynamic changes in the three-dimensional attitude in the prior art.

[0006] To achieve the above-mentioned purpose, the first aspect of the present disclosure provides a projection area determination method, which comprises: acquiring device parameters of an imaging device of an aircraft, wherein the device parameters comprise attitude angle parameters and field of view angle parameters; determining a direction vector corresponding to a viewing pyramid of the imaging device according to the field of view angle parameters, the direction vector being used to represent a boundary ray of the viewing pyramid, and the direction vector corresponding to a local coordinate system of the imaging device; constructing a three-dimensional rotation matrix according to the attitude angle parameters, and converting the direction vector from the local coordinate system to a world coordinate system based on the three-dimensional rotation matrix to obtain a target direction vector; determining position information of a projection intersection point of the boundary ray and the ground according to the vector value of the target direction vector, and determining the ground projection area of the imaging device according to the position information of the projection intersection point.

[0007] In an optional implementation, the method further comprises obtaining a position parameter of the aerial vehicle, the position parameter comprising a planar coordinate parameter and an altitude parameter, the target direction vector comprising an X-axis component, a Y-axis component and a Z-axis component of the boundary ray in the world coordinate system; and determining the position information of the intersection point of the projection of the boundary ray on the ground according to the vector value of the target direction vector comprises: calculating a parameter value of the boundary ray reaching the ground according to the Z-axis component in the target direction vector and the altitude parameter; determining a first coordinate value of the intersection point according to the parameter value, the X-axis component and a first coordinate in the planar coordinate parameter; determining a second coordinate value of the intersection point according to the parameter value, the Y-axis component and a second coordinate in the planar coordinate parameter; and determining the position information of the intersection point according to the first coordinate value and the second coordinate value.

[0008] In an optional implementation, the planar coordinate parameter is obtained by Mercator projection conversion on the latitude and longitude coordinate of the aerial vehicle, and the position information of the intersection point is determined according to the first coordinate value and the second coordinate value, comprising: performing inverse Mercator projection conversion on the first coordinate value and the second coordinate value to obtain the latitude and longitude coordinate of the intersection point; and determining the position information of the intersection point according to the latitude and longitude coordinate of the intersection point.

[0009] In an optional implementation, the device parameter further comprises an imaging scale parameter, and the direction vector corresponding to the viewing pyramid of the imaging device is determined according to the field of view angle parameter, comprising: determining a vertical half field of view angle and a horizontal half field of view angle of the imaging device according to the field of view angle parameter and the imaging scale parameter; and determining the direction vector corresponding to the viewing pyramid of the imaging device according to the vertical half field of view angle and the horizontal half field of view angle in the local coordinate system of the imaging device.

[0010] In an optional implementation, the imaging scale parameter is used to represent the aspect ratio of an imaging picture of the imaging device, and the vertical half field of view angle and the horizontal half field of view angle of the imaging device are determined according to the field of view angle parameter and the imaging scale parameter, comprising: determining the vertical half field of view angle according to the half value of the field of view angle parameter; and determining the horizontal half field of view angle according to the vertical half field of view angle and the aspect ratio.

[0011] In an optional implementation, the horizontal direction, the vertical direction and the shooting direction of the imaging device are respectively the X-axis direction, the Y-axis direction and the Z-axis direction of the local coordinate system; and the determining, according to the vertical half field of view and the horizontal half field of view, of the direction vector corresponding to the view pyramid of the imaging device in the local coordinate system of the imaging device includes: for any vertex of the view pyramid, determining a first component of the X-axis of the vertex in the local coordinate system according to the horizontal half field of view; determining a second component of the Y-axis of the vertex in the local coordinate system according to the vertical half field of view; determining a third component of the Z-axis of the vertex in the local coordinate system according to a preset component value; and determining the direction vector corresponding to the vertex according to the first component, the second component and the third component; and the direction vector corresponding to the view pyramid includes the direction vector corresponding to each vertex of the view pyramid.

[0012] In an optional implementation, the three-dimensional rotation matrix is an Euler angle rotation matrix, the attitude angle parameter includes a pitch angle parameter, an azimuth angle parameter and a roll angle parameter, and the constructing the three-dimensional rotation matrix according to the attitude angle parameter includes: constructing a first rotation matrix for rotation around the Z-axis of the world coordinate system according to the azimuth angle parameter; constructing a second rotation matrix for rotation around the Y-axis of the world coordinate system according to the pitch angle parameter; constructing a third rotation matrix for rotation around the X-axis of the world coordinate system according to the roll angle parameter; and multiplying the first rotation matrix, the second rotation matrix and the third rotation matrix in a preset rotation order to obtain the three-dimensional rotation matrix.

[0013] In an optional implementation, the number of the projection intersection points is multiple, and the determining the ground projection area of the imaging device according to the position information of the projection intersection points includes: sorting the multiple projection intersection points according to the position size relationship of the position information of the multiple projection intersection points to obtain an intersection point sequence, and adding a first projection intersection point in the intersection point sequence to the end of the intersection point sequence to make the first and the end of the intersection point sequence consistent; for each projection intersection point, constructing an intersection point entity of the projection intersection point according to the position information of the projection intersection point; and sequentially connecting the intersection point entities corresponding to adjacent two projection intersection points in the intersection point sequence to form a region boundary line, and determining a region formed by the region boundary line as the ground projection area of the imaging device.

[0014] In a second aspect, the present disclosure provides a projection area determination apparatus, comprising: a parameter acquisition module configured to acquire device parameters of an imaging device of an aerial vehicle, wherein the device parameters comprise attitude angle parameters and field of view angle parameters; a first determination module configured to determine a direction vector corresponding to a viewing pyramid of the imaging device according to the field of view angle parameters, wherein the direction vector is used to represent a boundary ray of the viewing pyramid, and the direction vector corresponds to a local coordinate system of the imaging device; a vector conversion module configured to construct a three-dimensional rotation matrix according to the attitude angle parameters, convert the direction vector from the local coordinate system to a world coordinate system based on the three-dimensional rotation matrix, and obtain a target direction vector; and a second determination module configured to determine position information of a projection intersection point of the boundary ray and a ground surface according to a vector value of the target direction vector, and determine a ground projection area of the imaging device according to the position information of the projection intersection point.

[0015] In a third aspect, the present disclosure provides an electronic device, comprising: one or more processors; a memory having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the projection area determination method described above; and one or more I / O interfaces connected between the processor and the memory, configured to realize information interaction between the processor and the memory.

[0016] The present disclosure has the following advantages:

[0017] The projection area determination method and determination apparatus provided by the embodiments of the present disclosure first acquire device parameters of an imaging device of an aerial vehicle, wherein the device parameters comprise attitude angle parameters and field of view angle parameters; secondly, a direction vector corresponding to a viewing pyramid of the imaging device is determined according to the field of view angle parameters, wherein the direction vector is used to represent a boundary ray of the viewing pyramid, and the direction vector corresponds to a local coordinate system of the imaging device; then, a three-dimensional rotation matrix is constructed according to the attitude angle parameters, the direction vector is converted from the local coordinate system to a world coordinate system based on the three-dimensional rotation matrix, and a target direction vector is obtained; finally, position information of a projection intersection point of the boundary ray and a ground surface is determined according to a vector value of the target direction vector, and a ground projection area of the imaging device is determined according to the position information of the projection intersection point.

[0018] It can be seen that the embodiment of the present disclosure can construct a three-dimensional rotation matrix through the attitude angle parameter of the imaging device of the aircraft, so that the three-dimensional rotation matrix can represent any attitude of the imaging device in the three-dimensional space. Therefore, the direction vector of the view pyramid of the imaging device is constructed through the view angle field parameter of the imaging device, and the direction vector is converted from the local coordinate system of the imaging device to the world coordinate system based on the three-dimensional rotation matrix to obtain a target direction vector, so that the ground projection area of the imaging device is determined according to the vector value of the target direction vector. The projection area determination method of the embodiment of the present disclosure can effectively adapt to various attitude changes of the imaging device of the aircraft, and improve the projection area calculation accuracy in the attitude change scene.

[0019] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent to those skilled in the art through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings are intended to provide a further understanding of the present disclosure and constitute a part of the specification, together with the embodiments of the present disclosure, to explain the present disclosure, and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art through the description of the detailed example embodiments by referring to the accompanying drawings, in which:

[0021] Figure 1 A flowchart of a projection area determination method provided by an embodiment of the present disclosure;

[0022] Figure 2 A flowchart of a projection area determination method provided by an embodiment of the present disclosure;

[0023] Figure 3 A block diagram of a projection area determination device provided by an embodiment of the present disclosure;

[0024] Figure 4 A block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure, and do not limit the present disclosure.

[0026] As used in the present disclosure, the term "and / or" includes any and all combinations of one or more related listed items.

[0027] The terminology used by the disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in the description of the disclosure, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0028] When the term "comprising" or "made of" is used in the disclosure, it specifies the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used in the disclosure are intended to have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the disclosure, and will not be interpreted in an overly literal or overly formal sense unless expressly so defined in the disclosure.

[0030] The projection area refers to the area range formed by the intersection of the viewing pyramid of the imaging device carried by the aircraft and the ground plane under a specific position and attitude. That is, the ground range that can be covered when the imaging device is photographed at this position.

[0031] In the related art, in order to determine the ground projection area of the imaging device of the aircraft, a simplified model is usually used to improve the calculation efficiency. For example, in the related art, it is assumed that the pitch angle and roll angle of the imaging device are close to 0°, and the yaw angle changes slowly, so that the three-dimensional space problem is simplified to an approximate two-dimensional geometric model (such as orthographic projection). In the ground processing stage, the geometric error is corrected based on high-precision image registration algorithm (such as SIFT, SURF, etc.). That is, the related art determines the ground projection area of the imaging device by combining the coarse model with the fine registration. However, in the above-mentioned manner, it is not flexible to adapt to various three-dimensional attitudes of the imaging device, so that when the three-dimensional attitude of the imaging device changes dynamically, the calculation of the ground projection area is not accurate.

[0032] For example, when the pitch angle deviates from 90°, trapezoidal deformation occurs, when the roll angle is not 0°, parallelogram deformation occurs, and when both exist, general quadrilateral deformation occurs. Even if the processing method assisted by Kalman filtering or GPS is used, the projection area determination method in the related art cannot effectively adapt to the projection deformation caused by the dynamic change of the three-dimensional attitude.

[0033] Therefore, in a complex task (such as a tilt photography surround flight scene), the imaging device needs to rotate around three axes in real time, and the related technology is difficult to accurately adapt to the projection deformation caused by the attitude change, resulting in deviation of the determined imaging range, which limits the application in dynamic scenes.

[0034] Therefore, the embodiments of the present disclosure provide a projection area determination method and a determination device. First, the device parameters of the imaging device of the aircraft are obtained, which include attitude angle parameters and field of view angle parameters. Second, the direction vector corresponding to the viewing pyramid of the imaging device is determined according to the field of view angle parameters. The direction vector is used to represent the boundary ray of the viewing pyramid, and the direction vector corresponds to the local coordinate system of the imaging device. Then, a three-dimensional rotation matrix is constructed according to the attitude angle parameters, and the direction vector is converted from the local coordinate system to the world coordinate system based on the three-dimensional rotation matrix to obtain a target direction vector. Finally, the position information of the intersection point of the boundary ray and the ground is determined according to the vector value of the target direction vector, and the ground projection area of the imaging device is determined according to the position information of the intersection point.

[0035] Therefore, the embodiments of the present disclosure can construct a three-dimensional rotation matrix through the attitude angle parameters of the imaging device of the aircraft, so that the three-dimensional rotation matrix can represent any attitude of the imaging device in three-dimensional space. Therefore, the direction vector of the viewing pyramid of the imaging device is constructed through the field of view angle parameters, and the direction vector is converted from the local coordinate system of the imaging device to the world coordinate system based on the three-dimensional rotation matrix to obtain a target direction vector. Therefore, the ground projection area of the imaging device is determined according to the vector value of the target direction vector. The projection area determination method of the embodiments of the present disclosure can effectively adapt to various attitude changes of the imaging device of the aircraft, and improve the projection area calculation accuracy in the attitude change scene.

[0036] In a first aspect, the embodiments of the present disclosure provide a projection area determination method. Figure 1 For the flowchart of the projection area determination method provided by the embodiments of the present disclosure, refer to Figure 1 The method comprises the following steps.

[0037] Step S110: Obtain the device parameters of the imaging device of the aircraft, wherein the device parameters include attitude angle parameters and field of view angle parameters.

[0038] The imaging device is mounted on the aircraft, and the imaging device can be a camera or the like. The embodiments of the present disclosure do not limit this.

[0039] The attitude angle parameters are used to represent the spatial attitude of the imaging device, and can include a pitch angle parameter pitch, a yaw angle parameter yaw, and a roll angle parameter roll.

[0040] The pitch parameter represents an angle between the optical axis of the imaging device and the horizontal plane, and the pitch parameter ranges from 0 to 180 degrees. When the pitch parameter is 90 degrees, the optical axis of the imaging device is perpendicular to the downward direction, and the orthographic shooting state is achieved. The yaw parameter represents an angle between the horizontal direction of the imaging device and the north direction, and the yaw parameter ranges from 0 to 360 degrees. When the yaw parameter is 0 degrees, the imaging device is directed to the north direction. The roll parameter is used to represent a rotation angle of the imaging device around the optical axis thereof, and the roll parameter ranges from 0 to 360 degrees.

[0041] It should be noted that after the pitch parameter, the yaw parameter, and the roll parameter are obtained, the above attitude angle parameters can be subjected to angle preprocessing, so as to be converted into radians.

[0042] For example, for the yaw parameter, the processed yaw yawRad obtained through the angle preprocessing is: .

[0043] For example, for the pitch parameter, the processed pitch pitchRad obtained through the angle preprocessing is: The above processing method can convert 90 degrees of vertical downward direction into 0 degrees of local coordinate system reference.

[0044] For example, for the roll parameter, the processed roll rollRad obtained through the angle preprocessing is: .

[0045] The field of view parameter can be a vertical field of view of the imaging device, and is used to represent a maximum angle range of the imaging device in the vertical direction for effectively capturing an image. For example, the field of view parameter is 30 degrees.

[0046] In an optional implementation, the device parameters of the imaging device can further include an imaging scale parameter of the imaging device, and the imaging scale parameter is used to represent an aspect ratio of an imaging picture of the imaging device. The imaging scale parameter can be determined by an aspect ratio of a sensor of the imaging device, an image resolution of the imaging device, and the like, and the embodiments of the present disclosure do not limit this.

[0047] In an optional implementation, in addition to obtaining the device parameters of the imaging device, a position parameter of the aircraft can also be obtained, and the position parameter can include a plane coordinate parameter and a height from the ground parameter height.

[0048] The plane coordinate parameter can be obtained by coordinate conversion on the latitude and longitude coordinates of the aircraft, for example, by Mercator projection to obtain the plane coordinate parameter. The height from the ground parameter height is used to represent a height of the aircraft from the ground, for example, the height from the ground parameter is 500 m.

[0049] Step S120: determining, according to the field of view angle parameter, a direction vector corresponding to the view pyramid of the imaging device, the direction vector being used to represent a boundary ray of the view pyramid, and the direction vector corresponding to a local coordinate system of the imaging device.

[0050] In the step of determining, according to the field of view angle parameter, the direction vector corresponding to the view pyramid of the imaging device, only unit direction vectors corresponding to four corner points of the view pyramid can be determined. The four corner points of the view pyramid refer to four vertices constituting the spatial range of the view pyramid. That is, the direction vector is a unit direction vector in the local coordinate system of the imaging device, pointing from the optical center of the imaging device to the four corner points of the view pyramid.

[0051] Thus, the direction vectors corresponding to the four corner points can represent four boundary rays of the view pyramid of the imaging device, and the direction vectors of the four corner points can define the boundary range of the field of view of the imaging device, facilitating determination of the ground projection area of the imaging device.

[0052] In an optional implementation, the vertical half field of view angle and the horizontal half field of view angle of the imaging device can be determined according to the field of view angle parameter, so as to determine, in the local coordinate system of the imaging device, the direction vector corresponding to the view pyramid of the imaging device according to the vertical half field of view angle and the horizontal half field of view angle.

[0053] In an optional implementation, in order to flexibly adapt to different imaging ratios of the imaging device for shooting tasks, an imaging ratio parameter of the imaging device can be obtained, so as to dynamically determine the vertical half field of view angle and the horizontal half field of view angle of the imaging device according to the imaging ratio parameter and the field of view angle parameter, and then determine the direction vector corresponding to the view pyramid of the imaging device according to the vertical half field of view angle and the horizontal half field of view angle.

[0054] Correspondingly, the device parameter further includes an imaging ratio parameter, and the step of determining, according to the field of view angle parameter, the direction vector corresponding to the view pyramid of the imaging device includes: determining, according to the field of view angle parameter and the imaging ratio parameter, the vertical half field of view angle and the horizontal half field of view angle of the imaging device; and determining, in the local coordinate system of the imaging device, the direction vector corresponding to the view pyramid of the imaging device according to the vertical half field of view angle and the horizontal half field of view angle.

[0055] The imaging ratio parameter can be determined by the aspect ratio of the sensor of the imaging device. For example, the aspect ratio of the sensor is imagingRatio[width, height], and the imaging ratio parameter aspect is the ratio of imagingRatio[1] to imagingRatio[0] in the aspect ratio.

[0056] The imaging ratio parameter can also be determined by a physical size of a sensor of the imaging device. For example, the imaging ratio parameter aspect is a ratio of a physical height value to a physical width value of the sensor.

[0057] The imaging ratio parameter can also be determined according to an image resolution of the imaging device. For example, the resolution of the input image is width in horizontal pixels and height in vertical pixels, and the imaging ratio parameter aspect is determined according to a ratio of the height in vertical pixels to the width in horizontal pixels.

[0058] In an optional implementation, the imaging ratio parameter is used to represent an aspect ratio of an imaging view of the imaging device; and the vertical half field of view and the horizontal half field of view of the imaging device are determined according to the field of view parameter and the imaging ratio parameter, including: determining the vertical half field of view according to a half value of the field of view parameter; and determining the horizontal half field of view according to the vertical half field of view and the aspect ratio.

[0059] For example, the vertical half field of view verticalHalf, the horizontal half field of view horizontalHalf, and the direction vector of the view pyramid are determined according to the field of view parameter fov, the imaging ratio parameter aspect, and the half value of the field of view parameter. The direction vector of the view pyramid can be determined by the following formula:

[0060] Formula (1)

[0061] Formula (2)

[0062] Formula (3)

[0063] wherein fov is the field of view parameter, which is used to represent a vertical field of view of the imaging device; imagingRatio is a width-height array of a sensor of the imaging device, is a width value of the sensor, is a height value of the sensor, and aspect is the imaging ratio parameter, which is used to represent an aspect ratio of an imaging view of the imaging device.

[0064] After the horizontal half field of view and the vertical half field of view are determined, the direction vector of the view pyramid of the imaging device can be determined according to the horizontal half field of view and the vertical half field of view.

[0065] In an optional implementation, the direction vector of each corner point of the view pyramid can be determined according to the horizontal half field of view and the vertical half field of view in a local coordinate system of the imaging device.

[0066] Correspondingly, the horizontal direction, the vertical direction and the shooting direction of the imaging device are respectively the X-axis direction, the Y-axis direction and the Z-axis direction of the local coordinate system; in the local coordinate system of the imaging device, according to the vertical half field of view angle and the horizontal half field of view angle, a direction vector corresponding to a viewing pyramid of the imaging device is determined, including: for any vertex of the viewing pyramid, according to the horizontal half field of view angle, a first component of the vertex in the X-axis of the local coordinate system is determined; according to the vertical half field of view angle, a second component of the vertex in the Y-axis of the local coordinate system is determined; according to a preset component value, a third component of the vertex in the Z-axis of the local coordinate system is determined; according to the first component, the second component and the third component, a direction vector corresponding to the vertex is determined; wherein the direction vector corresponding to the viewing pyramid includes the direction vector corresponding to each vertex of the viewing pyramid.

[0067] Wherein the direction vector of each vertex of the viewing pyramid is [first component, second component, third component].

[0068] Wherein the first component and the second component can be determined according to the tangent values of the horizontal half field of view angle and the vertical half field of view angle. The third component can be determined according to a preset component value, such as a preset component value of -1.

[0069] Exemplarily, in the local coordinate system of the imaging device (the Z-axis of the local coordinate system points to the shooting direction of the imaging device, the X-axis is the horizontal direction, and the Y-axis is the vertical direction), the direction vectors of the four vertices of the viewing pyramid generated according to the horizontal half field of view angle and the vertical half field of view angle are:

[0070] The direction vector of the left upper vertex:

[0071] The direction vector of the right upper vertex:

[0072] The direction vector of the right lower vertex:

[0073] The direction vector of the left lower vertex:

[0074] Wherein verticalHalf is the vertical half field of view angle, is the horizontal half field of view angle. It can be seen that the direction vectors of the above four vertices define four boundary rays of the viewing pyramid of the imaging device.

[0075] In addition, it should be noted that the boundary of the viewing pyramid of the imaging device can also be parameterized, that is, N points are uniformly sampled along the boundary of the viewing pyramid, and a direction vector corresponding to each sampling point is determined, and the embodiments of the present disclosure do not limit this.

[0076] Step S130: constructing a three-dimensional rotation matrix according to the attitude angle parameter, and converting the direction vector from the local coordinate system to the world coordinate system based on the three-dimensional rotation matrix to obtain a target direction vector.

[0077] In the formula (1), the three-dimensional rotation matrix can be an Euler angle rotation matrix or a three-dimensional rotation matrix based on a quaternion representation, and the present disclosure does not limit this.

[0078] In the formula (1), the product of the direction vector and the three-dimensional rotation matrix can be taken as the target direction vector. The target direction vector is a conversion vector obtained by converting the direction vector from the local coordinate system of the imaging device to the world coordinate system. It should be noted that the direction vector and the target direction vector are only different in the corresponding coordinate systems, and both are used to represent the same boundary ray of the viewing frustum of the imaging device.

[0079] In an optional implementation, the three-dimensional rotation matrix is an Euler angle rotation matrix, and the attitude angle parameter includes a pitch angle parameter, an azimuth angle parameter and a roll angle parameter. The three-dimensional rotation matrix is constructed according to the attitude angle parameter, including: constructing a first rotation matrix rotating around the Z axis of the world coordinate system according to the azimuth angle parameter; constructing a second rotation matrix rotating around the Y axis of the world coordinate system according to the pitch angle parameter; constructing a third rotation matrix rotating around the X axis of the world coordinate system according to the roll angle parameter; and multiplying the first rotation matrix, the second rotation matrix and the third rotation matrix in a preset rotation order to obtain the three-dimensional rotation matrix.

[0080] For example, the first rotation matrix rotating around the Z axis of the world coordinate system constructed based on the azimuth angle parameter yaw is

[0081] Formula (4)

[0082] For example, the second rotation matrix rotating around the Y axis of the world coordinate system constructed based on the pitch angle parameter pitch is

[0083] Formula (5)

[0084] For example, the third rotation matrix rotating around the X axis of the world coordinate system constructed based on the roll angle parameter roll is

[0085] Formula (6)

[0086] Therefore, after multiplying the first rotation matrix, the second rotation matrix and the third rotation matrix in the preset rotation order, the final three-dimensional rotation matrix R can be obtained For example, the three-dimensional rotation matrix R obtained by matrix multiplication is:​​​

[0087] Formula (7)

[0088] wherein the preset rotation order can be adaptively set according to actual application needs, and the embodiments of the present disclosure do not limit this. For example, the preset rotation order is ZYX order, XYZ order, ZXY order, etc. Exemplarily, in actual application, according to the Euler angle defined by the flight control system or the sensor output, the corresponding rotation order is selected as the preset rotation order, so that the three-dimensional rotation matrix constructed can accurately represent the three-dimensional attitude of the imaging device.

[0089] After the three-dimensional rotation matrix is constructed, the direction vector can be converted from the local coordinate system to the world coordinate system based on the three-dimensional rotation matrix, so as to obtain a target direction vector.

[0090] Exemplarily, for the direction vector of any vertex of the frustum , the target direction vector converted based on the three-dimensional rotation matrix is:

[0091] Formula (8)

[0092] wherein, is the target direction vector, is the three-dimensional rotation matrix. Specifically, the calculation process of the above target direction vector can refer to the following formula:

[0093] Formula (9)

[0094] Formula (10)

[0095] Formula (11)

[0096] The related technology adopts a simplified model processing method, which can only process the direction change in the horizontal plane, so that the component of the optical axis of the imaging device in the vertical direction cannot be changed. For example, when the pitch angle deviates from 90° or the roll angle deviates from 0°, the simplified model will deviate significantly from the real physical state.

[0097] The embodiment of the present disclosure converts the direction vector by constructing an Euler angle rotation matrix, so that even if the pitch angle and the roll angle are offset, the projection component of the target direction vector of each corner point of the frustum of the imaging device in the Z direction will also adaptively change. Therefore, the embodiment of the present disclosure can effectively adapt to the three-dimensional attitude change of the imaging device, and can accurately determine the ground projection area of the imaging device under complex tasks such as tilt photography (for example, the pitch angle is 30° to 60°), surround flight (for example, the yaw angle continuously changes and the roll angle is not 0), building facade shooting (for example, the pitch angle is less than 45°), and the like.

[0098] In addition, in the embodiment of the present disclosure, the frustum direction inherent to the imaging device is converted into a real shooting direction considering the actual attitude of the aircraft. The coupling effect of the three Euler angles is encoded by the three-dimensional rotation matrix, so that complex three-dimensional space transformation can be completed by only one matrix multiplication. The above processing method replaces the step-by-step approximate calculation, and avoids the error accumulation phenomenon caused by multiple coordinate system conversions. Moreover, the processing method based on the three-dimensional rotation matrix in the embodiment of the present disclosure can comply with the mathematical description of rigid body kinematics, that is, the accuracy of the physical model is ensured, and the additional attitude estimation correction process in the related art is avoided.

[0099] In an optional implementation, the above-mentioned three-dimensional rotation matrix can also be represented by a quaternion. For example, the quaternion is wherein the attitude angle parameter can be an Euler angle parameter, or can be directly an attitude parameter in the form of a quaternion, and the embodiment of the present disclosure does not limit this.

[0100] For example, the attitude angle parameter includes a pitch angle parameter, an azimuth angle parameter, and a roll angle parameter. Then, the quaternion can be constructed according to the azimuth angle parameter yaw, the pitch angle parameter pitch, and the roll angle parameter roll, and the direction vector can be converted into the target direction vector according to the quaternion. For details, refer to the following formula for constructing the quaternion:

[0101] Formula (12)

[0102] Formula (13)

[0103] Formula (14)

[0104] Formula (15)

[0105] In addition, if the attitude angle parameter is attitude data in the form of a quaternion, the quaternion can also be directly constructed according to the attitude data. For example, the quaternion is constructed by the following formula:

[0106] Equation (16)

[0107] wherein, is a rotation angle, is a unit vector of the rotation axis.

[0108] Further, after constructing the quaternion, the direction vector can be converted into the target direction vector according to the following equation:

[0109] Equation (17)

[0110] wherein, is a target direction vector, is a quaternion, is a conjugate of the quaternion, is a quaternion multiplication.

[0111] In the embodiments of the present disclosure, the quaternion can represent a three-dimensional rotation transformation, has a high calculation efficiency, can effectively avoid the gimbal lock problem, and has good interpolation smoothness, and can effectively adapt to dynamic attitude smooth transition scenarios, such as smoothly rotating the imaging device from 0° to 360°.

[0112] With reference to Figure 1 , after converting the direction vector into the target direction vector according to the three-dimensional conversion matrix, step S140 can be performed.

[0113] Step S140: determining the position information of the projection intersection point of the boundary ray and the ground according to the vector value of the target direction vector, and determining the ground projection area of the imaging device according to the position information of the projection intersection point.

[0114] In this step, for each boundary ray starting from the imaging device, the projection intersection point of the ray and the ground (i.e., Z=0) can be calculated respectively. Specifically, the position information of the projection intersection point can be determined by constructing a ray parameter equation.

[0115] After determining the position information of each projection intersection point, such as the position information of the projection intersection point of the boundary ray of the four corner points of the frustum and the ground, the projection intersection points can be enclosed to form a ground projection area according to the position size relationship of the position information of each projection intersection point. For example, the ground projection area is a rectangle, a parallelogram, etc., and the embodiments of the present disclosure are not limited thereto.

[0116] The embodiment of the present disclosure provides a method for determining a projection area, which can construct a three-dimensional rotation matrix through an attitude angle parameter of an imaging device of a flying vehicle, so that the three-dimensional rotation matrix can represent an arbitrary attitude of the imaging device in a three-dimensional space. Thus, a direction vector of a viewing pyramid of the imaging device is constructed through a visual angle field parameter of the imaging device, and the direction vector is converted from a local coordinate system of the imaging device to a world coordinate system based on the three-dimensional rotation matrix to obtain a target direction vector, so that a ground projection area of the imaging device is determined according to a vector value of the target direction vector. The projection area determination method of the embodiment of the present disclosure can effectively adapt to various attitude changes of the imaging device of the flying vehicle, and improve the projection area calculation accuracy in the attitude change scene.

[0117] In an optional implementation, for each boundary ray, a corresponding ray parameter equation can be constructed, so that the position information of the projection intersection point of the boundary ray and the ground is determined according to the ray parameter equation.

[0118] Correspondingly, in the embodiment of the present disclosure, the position parameter of the flying vehicle can also be obtained, the position parameter including a plane coordinate parameter and a height-from-ground parameter, and the target direction vector including an X-axis component, a Y-axis component and a Z-axis component of the boundary ray in the world coordinate system. The position information of the projection intersection point of the boundary ray and the ground is determined according to the vector value of the target direction vector, including: calculating a parameter value of the boundary ray reaching the ground according to the Z-axis component in the target direction vector and the height-from-ground parameter; determining a first coordinate value of the projection intersection point according to the parameter value, the X-axis component and a first coordinate in the plane coordinate parameter; determining a second coordinate value of the projection intersection point according to the parameter value, the Y-axis component and a second coordinate in the plane coordinate parameter; and determining the position information of the projection intersection point according to the first coordinate value and the second coordinate value.

[0119] For example, according to the plane coordinate parameter of the flying vehicle , the target direction vector is , and the height-from-ground parameter of the flying vehicle is heightMeters, the ray parameter equation is constructed as follows:

[0120] Formula (18)

[0121] Where t is the parameter value of the boundary ray reaching the ground, that is, the elongation multiple required for the imaging device to reach the ground along the target direction vector.

[0122] Further, let z in the above formula (18) be 0, so that the relationship between the Z-axis component in the target direction vector, the height-from-ground parameter and the parameter value of the boundary ray reaching the ground is:

[0123] Formula (19)

[0124] Therefore, based on the above formula (19), the parameter value of the boundary ray reaching the ground can be calculated based on the Z-axis component in the target direction vector and the ground height parameter.

[0125] Furthermore, based on the above formulas (18) and (19), the first coordinate value and the second coordinate value of the projection intersection point can be obtained as follows:

[0126] Formula (20)

[0127] Formula (21)

[0128] in, The first coordinate value of the projection intersection point. This is the second coordinate value of the projection intersection point. It can be determined according to the above formula (20), based on the parameter value t and the X-axis components. and the first coordinate in the plane coordinate parameters Determine the first coordinate value of the projection intersection point. According to the above formula (21), based on the parameter value t and the Y-axis component... and the second coordinate in the plane coordinate parameters Determine the first coordinate value of the projection intersection point. .

[0129] Since the first and second coordinate values ​​of the aforementioned projection intersection point are planar coordinates, they can be further converted into latitude and longitude coordinates to obtain the location information of the projection intersection point.

[0130] In related technologies, simplified models typically use fixed geometric projection formulas to determine projection corner points. However, when the camera tilts or rotates, the projection area becomes trapezoidal or parallelogram-shaped instead of rectangular, making it impossible to accurately determine the projection area using the above methods.

[0131] In this embodiment, by employing ray tracing, the projection intersection point of each boundary ray of the visual frustum with the ground is calculated independently, thereby enabling flexible handling of projection deformation caused by attitude changes. For example, when the pitch angle parameter is 60°, the projection distances of the four corner points of the visual frustum are no longer equal, and the projection distance at its far end increases significantly. This embodiment, however, can flexibly calculate the parameter value (t-value) of the boundary ray reaching the ground for each corner point. This allows for precise determination of deformation differences, thereby improving the accuracy of determining the ground projection area.

[0132] In an optional implementation, the planar coordinate parameter is obtained by Mercator projection conversion on the longitude and latitude coordinates of the aircraft, and the position information of the projection intersection point is determined according to the first coordinate value and the second coordinate value, including: performing inverse Mercator projection conversion on the first coordinate value and the second coordinate value to obtain the longitude and latitude coordinates of the projection intersection point; and determining the position information of the projection intersection point according to the longitude and latitude coordinates of the projection intersection point.

[0133] For example, for the longitude and latitude coordinates (lng, lat) of the aircraft, the Web Mercator projection can be used to convert them into the planar coordinate parameter (x, y), which can be specifically referred to the following formula:

[0134] Formula (22)

[0135] Formula (23)

[0136] wherein, lng is the longitude of the aircraft, lat is the latitude of the aircraft, and R is the radius of the earth. In a wide range of flight tasks (such as a flight task across an area of tens of kilometers), cumulative errors will be caused by the curvature of the earth. The embodiment of the present disclosure converts the longitude and latitude coordinates of the aircraft by Mercator projection, thereby reducing the calculation errors caused by the curvature of the earth.

[0137] Further, by performing inverse Mercator projection conversion on the first coordinate value x0 of the projection intersection point and the second coordinate value y0 of the projection intersection point, the longitude and latitude coordinates (lng0, lat0) of the projection intersection point can be obtained, for example, based on the Web Mercator inverse projection, the longitude and latitude coordinates of the projection intersection point can be obtained according to the following formula:

[0138]

[0139] Formula (25)

[0140] wherein, lng0 is the longitude of the projection intersection point, lat0 is the latitude of the projection intersection point, and R is the radius of the earth.

[0141] In addition, the conversion between the longitude and latitude coordinates and the planar coordinate parameter can also be realized based on the transverse Mercator projection, which is not limited in the embodiment of the present disclosure.

[0142] Therefore, the longitude and latitude coordinates of the projection intersection point obtained by inverse Mercator projection conversion can be taken as the position information of the projection intersection point, and then the ground projection area can be determined according to the position information of the projection intersection point.

[0143] ​​​​In an optional implementation, the number of the projection intersection points is multiple, and the ground projection area of the imaging device is determined according to the position information of the projection intersection points, including: according to the position size relationship of the position information of the multiple projection intersection points, the multiple projection intersection points are sorted to obtain an intersection point sequence, and the first projection intersection point in the intersection point sequence is added to the end of the intersection point sequence to make the first and the last of the intersection point sequence consistent; for each projection intersection point, the intersection entity of the projection intersection point is constructed according to the position information of the projection intersection point; the intersection entities corresponding to adjacent two projection intersection points in the intersection point sequence are sequentially connected to form a region boundary line, and the region formed by the region boundary line is determined as the ground projection area of the imaging device.

[0144] For example, for the four boundary rays corresponding to the four corner points of the imaging device's frustum, the four boundary rays and the four projection intersection points on the ground are sequentially arranged according to the position size relationship of the position information, and the first projection intersection point in the intersection point sequence is added to the end of the sequence, so that the adjacent projection intersection points in the sequence can be connected to form a closed polygon.

[0145] For example, the intersection point sequence is {left upper intersection point, right upper intersection point, right lower intersection point, left lower intersection point, closed point (the same as the first projection intersection point)}.

[0146] For example, for the above intersection point sequence, the output coordinate string can be .

[0147] Correspondingly, for any projection intersection point, the intersection entity of the projection intersection point can be drawn in the visualization scene (such as a geographic information system, a Web map, etc.) corresponding to the ground projection area according to the position information of the projection intersection point. Thus, the intersection entities of the adjacent two projection intersection points in the intersection point sequence are connected to form a region boundary line. The region surrounded by the region boundary lines is the ground projection area of the imaging device.

[0148] In an optional implementation, in addition to constructing the ray parameter equation, the position information of the projection intersection point can also be determined by constructing a perspective projection matrix, constructing a homogeneous coordinate based on the vector value of the target direction vector, and then applying the perspective projection matrix on the homogeneous coordinate.

[0149] For example, by constructing a perspective projection matrix P, a point in a three-dimensional space is projected to a ground plane to obtain a projection intersection point of a boundary ray corresponding to a target direction vector, and the specific formula is as follows:

[0150] Formula (26)

[0151] wherein, T is a translation matrix (encoding the position of the imaging device), is a rotation matrix, and h is the height of the imaging device to the ground. Correspondingly, the homogeneous coordinates of the target direction vector are constructed as The projection matrix is applied to obtain

[0152] Formula (27)

[0153] Correspondingly, the coordinates of the projection intersection point are obtained as In addition, the internal and external parameters of the imaging device can be encoded into the projection matrix, so that the coordinates of the projection intersection point are calculated through one matrix multiplication, which is not limited in the embodiments of the present disclosure.

[0154] In the embodiments of the present disclosure, the perspective projection matrix is constructed to determine the projection intersection point, which can be directly connected to the computer graphics rendering pipeline, and is convenient for GPU hardware acceleration. It can be applied to flight simulators that need real-time three-dimensional visualization, and application scenarios integrated with game engines such as Unity and Unreal Engine.

[0155] In an optional implementation, the direction vector and the position vector in the local coordinate system of the imaging device can be respectively transformed based on the homogeneous transformation matrix, and then the transformed vectors are used to determine the projection intersection point of the boundary ray and the ground.

[0156] For example, a 4x4 homogeneous transformation matrix H can be used to represent the combination of rotation and translation, as shown in the following formula:

[0157] Formula (28)

[0158] wherein, is used to represent the position of the imaging device. Thus, the direction vector (direction without translation) and the position vector are respectively transformed based on the above homogeneous transformation matrix, and then the projection intersection point of the boundary ray and the ground is calculated according to the transformed vectors.

[0159] In order to facilitate understanding, the specific implementation details of the above embodiments are described below with a specific example:

[0160] In the related art, the ground projection area of the imaging device is usually determined based on a simplified model, but the above method cannot guarantee the calculation accuracy of the ground projection area, and the specific limitations are as follows:

[0161] (1) Unable to effectively deal with the projection deformation caused by the dynamic change of three-dimensional attitude. When the imaging device such as the camera rotates around the three axes, the projection of its visual pyramid on the ground will change from a regular rectangle to a trapezoid or a parallelogram. The above method lacks the ability to calculate high-precision three-dimensional attitude parameters in real time, so it is difficult to accurately represent the projection deformation, resulting in estimation deviation of the ground projection area, which limits its application in dynamic scenes.

[0162] (2) Unable to flexibly adapt to the shooting tasks of different imaging ratios of the imaging device, resulting in a large error in the calculation of the ground projection area.

[0163] (3) Unable to differentiate the corner points of the visual pyramid of the imaging device, so that non-rectangular deformation cannot be modeled, affecting the accuracy of the ground projection area calculation.

[0164] (4) Using latitude and longitude spherical coordinates to calculate the straight-line distance will ignore the curvature of the earth, and when the aircraft performs a super-long distance shooting task, it will produce an error in the calculation of the ground projection area.

[0165] Therefore, the embodiments of the present disclosure provide a projection area determination method, which aims to solve the problems of inaccurate projection deformation calculation caused by the dynamic change of three-dimensional attitude, the inability to adapt to each imaging ratio of the imaging device, and the like in the related art. By constructing a three-dimensional Euler angle rotation matrix (Yaw-Pitch-Roll) and introducing a dynamic imaging scale parameter, the accurate calculation of the ground projection area of the imaging device of the aircraft under any three-dimensional attitude is realized.

[0166] Figure 2 A flowchart of a projection area determination method provided by the embodiments of the present disclosure is shown, referring to Figure 2 The method comprises the following steps.

[0167] Step S200: Obtain the device parameters of the imaging device of the aircraft and the position parameters of the aircraft, wherein the device parameters include attitude angle parameters and field of view angle parameters, and the position parameters include latitude and longitude coordinates and height above ground parameters.

[0168] Step S201: Verify whether the parameters are valid. If the parameters are valid, execute step S202; if the parameters are invalid, execute step S211.

[0169] In this step, the validity of each parameter can be verified. For example, verify whether the latitude and longitude coordinates are within a preset range. For example, the longitude coordinate should be between -180° and 180°, and the latitude coordinate should be between -90° and 90°. For example, verify whether the imaging scale parameter is a positive number, verify the parameter integrity of each parameter, and the like. The embodiments of the present disclosure do not limit this.

[0170] Step S202: Perform Web Mercator projection on the latitude and longitude coordinates to obtain the plane coordinate parameters.

[0171] Step S203: Determine the vertical half-field angle and horizontal half-field angle of the imaging device based on the field of view parameters and imaging scale parameters.

[0172] Step S204: In the local coordinate system of the imaging device, determine the direction vectors corresponding to the four corner points of the imaging device's field of view based on the vertical half-field angle and the horizontal half-field angle.

[0173] Step S205: Construct a three-dimensional rotation matrix based on the pitch angle parameters, azimuth angle parameters, and roll angle parameters.

[0174] Step S206: Based on the three-dimensional rotation matrix, the direction vectors corresponding to the four corner points are transformed from the local coordinate system to the world coordinate system to obtain the target direction vectors corresponding to the four corner points.

[0175] Step S207: Determine the plane coordinates of the intersection point of the projection of the boundary ray and the ground based on the vector values ​​of the target direction vectors corresponding to the four corner points, the plane coordinate parameters, and the ground clearance parameters.

[0176] This step can also include security checks, such as checking the vector value z of the target direction vector. w Check if the value is less than zero, meaning the boundary ray must point downwards. Also, verify the parameter values.

[0177] Step S208: Perform Web Mercator inverse projection on the planar coordinates of the projection intersection points to obtain the position information of the projection intersection points.

[0178] Step S209: Verify whether the position information of the projection intersection is valid. If the position information is valid, proceed to step S210. If the position information is invalid, proceed to step S211.

[0179] This can be achieved by verifying whether the latitude and longitude coordinates of the projection intersection point are within a preset range. For example, the longitude coordinates of the projection intersection point must be between -180° and 180°, and the latitude coordinates must be between -90° and 90°. Alternatively, it can verify whether the latitude and longitude coordinates of the projection intersection point are empty values, etc., but this embodiment does not impose any limitations on this.

[0180] Step S210: Determine the ground projection area of ​​the imaging device based on the location information of the projection intersection point.

[0181] For example, when the image acquisition is performed for a certain position based on the tilt photography, the parameters are as follows: the height is 100 meters, the pitch is 60° (i.e., the tilt is 30°), the yaw is 45°, the roll is 0°, the field of view (fov) of the imaging device is 30°, and the imaging ratio is 16:9. Thus, the calculation result of the ground projection area for the above parameters is as follows: the ground projection area is a trapezoid, the proximal width is about 40 meters, the distal width is about 40 meters, the proximal distance is about 58 meters, and the distal distance is about 100 meters.

[0182] When the image acquisition is performed for the above position based on the surround flight, the parameters are as follows: the height is 20 meters, the pitch is 60° (i.e., the tilt is 30°), the yaw is 90°, the roll is 15° (i.e., the body is laterally tilted), the fov of the imaging device is 40°, and the imaging ratio is 4:3. Thus, the calculation result of the ground projection area for the above parameters is as follows: the ground projection area is a parallelogram.

[0183] Therefore, the disclosed embodiments can effectively adapt to the three-dimensional attitude change of the imaging device when determining the ground projection area, and improve the determination accuracy of the ground projection area.

[0184] Step S211: performing abnormality processing and error feedback.

[0185] In this step, the error can be captured and the detailed information can be recorded, and the specific error reason (e.g., the Z component in the direction vector of the imaging device must be negative) can be fed back, thereby ensuring the robustness of the system under the boundary condition.

[0186] In addition, the scheme in the disclosed embodiments can also be used for attitude verification of the imaging device. For example, after the attitude of the imaging device is calculated, the scheme in the disclosed embodiments is used to determine the ground projection area based on the attitude, so that the attitude calculation of the imaging device is determined to be correct when the ground projection area matches the target area of the imaging device.

[0187] The determination method of the projection area provided by the disclosed embodiments has the following advantages:

[0188] (1) Compared with the attitude simplification assumption in the related art, the disclosed embodiments can realize the projection calculation of the complete three-dimensional attitude. Specifically, the disclosed embodiments construct a complete three-dimensional Euler angle rotation matrix, which can realize accurate projection area calculation under complex tasks such as tilt photography, surround flight, and building facade shooting.

[0189] And, the embodiment of the present disclosure replaces the step-by-step approximate calculation with a one-time transformation of matrix algebra, avoiding the cumulative error introduced by each angle simplification. In addition, the rotation matrix method of the embodiment of the present disclosure conforms to the mathematical description of rigid body kinematics, avoiding the additional attitude estimation correction process in the related art, and improving the calculation efficiency.

[0190] (2) The embodiment of the present disclosure realizes cross-device universal calculation by introducing an imaging scale parameter. In the related art, the field of view angle parameter is usually directly regarded as the horizontal field of view angle or the vertical field of view angle, which implicitly assumes that the sensor is square. However, in fact, there are various scales of sensors of imaging devices: such as 4:3 sensors, 16:9 sensors, and 3:2 sensors. If a fixed scale is used for calculation, calculation errors will be introduced. Such errors will directly affect the flight path interval setting in the flight task planning. The embodiment of the present disclosure can adapt to imaging devices with different imaging scales by obtaining dynamic imaging scale parameters, reduce the redundancy rate of flight path planning, and provide more accurate initial registration parameters for subsequent calculation of the ground projection area, thereby improving the calculation accuracy of the ground projection area.

[0191] (3) The embodiment of the present disclosure uses an independent ray intersection method to accurately model the projection deformation. In the related art, a unified geometric formula is usually used to calculate the projection area. Under the above calculation method, the projection distances of the four corner points of the viewing pyramid to the imaging device are equal, that is, the determined ground projection is only a regular rectangle, which cannot model various projection deformations. The embodiment of the present disclosure independently establishes a parameter equation for each boundary ray of the viewing pyramid, respectively calculates the intersection points of each ray and the ground plane, and thereby through independently solving the parameter values of each ray, the Z-direction component difference caused by the attitude change can be reflected to the projection distance, without manually deriving complex geometric correction formulas, thereby accurately modeling various projection shapes (such as trapezoids, matrices, parallelograms, and general quadrilaterals).

[0192] (4) The embodiment of the present disclosure performs coordinate transformation and geometric operation based on the Web Mercator projection coordinate system, avoiding the calculation error caused by the curvature of the earth, and improving the calculation accuracy of the subsequent ground projection area.

[0193] The step division of the above various methods is only for clear description, and can be combined into one step or split into multiple steps in implementation, as long as the same logical relationship is included, which is within the protection scope of the present patent; adding insignificant modifications or introducing insignificant designs in the algorithm or flow, but not changing the core design of the algorithm and flow, are within the protection scope of the present patent.

[0194] In a second aspect, the embodiment of the present disclosure provides a projection area determination device, Figure 3A block diagram of a projection area determination apparatus is provided for embodiments of the present disclosure, referring to Figure 3 The apparatus comprises:

[0195] A parameter acquisition module 31 is configured to acquire device parameters of an imaging device of an aerial vehicle, wherein the device parameters comprise attitude angle parameters and field of view angle parameters.

[0196] A first determination module 32 is configured to determine a direction vector corresponding to a viewing pyramid of the imaging device according to the field of view angle parameters, wherein the direction vector is used to represent a boundary ray of the viewing pyramid, and the direction vector corresponds to a local coordinate system of the imaging device.

[0197] A vector conversion module 33 is configured to construct a three-dimensional rotation matrix according to the attitude angle parameters, and convert the direction vector from the local coordinate system to a world coordinate system based on the three-dimensional rotation matrix to obtain a target direction vector.

[0198] A second determination module 34 is configured to determine position information of a projection intersection point of the boundary ray and the ground surface according to a vector value of the target direction vector, and determine a ground projection area of the imaging device according to the position information of the projection intersection point.

[0199] In an optional implementation, the method further comprises acquiring position parameters of the aerial vehicle, wherein the position parameters comprise plane coordinate parameters and a height-from-ground parameter, and the target direction vector comprises an X-axis component, a Y-axis component and a Z-axis component of the boundary ray in the world coordinate system; and the determining the position information of the projection intersection point of the boundary ray and the ground surface according to the vector value of the target direction vector comprises: calculating a parameter value of the boundary ray reaching the ground surface according to the Z-axis component of the target direction vector and the height-from-ground parameter; determining a first coordinate value of the projection intersection point according to the parameter value, the X-axis component and a first coordinate in the plane coordinate parameters; determining a second coordinate value of the projection intersection point according to the parameter value, the Y-axis component and a second coordinate in the plane coordinate parameters; and determining the position information of the projection intersection point according to the first coordinate value and the second coordinate value.

[0200] In an optional implementation, the plane coordinate parameters are obtained by Mercator projection conversion on latitude and longitude coordinates of the aerial vehicle, and the determining the position information of the projection intersection point according to the first coordinate value and the second coordinate value comprises: performing Mercator inverse projection conversion on the first coordinate value and the second coordinate value to obtain latitude and longitude coordinates of the projection intersection point; and determining the position information of the projection intersection point according to the latitude and longitude coordinates of the projection intersection point.

[0201] In an optional implementation manner, the device parameter further comprises an imaging scale parameter, and the determining the direction vector corresponding to the view pyramid of the imaging device according to the field of view angle parameter comprises: determining a vertical half field of view angle and a horizontal half field of view angle of the imaging device according to the field of view angle parameter and the imaging scale parameter; and determining the direction vector corresponding to the view pyramid of the imaging device according to the vertical half field of view angle and the horizontal half field of view angle in the local coordinate system of the imaging device.

[0202] In an optional implementation manner, the imaging scale parameter is used to represent an aspect ratio of an imaging picture of the imaging device, and the determining the vertical half field of view angle and the horizontal half field of view angle of the imaging device according to the field of view angle parameter and the imaging scale parameter comprises: determining the vertical half field of view angle according to a half value of the field of view angle parameter; and determining the horizontal half field of view angle according to the vertical half field of view angle and the aspect ratio.

[0203] In an optional implementation manner, the horizontal direction, the vertical direction and the shooting direction of the imaging device are respectively an X-axis direction, a Y-axis direction and a Z-axis direction of the local coordinate system, and the determining the direction vector corresponding to the view pyramid of the imaging device according to the vertical half field of view angle and the horizontal half field of view angle in the local coordinate system of the imaging device comprises: for any vertex of the view pyramid, determining a first component of the X-axis of the vertex in the local coordinate system according to the horizontal half field of view angle; determining a second component of the Y-axis of the vertex in the local coordinate system according to the vertical half field of view angle; determining a third component of the Z-axis of the vertex in the local coordinate system according to a preset component value; and determining a direction vector corresponding to the vertex according to the first component, the second component and the third component; wherein the direction vector corresponding to the view pyramid comprises the direction vector corresponding to each vertex of the view pyramid.

[0204] In an optional implementation manner, the three-dimensional rotation matrix is an Euler angle rotation matrix, the attitude angle parameter comprises a pitch angle parameter, an azimuth angle parameter and a roll angle parameter, and the constructing the three-dimensional rotation matrix according to the attitude angle parameter comprises: constructing a first rotation matrix of rotation around a Z-axis of a world coordinate system according to the azimuth angle parameter; constructing a second rotation matrix of rotation around a Y-axis of the world coordinate system according to the pitch angle parameter; constructing a third rotation matrix of rotation around an X-axis of the world coordinate system according to the roll angle parameter; and multiplying the first rotation matrix, the second rotation matrix and the third rotation matrix in a preset rotation order to obtain the three-dimensional rotation matrix.

[0205] In an optional implementation, the number of the projection intersection points is multiple, and the determining the ground projection area of the imaging device according to the position information of the projection intersection points comprises: sorting the multiple projection intersection points according to the position size relationship of the position information of the multiple projection intersection points to obtain an intersection point sequence, and adding a first projection intersection point in the intersection point sequence to the end of the intersection point sequence to make the first and the end of the intersection point sequence consistent; for each projection intersection point, constructing an intersection point entity of the projection intersection point according to the position information of the projection intersection point; sequentially connecting the intersection point entities corresponding to adjacent two projection intersection points in the intersection point sequence to form a region boundary line, and determining a region formed by the region boundary line as the ground projection area of the imaging device.

[0206] The apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the method embodiments of the first aspect, and the specific implementation and technical effects can refer to the description of the method embodiments. For brevity, they will not be described here.

[0207] It should be noted that each module involved in the present embodiment is a logical module. In actual application, one logical unit can be one physical unit, or a part of a physical unit, or realized by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present disclosure, units not closely related to solving the technical problems proposed by the present disclosure are not introduced in the present embodiment, but this does not mean that there are no other units in the present embodiment.

[0208] With reference to Figure 4 The present embodiment provides an electronic device, which comprises:

[0209] one or more processors 401;

[0210] a memory 402, on which one or more programs are stored, when the one or more programs are executed by the one or more processors, the one or more processors implement the projection area determination method of any one of the above.

[0211] one or more I / O interfaces 403 connected between the processor and the memory, configured to realize the information interaction of the processor and the memory.

[0212] The processor 401 is a device with data processing capability, including but not limited to a central processing unit (CPU) and the like; the memory 402 is a device with data storage capability, including but not limited to a random access memory (RAM, more specifically SDRAM, DDR, and the like), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), and a flash memory (FLASH); the I / O interface (read-write interface) 403 is connected between the processor 401 and the memory 402, and can realize information interaction between the processor 401 and the memory 402, including but not limited to a data bus and the like.

[0213] In some embodiments, the processor 401, the memory 402, and the I / O interface 403 are connected to each other through a bus, and further connected to other components of the computing device.

[0214] The embodiment also provides a computer readable medium, which stores a computer program, and the program is executed by a processor to implement the method for determining a projection area provided in the embodiment. To avoid repeated description, the specific steps of the method for determining a projection area are not described herein.

[0215] The embodiment of the present disclosure further provides a computer program product, including computer readable code or a non-volatile computer readable storage medium carrying the computer readable code. When the computer readable code is run in a processor of an electronic device, the processor in the electronic device executes the above method for determining a projection area.

[0216] Those skilled in the art can understand that all or some of the steps in the method, the functional modules / units in the system and the device described above can be implemented by software, firmware, hardware, or a combination thereof. In the hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer-readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and can include any information delivery media.

[0217] It should be noted that the terms "comprising", "including", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0218] Those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments but not others, the combination of features of different embodiments means that it is within the scope of the embodiments and forms different embodiments.

[0219] It is understood that the above embodiments are only exemplary for illustrating the principles of the present disclosure, and the present disclosure is not limited thereto. Various modifications and improvements can be made by those of ordinary skill in the art without departing from the spirit and essence of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A method for determining a projection area, characterized in that, The method includes: Acquire the device parameters of the imaging equipment of the aircraft, wherein the device parameters include attitude angle parameters and field of view angle parameters; Based on the field of view parameters, the direction vector corresponding to the visual cone of the imaging device is determined. The direction vector is used to characterize the boundary ray of the visual cone, and the direction vector corresponds to the local coordinate system of the imaging device. A three-dimensional rotation matrix is ​​constructed based on the attitude angle parameters. The direction vector is then transformed from the local coordinate system to the world coordinate system based on the three-dimensional rotation matrix to obtain the target direction vector. Based on the vector value of the target direction vector, the position information of the projection intersection point of the boundary ray and the ground is determined, and based on the position information of the projection intersection point, the ground projection area of ​​the imaging device is determined.

2. The method according to claim 1, characterized in that, The method further includes obtaining the position parameters of the aircraft, including planar coordinate parameters and ground altitude parameters, and the target direction vector including the X-axis component, Y-axis component, and Z-axis component of the boundary ray in the world coordinate system; determining the position information of the projection intersection point of the boundary ray and the ground based on the vector value of the target direction vector includes: Based on the Z-axis component of the target direction vector and the ground clearance parameter, calculate the parameter value of the boundary ray reaching the ground. The first coordinate value of the projection intersection point is determined based on the parameter value, the X-axis component, and the first coordinate in the plane coordinate parameters; The second coordinate value of the projection intersection point is determined based on the parameter value, the Y-axis component, and the second coordinate in the plane coordinate parameters; The location information of the projection intersection point is determined based on the first coordinate value and the second coordinate value.

3. The method according to claim 2, characterized in that, The planar coordinate parameters are obtained by performing Mercator projection transformation on the latitude and longitude coordinates of the aircraft. Determining the position information of the projection intersection point based on the first coordinate value and the second coordinate value includes: Perform Mercator inverse projection transformation on the first coordinate value and the second coordinate value to obtain the latitude and longitude coordinates of the projection intersection point; The location information of the projection intersection point is determined based on the latitude and longitude coordinates of the projection intersection point.

4. The method according to claim 1, characterized in that, The device parameters also include imaging scale parameters. Determining the direction vector corresponding to the visual cone of the imaging device based on the field of view parameters includes: The vertical half-field of view and the horizontal half-field of view of the imaging device are determined based on the field of view parameters and the imaging scale parameters. In the local coordinate system of the imaging device, the direction vector corresponding to the visual cone of the imaging device is determined based on the vertical half-field angle and the horizontal half-field angle.

5. The method according to claim 4, characterized in that, The imaging ratio parameter is used to characterize the aspect ratio of the image displayed by the imaging device; determining the vertical half-field of view and the horizontal half-field of view of the imaging device based on the field of view parameter and the imaging ratio parameter includes: The vertical half-field angle is determined based on the half-value of the field of view parameter; The horizontal half-field of view is determined based on the vertical half-field of view and the aspect ratio.

6. The method according to claim 4, characterized in that, The horizontal direction, vertical direction, and shooting orientation of the imaging device are respectively the X-axis direction, Y-axis direction, and Z-axis direction of the local coordinate system; the determination of the direction vector corresponding to the visual frustum of the imaging device in the local coordinate system of the imaging device, based on the vertical half-field angle and the horizontal half-field angle, includes: For any corner point of the visual cone, the first component of the X-axis of the corner point in the local coordinate system is determined according to the horizontal half-field angle; Based on the vertical half-field angle, determine the second component of the Y-axis of the corner point in the local coordinate system; Based on the preset component values, determine the third component of the Z-axis of the corner point in the local coordinate system; Based on the first component, the second component, and the third component, the direction vector corresponding to the corner point is determined; wherein, the direction vector corresponding to the visual cone includes the direction vector corresponding to each corner point of the visual cone.

7. The method according to claim 1, characterized in that, The three-dimensional rotation matrix is ​​an Euler angle rotation matrix, and the attitude angle parameters include pitch angle parameters, azimuth angle parameters, and roll angle parameters. Constructing the three-dimensional rotation matrix based on the attitude angle parameters includes: Based on the azimuth parameters, construct a first rotation matrix for rotation around the Z-axis in the world coordinate system; Based on the pitch angle parameters, construct a second rotation matrix for rotation around the Y-axis in the world coordinate system; Based on the roll angle parameter, construct a third rotation matrix for rotation around the X-axis in the world coordinate system; The first rotation matrix, the second rotation matrix, and the third rotation matrix are multiplied together according to a preset rotation order to obtain the three-dimensional rotation matrix.

8. The method according to any one of claims 1-7, characterized in that, The number of projection intersection points is multiple, and the step of determining the ground projection area of ​​the imaging device based on the position information of the projection intersection points includes: Based on the positional relationship of multiple projection intersections, the multiple projection intersections are sorted to obtain an intersection sequence, and the first projection intersection in the intersection sequence is added to the end of the intersection sequence to make the beginning and end of the intersection sequence consistent. For each projection intersection point, construct the intersection point entity based on the position information of the projection intersection point; By sequentially connecting the intersection entities corresponding to two adjacent projection intersections in the intersection sequence, a region boundary line is formed, and the region formed by the region boundary line is determined as the ground projection region of the imaging device.

9. A device for determining a projection area, characterized in that, include: The parameter acquisition module is used to acquire the device parameters of the imaging equipment of the aircraft, wherein the device parameters include attitude angle parameters and field of view parameters; The first determining module is used to determine the direction vector corresponding to the visual cone of the imaging device according to the field of view parameter. The direction vector is used to characterize the boundary ray of the visual cone, and the direction vector corresponds to the local coordinate system of the imaging device. The vector transformation module is used to construct a three-dimensional rotation matrix based on the attitude angle parameters, and to convert the direction vector from the local coordinate system to the world coordinate system based on the three-dimensional rotation matrix to obtain the target direction vector. The second determining module is used to determine the position information of the projection intersection point of the boundary ray and the ground based on the vector value of the target direction vector, and to determine the ground projection area of ​​the imaging device based on the position information of the projection intersection point.

10. An electronic device, characterized in that, include: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method for determining the projection area according to any one of claims 1-8; One or more I / O interfaces are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.

Images