Target point geographical position determination method and system, electronic device and storage medium

Abstract

The application discloses a target point geographical position determination method and system, electronic equipment and a storage medium, and the method comprises the following steps: acquiring a first inclination angle and a second inclination angle; acquiring a first coordinate position of a target point in a target image under a first coordinate system; wherein the first coordinate system is a camera equipment coordinate system of the camera equipment shooting the target image under a current pose of the camera equipment; converting the first coordinate position of the target point into a second coordinate position under a second coordinate system based on the first inclination angle, the second inclination angle and the current pose of the camera equipment; wherein the second coordinate system is a reference coordinate system of the camera equipment under an ideal state; determining the geographical position of the target point based on the second coordinate position and information of the camera equipment; through the above scheme, the pan-tilt inclination angle can be acquired based on the composite inclination angle and the camera inclination angle, and then the target point coordinate conversion is performed to determine the geographical position of the target point, thereby improving the positioning efficiency and the positioning accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a target point geographical position determination method and system, electronic equipment and storage medium. BACKGROUND

[0002] With the maturity and popularity of image processing technology, its application range is also more and more wide, especially in the event of an accident, the geographical position of the accident site is determined through image processing, and the geographical position of the target point is generally determined based on the yaw angle, the pitch angle, the geographical position of the device installation and the internal parameters.

[0003] In the research and practice process of the prior art, the inventors of the present application found that the horizontal installation of the device platform cannot be guaranteed, and the horizontal installation of the device cannot be guaranteed in some cases. The prior art sets a reference coordinate system at an arbitrary position for a flight device, and then corrects the attitude of the image sensor. However, in the matrix mapping process, due to the particularity of Euler angles, the problem of rotation axis merging (i.e. gimbal lock) may occur, and thus an accurate tilt angle cannot be obtained, affecting the accuracy of the geographical position of the target point. SUMMARY

[0004] The technical problem solved by the present application is to provide a target point geographical position determination method and system, electronic equipment and storage medium, which can obtain the compound tilt angle of the holder and the camera device, determine the first tilt angle corresponding to the camera device and the second tilt angle corresponding to the holder, and then obtain the first coordinate position corresponding to the camera device coordinate system of the target point in the target image, so as to determine the second coordinate position of the first coordinate position of the target point converted to the reference coordinate system, and then determine the geographical position of the target point through the second coordinate position, realize the positioning of the target point, and improve the efficiency and accuracy of the position determination.

[0005] To solve the above technical problems, one technical scheme adopted by the present application is to provide a target point geographical position determination method, which comprises the following steps: acquiring a first inclination angle and a second inclination angle; wherein the first inclination angle is an installation inclination angle of a camera device relative to a holder in a real geographical environment where the camera device is located; the second inclination angle is an installation inclination angle of the holder relative to a reference surface in the real geographical environment where the camera device is located; acquiring a first coordinate position of a target point in a target image in a first coordinate system; wherein the first coordinate system is a camera device coordinate system in which the camera device captures the target image in a current pose; converting the first coordinate position of the target point into a second coordinate position in a second coordinate system based on the first inclination angle, the second inclination angle, and a current pose of the camera device; wherein the second coordinate system is a reference coordinate system of the camera device in an ideal state; and determining a geographical position of the target point based on the second coordinate position and information of the camera device.

[0006] In an embodiment of the present application, the step of acquiring the first inclination angle and the second inclination angle comprises the following steps: acquiring a composite inclination angle of the camera device; acquiring the first inclination angle; and calculating the second inclination angle based on the composite inclination angle and the first inclination angle.

[0007] In an embodiment of the present application, the step of acquiring the composite inclination angle of the camera device comprises the following steps: fixing a pitch angle of the camera device, changing a yaw angle of the camera device by a fixed step, and acquiring a plurality of images; extracting a reference line in each of the images, calculating a slope of the reference line and a corresponding yaw angle in each of the images; wherein the reference line appears in each of the images and is always parallel to an imaging plane of the camera device; selecting a slope extreme value and a corresponding yaw angle from the slopes of the reference lines in each of the images; and calculating the composite inclination angle of the camera device according to the slope extreme value.

[0008] In an embodiment of the present application, the step of acquiring the first inclination angle comprises the following steps: fixing a first pose angle of the camera device, changing a second pose angle of the camera device, and acquiring two images; wherein the first pose angle is one of a pitch angle and a yaw angle of the camera device, the second pose angle is the other of the pitch angle and the yaw angle of the camera device, and the two images have an overlapping area; performing feature point detection and matching on the two images, and acquiring three-dimensional coordinate values of the matched feature points in two camera device coordinate systems corresponding to the two images; and acquiring the first inclination angle based on a minimum error sum of squares of coordinate values of a rotation axis in the three-dimensional coordinate values of the matched feature points in the two camera device coordinate systems.

[0009] In an embodiment of the present application, the second tilt angle is calculated based on the composite tilt angle and the first tilt angle, including: obtaining the second tilt angle based on the composite tilt angle and the first tilt angle; wherein the composite tilt angle is equal to the sum of the first tilt angle and the second tilt angle.

[0010] In an embodiment of the present application, the first tilt angle, the second tilt angle, and the current pose of the camera device are used to convert the first coordinate position of the target point into a second coordinate position in a second coordinate system, including: calculating the first coordinate position of the target point projected and converted into a third coordinate position in a first ideal preset pose; wherein in the first ideal preset pose, the first pose angle and the second pose angle of the camera device are both 0, the installation tilt angle of the camera device relative to the holder is 0, and the installation tilt angle of the holder relative to the reference reference surface is the second tilt angle; projecting and converting the third coordinate position of the target point into a fourth coordinate position in a second ideal preset pose; wherein in the second ideal preset pose, the first pose angle of the camera device is 0, the second pose angle is the second pose angle corresponding to the slope extreme value, the installation tilt angle of the camera device relative to the holder is 0, and the installation tilt angle of the holder relative to the reference reference surface is the second tilt angle; projecting and converting the fourth coordinate position of the target point into a fifth coordinate position in a third ideal preset pose; wherein in the third ideal preset pose, the first pose angle of the camera device is 0, the second pose angle is the second pose angle corresponding to the slope extreme value, the installation tilt angle of the camera device relative to the holder is 0, and the installation tilt angle of the holder relative to the reference reference surface is 0, and the fifth coordinate position is taken as the second coordinate position in the second coordinate system.

[0011] In an embodiment of the present application, the first pose angle is a pitch angle, and the second pose angle is a yaw angle.

[0012] To solve the above technical problems, another technical solution adopted by the present application is to provide a target point geographical position determination system, the system comprising: a first acquisition module, a second acquisition module, a conversion module, a determination module; wherein the first acquisition module is configured to acquire a first tilt angle and a second tilt angle, wherein the first tilt angle is the installation tilt angle of a camera device relative to a holder in a real geographical environment where the camera device is located; the second tilt angle is the installation tilt angle of the holder relative to a reference surface in the real geographical environment where the camera device is located; the second acquisition module is configured to acquire a first coordinate position of a target point in a target image in a first coordinate system, wherein the first coordinate system is a camera device coordinate system in which the camera device captures the target image in a current pose; the conversion module is configured to convert the first coordinate position of the target point into a second coordinate position in a second coordinate system based on the first tilt angle, the second tilt angle, and the current pose of the camera device, wherein the second coordinate system is a reference coordinate system of the camera device in an ideal state; and the determination module is configured to determine the geographical position of the target point based on the second coordinate position and information of the camera device.

[0013] To solve the above technical problems, still another technical solution adopted by the present application is to provide an electronic device, the electronic device comprising a memory and a processor coupled with the memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the target point geographical position determination method described above.

[0014] To solve the above technical problems, still another technical solution adopted by the present application is to provide a computer readable storage medium, the storage medium storing a program, the program being loaded and executed by a processor to implement the target point geographical position determination method described above.

[0015] Unlike existing technologies, the method for determining the geographical location of a target point provided in this application includes: obtaining a first tilt angle and a second tilt angle; wherein, the first tilt angle is the installation tilt angle of the camera device relative to the gimbal in the real geographical environment where the camera device is located; the second tilt angle is the installation tilt angle of the gimbal relative to a reference surface in the real geographical environment where the camera device is located; obtaining the first coordinate position of a target point in a target image in a first coordinate system; wherein, the first coordinate system is the camera device coordinate system in which the camera device captures the target image in its current pose; converting the first coordinate position of the target point into a second coordinate position in a second coordinate system based on the first tilt angle, the second tilt angle, and the current pose of the camera device; wherein, the second coordinate system is the reference coordinate system of the camera device in an ideal state; determining the geographical location of the target point based on the second coordinate position and the information of the camera device; thereby realizing the positioning of the geographical location of the target point and improving positioning efficiency and accuracy. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an embodiment of the method for determining the geographical location of a target point according to the present invention;

[0017] Figure 2 This is a flowchart illustrating an embodiment of step S1 of the present invention;

[0018] Figure 3 This is a flowchart illustrating an embodiment of step S11 of the present invention;

[0019] Figure 4 This is a flowchart illustrating an embodiment of step S12 of the present invention;

[0020] Figure 5 This is a flowchart illustrating an embodiment of step S13 of the present invention;

[0021] Figure 6 This is a flowchart illustrating an embodiment of step S3 of the present invention;

[0022] Figure 7 This is a schematic diagram of the structure of an embodiment of the target point geographical location determination system of the present invention;

[0023] Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device of the present invention;

[0024] Figure 9 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of the present invention. Detailed Implementation

[0025] The application will be described in further detail below with reference to the drawings and embodiments. It is to be specifically pointed out that the following embodiments are only used to illustrate the application, but do not limit the scope of the application. Similarly, the following embodiments are only part of the embodiments of the application, not all embodiments, and all other embodiments obtained by those skilled in the art without creative labor are within the scope of the application.

[0026] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase that the phrase in various places in the specification are not necessarily all referring to the same embodiment, or are necessarily referring to different or alternative embodiments. It is explicitly and implicitly understood that the application described herein is capable of combination with other embodiments.

[0027] The steps in the embodiments of the present application are not necessarily processed in the order described, and the steps can be selectively rearranged, deleted, or added according to requirements. The step description in the embodiments of the present application is only an optional sequence combination, and does not represent all sequence combinations of the embodiments of the present application. The sequence of steps in the embodiments cannot be considered as a limitation of the present application.

[0028] The term "and / or" in the embodiments of the present application means any and all possible combinations of the associated listed items. It should also be noted that when used in the specification, "comprise / comprising" specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or components and / or groups thereof.

[0029] The terms "first", "second", and the like in the present application are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units that are not listed, or can optionally include other steps or units inherent to the process, method, product, or device.

[0030] In addition, although the terms "first", "second", and the like are used repeatedly in the present application to describe various data (or various elements or various applications or various instructions or various operations), and the like, these data (or elements or applications or instructions or operations) should not be limited by these terms. These terms are only used to distinguish one data (or element or application or instruction or operation) from another data (or element or application or instruction or operation). For example, first position information can be referred to as second position information, and second position information can also be referred to as first position information, only the scope included by the two is different, without departing from the scope of the present application, and the first position information and the second position information are both a collection of various positions and attitudes, only the two are not the same collection of positions and attitudes.

[0031] The conventional target point geographical position determination method usually erects a detection device in a specific area to realize monitoring of the environment, automatic identification and positioning of the target point in an ideal state. However, the horizontal installation of the detection device position cannot be guaranteed, and the horizontal installation of the sensor cannot be effectively guaranteed either, so there are multiple inclination angles, which cause the positioning accuracy to decrease; and natural factors also affect the inclination angle. In the prior art, a reference coordinate system on the intersection is designed for a flight device to determine the corresponding position information, and then determine the offset attitude angle, that is, the reference coordinate system is at an arbitrary position, but it is not applicable to the fixed detection device, and in the matrix mapping process, due to the particularity of the Euler angle, the problem of rotation axis merging (i.e. gimbal lock) may occur; and a sufficient number of reference line segments need to be obtained, and clustering is performed according to the angle, and then the object inclination angle is obtained, and when the reference line segment is insufficient, the clustering fails, and it depends on certain prior knowledge to complete, so that the positioning effect is not ideal, the positioning efficiency is not high, and the positioning accuracy is insufficient.

[0032] The applicant found in the research that for the problem of inclination angle, the composite inclination angle can be obtained to determine the camera inclination angle and the corresponding gimbal inclination angle, and then the coordinate system conversion of the target point is performed to determine the geographical position information of the target point, improve the positioning efficiency and the positioning accuracy.

[0033] Therefore, a method for determining a geographic position of a target point is provided. The method includes the following steps. A first tilt angle and a second tilt angle are obtained. The first tilt angle is an installation tilt angle of a camera device relative to a gimbal in a real geographic environment in which the camera device is located. The second tilt angle is an installation tilt angle of the gimbal relative to a reference surface in the real geographic environment in which the camera device is located. A first coordinate position of a target point in a target image in a first coordinate system is obtained. The first coordinate system is a camera device coordinate system in which the camera device captures the target image in a current pose. The first coordinate position of the target point is converted into a second coordinate position in a second coordinate system based on the first tilt angle, the second tilt angle, and the current pose of the camera device. The second coordinate system is a reference coordinate system in which the camera device is in an ideal state. A geographic position of the target point is determined based on the second coordinate position and information of the camera device.

[0034] Referring to Figure 1 , Figure 1 is a flowchart of an embodiment of the method for determining a geographic position of a target point. It should be noted that the method of the present application is not limited to the order of the steps shown in Figure 1 . For example, the method can include the following steps. Figure 1

[0035] S1, obtaining a first tilt angle and a second tilt angle.

[0036] The first tilt angle is an installation tilt angle of a camera device relative to a gimbal in a real geographic environment in which the camera device is located. The second tilt angle is an installation tilt angle of the gimbal relative to a reference surface in the real geographic environment in which the camera device is located.

[0037] The camera device is a shooting device that converts an optical image signal into an electrical signal, such as a video camera, a camera, a camera head, etc., and is used to obtain image data. The real geographic environment is a specific geographic scene with specific geographic position information. The gimbal is a support platform for the shooting device, which is composed of two alternating current motors or direct current motors and can move horizontally and vertically. The first tilt angle corresponds to an installation tilt angle of a device camera image plane and a gimbal yaw angle. The second tilt angle corresponds to an installation tilt angle of a gimbal plane and a reference surface. The reference surface is a set reference surface for the real geographic environment.

[0038] In some embodiments, the first tilt angle is obtained by obtaining an included angle between an image plane of the camera device and a gimbal yaw angle in the real geographic environment in which the camera device is located. The second tilt angle is obtained by obtaining an included angle between the gimbal yaw angle and the reference surface in the real geographic environment in which the camera device is located.

[0039] Referring to Figure 2 ,​Figure 2 is a flowchart of an embodiment of step S1 of the present application, and step S1 includes:

[0040] S11, obtain a compound tilt angle of the camera device;

[0041] The compound tilt angle is a combination of the first tilt angle and the second tilt angle.

[0042] In some embodiments, the first tilt angle is a camera installation tilt angle, i.e., an angle formed between a pan yaw rotation axis corresponding to a pan angle of the gimbal and an image plane of the camera device; for example, an angle formed between a plane perpendicular to a xy plane of a gimbal coordinate system and parallel to a yz plane of the gimbal coordinate system and a plane perpendicular to an image plane of the camera device and parallel to a yz plane of a camera coordinate system, or an angle formed between the pan yaw rotation axis and a Y axis of the camera image plane. The second tilt angle is a gimbal installation tilt angle, i.e., an angle formed between the pan yaw rotation axis corresponding to the pan angle of the gimbal and a reference plane in a real geographical environment where the camera device is located; for example, an angle formed between a plane perpendicular to the xy plane of the gimbal coordinate system and parallel to the yz plane of the gimbal coordinate system and a plane perpendicular to an xy plane of a reference coordinate system in the real geographical environment where the camera device is located and parallel to a yz plane of the reference coordinate system, or an angle formed between the pan yaw rotation axis and a Y axis of a world coordinate system.

[0043] Referring to Figure 3 , Figure 3 is a flowchart of an embodiment of step S11 of the present application, and step S11 includes:

[0044] S111, fix a first pose angle of the camera device, change a second pose angle of the camera device at a fixed step length, and obtain a plurality of images;

[0045] Specifically, the first pose angle of the camera device is fixed, a fixed step length is set, the second pose angle of the camera device is changed by the fixed step length, an image is obtained each time the second pose angle is changed, and a plurality of images are obtained by changing the second pose angle multiple times.

[0046] In some embodiments, the first pose angle can be a pitch angle, and the second pose angle can be a yaw angle; then, the pitch angle of the camera device is fixed, a fixed step length is set, the yaw angle of the camera device is changed based on the fixed step length, an image is obtained each time the yaw angle of the camera device is changed, and a plurality of images are obtained by changing the yaw angle of the camera device multiple times.

[0047] S112, extract a reference line in each image, and calculate a slope of the reference line in each image and a corresponding second pose angle;

[0048] The slope of the reference line is a degree of inclination of the reference line relative to an imaging plane.

[0049] Optionally, the reference line appears in each image and is always parallel to the imaging plane of the camera device;

[0050] In some embodiments, a reference line appearing in each image and always parallel to the imaging plane of the camera device is extracted, the degree of inclination of the corresponding reference line in each image relative to the imaging plane is calculated, and the slope of the corresponding reference line and the corresponding second pose angle in each image are obtained, where the second pose angle is the yaw angle.

[0051] S113, selecting a slope extreme value and the corresponding second pose angle from the slopes of the reference lines in each image;

[0052] In some embodiments, the slopes of the reference lines in all images form a slope array, and a slope extreme value and the corresponding second pose angle are selected from the slope array.

[0053] Optionally, the slope extreme value is a slope maximum value or a slope minimum value.

[0054] In some embodiments, the slopes of the reference lines in all images form a slope array, and a slope maximum value and the corresponding second pose angle are selected from the slope array.

[0055] In some embodiments, the slopes of the reference lines in all images form a slope array, and a slope minimum value and the corresponding second pose angle are selected from the slope array.

[0056] S114, calculating a composite tilt angle of the camera device according to the slope extreme value.

[0057] In some embodiments, the slope extreme value can be used to calculate the composite tilt angle.

[0058] In some embodiments, the composite tilt angle is calculated as follows:

[0059] θ=tan k=θ1+θ2

[0060] Where θ is the composite tilt angle, k is the slope extreme value, θ1 is the first tilt angle, and θ2 is the second tilt angle.

[0061] For example, θ is the composite tilt angle, k is the slope maximum value, θ1 is the camera installation tilt angle, and θ2 is the gimbal installation tilt angle. At this time, the pose of the camera device coordinate system can be represented as: P=α k , T=0, θ1=0, θ2=θ2; P is the yaw angle, and T is the pitch angle.

[0062] S12, obtaining the first tilt angle;

[0063] In some embodiments, the first tilt angle is a camera installation tilt angle, i.e., an angle between a yaw rotation axis of the gimbal corresponding to the gimbal yaw angle and an image plane of the camera device; for example, an angle between a plane perpendicular to a xy plane of the gimbal coordinate system and parallel to a yz plane of the gimbal coordinate system and a plane perpendicular to the image plane of the camera device and parallel to a yz plane of the camera device coordinate system or an angle between the yaw rotation axis of the gimbal and a Y axis of the camera image plane.

[0064] Referring to Figure 4 , Figure 4 is a flowchart of an embodiment of step S12, which includes:

[0065] S121, fixing a first pose angle of the camera device, changing a second pose angle of the camera device, and obtaining two images;

[0066] wherein the first pose angle is one of a pitch angle and a yaw angle of the camera device, and the second pose angle is the other of the pitch angle and the yaw angle of the camera device, and the two images have an overlapping region.

[0067] In some embodiments, the first pose angle is the pitch angle, and the second pose angle is the yaw angle; then, fixing the pitch angle of the camera device, changing the yaw angle of the camera device, and obtaining two images with an overlapping region, the pose representations of the two images are P = a1, T = b and P = a2, T = b respectively.

[0068] In some embodiments, the first pose angle is the yaw angle, and the second pose angle is the pitch angle; then, fixing the yaw angle of the camera device, changing the pitch angle of the camera device, and obtaining two images with an overlapping region, the pose representations of the two images are P = a, T = b1 and P = a, T = b2 respectively.

[0069] S122, performing feature point detection and matching on the two images to obtain three-dimensional coordinate values of the matched feature points in two camera device coordinate systems corresponding to the two images;

[0070] wherein each image corresponds to a camera device coordinate system, so the two images correspond to two camera device coordinate systems, the XY axis plane of each camera device coordinate system is parallel to the corresponding image, and the Z axis is perpendicular to the corresponding image, so the camera device coordinate system to image distance is expressed as the length of the Z axis of the image in the corresponding camera device coordinate system.

[0071] Specifically, a common rotation coordinate axis of the two images is determined, a first feature point in the first image in the corresponding camera device coordinate system and its three-dimensional coordinate value are obtained, the common rotation axis is rotated, and a second feature point in the second image in the corresponding camera device coordinate system and its three-dimensional coordinate value are determined based on the first feature point.

[0072] Optionally, the coexisting rotation coordinate axis can be a coexisting Y axis or a coexisting X axis, which can be set according to actual conditions; for example, the coexisting rotation coordinate axis is the Y axis, and the three-dimensional coordinate values corresponding to the two images in the two camera coordinate systems can be obtained according to the rotation matrix;

[0073] For example, there is only one Y axis rotation between the two different camera coordinate systems, and according to the rotation matrix, the Y coordinates of the coordinates of the feature points after rotation in the two camera coordinate systems should be equal. However, due to the existence of the camera tilt angle, the Y coordinates cannot be equal. Therefore, the minimum error sum of squares of the Y coordinates is obtained, and then the camera tilt angle is obtained as the first tilt angle; wherein the rotation matrix is:

[0074]

[0075] S123, based on the minimum error sum of squares of the coordinates of the rotation coordinate axis in the three-dimensional coordinate values of the matched feature points in the two camera coordinate systems, the first tilt angle is obtained.

[0076] Wherein, the minimum error sum of squares is the optimization target, and the minimum error sum of squares of the coordinates is obtained based on the optimization target, that is, the first tilt angle is obtained.

[0077] In some embodiments, the rotation coordinate axis is the coexisting Y axis, and the Y axis coordinates of the first feature point in the corresponding camera coordinate system of the first image and the Y axis coordinates of the second feature point in the corresponding camera coordinate system of the second image are obtained, and then the square sum of the Y axis coordinates of the first feature point and the Y axis coordinates of the second feature point is obtained, and then the minimum error sum of squares of the coordinates is obtained, and finally the approximate value of the Y axis included angle between the imaging plane of the camera and the Y axis of the gimbal yaw angle is obtained as the first tilt angle.

[0078] S13, based on the composite tilt angle and the first tilt angle, the second tilt angle is calculated.

[0079] Referring to Figure 5 , Figure 5 is a flowchart of an embodiment of step S13, and step S13 includes:

[0080] S131, the angle relationship between the composite tilt angle and the first tilt angle and the second tilt angle is obtained;

[0081] Wherein, the composite tilt angle is equal to the sum of the first tilt angle and the second tilt angle;

[0082] Specifically, the composite tilt angle is the sum of the tilt angles common to the camera and the camera platform, and therefore, the angle relationship between the composite tilt angle and the first tilt angle and the second tilt angle needs to be obtained.

[0083] S132, obtain the second tilt angle based on the angle relationship, the compound tilt angle and the first tilt angle.

[0084] Specifically, the second tilt angle can be determined by the obtained compound tilt angle, the first tilt angle and the corresponding angle relationship:

[0085] θ2 = θ - θ1

[0086] wherein θ is the compound tilt angle, θ1 is the first tilt angle, and θ2 is the second tilt angle.

[0087] S2, obtain the first coordinate position of the target point in the target image in the first coordinate system;

[0088] wherein the first coordinate system is a camera device coordinate system of the camera device for shooting the target image in the current pose;

[0089] Specifically, the target image is an image with event information, i.e. an image from which event information can be obtained is the target image; the event information can be a natural event or a non-natural event, for example: flood or fire; the target point is the position of the event, for example: the position of the flood or the position of the fire.

[0090] In some embodiments, the image with the event is obtained, and the position of the event point in the camera device coordinate system corresponding to the image shot by the camera device in the current pose when the event occurs in the image is the first coordinate position.

[0091] S3, convert the first coordinate position of the target point into a second coordinate position in the second coordinate system based on the first tilt angle, the second tilt angle and the current pose of the camera device;

[0092] wherein the second coordinate system is a reference coordinate system of the camera device in the ideal state;

[0093] In some embodiments, the current pose of the camera device is obtained, and the first coordinate position of the event point is converted into the second coordinate position in the reference coordinate system of the camera device in the ideal state based on the installation tilt angle of the camera device, the installation tilt angle of the camera platform and the current pose of the camera device.

[0094] Referring to Figure 6 , Figure 6 is a flowchart of an embodiment of step S3 of the present application, and step S3 includes:

[0095] S31, calculate the first coordinate position of the target point to be projected and converted into a third coordinate position in the first ideal preset pose;

[0096] Wherein, under the first ideal preset pose, the first pose angle and the second pose angle of the camera equipment are 0 respectively, the installation tilt angle of the camera equipment relative to the holder is 0, and the installation tilt angle of the holder relative to the reference plane is the second tilt angle, that is, the first ideal preset pose is: P=0, T=0, θ1=0, θ2=θ2.

[0097] In some embodiments, the position of the corresponding event occurrence point in the camera equipment coordinate system corresponding to the image taken by the camera equipment under the current pose is set as p(x, y, z), and the pose is P=α, T=β, θ1=θ 1, θ2=θ2; that is, the first pose angle of the camera equipment is the first pose angle corresponding to the slope of the line segment corresponding to the image, the second pose angle of the camera equipment is the second pose angle corresponding to the slope of the line segment corresponding to the image, the installation tilt angle of the camera equipment relative to the holder is the first tilt angle, and the installation tilt angle of the holder relative to the reference plane is the second tilt angle.

[0098] The first coordinate position p(x, y, z) of the target point is projected and converted into the third coordinate position p'(x', y', z') under the first ideal preset pose, and the following equation is obtained:

[0099] R α *R θ1 *R β *p=p'

[0100] Wherein, R α *R θ1 *R β is the rotation matrix of the first ideal preset pose.

[0101] S32, the third coordinate position of the target point is projected and converted into the fourth coordinate position under the second ideal preset pose.

[0102] Wherein, under the second ideal preset pose, the first pose angle of the camera equipment is 0, the second pose angle is the second pose angle corresponding to the slope extreme value, the installation tilt angle of the camera equipment relative to the holder is 0, and the installation tilt angle of the holder relative to the reference plane is the second tilt angle, that is, the second ideal preset pose is: P=α k , T=0, θ1=0, θ2=θ2.

[0103] In some embodiments, the third coordinate position p'(x', y', z') is projected and converted into the fourth coordinate position p''(x'', y'', z'') under the second ideal preset pose, and the following equation is obtained:

[0104] R αk *p''=p'

[0105] Wherein, R αk is the rotation matrix of the second ideal preset pose.

[0106] S33, project and convert the fourth coordinate position of the target point into a fifth coordinate position under a third ideal preset pose;

[0107] Wherein, under the third ideal preset pose, the first pose angle of the camera equipment is 0, the second pose angle is the second pose angle corresponding to the slope extreme value, the installation inclination angle of the camera equipment relative to the holder is 0, and the installation inclination angle of the holder relative to the reference surface is 0, that is, the third ideal preset pose is: P = a k , T = 0, θ1 = 0, θ2 = 0;

[0108] In some embodiments, the fourth coordinate position p"(x", y", z") is projected and converted into a fifth coordinate position p'"(x'", y'", z'" ) under the third ideal pose, and there is:

[0109] R θ2 * p" = p'"

[0110] Wherein, R θ2 is the rotation matrix of the third ideal preset pose.

[0111] S34, the fifth coordinate position is a second coordinate position under the second coordinate system.

[0112] The first coordinate position p(x, y, z) of the target point is converted into the fifth coordinate position p'"(x'", y'", z'") as:

[0113]

[0114] That is, the fifth coordinate position p'"(x'", y'", z'") is the second coordinate position of the camera equipment under the reference coordinate system in the ideal state, and the corrected target point corresponds to the pitch angle T = 0 and the yaw angle P = a k .

[0115] S4, determine the geographic position of the target point based on the second coordinate position and the information of the camera equipment.

[0116] Wherein, the information of the camera equipment can include the geographic position information of the camera equipment installation, the installation height information of the camera equipment.

[0117] In some embodiments, it is necessary to obtain the geographic position information of the camera equipment installation, the installation height information of the camera equipment and other camera equipment related information; and then based on the camera equipment related information and the fifth coordinate position p'"(x'", y'", z'") and its corresponding pose: pitch angle T = 0 and yaw angle P = a k , determine the geographic position of the target point.

[0118] Differing from the prior art, in the embodiment, a target point geographical position determination method is provided, comprising: acquiring a first inclination angle and a second inclination angle; wherein the first inclination angle is an installation inclination angle of a camera device relative to a holder in a real geographical environment in which the camera device is located; the second inclination angle is an installation inclination angle of the holder relative to a reference surface in the real geographical environment in which the camera device is located; acquiring a first coordinate position of a target point in a target image in a first coordinate system; wherein the first coordinate system is a camera device coordinate system in which the camera device captures the target image in a current pose; converting the first coordinate position of the target point into a second coordinate position in a second coordinate system based on the first inclination angle, the second inclination angle, and the current pose of the camera device; wherein the second coordinate system is a reference coordinate system of the camera device in an ideal state; determining a geographical position of the target point based on the second coordinate position and information of the camera device; that is, the application determines the second inclination angle by acquiring a composite inclination angle and the first inclination angle, acquires a first coordinate position of a target point in a target image captured by a camera device in a current pose of the camera device in a camera device coordinate system, converts the first coordinate position into a second coordinate position in a reference coordinate system of the camera device in an ideal state, and then determines a geographical position of the target point in combination with information of the camera device, so that accurate positioning of the geographical position of the target point in the image is realized, and positioning efficiency and positioning accuracy are improved.

[0119] Please refer to Figure 7 , Figure 7 is a structural schematic diagram of an embodiment of a target point geographical position determination system of the application. The system can execute the steps of the target point geographical position determination method described above. For related content, please refer to the detailed description in the method described above, which will not be repeated here.

[0120] The target point geographical position determination system 200 comprises: a first acquisition module 210, a second acquisition module 220, a conversion module 230, and a determination module 240. The first acquisition module 210 is configured to acquire a first inclination angle and a second inclination angle, the first inclination angle being an installation inclination angle of a camera device relative to a holder in a real geographical environment in which the camera device is located; the second inclination angle being an installation inclination angle of the holder relative to a reference surface in the real geographical environment in which the camera device is located. The second acquisition module 220 is configured to acquire a first coordinate position of a target point in a target image in a first coordinate system, the first coordinate system being a camera device coordinate system in which the camera device captures the target image in a current pose. The conversion module 230 is configured to convert the first coordinate position of the target point into a second coordinate position in a second coordinate system based on the first inclination angle, the second inclination angle, and the current pose of the camera device, the second coordinate system being a reference coordinate system of the camera device in an ideal state. The determination module 240 is configured to determine a geographical position of the target point based on the second coordinate position and information of the camera device.

[0121] Please refer to Figure 8 , Figure 8 is a structural schematic diagram of an embodiment of an electronic device in the present application. The electronic device can execute the steps in the target point geographical position determination method described above. The electronic device 300 comprises a memory 310 and a processor 320 coupled with the memory, and the memory 310 stores a computer program, which is loaded and executed by the processor 320 to implement the target point geographical position determination method described above.

[0122] Please refer to Figure 9 , Figure 9 is a structural schematic diagram of an embodiment of a computer readable storage medium in the present application. The computer readable storage medium 400 stores a program 410, which is loaded and executed by a processor to implement the target point geographical position determination method described above.

[0123] The above scheme automatically calibrates the input image, determines the camera tilt angle and the corresponding gimbal tilt angle by obtaining the composite tilt angle, can avoid the problem of loss of degrees of freedom caused by the universal lock, and then converts the coordinates between the target points. And no special feature points are used, that is, relatively common feature points are used for processing, without the need for manual selection, avoiding the influence of human factors; and it does not rely on too much prior knowledge and preset threshold, avoiding interference of other factors, and then determining the geographical position information of the target point; Therefore, the composite tilt angle and the camera tilt angle are used to obtain the gimbal tilt angle, so as to perform coordinate transformation on the target point to determine the geographical position information of the target point, improve the positioning efficiency and positioning accuracy.

[0124] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. For example, the device embodiments described above are only schematic, and the division of the modules or units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0125] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the present embodiment scheme.

[0126] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0127] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0128] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent flow transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for determining the geographical location of a target point, characterized in that, include: Obtain a first tilt angle and a second tilt angle; wherein, the first tilt angle is the installation tilt angle of the camera device relative to the gimbal in the actual geographical environment where the camera device is located; the second tilt angle is the installation tilt angle of the gimbal relative to the reference surface in the actual geographical environment where the camera device is located; obtain the composite tilt angle of the camera device, which is determined by the extreme value of the slope of the reference line corresponding to multiple images obtained when the pitch angle of the camera device is fixed and the yaw angle is changed; the second tilt angle is calculated based on the composite tilt angle and the first tilt angle; Obtain the first coordinate position of the target point in the target image in the first coordinate system; wherein, the first coordinate system is the camera device coordinate system in which the camera device captures the target image in the current pose; Based on the first tilt angle, the second tilt angle, and the current pose of the camera device, the first coordinate position of the target point is converted into a second coordinate position in a second coordinate system; wherein, the second coordinate system is the reference coordinate system of the camera device in an ideal state; The geographical location of the target point is determined based on the second coordinate position and the information from the camera device.

2. The method according to claim 1, characterized in that, The process of obtaining the composite tilt angle of the camera device includes: The pitch angle of the camera device is fixed, and the yaw angle of the camera device is changed by a fixed step size to acquire multiple images; Extract reference lines from each of the images, and calculate the slope and corresponding yaw angle of the reference lines in each image; wherein the reference lines appear in each of the images and are always parallel to the imaging plane of the camera device; Select the extreme slope values ​​and corresponding yaw angles from the slope of the reference line in each of the images; The composite tilt angle of the camera device is calculated based on the extreme value of the slope.

3. The method according to claim 1, characterized in that, Obtaining the first tilt angle includes: The first pose angle of the camera device is fixed, and the second pose angle of the camera device is changed to acquire two images; wherein, the first pose angle is one of the pitch angle and yaw angle of the camera device, and the second pose angle is the other of the pitch angle and yaw angle of the camera device, and the two images have overlapping areas; Feature point detection and matching are performed on the two images to obtain the three-dimensional coordinate values ​​of the matched feature points in the coordinate systems of the two cameras corresponding to the two images; The first tilt angle is obtained based on the minimum squared error of the coordinate values ​​of the matched feature points on a rotating axis in the three-dimensional coordinates of the two camera device coordinate systems.

4. The method according to claim 1, characterized in that, The calculation of the second tilt angle based on the composite tilt angle and the first tilt angle includes: The second tilt angle is obtained based on the composite tilt angle and the first tilt angle; wherein the composite tilt angle is equal to the sum of the first tilt angle and the second tilt angle.

5. The method according to claim 1, characterized in that, The step of converting the first coordinate position of the target point into a second coordinate position in a second coordinate system based on the first tilt angle, the second tilt angle, and the current pose of the camera device includes: The first coordinate position of the target point is projected and converted into the third coordinate position under the first ideal preset pose; wherein, under the first ideal preset pose, the first pose angle and the second pose angle of the camera device are 0 respectively, the installation tilt angle of the camera device relative to the gimbal is 0, and the installation tilt angle of the gimbal relative to the reference surface is the second tilt angle. The third coordinate position of the target point is projected and converted into the fourth coordinate position under the second ideal preset pose; wherein, under the second ideal preset pose, the first pose angle of the camera device is 0, the second pose angle is the second pose angle corresponding to the slope extreme value, the installation tilt angle of the camera device relative to the gimbal is 0, and the installation tilt angle of the gimbal relative to the reference surface is the second tilt angle. The fourth coordinate position of the target point is projected and converted into the fifth coordinate position under the third ideal preset pose; wherein, under the third ideal preset pose, the first pose angle of the camera device is 0, the second pose angle is the second pose angle corresponding to the slope extreme value, the installation tilt angle of the camera device relative to the gimbal is 0, the installation tilt angle of the gimbal relative to the reference surface is 0, and the fifth coordinate position is used as the second coordinate position under the second coordinate system.

6. The method according to claim 5, characterized in that, The first attitude angle is the pitch angle, and the second attitude angle is the yaw angle.

7. A system for determining the geographical location of a target point, characterized in that, The system includes: A first acquisition module is used to acquire a first tilt angle and a second tilt angle; wherein, the first tilt angle is the installation tilt angle of the camera device relative to the gimbal in the real geographical environment where the camera device is located; the second tilt angle is the installation tilt angle of the gimbal relative to the reference surface in the real geographical environment where the camera device is located; the module acquires a composite tilt angle of the camera device, wherein the composite tilt angle is determined by the extreme value of the slope of the reference line corresponding to multiple images acquired when the pitch angle of the camera device is fixed and the yaw angle is changed; and the second tilt angle is calculated based on the composite tilt angle and the first tilt angle. The second acquisition module is used to acquire the first coordinate position of the target point in the target image in the first coordinate system; wherein, the first coordinate system is the camera device coordinate system in which the camera device captures the target image in the current pose; The conversion module converts the first coordinate position of the target point into a second coordinate position in a second coordinate system based on the first tilt angle, the second tilt angle, and the current pose of the camera device; wherein the second coordinate system is the reference coordinate system of the camera device in an ideal state; The determination module determines the geographical location of the target point based on the second coordinate position and the information of the camera device.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor coupled to the memory. The memory stores a computer program, which, when loaded and executed by the processor, is used to implement the method for determining the geographical location of a target point as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program, which, when loaded and executed by a processor, is used to implement the method for determining the geographical location of a target point as described in any one of claims 1-6.

Images