Scanning search planning method, system and equipment for roll-on-pitch visual camera and medium

Abstract

The invention provides a rolling and pitching type visual camera scanning search planning method, system and device and a medium, and relates to the technical field of automatic control, and the planning method comprises the steps: after a scanning task is received, constructing an auxiliary coordinate system and a row scanning line according to scanning task information, and carrying out offline calculation to obtain a reference value of a related variable; initializing the related variable according to the reference value of the related variable, controlling the visual axis of the camera to point to the midpoint of the line scanning line, and after entering a scanning mode, controlling the camera to adjust the angle by acquiring the midpoint coordinate of the line scanning line and the planning position coordinate in the auxiliary coordinate system in real time and converting the two coordinates to the earth coordinate system. The scanning search in the rail penetrating direction is realized; according to the invention, through rolling and pitching motion of the rolling and pitching type visual camera and flight motion of the flight platform, comprehensive scanning can be carried out in rail penetrating and along-rail directions, and the range of scanning imaging is greatly expanded; besides, according to different search scenes and task requirements, the width of the line scanning line can be flexibly adjusted, the scanning position can be flexibly planned and the like according to factors such as the scanning range in the rail penetrating direction and the flight space state, various complex environmental conditions are adapted, and the applicability is improved.

Description

Technical Field

[0001] The present application relates to the field of automatic control technology, and in particular to a scanning search planning method, system, device and medium for a roll-and-pitch visual camera. Background Art

[0002] Vision cameras, mounted on a flying platform, scan and search designated areas, such as land and sea, to detect and identify targets. Currently, a common search method for rolling-tilt platforms for vision cameras involves a fixed pitch angle and a forward sweep along a roll axis perpendicular to the flight direction. For low-altitude, long-range, and other flight-space search applications, the imaging range of the through-track scan is limited to varying degrees, impacting search efficiency and flight platform trajectory planning. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, the present application aims to provide a rolling visual camera scanning search planning method, system, device and medium.

[0004] In a first aspect, the present application provides a scanning search planning method for a rolling visual camera, comprising the following steps: S100. Receive a scanning task and obtain scanning task information, wherein the scanning task information includes a scanning range in the through-track direction and flight space status information, and the flight space status information includes a flight altitude and a working distance; wherein a dual-axis stabilized platform is provided on the flight platform for mounting the visual camera, and the visual camera performs rolling and pitching motions on the dual-axis stabilized platform; the flight platform is also provided with a navigation system, and the navigation system is used to obtain the flight space status information, and the navigation system has a navigation coordinate system, the origin of the navigation coordinate system is the navigation system origin, the X-axis is along the north direction, the Z-axis is along the plumb line direction, and the Y-axis satisfies the right-hand rule with the X-axis and the Z-axis and points to the east direction; S200. Construct an auxiliary coordinate system based on the scanning task information, and construct a row scan line in the auxiliary coordinate system, wherein the origin of the auxiliary coordinate system is the origin of the navigation system, the X-axis is parallel to the flight direction and the horizontal plane, the Z-axis points to the center of the earth along the plumb line, and the Y-axis satisfies the right-hand rule with the X-axis and the Z-axis; the row scan line is perpendicular to the X-axis of the auxiliary coordinate system, and the distance between the row scan line and the origin of the auxiliary coordinate system is determined according to the working distance, and the projection of the X-axis of the auxiliary coordinate system on the row scan line is the midpoint of the row scan line; S300. Obtaining a set of related variable reference values ​​in the auxiliary coordinate system by offline calculation based on the scanning task information, the set of related variable reference values ​​including a plurality of related variables and related variable reference values ​​corresponding to each of the related variables, the related variables including at least a row scanning line width, a planned position coordinate, a planned position imaging time, and a row scanning line midpoint, and the corresponding related variable reference values ​​including a row scanning line width reference value, a planned position reference coordinate, a planned position reference imaging time, and a reference pitch pointing angle of the row scanning line midpoint; S400. Initializing each of the related variables according to the reference value of the related variables to obtain a plurality of initialized variables, the visual camera is in a waiting state for scanning; controlling the visual camera to switch to a scanning search state, and controlling its movement according to the initialized variables; S500. Real-time acquisition of the real-time coordinates of the midpoint of the scan line in the auxiliary coordinate system, recorded as the real-time auxiliary coordinates of the midpoint of the scan line; S600. Obtain the real-time coordinates of the planned position in the auxiliary coordinate system according to the real-time auxiliary coordinates of the midpoint, recorded as the real-time auxiliary coordinates of the planned position; S700. The real-time auxiliary coordinates of the midpoint of the scan line and the real-time auxiliary coordinates of the planned position are converted to the earth coordinate system to obtain the real-time earth coordinates of the midpoint of the scan line and the planned position in real time; S800. Control the visual camera to adjust its angle according to the real-time earth coordinates of the midpoint of the row scan line and the real-time earth coordinates of the planned position.

[0005] According to the technical solution provided in the embodiment of the present application, step S800 further includes the following steps: S900. Set the time interval; S1000. After the time interval has passed, repeat steps S500-S800.

[0006] According to the technical solution provided in the embodiment of the present application, obtaining the planned position reference coordinates includes the following steps: S310. Evenly divide the scanning range in the cross-track direction into a plurality of search reference sub-areas, each of which has a visual axis reference pointing point; S320. Get the distance between each of the visual axis reference pointing point and the midpoint of the row scan line, recorded as the planning distance; S330. According to the planned distance, the coordinates of each of the visual axis reference pointing points are obtained, which are the planned position reference coordinates.

[0007] According to the technical solution provided in the embodiment of the present application, step S500 includes the following steps: S510. Obtain the angle between the viewing axis and the X-axis in the auxiliary coordinate system, recorded as the first angle; S520. Obtain the distance between the visual camera and the target point, recorded as the real-time distance; S530. Obtain the X-axis coordinate and the Z-axis coordinate of the real-time auxiliary coordinate of the midpoint of the line scan in the auxiliary coordinate system according to the first angle and the real-time distance, and obtain the real-time auxiliary coordinate of the midpoint of the line scan line.

[0008] According to the technical solution provided in the embodiment of the present application, step S800 includes the following steps: S810. Continuously obtain the position information and attitude information of the flight platform, as well as the frame angle information of the dual-axis stabilized platform; S820. According to the position information and the posture information, the real-time earth coordinates of the planned position are converted into the camera base coordinate system or the line of sight coordinate system to obtain the real-time conversion coordinates of the planned position; S830. Calculate the target angle according to the real-time conversion coordinates of the planned position and the frame angle information, where the target angle is the angle at which the dual-axis stabilized platform needs to be adjusted; S840. Control the movement of the dual-axis stabilized platform according to the target angle, thereby adjusting the pitch angle and roll angle of the visual camera.

[0009] According to the technical solution provided in the embodiment of the present application, the row scan line width reference value is obtained by the following formula: Formula (1) Where, D is the working distance of the visual camera, θ represents the field of view of the visual camera; ε Represents the margin value.

[0010] In a second aspect, the present application provides a roll-pitch visual camera scanning search planning system, which is used to implement the roll-pitch visual camera scanning search planning method described above, including: A receiving module, wherein the receiving module is configured to receive a scanning task and obtain scanning task information, wherein the scanning task information includes a scanning range in a through-track direction and flight space status information, and the flight space status information includes a flight altitude and a working distance; wherein a dual-axis stabilized platform is provided on the flight platform for mounting the visual camera, and the visual camera performs a rolling and pitching motion on the dual-axis stabilized platform; the flight platform is also provided with a navigation system, and the navigation system is used to obtain the flight space status information, and the navigation system has a navigation coordinate system, the origin of the navigation coordinate system is the origin of the navigation system, the X-axis is along the north direction, the Z-axis is along the plumb line direction, and the Y-axis, the X-axis, and the Z-axis satisfy the right-hand rule and point to the east direction; a construction module configured to construct an auxiliary coordinate system based on the scanning task information and to construct a row scan line in the auxiliary coordinate system, wherein the origin of the auxiliary coordinate system is the origin of the navigation system, the X-axis is parallel to the flight direction and the horizontal plane, the Z-axis points to the center of the earth along the plumb line, and the Y-axis satisfies the right-hand rule with the X-axis and the Z-axis; the row scan line is perpendicular to the X-axis of the auxiliary coordinate system, and the distance between the row scan line and the origin of the auxiliary coordinate system is determined according to the working distance, and the projection of the X-axis of the auxiliary coordinate system on the row scan line is the midpoint of the row scan line; a first calculation module, configured to obtain, by offline calculation according to the scanning task information, a set of related variable reference values ​​in the auxiliary coordinate system, the set of related variable reference values ​​including a plurality of related variables and related variable reference values ​​corresponding to each of the related variables, the related variables including at least a row scanning line width, a planned position coordinate, a planned position imaging moment, and a row scanning line midpoint, and the corresponding related variable reference values ​​including a row scanning line width reference value, a planned position reference coordinate, a planned position reference imaging moment, and a reference pitch pointing angle of the row scanning line midpoint; An initialization module, wherein the initialization module is configured to initialize each of the related variables according to the reference value of the related variables to obtain a plurality of initialized variables, wherein the visual camera is in a waiting state for scanning; control the visual camera to switch to a scanning search state, and control its movement according to the initialized variables; A first acquisition module, wherein the first acquisition module is configured to acquire real-time coordinates of the midpoint of the scan line in the auxiliary coordinate system, recorded as the real-time auxiliary coordinates of the midpoint of the scan line; a second acquisition module configured to obtain the real-time coordinates of the planned position in the auxiliary coordinate system according to the real-time auxiliary coordinates of the midpoint, recorded as the real-time auxiliary coordinates of the planned position; A conversion module configured to convert the real-time auxiliary coordinates of the midpoint of the row scan line and the real-time auxiliary coordinates of the planned position into the earth coordinate system to obtain the real-time earth coordinates of the midpoint of the row scan line and the real-time earth coordinates of the planned position; A control module is configured to control the visual camera to adjust its angle according to the real-time earth coordinates of the midpoint of the row scan line and the real-time earth coordinates of the planned position.

[0011] In a third aspect, the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the rolling visual camera scanning search planning method as described above are implemented.

[0012] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the rolling-tilt visual camera scanning search planning method as described above are implemented.

[0013] In summary, the present application proposes a rolling and pitching visual camera scanning and search planning method. After receiving the scanning task, an auxiliary coordinate system and a row scanning line are constructed according to the scanning task information, and the reference values ​​of the relevant variables are obtained by offline calculation. The relevant variables are initialized according to the reference values ​​of the relevant variables, and the camera's visual axis is controlled to point to the midpoint of the row scanning line. After entering the scanning mode, the midpoint coordinates of the row scanning line and the planned position coordinates in the auxiliary coordinate system are obtained in real time, and the two are converted into the earth coordinate system, and then the camera adjustment angle is controlled to realize the scanning search along the through-track direction; the present application can perform comprehensive scanning in the through-track and along-track directions through the rolling and pitching movements of the visual camera and the flight movement of the flying platform, greatly expanding the range of scanning imaging; in addition, for different search scenarios and task requirements, the present application can flexibly adjust the width of the row scanning line, the planned scanning position, etc. according to factors such as the scanning range in the through-track direction and the flight space state, adapt to various complex environmental conditions, and improve applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of a scanning search planning method for a roll-and-pitch visual camera provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a dual-axis stabilization platform provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of the terminal device provided in an embodiment of the present application.

[0015] The text annotations in the figure represent: 1. First rotating shaft; 2. Second rotating shaft; 3. Rolling outer frame; 4. Visual camera; 700. Computer system; 701. CPU; 702. ROM; 703. RAM; 704. Bus; 705. I / O interface; 706. Input section; 707. Output section; 708. Storage section; 709. Communication section; 710. Drive; 711. Removable media DETAILED DESCRIPTION The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0016] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0017] Example 1 As mentioned in the background technology, the present application proposes a rolling and pitching visual camera scanning search planning method. The flight platform has a navigation system, the navigation system has a navigation coordinate system, and the flight platform is provided with a dual-axis stabilized platform. The dual-axis stabilized platform is used to mount the visual camera 4. The visual camera 4 performs rolling and pitching motions on the dual-axis stabilized platform. The planning method includes the following steps: S100. Receive a scanning task and obtain scanning task information, wherein the scanning task information includes a scanning range in the through-track direction and flight space status information, and the flight space status information includes a flight altitude and a working distance; wherein a dual-axis stabilized platform is provided on the flight platform for mounting the visual camera 4, and the visual camera 4 performs a rolling and pitching motion on the dual-axis stabilized platform; the flight platform is also provided with a navigation system, the navigation system is used to obtain the flight space status information, and the navigation system has a navigation coordinate system, the origin of the navigation coordinate system is the navigation system origin, the X-axis is along the north direction, the Z-axis is along the plumb line direction, and the Y-axis, the X-axis, and the Z-axis satisfy the right-hand rule and point to the east direction; Please refer to Figure 2 As shown, the dual-axis stabilized platform has a rolling outer frame 3, which can rotate along a first rotating shaft 1, and the axis extension direction of the first rotating shaft 1 is parallel to the horizontal plane and parallel to the along-track direction; the pitch inner frame is connected to the first rotating shaft 1, and the pitch inner frame has a second rotating shaft 2, and the axis extension direction of the second rotating shaft 2 is parallel to the horizontal plane and parallel to the through-track direction. The visual camera can perform rolling and pitching movements on the two-axis stabilized platform, thereby realizing line scanning in the through-track direction; the working distance is the distance between the target point and the visual camera 4 along the X-axis direction of the navigation coordinate system; optionally, the scanning range in the through-track direction is 10 kilometers; the visual camera 4 has a camera base coordinate system, which is used to determine the relative position and direction of the visual camera 4 on the flight platform. Through this coordinate system, the installation position and angle relationship of the visual camera 4 relative to the flight platform can be clearly determined; wherein, its origin is the origin of the navigation system, the X-axis is forward along the flight platform, the Z-axis points to the bottom of the flight platform, and the Y-axis satisfies the right-hand rule with the X-axis and the Z-axis. S200. Construct an auxiliary coordinate system based on the scanning task information, and construct a row scan line in the auxiliary coordinate system, wherein the origin of the auxiliary coordinate system is the origin of the navigation system, the X-axis is parallel to the flight direction and the horizontal plane, the Z-axis is perpendicular to the center of the earth, and the Y-axis satisfies the right-hand rule with the X-axis and the Z-axis; the row scan line is perpendicular to the X-axis of the auxiliary coordinate system, and the distance between the row scan line and the origin of the auxiliary coordinate system is determined according to the working distance, and the projection of the X-axis of the auxiliary coordinate system on the row scan line is the midpoint of the row scan line; Since the projection of the X-axis of the auxiliary coordinate system on the row scan line is the midpoint of the row scan line, when the cross-track scanning range is 10 kilometers, the range of the row scan line along the Y-axis direction is [-5 kilometers, 5 kilometers]; S300. Obtaining a set of related variable reference values ​​in the auxiliary coordinate system by offline calculation based on the scanning task information, the set of related variable reference values ​​including a plurality of related variables and related variable reference values ​​corresponding to each of the related variables, the related variables including at least a row scanning line width, a planned position coordinate, a planned position imaging time, and a row scanning line midpoint, and the corresponding related variable reference values ​​including a row scanning line width reference value, a planned position reference coordinate, a planned position reference imaging time, and a reference pitch pointing angle of the row scanning line midpoint; The maximum width range that the row scan line needs to cover is determined according to the scanning range in the track crossing direction. The reference value of the row scan line width is obtained by the following formula: Formula (1) Where, D is the working distance of the visual camera 4, θ represents the field of view of the visual camera 4; ε Represents the margin value; wherein, the field of view angle is the inherent attribute of the visual camera 4, and the margin value is used to ensure that the scanning range in the cross-track direction can be completely covered.

[0018] Furthermore, obtaining the planned position reference coordinates includes the following steps: S310. Evenly divide the cross-track scanning range into a plurality of search reference sub-areas, each of which has a visual axis reference pointing point; wherein the visual axis reference pointing point of each search reference sub-area is relatively far from the midpoint of the row scan; The midpoint of the row scan line is marked as L0, and the coordinates are (l v , 0, h) A , where A represents the auxiliary coordinate system; S320. Obtain the distance between each of the visual axis reference pointing points and the midpoint of the row scan line, recorded as the planned distance dn , wherein a total of n said visual axis reference pointing points are included; S330. According to the planned distance, the coordinates of each of the visual axis reference pointing points are obtained, which are the planned position reference coordinates, denoted as (l v , d n , h) A .

[0019] Furthermore, the imaging moment is the moment when the image is scanned to each of the visual axis reference pointing points. The size of the imaging moment is related to the speed of the flight platform. When the speed of the flight platform is fast, the time interval between two adjacent visual axis reference pointing points is short. The imaging moments corresponding to each of the visual axis reference pointing points are recorded as t0, t1, t2, t3...t n , wherein the visual axis reference pointing point of the first visual axis of the visual camera 4 is set as the midpoint of the row scan line, and its corresponding imaging time is t0. The subsequent imaging time can be obtained by the following formula: Formula (2) Where, L is the width of each search reference sub-region, V is the speed of the flying platform.

[0020] Furthermore, the pitch pointing angle of the midpoint of the row scan line is obtained by the following formula: Formula (3) Where, P 0 represents the pitch angle of the visual axis of the visual camera 4 pointing to the midpoint of the row scan line, H represents the flight altitude, D is the working distance. In addition, when the visual axis of the visual camera 4 points to the midpoint of the row scan line, its rolling pointing angle is 0; for example P 0 is 30°.

[0021] S400. Initializing the related variables according to the reference values ​​of the related variables to obtain a plurality of initialized variables, the visual camera 4 is in a waiting state for scanning; controlling the visual camera 4 to switch to a scanning search state, and controlling its movement according to the initialized variables; Optionally, the pitch pointing angle of the midpoint of the row scan line is initialized to 30°, and the roll pointing angle is 0°; the driving mechanism drives the visual camera 4 to move through the dual-axis stabilized platform, so that at the initial moment, the visual axis of the visual camera 4 points to the midpoint of the row scan line, wherein the pitch pointing angle is 30° and the roll pointing angle is 0°.

[0022] S500. Real-time acquisition of the real-time coordinates of the midpoint of the scan line in the auxiliary coordinate system, recorded as the real-time auxiliary coordinates of the midpoint of the scan line; During the scanning process, parameters such as the flight altitude, working distance, imaging time, the pitch pointing angle of the midpoint of the line scan line, and the planned scanning position may change due to factors such as changes in the actual flight state, navigation errors, and dynamic changes in the target area. What remains unchanged is the camera imaging field of view. Obtaining the real-time auxiliary coordinates of the midpoint of the line scan line includes the following steps: S510. Obtain the angle between the viewing axis and the X-axis in the auxiliary coordinate system, recorded as the first angle; S520. Obtain the distance between the visual camera and the target point, recorded as the real-time distance; S530. Obtain the X-axis coordinate and the Z-axis coordinate of the real-time auxiliary coordinate of the midpoint of the line scan in the auxiliary coordinate system according to the first angle and the real-time distance, and obtain the real-time auxiliary coordinate of the midpoint of the line scan line.

[0023] The visual camera 4 is usually provided with a distance measuring device, which is used to obtain the real-time distance. The technology for obtaining the first angle is an existing technology and will not be described here. The X-axis coordinate and the Z-axis coordinate of the real-time auxiliary coordinate of the midpoint of the line scan in the auxiliary coordinate system are obtained by trigonometric calculation based on the first angle and the real-time distance, respectively. m and Z m .

[0024] S600. Obtain the real-time coordinates of the planned position in the auxiliary coordinate system according to the real-time auxiliary coordinates of the midpoint of the row scan line, recorded as the real-time auxiliary coordinates of the planned position; The real-time auxiliary coordinates of the planned positions are the coordinates of multiple planned positions on the row scan line during the real-time scanning process. Since the planned positions have no offset relative to the midpoint of the row scan line in the X-axis and Y-axis directions, the distance between each planned position and the midpoint of the row scan line is d. n , so the coordinates of each planned position, that is, the real-time auxiliary coordinates of the planned position are (X m , d n , Z m ).

[0025] S700. The real-time auxiliary coordinates of the midpoint of the scanning line and the real-time auxiliary coordinates of the planned position are converted into the earth coordinate system to obtain the real-time earth coordinates of the midpoint of the scanning line and the real-time earth coordinates of the planned position; Among them, the origin of the earth coordinate system is at the geometric center of the earth, the X-axis points to the 0-degree longitude of the equatorial plane, the Z-axis points to the north through the polar axis, and the Y-axis, X-axis, and Z-axis satisfy the right-hand rule; through the rotation matrix, the auxiliary coordinate system is translated and rotated to obtain the earth coordinate system, which is a prior art and will not be repeated here.

[0026] S800. Control the visual camera 4 to adjust the angle and direction according to the real-time earth coordinates of the midpoint of the scan line and the real-time earth coordinates of the planned position; The specific steps include: S810. Continuously obtain the position information and attitude information of the flight platform, as well as the frame angle information of the dual-axis stabilized platform; wherein the position information and attitude information are obtained by the navigation system, and the frame angle information is obtained by an angle sensor installed on the aircraft platform; S820. According to the position information and the posture information, the real-time earth coordinates of the planned position are converted into the camera base coordinate system or the line of sight coordinate system to obtain the real-time conversion coordinates of the planned position; Among them, the visual camera 4 has a line of sight coordinate system, which provides a precise reference frame for controlling the line of sight of the visual camera 4. By adjusting the relationship between the line of sight coordinate system and other coordinate systems, the line of sight of the visual camera 4 can be precisely controlled to point to a specific target position; the origin of the line of sight coordinate system is the intersection of the sensitive axes of the IMU (Inertial Measurement Unit) inside the two-axis stabilized platform. When the roll and pitch of the two-axis stabilized platform are both at zero position, the line of sight coordinate system coincides with the camera base coordinate system.

[0027] The real-time earth coordinates of the planned position are converted into the real-time conversion coordinates of the planned position by translation or rotation.

[0028] S830. Calculate the target angle according to the real-time conversion coordinates of the planned position and the frame angle information, where the target angle is the angle at which the dual-axis stabilized platform needs to be adjusted; The coordinates of the planned position are converted in real time to calculate the corresponding angle of the dual-axis stabilized platform, which is then compared with the frame angle information, and the difference between the two is the target angle.

[0029] S840. According to the target angle, the dual-axis stabilization platform is driven to move, thereby adjusting the pitch angle and roll angle of the visual camera 4; Optionally, the dual-axis stabilized platform is adjusted by a control device such as a motor or a servo system to achieve geographic pointing control.

[0030] In a preferred embodiment, step S800 further includes the following steps: S900. Set the time interval; S1000. After the time interval has passed, repeat steps S500-S800.

[0031] The time interval is the time it takes to complete a complete process (i.e., step 800) and start the next execution of the same process. By repeatedly executing steps S500-S800, the system will make real-time adjustments and optimizations based on current conditions, such as the position of the flight platform, the state of the target, and environmental factors, to ensure the accuracy and effectiveness of the scan.

[0032] In summary, this application proposes a rolling and pitching visual camera scanning and search planning method. Through the rolling and pitching motions of the visual camera and the flying motions of the flying platform, it can perform comprehensive scanning in the through-track and along-track directions, greatly expanding the scope of scanning imaging. In addition, by establishing multiple coordinate systems and performing coordinate conversion, the coordinates of the midpoint of the line scan line and the coordinates of the planned position are accurately calculated, so that during the scanning search process, the visual axis can be accurately controlled to perform geographic pointing according to the spatial coordinates of the planned position, thereby improving the accuracy and pertinence of the scan. Moreover, for different search scenarios and task requirements, this application can flexibly adjust the width of the line scan line, the planned scanning position, etc. according to factors such as the scanning range in the through-track direction and the flight space state, adapt to various complex environmental conditions, and improve applicability.

[0033] Example 2 Based on Example 1, this application proposes a rolling visual camera scanning search planning system, including: A receiving module, wherein the receiving module is configured to receive a scanning task and obtain scanning task information, wherein the scanning task information includes a scanning range in a through-track direction and flight space status information, and the flight space status information includes a flight altitude and a working distance; wherein a dual-axis stabilized platform is provided on the flight platform for mounting the visual camera 4, and the visual camera 4 performs a rolling and pitching motion on the dual-axis stabilized platform; the flight platform is also provided with a navigation system, and the navigation system is used to obtain the flight space status information, and the navigation system has a navigation coordinate system, the origin of the navigation coordinate system is the origin of the navigation system, the X-axis is along the north direction, the Z-axis is along the plumb line direction, and the Y-axis, the X-axis, and the Z-axis satisfy the right-hand rule and point to the east direction; a construction module configured to construct an auxiliary coordinate system based on the scanning task information and to construct a row scan line in the auxiliary coordinate system, wherein the origin of the auxiliary coordinate system is the origin of the navigation system, the X-axis is parallel to the flight direction and the horizontal plane, the Z-axis points to the center of the earth along the plumb line, and the Y-axis satisfies the right-hand rule with the X-axis and the Z-axis; the row scan line is perpendicular to the X-axis of the auxiliary coordinate system, and the distance between the row scan line and the origin of the auxiliary coordinate system is determined according to the working distance, and the projection of the X-axis of the auxiliary coordinate system on the row scan line is the midpoint of the row scan line; a first calculation module, configured to obtain, by offline calculation according to the scanning task information, a set of related variable reference values ​​in the auxiliary coordinate system, the set of related variable reference values ​​including a plurality of related variables and related variable reference values ​​corresponding to each of the related variables, the related variables including at least a row scanning line width, a planned position coordinate, a planned position imaging moment, and a row scanning line midpoint, and the corresponding related variable reference values ​​including a row scanning line width reference value, a planned position reference coordinate, a planned position reference imaging moment, and a reference pitch pointing angle of the row scanning line midpoint; An initialization module, wherein the initialization module is configured to initialize each of the related variables according to the reference value of the related variables to obtain a plurality of initialized variables, wherein the visual camera 4 is in a waiting state for scanning; controlling the visual camera 4 to switch to a scanning search state and controlling its movement according to the initialized variables; A first acquisition module, wherein the first acquisition module is configured to acquire real-time coordinates of the midpoint of the scan line in the auxiliary coordinate system, recorded as the real-time auxiliary coordinates of the midpoint of the scan line; a second acquisition module configured to obtain the real-time coordinates of the planned position in the auxiliary coordinate system according to the real-time auxiliary coordinates of the midpoint, recorded as the real-time auxiliary coordinates of the planned position; A conversion module configured to convert the real-time auxiliary coordinates of the midpoint of the row scan line and the real-time auxiliary coordinates of the planned position into the earth coordinate system to obtain the real-time earth coordinates of the midpoint of the row scan line and the real-time earth coordinates of the planned position; A control module is configured to control the visual camera 4 to adjust its angle according to the real-time earth coordinates of the midpoint of the row scan line and the real-time earth coordinates of the planned position.

[0034] Example 3 like Figure 3As shown, the computer system 700 of the terminal device includes a CPU 701, which can perform various appropriate actions and processes according to the programs stored in the ROM 702 or the programs loaded from the storage part 708 into the RAM 703. Various programs and data required for system operation are also stored in the RAM 703. The CPU 701, ROM 702 and RAM 703 are connected to each other via a bus 704. An I / O interface 705 is also connected to the bus 704. The following components are connected to the I / O interface 705: an input part 706 including a keyboard, a mouse, etc.; an output part 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part 708 including a hard disk, etc.; and a communication part 709 including a network interface card such as a LAN card, a modem, etc. The communication part 709 performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is mounted on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0035] In particular, according to the embodiments of the present application, the above reference process Figure 1 The described process can be implemented as a computer software program. For example, embodiment 3 of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component and / or installed from a removable medium. When the computer program is executed by CPU 701, the above-described functions defined in the present computer system 700 are performed.

[0036] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0037] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments 3 of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0038] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs, and when executed by the electronic device, the electronic device implements the roll-and-pitch visual camera scanning search planning method described in the above embodiments.

[0039] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments disclosed in the present application, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0040] Furthermore, although the steps of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0041] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by combining software with necessary hardware.

[0042] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features having similar functions disclosed in this application.

Claims

1. A scanning search planning method for a rolling visual camera, comprising the following steps: S100. Receive a scanning task and obtain scanning task information, wherein: The scanning task information includes a scanning range in the through-track direction and flight space status information, and the flight space status information includes a flight altitude and a working distance; wherein a dual-axis stabilization platform is provided on the flight platform for mounting the visual camera (4), and the visual camera (4) performs rolling and pitching motions on the dual-axis stabilization platform; the flight platform is also provided with a navigation system, and the navigation system is used to obtain the flight space status information, and the navigation system has a navigation coordinate system, the origin of the navigation coordinate system is the origin of the navigation system, the X-axis is along the due north direction, the Z-axis is along the plumb line direction, and the Y-axis, the X-axis, and the Z-axis satisfy the right-hand rule and point to the due east direction; S200. Construct an auxiliary coordinate system according to the scanning task information, and construct a row scanning line under the auxiliary coordinate system, wherein the origin of the auxiliary coordinate system is the origin of the navigation system, the X-axis is parallel to the flight direction and the horizontal plane, the Z-axis points to the center of the earth along the plumb line direction, and the Y-axis satisfies the right-hand rule with the X-axis and the Z-axis; the row scanning line is perpendicular to the X-axis of the auxiliary coordinate system, and the distance between the row scanning line and the origin of the auxiliary coordinate system is determined according to the working distance, and the projection of the X-axis of the auxiliary coordinate system on the row scanning line is the midpoint of the row scanning line; S300. Obtain a set of related variable reference values ​​in the auxiliary coordinate system according to the scanning task information by offline calculation, wherein the set of related variable reference values ​​includes a plurality of related variables and related variable reference values ​​corresponding to each of the related variables, wherein the related variables at least include a row scanning line width, a planned position coordinate, a planned position imaging time, and a row scanning line midpoint, and the corresponding related variable reference values ​​include a row scanning line width reference value, a planned position reference coordinate, a planned position reference imaging time, and a reference pitch pointing angle of the row scanning line midpoint; S400. Initializing each of the related variables according to the reference value of the related variables to obtain a plurality of initialized variables, wherein the visual camera (4) is in a waiting scanning state; controlling the visual camera (4) to switch to a scanning search state, and controlling its movement according to the initialized variables; S500. Real-time acquisition of the real-time coordinates of the midpoint of the scan line in the auxiliary coordinate system, recorded as the real-time auxiliary coordinates of the midpoint of the scan line; S600. Obtain the real-time coordinates of the planned position in the auxiliary coordinate system according to the real-time auxiliary coordinates of the midpoint, recorded as the real-time auxiliary coordinates of the planned position; S700. The real-time auxiliary coordinates of the midpoint of the scanning line and the real-time auxiliary coordinates of the planned position are converted to the earth coordinate system to obtain the real-time earth coordinates of the midpoint of the scanning line and the real-time earth coordinates of the planned position; S800. Control the visual camera (4) to adjust its angle according to the real-time earth coordinates of the midpoint of the row scanning line and the real-time earth coordinates of the planned position.

2. The scanning search planning method of a rolling visual camera according to claim 1, characterized in that: After step S800, the following steps are also included: S900. Set the time interval; S1000. After the time interval has passed, steps S500-S800 are repeated.

3. The scanning search planning method of a rolling visual camera according to claim 1, characterized in that: Obtaining the planned position reference coordinates comprises the following steps: S310. Evenly divide the scanning range in the cross-track direction into a plurality of search reference sub-areas, each of which has a visual axis reference pointing point; S320. Get the distance between each of the visual axis reference pointing points and the midpoint of the row scan line, recorded as the planning distance; S330. According to the planned distance, the coordinates of each visual axis reference pointing point are obtained, which are the planned position reference coordinates.

4. The scanning search planning method of a rolling visual camera according to claim 1, characterized in that: Step S500 includes the following steps: S510. Obtain the angle between the visual axis and the X axis in the auxiliary coordinate system, recorded as the first angle; S520. Obtain the distance between the visual camera 4 and the target point, recorded as the real-time distance; S530. Obtain the X-axis coordinate and the Z-axis coordinate of the real-time auxiliary coordinate of the midpoint of the line scan in the auxiliary coordinate system according to the first angle and the real-time distance, and obtain the real-time auxiliary coordinate of the midpoint of the line scan.

5. The scanning search planning method of a rolling visual camera according to claim 1, characterized in that: Step S800 includes the following steps: S810. Continuously acquiring the position information and attitude information of the flight platform, and the frame angle information of the dual-axis stabilized platform; S820. According to the position information and the posture information, the real-time earth coordinates of the planned position are converted into a camera base coordinate system or a visual axis coordinate system to obtain real-time conversion coordinates of the planned position; S830. According to the real-time conversion coordinates of the planned position and the frame angle information, a target angle is calculated, where the target angle is the angle that the dual-axis stabilized platform needs to be adjusted; S840. Controlling the movement of the dual-axis stabilization platform according to the target angle, thereby adjusting the pitch angle and roll angle of the visual camera (4).

6. The scanning search planning method of a rolling visual camera according to claim 1, characterized in that: The row scan line width reference value is obtained by the following formula: Formula (1) In the formula, D is the working distance of the visual camera (4), θ represents the field of view of the visual camera (4); ε Represents the margin value.

7. A rolling visual camera scanning search planning system, used to implement the rolling visual camera scanning search planning method according to any one of claims 1 to 6, characterized in that: The system comprises: A receiving module, the receiving module is configured to receive a scanning task and obtain scanning task information, wherein the scanning task information includes a scanning range in a through-track direction and flight space status information, and the flight space status information includes a flight altitude and a working distance; wherein a dual-axis stabilization platform is provided on the flight platform for mounting the visual camera (4), and the visual camera (4) performs rolling and pitching motions on the dual-axis stabilization platform; the flight platform is also provided with a navigation system, the navigation system is used to obtain the flight space status information, the navigation system has a navigation coordinate system, the origin of the navigation coordinate system is the origin of the navigation system, the X-axis is along the due north direction, the Z-axis is along the plumb line direction, and the Y-axis, the X-axis, and the Z-axis satisfy the right-hand rule and point to the due east direction; A construction module, wherein the construction module is configured to construct an auxiliary coordinate system according to the scanning task information, and construct a row scanning line in the auxiliary coordinate system, wherein the origin of the auxiliary coordinate system is the origin of the navigation system, the X-axis is parallel to the flight direction and the horizontal plane, the Z-axis points to the center of the earth along the plumb line direction, and the Y-axis satisfies the right-hand rule with the X-axis and the Z-axis; the row scanning line is perpendicular to the X-axis of the auxiliary coordinate system, and the distance between the row scanning line and the origin of the auxiliary coordinate system is determined according to the working distance, and the projection of the X-axis of the auxiliary coordinate system on the row scanning line is the midpoint of the row scanning line; A first calculation module, wherein the first calculation module is configured to obtain a set of related variable reference values ​​in the auxiliary coordinate system by offline calculation according to the scanning task information, wherein the set of related variable reference values ​​includes a plurality of related variables and related variable reference values ​​corresponding to each of the related variables, wherein the related variables at least include a row scanning line width, a planned position coordinate, a planned position imaging time, and a row scanning line midpoint, and the corresponding related variable reference values ​​include a row scanning line width reference value, a planned position reference coordinate, a planned position reference imaging time, and a reference pitch pointing angle of the row scanning line midpoint; An initialization module, wherein the initialization module is configured to initialize each of the related variables according to the reference value of the related variables to obtain a plurality of initialization variables, wherein the visual camera (4) is in a waiting scanning state; control the visual camera (4) to switch to a scanning search state, and control its movement according to the initialization variables; A first acquisition module, the first acquisition module is configured to acquire in real time the real-time coordinates of the midpoint of the row scan line in the auxiliary coordinate system, recorded as the real-time auxiliary coordinates of the midpoint of the row scan line; A second acquisition module, the second acquisition module is configured to obtain the real-time coordinates of the planned position in the auxiliary coordinate system according to the real-time auxiliary coordinates of the midpoint, recorded as the real-time auxiliary coordinates of the planned position; A conversion module, wherein the conversion module is configured to convert the real-time auxiliary coordinates of the midpoint of the row scan line and the real-time auxiliary coordinates of the planned position into the earth coordinate system to obtain the real-time earth coordinates of the midpoint of the row scan line and the real-time earth coordinates of the planned position; A control module, the control module being configured to control the visual camera (4) to adjust its angle according to the real-time earth coordinates of the midpoint of the row scanning line and the real-time earth coordinates of the planned position.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the rolling visual camera scanning search planning method as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the rolling visual camera scanning search planning method as described in any one of claims 1 to 6 are implemented.

Images