Unmanned aerial vehicle reconnaissance positioning system, method and equipment

Abstract

The invention discloses an unmanned aerial vehicle reconnaissance positioning system, method and device, and relates to the technical field of unmanned aerial vehicle positioning, and the system comprises an unmanned aerial vehicle task management device which is used for controlling an airborne photoelectric device to carry out stepping staring search on a to-be-scanned region based on prior coordinate information, and transmitting a generated first search video to an image recognition device; the image recognition equipment is used for carrying out frame-by-frame image recognition on the first search video to obtain coarse positioning coordinates of the positioning target; the unmanned aerial vehicle task management equipment is used for controlling the airborne optoelectronic equipment to perform staring investigation according to the coarse positioning coordinates to obtain a second search video; the image recognition equipment is used for carrying out secondary image recognition on the second search video to obtain pixel position information in the positioning target display picture; and the unmanned aerial vehicle task management equipment is used for controlling the airborne photoelectric equipment to track the positioning target according to the pixel position information, and acquiring a fine positioning coordinate of the positioning target. And automatic accurate positioning is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) positioning, and in particular to a UAV reconnaissance and positioning system, method and equipment. Background Art

[0002] The UAV can cruise at high altitude for a long time, and the operator controls the onboard optoelectronic equipment to conduct continuous reconnaissance and surveillance of a wide area. The operator manually controls the azimuth, pitch, focal length, field of view and other parameters of the onboard optoelectronic equipment through the command panel and joystick to achieve detailed reconnaissance of a given area and conduct manual interpretation to confirm whether there are any suspicious targets.

[0003] However, as the time for using drones for persistent ground surveillance and reconnaissance continues to increase, the pressure on personnel is gradually increasing. First, the current reconnaissance time of medium and large drones is generally 20 to 40 hours, which seriously increases the fatigue of personnel working for a long time; second, the control method of airborne optoelectronic equipment based on existing technology is mostly human-in-the-loop mode, that is, personnel need to operate the joystick non-stop to achieve reconnaissance and search. Third, the judgment of suspicious targets is all manual interpretation, and the fatigue caused by long-term reconnaissance will cause personnel to be unable to concentrate, resulting in misinterpretation of targets, missed targets, etc.

[0004] In summary, how to achieve automatic tracking and precise positioning of a target by a UAV is a technical problem to be solved in this field. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a UAV reconnaissance and positioning system, method, and device, which can realize the automatic tracking and precise positioning of the UAV to the positioning target. The specific scheme is as follows:

[0006] In the first aspect, the present application discloses a UAV reconnaissance and positioning system, including: a UAV mission management device, an airborne optoelectronic device, and an image recognition device, wherein:

[0007] The UAV task management device is used to control the airborne optoelectronic device to perform a step-and-gaze search on the area to be scanned in the priori coordinate information based on the received priori coordinate information, calibrate the reference point, and send the first search video generated during the search process to the image recognition device;

[0008] The image recognition device is used to perform frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and calculate the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain a rough positioning coordinate of each of the positioning targets, and then send the rough positioning coordinates to the UAV mission management device;

[0009] The UAV mission management device is used to control the airborne optoelectronic device to perform staring detection according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device;

[0010] The image recognition device is further used to perform secondary image recognition on the second search video to obtain pixel position information of the positioning target in the display picture, and send the pixel position information to the drone mission management device;

[0011] The UAV mission management device is also used to control the airborne optoelectronic device to track the positioning target according to the pixel position information, and obtain the precise positioning coordinates of the positioning target through laser ranging and a preset positioning algorithm.

[0012] Optionally, the UAV mission management device includes:

[0013] The instruction sending module is used to send a first search instruction and a priori coordinate information to the airborne optoelectronic device when it is detected that the relative distance between the positioning target and the drone where the airborne optoelectronic device is located meets a preset distance condition.

[0014] Optionally, the airborne optoelectronic device includes:

[0015] The first search module is used to adjust the detection direction according to the first search instruction and the prior coordinate information, so as to perform a grid-type spiral step scan on the area to be scanned, and use a laser to locate and calibrate the reference points in the video screen formed by the scan, so as to obtain a first search video carrying a timestamp for each frame of the image; wherein the first search instruction includes a reference point selection rule, so as to determine the reference point located in the video screen based on the reference point selection rule.

[0016] Optionally, the image recognition device includes:

[0017] A first image recognition module is used to perform frame-by-frame image recognition operations on the first search video by using the target shape features of each positioning target stored in a preset model database to obtain a number of positioning targets in each first search image, and calculate the coarse positioning coordinates of each positioning target in the corresponding first search image based on the timestamp of each frame image and the coordinate information of the reference point; wherein the coarse positioning coordinates include a timestamp, a positioning target ID, a positioning target longitude, a positioning target latitude, and a positioning target altitude;

[0018] The coordinate sending module is used to send the corresponding coarse positioning coordinates to the UAV mission management device according to the timestamp of each frame image.

[0019] Optionally, the airborne optoelectronic device includes:

[0020] A direction adjustment module, used for adjusting the detection direction of the airborne optoelectronic device according to the coarse positioning coordinates;

[0021] The second search module is used to perform a focus operation and perform a gaze detection in the detection direction to obtain a second search video.

[0022] Optionally, the second search module includes:

[0023] The gaze detection unit is used to perform a focusing operation and to perform gaze detection on a scene where a positioning target is located in a detection direction, so as to track a target scene including the positioning target.

[0024] Optionally, the image recognition device includes:

[0025] The second image recognition module is used to perform image recognition on each target scene through the target shape features of each positioning target stored in the preset model database, so as to determine the pixel position information of the positioning target in the display screen according to the difference between the current pixel position of the positioning target of each target scene and the coordinates of the reference point, and send the pixel position information to the UAV mission management device.

[0026] Optionally, the airborne optoelectronic device includes:

[0027] A laser ranging module, used to receive a laser ranging positioning instruction sent by the UAV mission management device based on the pixel position information, to emit a laser beam to the positioning target according to the laser ranging positioning instruction, and to receive a laser signal reflected by the positioning target, so as to calculate a target distance between the positioning target and the UAV;

[0028] The precise positioning module is used to calculate the precise positioning coordinates of the positioning target by using a preset positioning algorithm and according to the pixel position information, the target distance, the drone position, and the drone posture.

[0029] In a second aspect, the present application discloses a method for reconnaissance and positioning of a drone, comprising:

[0030] Based on the received priori coordinate information, control the airborne optoelectronic device to perform a step-and-gaze search on the area to be scanned in the priori coordinate information, calibrate the reference point, and send a first search video generated during the search process to the image recognition device;

[0031] The image recognition device performs frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and calculates the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain a rough positioning coordinate of each of the positioning targets, and then sends the rough positioning coordinates to the UAV mission management device;

[0032] Controlling the airborne optoelectronic device to perform staring detection according to the rough positioning coordinates through the UAV mission management device to obtain a second search video, and sending the second search video to the image recognition device;

[0033] Performing secondary image recognition on the second search video by the image recognition device to obtain pixel position information of the positioning target in the display screen, and sending the pixel position information to the UAV task management device;

[0034] The unmanned aerial vehicle task management device is used to control the airborne optoelectronic device to track the positioning target according to the pixel position information, and the precise positioning coordinates of the positioning target are obtained through laser ranging and a preset positioning algorithm.

[0035] In a third aspect, the present application discloses a UAV reconnaissance and positioning method, which is applied to a UAV mission management device, comprising:

[0036] Based on the received priori coordinate information, the airborne optoelectronic device is controlled to perform a step-and-gaze search on the area to be scanned in the priori coordinate information, and the reference point is calibrated, and a first search video generated in the search process is sent to an image recognition device, so that the image recognition device performs frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and the positioning targets in each of the first search images are calculated based on the coordinate information of the reference point to obtain a rough positioning coordinate of each of the positioning targets, and then the rough positioning coordinates are sent to the UAV mission management device;

[0037] Controlling the airborne optoelectronic device to perform staring detection according to the coarse positioning coordinates to obtain a second search video, and sending the second search video to the image recognition device so that the image recognition device performs secondary image recognition on the second search video to obtain pixel position information of the positioning target in the display screen, and sending the pixel position information to the UAV mission management device;

[0038] The airborne optoelectronic device is controlled to track the positioning target according to the pixel position information, and the precise positioning coordinates of the positioning target are obtained through laser ranging and a preset positioning algorithm.

[0039] In a fourth aspect, the present application discloses an electronic device, including:

[0040] Memory, used to store computer programs;

[0041] A processor is used to execute the computer program to implement the steps of the aforementioned UAV reconnaissance and positioning method disclosed above.

[0042] It can be seen that the present application discloses a UAV reconnaissance and positioning system, including: a UAV task management device, an airborne optoelectronic device, and an image recognition device, wherein the UAV task management device is used to control the airborne optoelectronic device to perform a step-and-gaze search on the area to be scanned in the priori coordinate information based on the received priori coordinate information, and to calibrate the reference point, and send the first search video generated by the search process to the image recognition device; the image recognition device is used to perform frame-by-frame image recognition on the first search video to obtain a first search image containing several positioning targets, and calculate the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the coarse positioning coordinates of each of the positioning targets, and then The coarse positioning coordinates are sent to the UAV task management device; the UAV task management device is used to control the airborne optoelectronic device to perform gaze detection according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device; the image recognition device is also used to perform secondary image recognition on the second search video to obtain the pixel position information of the positioning target in the display screen, and send the pixel position information to the UAV task management device; the UAV task management device is also used to control the airborne optoelectronic device to track the positioning target according to the pixel position information, and obtain the precise positioning coordinates of the positioning target through laser ranging and a preset positioning algorithm. It can be seen that the system successively determines each positioning target by performing frame-by-frame image recognition on the first search video through the image recognition device, and calculates the relative coordinates based on the pre-defined reference point to obtain the coarse positioning coordinates of each positioning target, and then performs secondary image recognition based on this gaze detection to obtain the pixel position information of the positioning target in the display screen, and finally obtains the precise positioning coordinates by combining laser ranging and a preset positioning algorithm. Through multiple rounds of image recognition and the fusion processing of pixel position information and laser ranging information, the determination of the target position can be gradually converged, which greatly improves the positioning accuracy compared to the single recognition or single information positioning method. And a series of automated processes from obtaining prior coordinate information to final precise positioning coordinate determination are realized. Compared with the traditional method of relying on manual manipulation for reconnaissance and positioning, it greatly reduces the manual intervention link, can search, identify and locate the target area more quickly, and effectively improves the overall reconnaissance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0044] Figure 1 This is a structural diagram of a UAV reconnaissance and positioning system disclosed in this application;

[0045] Figure 2 A schematic diagram of a grid-type step scanning disclosed in this application;

[0046] Figure 3 A pixel tracking result diagram disclosed in this application;

[0047] Figure 4 This is a flow chart of a UAV reconnaissance and positioning method disclosed in this application;

[0048] Figure 5 A flowchart of a specific UAV reconnaissance and positioning method disclosed in this application;

[0049] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] Drones can cruise at high altitudes for long periods of time, and operators control airborne optoelectronic equipment to conduct continuous reconnaissance and surveillance of wide areas. Specifically, the operator controls the airborne optoelectronic equipment by operating the joystick at the ground station. The data stream is the remote control command from the ground station sent to the airborne measurement and control system of the drone through a wireless link. The airborne measurement and control system transmits the control command to the drone mission management device, and the drone mission management device dispatches the airborne optoelectronic equipment to execute the corresponding command to perform reconnaissance operations. In this way, the operator manually controls the azimuth, pitch, focal length, field of view and other parameters of the airborne optoelectronic equipment through the command panel and joystick to achieve detailed reconnaissance of a given area, and conducts manual interpretation to confirm whether there are suspicious targets.

[0052] However, as the time for using drones for persistent ground surveillance and reconnaissance continues to increase, the pressure on personnel is gradually increasing. First, the current reconnaissance time of medium and large drones is generally 20 to 40 hours, which seriously increases the fatigue of personnel working for a long time; second, the control method of airborne optoelectronic equipment based on existing technology is mostly human-in-the-loop mode, that is, personnel need to operate the joystick non-stop to achieve reconnaissance and search. Third, the judgment of suspicious targets is all manual interpretation, and the fatigue caused by long-term reconnaissance will cause personnel to be unable to concentrate, resulting in misinterpretation of targets, missed targets, etc.

[0053] To this end, the present invention provides a UAV reconnaissance and positioning solution, which can realize automatic tracking and precise positioning of the UAV to the positioning target.

[0054] Reference Figure 1 As shown, the embodiment of the present application discloses a drone reconnaissance and positioning system, including: a drone mission management device 11, an airborne optoelectronic device 12, and an image recognition device 13, wherein:

[0055] The UAV task management device 11 is used to control the airborne optoelectronic device 12 to perform a step gaze search on the to-be-scanned area in the priori coordinate information based on the received priori coordinate information, and to calibrate the reference point, and to send the first search video generated during the search process to the image recognition device 13;

[0056] The image recognition device 13 is used to perform frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and calculate the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the rough positioning coordinates of each of the positioning targets, and then send the rough positioning coordinates to the UAV task management device 11;

[0057] The UAV mission management device 11 is used to control the airborne optoelectronic device 12 to perform staring detection according to the rough positioning coordinates to obtain a second search video, and send the second search video to the image recognition device 13;

[0058] The image recognition device 13 is further used to perform secondary image recognition on the second search video to obtain pixel position information of the positioning target in the display screen, and send the pixel position information to the drone mission management device 11;

[0059] The UAV mission management device 11 is also used to control the airborne optoelectronic device 12 to track the positioning target according to the pixel position information, and obtain the precise positioning coordinates of the positioning target through laser ranging and a preset positioning algorithm.

[0060] In this embodiment, the drone reconnaissance and positioning system includes three parts: drone task management equipment 11, airborne optoelectronic equipment 12, and image recognition equipment 13. Among them, the drone task management equipment 11 is responsible for comprehensive business management such as condition judgment, logic execution, data conversion and processing, and the airborne optoelectronic equipment 12 implements grid-type step-by-step gaze search, field of view matching adjustment, target tracking and precise positioning, and the image recognition equipment 13 completes the real-time target detection and recognition of the video stream output by the airborne optoelectronic equipment 12. It should be noted that the airborne optoelectronic equipment 12 searches and detects the target based on visible light or infrared imaging. Traditionally, the airborne optoelectronic equipment 12 itself does not have the ability of autonomous target detection and precise positioning, and requires the operator to complete target discrimination in the loop, manually track and start laser ranging positioning. Therefore, the drone reconnaissance and positioning system constructed by the present invention can achieve autonomous precise positioning of the airborne optoelectronic equipment 12 through the mutual cooperation of various parts.

[0061] Specifically, in order to realize the intelligent target recognition and precise positioning process, the UAV task management device 11 is used as the control center, and the target information is first obtained through the long-range detection payloads such as synthetic aperture radar (hereinafter referred to as an example), communication reconnaissance equipment, and electronic reconnaissance equipment. However, due to the hardware conditions, its accuracy has a large error. Synthetic aperture radar is used to obtain the prior coordinate information containing the positioning target, and send the prior coordinate information to the UAV task management device 11. The UAV task management device 11 sends the acquired prior coordinate information to the airborne optoelectronic device 12, and performs continuous stepping and staring search on the area to be scanned where the prior coordinate information is located. The image recognition device 13 combines the shape features of each positioning target in the preset model database to perform frame-by-frame target recognition on the video stream during the continuous search process, and performs target pixel deviation solution based on the reference point information determined in the area to be scanned and the identified positioning target, and outputs the target coordinates. The UAV task management device 11 guides the airborne optoelectronic device 12 to the image recognition device 13 to output the target coordinates and maintain scene tracking. The image recognition device 13 performs secondary target recognition on the image frames of the scene tracking process and outputs the target pixel deviation. The UAV mission management device 11 controls the airborne optoelectronic device 12 to perform pixel tracking according to the target pixel deviation and start laser ranging to calculate the precise coordinates of the current positioning target.

[0062] The drone mission management device 11 includes:

[0063] The instruction issuing module 111 is used to issue a first search instruction and a priori coordinate information to the airborne optoelectronic device 12 when it is detected that the relative distance between the positioning target and the UAV where the airborne optoelectronic device 12 is located meets a preset distance condition.

[0064] It can be understood that when the UAV mission management device 11 detects that the relative distance between the target to be located and the carrier is ≤30km, it sends a first search instruction to the airborne optoelectronic device 12. At the same time, it sends the prior coordinate information received by itself to the airborne optoelectronic device 12, wherein the prior coordinate information is detected and provided by the airborne radar SAR or other information sources, and the prior coordinate information is specifically the rough position information of the location of the positioning target, including the position and possible range.

[0065] Moreover, the preset distance condition is determined by taking the average value of the relative distance between the positioning target and the carrier corresponding to the sampled image obtained by sampling the airborne optoelectronic device 12 to meet the recognition pixel usage requirements. In this way, it is ensured that the image collected by the airborne optoelectronic device 12 that meets the preset distance condition can be recognized at the pixel level.

[0066] The airborne optoelectronic device 12 comprises:

[0067] The first search module 121 is used to adjust the detection direction according to the first search instruction and the prior coordinate information to perform a grid-type spiral step scan on the area to be scanned, and use a laser to locate and calibrate the reference points in the video screen formed by the scan to obtain a first search video carrying a timestamp for each frame of the image; wherein the first search instruction includes a reference point selection rule so as to determine the reference point located in the video screen based on the reference point selection rule.

[0068] It can be understood that after the first search module 121 in the airborne optoelectronic device 12 receives the first search instruction and the priori coordinate information, the first search module 121 directly controls the airborne optoelectronic device 12 to perform a range area scan toward the priori coordinate information, i.e., a cell search+;

[0069] It should be noted that the first search module 121 sends a cell search + instruction and the approximate coordinates of the target to the airborne optoelectronic device 12, and determines the reference point in the entire search process according to the reference point selection rule in the first search instruction. For example, at the beginning of a grid-type step-and-gaze search, it is stipulated that the upper left corner or other fixed corner position of the area to be searched is used as the reference point. Such a rule can ensure that the selection of the reference point is consistent and certain each time the search is performed, and then the airborne optoelectronic device 12 enters the cell search + mode according to the step dwell time of 80ms and the overlap rate of 20%;

[0070] The first step is geographic tracking to the target location to be located;

[0071] The second step is auto focus;

[0072] The third step is to complete laser ranging and positioning, and calculate the position coordinate information of the target to be located;

[0073] The fourth step is to use the location of the target to be located as the initial grid and perform a grid-type spiral step scan. The process of the grid-type spiral step scan is as follows: Figure 2 As shown, the area to be scanned, which has been divided into grids, is scanned in a spiral step-by-step manner in the direction of the arrow. The current working status and cell search + status are broadcast on the system bus. The default initialization is N / A, the ongoing status is ING, the complete cycle is END, and the complete cycle is not completed is FAILURE;

[0074] In the fifth step, the onboard optoelectronic device 12 sends video data to the image recognition device 13 via a digital signal, and the timestamp and reference point of each frame are added to the video stream. In this way, a first search video carrying the timestamp and reference point of each frame image is obtained.

[0075] The image recognition device 13 includes:

[0076] The first image recognition module 131 is used to perform frame-by-frame image recognition operations on the first search video through the target shape features of each positioning target stored in the preset model database to obtain a number of positioning targets in each first search image, and calculate the coarse positioning coordinates of each positioning target in the corresponding first search image based on the timestamp of each frame image and the coordinate information of the reference point; wherein the coarse positioning coordinates include a timestamp, a positioning target ID, a positioning target longitude, a positioning target latitude, and a positioning target altitude;

[0077] The coordinate sending module 132 is used to send the corresponding coarse positioning coordinates to the UAV mission management device 11 according to the timestamp of each frame of image.

[0078] It can be understood that the first image recognition module 131 of the image recognition device 13 performs image target matching and recognition on the first search video frame by frame according to the target shape feature information of several types of positioning targets in the model database to identify each positioning target existing in the first search image (which may be 0, 1, or more, and this is not specifically limited), and then calculates the coordinate information of each positioning target located in each first search image through the image algorithm module and the coordinates of the reference point attached to the first search video as the rough positioning coordinates, and then the coordinate sending module 12 sends the rough positioning coordinates to the bus according to the timestamp information. Specifically, the execution steps of the first image recognition module 131 and the coordinate sending module 132 are as follows:

[0079] a) receiving 3G-SDI video stream data from the airborne optoelectronic device 12, i.e., the first search video;

[0080] b) performing image target recognition on the first search video frame by frame, and receiving additional information (coordinates of the reference point);

[0081] c) Calculate the target position through the image algorithm module;

[0082] d) Output rough positioning coordinates; wherein the output rough positioning coordinates include timestamp, target ID, target longitude, target latitude, target altitude and data validity, etc.

[0083] The airborne optoelectronic device 12 comprises:

[0084] A direction adjustment module 122, used to adjust the detection direction of the recording photoelectric device according to the rough positioning coordinates;

[0085] The second search module 123 is used to perform a focus operation and perform a gaze detection in the detection direction to obtain a second search video.

[0086] The second search module 122 includes:

[0087] The gaze detection unit is used to perform a focusing operation and to perform gaze detection on a scene where a positioning target is located in a detection direction, so as to track a target scene including the positioning target.

[0088] It can be understood that after the image recognition device 13 calculates the coarse positioning coordinates, the image recognition device 13 follows the airborne optoelectronic device 12 to the coarse positioning target position to maintain the scene tracking process. Specifically, after the UAV mission management device 11 sends a gaze command and coarse positioning coordinates to the airborne optoelectronic device 12, the gaze detection steps are as follows:

[0089] a) the airborne optoelectronic device 12 performs a gaze process according to the gaze instruction and the rough positioning coordinates;

[0090] b) Geographic tracking to the location of the coarse positioning coordinates;

[0091] c) performing lens auto-focus operation;

[0092] d) Perform scene tracking.

[0093] The image recognition device 13 includes:

[0094] The second image recognition module 133 is used to perform image recognition on each target scene through the target shape features of each positioning target stored in the preset model database, so as to determine the pixel position information of the positioning target in the display screen according to the difference between the current pixel position of the positioning target of each target scene and the coordinates of the reference point, and send the pixel position information to the UAV mission management device.

[0095] It can be understood that, like the recognition process of the first image recognition module 131, image recognition is performed on each target scene of the second search video generated under coarse positioning, the pixel position information of the positioned target is determined, and the information is sent to the UAV task management device 11, and then the UAV task management device 11 sends the pixel position information to the airborne optoelectronic device 12.

[0096] Furthermore, the onboard optoelectronic device 12 performs pixel tracking on the positioning target identified during the gaze process, and the steps are as follows:

[0097] a) The UAV mission management device 11 sends a gaze command and a target pixel deviation value to the airborne optoelectronic device 12; wherein the target pixel deviation value is the pixel position information of the positioning target in the display screen determined by the difference between the current pixel position of the positioning coordinates and the coordinates of the reference point.

[0098] b) The airborne optoelectronic device 12 performs pixel tracking according to the target pixel deviation value and maintains an automatic tracking state, such as Figure 3 shown.

[0099] The airborne optoelectronic device 12 comprises:

[0100] The laser ranging module 124 is used to receive the laser ranging positioning instruction sent by the UAV task management device based on the pixel position information, to emit a laser beam to the positioning target according to the laser ranging positioning instruction, and to receive the laser signal reflected by the positioning target to calculate the target distance between the positioning target and the UAV;

[0101] The precise positioning module 125 is used to calculate the precise positioning coordinates of the positioning target by using a preset positioning algorithm and according to the pixel position information, the target distance, the drone position, and the drone attitude.

[0102] It can be understood that the airborne optoelectronic device 12 performs laser ranging positioning on the automatically tracked positioning target and sends the precise positioning coordinates to the bus. Specifically:

[0103] a) The UAV mission management device 11 sends a Mark instruction to the airborne optoelectronic device 12;

[0104] b) Perform laser ranging positioning on the positioning target;

[0105] c) Calculate the precise positioning coordinates of the positioning target;

[0106] d) Output precise positioning coordinates to the bus.

[0107] It can be seen that the present application discloses a UAV reconnaissance and positioning system, including: a UAV task management device, an airborne optoelectronic device, and an image recognition device, wherein the UAV task management device is used to control the airborne optoelectronic device to perform a step-and-gaze search on the area to be scanned in the priori coordinate information based on the received priori coordinate information, and to calibrate the reference point, and send the first search video generated by the search process to the image recognition device; the image recognition device is used to perform frame-by-frame image recognition on the first search video to obtain a first search image containing several positioning targets, and calculate the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the coarse positioning coordinates of each of the positioning targets, and then The coarse positioning coordinates are sent to the UAV task management device; the UAV task management device is used to control the airborne optoelectronic device to perform gaze detection according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device; the image recognition device is also used to perform secondary image recognition on the second search video to obtain the pixel position information of the positioning target in the display screen, and send the pixel position information to the UAV task management device; the UAV task management device is also used to control the airborne optoelectronic device to track the positioning target according to the pixel position information, and obtain the precise positioning coordinates of the positioning target through laser ranging and a preset positioning algorithm. It can be seen that the system successively determines each positioning target by performing frame-by-frame image recognition on the first search video through the image recognition device, and calculates the relative coordinates based on the pre-defined reference point to obtain the coarse positioning coordinates of each positioning target, and then performs secondary image recognition based on this gaze detection to obtain the pixel position information of the positioning target in the display screen, and finally obtains the precise positioning coordinates by combining laser ranging and a preset positioning algorithm. Through multiple rounds of image recognition and the fusion processing of pixel position information and laser ranging information, the determination of the target position can be gradually converged, which greatly improves the positioning accuracy compared to the single recognition or single information positioning method. And a series of automated processes from obtaining prior coordinate information to final precise positioning coordinate determination are realized. Compared with the traditional method of relying on manual manipulation for reconnaissance and positioning, it greatly reduces the manual intervention link, can search, identify and locate the target area more quickly, and effectively improves the overall reconnaissance efficiency.

[0108] Reference Figure 4 As shown, the present invention also discloses a UAV reconnaissance and positioning method, including:

[0109] Step S11: Based on the received priori coordinate information, the airborne optoelectronic device is controlled to perform a step-and-gaze search on the area to be scanned in the priori coordinate information, and a reference point is calibrated, and a first search video generated during the search process is sent to the image recognition device;

[0110] Step S12: performing frame-by-frame image recognition on the first search video through the image recognition device to obtain a first search image containing a plurality of positioning targets, and calculating the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the rough positioning coordinates of each of the positioning targets, and then sending the rough positioning coordinates to the UAV mission management device;

[0111] Step S13: controlling the airborne optoelectronic device to perform staring detection according to the rough positioning coordinates through the UAV mission management device to obtain a second search video, and sending the second search video to the image recognition device;

[0112] Step S14: performing secondary image recognition on the second search video by the image recognition device to obtain pixel position information of the positioning target in the display screen, and sending the pixel position information to the UAV mission management device;

[0113] Step S15: The UAV mission management device is used to control the airborne optoelectronic device to track the positioning target according to the pixel position information, and the precise positioning coordinates of the positioning target are obtained through laser ranging and a preset positioning algorithm.

[0114] It can be seen that the present application discloses controlling an airborne optoelectronic device to perform a step gaze search on the area to be scanned in the priori coordinate information based on the received priori coordinate information, and calibrating the reference point, and sending the first search video generated by the search process to an image recognition device; performing frame-by-frame image recognition on the first search video by the image recognition device to obtain a first search image containing several positioning targets, and calculating the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the coarse positioning coordinates of each of the positioning targets, and then sending the coarse positioning coordinates to a drone task management device; controlling the airborne optoelectronic device to perform gaze detection by the drone task management device and according to the coarse positioning coordinates to obtain a second search video, and sending the second search video to the image recognition device; performing secondary image recognition on the second search video by the image recognition device to obtain the pixel position information of the positioning target in the display screen, and sending the pixel position information to the drone task management device; controlling the airborne optoelectronic device to track the positioning target by the drone task management device and according to the pixel position information, and obtaining the precise positioning coordinates of the positioning target by laser ranging and a preset positioning algorithm. It can be seen that the system first uses the image recognition device to perform frame-by-frame image recognition on the first search video to determine each positioning target, and calculates the relative coordinates based on the pre-defined reference point to obtain the rough positioning coordinates of each positioning target, and then performs secondary image recognition based on this gaze detection to obtain the pixel position information of the positioning target in the display screen, and finally combines laser ranging and preset positioning algorithms to obtain precise positioning coordinates. Through multiple rounds of image recognition and the fusion processing of pixel position information and laser ranging information, the determination of the target position can be gradually converged, which greatly improves the positioning accuracy compared to the single recognition or single information positioning method. And a series of automated processes from obtaining prior coordinate information to determining the final precise positioning coordinates are realized. Compared with the traditional method of relying on manual manual operation for reconnaissance and positioning, it greatly reduces the manual intervention link, can search, identify and locate the target area more quickly, and effectively improves the overall reconnaissance efficiency.

[0115] Reference Figure 5 As shown, the present invention also discloses a UAV reconnaissance and positioning method, which is applied to UAV mission management equipment, including:

[0116] Step S21: Based on the received prior coordinate information, control the airborne optoelectronic device to perform a step gaze search on the area to be scanned in the prior coordinate information, and calibrate the reference point, and send the first search video generated by the search process to the image recognition device, so that the image recognition device performs frame-by-frame image recognition on the first search video to obtain a first search image containing several positioning targets, and calculate the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the rough positioning coordinates of each of the positioning targets, and then send the rough positioning coordinates to the UAV mission management device.

[0117] Step S22: Control the airborne optoelectronic device to perform gaze detection according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device so that the image recognition device performs secondary image recognition on the second search video to obtain pixel position information of the positioning target in the display screen, and send the pixel position information to the UAV mission management device.

[0118] Step S23: controlling the airborne optoelectronic device to track the positioning target according to the pixel position information, and obtaining the precise positioning coordinates of the positioning target through laser ranging and a preset positioning algorithm.

[0119] Among them, for the detailed contents of the above steps, please refer to the aforementioned disclosed contents, which will not be repeated here.

[0120] It can be seen that the present application discloses controlling an airborne optoelectronic device to perform a step gaze search on the area to be scanned in the priori coordinate information based on the received priori coordinate information, and calibrating the reference point, sending the first search video generated in the search process to an image recognition device, so that the image recognition device performs frame-by-frame image recognition on the first search video to obtain a first search image containing several positioning targets, and calculates the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the coarse positioning coordinates of each of the positioning targets, and then sends the coarse positioning coordinates to the UAV task management device; controlling the airborne optoelectronic device to perform gaze detection according to the coarse positioning coordinates to obtain a second search video, and sending the second search video to the image recognition device, so that the image recognition device performs secondary image recognition on the second search video to obtain the pixel position information of the positioning target in the display screen, and sends the pixel position information to the UAV task management device; controlling the airborne optoelectronic device to track the positioning target according to the pixel position information, and obtaining the precise positioning coordinates of the positioning target through laser ranging and a preset positioning algorithm. It can be seen that the system first uses the image recognition device to perform frame-by-frame image recognition on the first search video to determine each positioning target, and calculates the relative coordinates based on the pre-defined reference point to obtain the rough positioning coordinates of each positioning target, and then performs secondary image recognition based on this gaze detection to obtain the pixel position information of the positioning target in the display screen, and finally combines laser ranging and preset positioning algorithms to obtain precise positioning coordinates. Through multiple rounds of image recognition and the fusion processing of pixel position information and laser ranging information, the determination of the target position can be gradually converged, which greatly improves the positioning accuracy compared to the single recognition or single information positioning method. And a series of automated processes from obtaining prior coordinate information to determining the final precise positioning coordinates are realized. Compared with the traditional method of relying on manual manual operation for reconnaissance and positioning, it greatly reduces the manual intervention link, can search, identify and locate the target area more quickly, and effectively improves the overall reconnaissance efficiency.

[0121] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0122] Figure 6A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the drone reconnaissance and positioning method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0123] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0124] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0125] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0126] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the drone reconnaissance and positioning method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. In addition to data received by the electronic device and transmitted from an external device, the data 223 can also include data collected by its own input and output interface 25, etc.

[0127] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned disclosed drone reconnaissance and positioning method. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0128] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0129] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly with hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory RAM (Random Access Memory), memory, read-only memory ROM (Read Only Memory), electrically programmable EPROM (Electrically Programmable Read Only Memory), electrically erasable programmable EEPROM (ElectricErasable Programmable Read Only Memory), register, hard disk, removable disk, CD-ROM (CompactDisc-Read Only Memory), or any other form of storage medium known in the technical field.

[0130] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0131] The scheme provided by the present invention is introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A UAV reconnaissance and positioning system, characterized in that: include: UAV mission management equipment, airborne optoelectronic equipment, image recognition equipment, including: The UAV task management device is used to control the airborne optoelectronic device to perform a step-and-gaze search on the area to be scanned in the priori coordinate information based on the received priori coordinate information, calibrate the reference point, and send the first search video generated during the search process to the image recognition device; The image recognition device is used to perform frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and calculate the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain a rough positioning coordinate of each of the positioning targets, and then send the rough positioning coordinates to the UAV mission management device; The UAV mission management device is used to control the airborne optoelectronic device to perform staring detection according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device; The image recognition device is further used to perform secondary image recognition on the second search video to obtain pixel position information of the positioning target in the display picture, and send the pixel position information to the drone mission management device; The UAV mission management device is also used to control the airborne optoelectronic device to track the positioning target according to the pixel position information, and obtain the precise positioning coordinates of the positioning target through laser ranging and a preset positioning algorithm.

2. The unmanned aerial vehicle reconnaissance and positioning system according to claim 1, characterized in that: The UAV mission management device comprises: The instruction sending module is used to send a first search instruction and a priori coordinate information to the airborne optoelectronic device when it is detected that the relative distance between the positioning target and the drone where the airborne optoelectronic device is located meets a preset distance condition.

3. The unmanned aerial vehicle reconnaissance and positioning system according to claim 2, characterized in that: The airborne optoelectronic device comprises: The first search module is used to adjust the detection direction according to the first search instruction and the prior coordinate information, so as to perform a grid-type spiral step scan on the area to be scanned, and use a laser to locate and calibrate the reference points in the video screen formed by the scan, so as to obtain a first search video carrying a timestamp for each frame of the image; wherein the first search instruction includes a reference point selection rule, so as to determine the reference point located in the video screen based on the reference point selection rule.

4. The unmanned aerial vehicle reconnaissance and positioning system according to claim 3, characterized in that: The image recognition device comprises: A first image recognition module is used to perform frame-by-frame image recognition operations on the first search video by using the target shape features of each positioning target stored in a preset model database to obtain a number of positioning targets in each first search image, and calculate the coarse positioning coordinates of each positioning target in the corresponding first search image based on the timestamp of each frame image and the coordinate information of the reference point; wherein the coarse positioning coordinates include a timestamp, a positioning target ID, a positioning target longitude, a positioning target latitude, and a positioning target altitude; The coordinate sending module is used to send the corresponding coarse positioning coordinates to the UAV mission management device according to the timestamp of each frame image.

5. The unmanned aerial vehicle reconnaissance and positioning system according to claim 3, characterized in that: The airborne optoelectronic device comprises: A direction adjustment module, used for adjusting the detection direction of the airborne optoelectronic device according to the coarse positioning coordinates; The second search module is used to perform a focus operation and perform a gaze detection in the detection direction to obtain a second search video.

6. The unmanned aerial vehicle reconnaissance and positioning system according to claim 5, characterized in that: The second search module includes: The gaze detection unit is used to perform a focusing operation and to perform gaze detection on a scene where a positioning target is located in a detection direction, so as to track a target scene including the positioning target.

7. The unmanned aerial vehicle reconnaissance and positioning system according to claim 6, characterized in that: The image recognition device comprises: The second image recognition module is used to perform image recognition on each target scene through the target shape features of each positioning target stored in the preset model database, so as to determine the pixel position information of the positioning target in the display screen according to the difference between the current pixel position of the positioning target of each target scene and the coordinates of the reference point, and send the pixel position information to the UAV mission management device.

8. The unmanned aerial vehicle reconnaissance and positioning system according to claim 7, characterized in that: The airborne optoelectronic device comprises: A laser ranging module, used to receive a laser ranging positioning instruction sent by the UAV mission management device based on the pixel position information, to emit a laser beam to the positioning target according to the laser ranging positioning instruction, and to receive a laser signal reflected by the positioning target, so as to calculate a target distance between the positioning target and the UAV; The precise positioning module is used to calculate the precise positioning coordinates of the positioning target by using a preset positioning algorithm and according to the pixel position information, the target distance, the drone position, and the drone posture.

9. A method for reconnaissance and positioning of an unmanned aerial vehicle, characterized in that: include: Based on the received priori coordinate information, control the airborne optoelectronic device to perform a step-and-gaze search on the area to be scanned in the priori coordinate information, calibrate the reference point, and send a first search video generated during the search process to the image recognition device; The image recognition device performs frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and calculates the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain a rough positioning coordinate of each of the positioning targets, and then sends the rough positioning coordinates to the UAV mission management device; Controlling the airborne optoelectronic device to perform staring detection according to the rough positioning coordinates through the UAV mission management device to obtain a second search video, and sending the second search video to the image recognition device; Performing secondary image recognition on the second search video by the image recognition device to obtain pixel position information of the positioning target in the display screen, and sending the pixel position information to the UAV task management device; The unmanned aerial vehicle task management device is used to control the airborne optoelectronic device to track the positioning target according to the pixel position information, and the precise positioning coordinates of the positioning target are obtained through laser ranging and a preset positioning algorithm.

10. A method for reconnaissance and positioning of an unmanned aerial vehicle, characterized in that: Applied to UAV mission management equipment, including: Based on the received priori coordinate information, the airborne optoelectronic device is controlled to perform a step-and-gaze search on the area to be scanned in the priori coordinate information, and the reference point is calibrated, and a first search video generated in the search process is sent to an image recognition device, so that the image recognition device performs frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and the positioning targets in each of the first search images are calculated based on the coordinate information of the reference point to obtain a rough positioning coordinate of each of the positioning targets, and then the rough positioning coordinates are sent to the UAV mission management device; Controlling the airborne optoelectronic device to perform staring detection according to the coarse positioning coordinates to obtain a second search video, and sending the second search video to the image recognition device so that the image recognition device performs secondary image recognition on the second search video to obtain pixel position information of the positioning target in the display screen, and sending the pixel position information to the UAV mission management device; The airborne optoelectronic device is controlled to track the positioning target according to the pixel position information, and the precise positioning coordinates of the positioning target are obtained through laser ranging and a preset positioning algorithm.

11. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the drone reconnaissance and positioning method as described in claim 10.

Images