An unmanned aerial vehicle reconnaissance positioning system, method and device

Abstract

The application discloses a kind of unmanned vehicle reconnaissance positioning system, method, equipment, it is related to unmanned vehicle positioning technical field, including: unmanned vehicle task management equipment, for controlling airborne photoelectric equipment to the region to be scanned based on priori coordinate information step-by-step stare search, first search video generated is sent to image recognition equipment;Image recognition equipment is used to carry out frame-by-frame image recognition to first search video, and the coarse positioning coordinate of positioning target is obtained;Unmanned vehicle task management equipment is used to control airborne photoelectric equipment according to coarse positioning coordinate and carry out stare reconnaissance, and second search video is obtained;Image recognition equipment is used to carry out secondary image recognition to second search video, and the pixel position information in the display picture of positioning target is obtained;Unmanned vehicle task management equipment is used to control airborne photoelectric equipment according to pixel position information and carry out tracking processing to positioning target, and the fine positioning coordinate of positioning target is acquired. Realize automation accurate positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle positioning, and in particular to an unmanned aerial vehicle reconnaissance positioning system, method and device. BACKGROUND

[0002] An unmanned aerial vehicle can cruise at high altitude for a long time, and an operator controls an airborne photoelectric device to continuously monitor and survey a wide area. The operator manually controls the azimuth, pitch, focal length, field of view and other parameters of the airborne photoelectric device through an instruction panel and a joystick to achieve detailed reconnaissance of a given area and to make a human judgment to confirm whether there is a suspicious target.

[0003] However, as the time for using an unmanned aerial vehicle to monitor and survey the ground is continuously increasing, the pressure on personnel is gradually increasing. First, the current medium and large unmanned aerial vehicle reconnaissance time is generally twenty to forty hours, and the fatigue of personnel working for a long time is seriously increased. Second, the control mode of the existing airborne photoelectric device is mostly a human-in-the-loop mode, that is, personnel need to continuously manipulate the control stick to achieve reconnaissance and search. Third, the discrimination of suspicious targets is a human judgment, and fatigue caused by long-time reconnaissance may cause personnel to be unable to concentrate, resulting in target misjudgment and missed judgment.

[0004] In summary, how to realize automatic tracking and precise positioning of the positioning target by the unmanned aerial vehicle is a technical problem to be solved in the field. SUMMARY

[0005] Therefore, the purpose of the present application is to provide an unmanned aerial vehicle reconnaissance positioning system, method and device, which can realize automatic tracking and precise positioning of the positioning target by the unmanned aerial vehicle. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses an unmanned aerial vehicle reconnaissance positioning system, comprising: an unmanned aerial vehicle task management device, an airborne photoelectric device and an image recognition device, wherein

[0007] The unmanned aerial vehicle task management device is configured to control the airborne photoelectric device to perform step-by-step staring search on a to-be-scanned area in the received prior coordinate information, and to calibrate a reference point, and to send a first search video generated in the search process to the image recognition device.

[0008] The image recognition device is configured to perform frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and to calculate the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain coarse positioning coordinates of each of the positioning targets, and then to send the coarse positioning coordinates to the unmanned aerial vehicle task management device.

[0009] The unmanned aerial vehicle task management device is configured to control the airborne photoelectric device to perform gaze detection according to the coarse positioning coordinates, to obtain a second search video, and to send the second search video to the image recognition device.

[0010] The image recognition device is further configured to perform secondary image recognition on the second search video to obtain pixel position information of the positioning target in a display screen, and to send the pixel position information to the unmanned aerial vehicle task management device.

[0011] The unmanned aerial vehicle task management device is further configured to control the airborne photoelectric device to perform tracking processing on the positioning target according to the pixel position information, and to obtain precise positioning coordinates of the positioning target by laser ranging and a preset positioning algorithm.

[0012] Optionally, the unmanned aerial vehicle task management device comprises:

[0013] The instruction issuing module is configured to issue a first search instruction and prior coordinate information to the airborne photoelectric device when it is detected that the relative distance between the positioning target and the unmanned aerial vehicle on which the airborne photoelectric device is located satisfies a preset distance condition.

[0014] Optionally, the airborne photoelectric device comprises:

[0015] The first search module is configured to adjust a detection direction according to the first search instruction and the prior coordinate information, to perform a grid spiral step scanning on the to-be-scanned region, and to use a laser to position and calibrate a reference point in a video screen formed by the scanning, to obtain a first search video carrying a time stamp of each frame of image; wherein the first search instruction comprises a reference point selection rule, so as to determine a reference point in the video screen based on the reference point selection rule.

[0016] Optionally, the image recognition device comprises:

[0017] The first image recognition module is configured to perform a frame-by-frame image recognition operation on the first search video by using target shape features of each positioning target stored in a preset model database, to obtain a plurality of positioning targets in each first search image, and to calculate coarse positioning coordinates of each positioning target in a corresponding first search image based on a time stamp of each frame of image and coordinate information of the reference point; wherein the coarse positioning coordinates comprise a time stamp, a positioning target ID, a positioning target longitude, a positioning target latitude, and a positioning target height.

[0018] The coordinate sending module is configured to send the corresponding coarse positioning coordinates to the unmanned aerial vehicle task management device according to a time stamp of each frame of image.

[0019] Optionally, the airborne photoelectric device comprises:

[0020] a direction adjusting module, configured to adjust a detection direction of the airborne photoelectric device according to the coarse positioning coordinates;

[0021] a second searching module, configured to perform focusing operation and gaze detection on the detection direction to obtain a second search video.

[0022] Optionally, the second searching module comprises:

[0023] a gaze detection unit, configured to perform focusing operation and gaze detection on a scene where a positioning target in the detection direction is located to track a target scene containing the positioning target.

[0024] Optionally, the image recognition device comprises:

[0025] a second image recognition module, configured to perform image recognition on each of the target scenes by target shape features of each of the positioning targets stored in a preset model database, to determine pixel position information of each of the positioning targets in a display picture according to a difference between a current pixel position of each of the positioning targets in each of the target scenes and the coordinates of the reference point, and send the pixel position information to the UAV task management device.

[0026] Optionally, the airborne photoelectric device comprises:

[0027] a laser ranging module, configured to receive laser ranging positioning instructions sent by the UAV task management device based on the pixel position information, to emit a laser beam to the positioning target according to the laser ranging positioning instructions, and receive a laser signal reflected by the positioning target to calculate a target distance between the positioning target and the UAV;

[0028] a fine positioning module, configured to calculate fine positioning coordinates of the positioning target by using a preset positioning algorithm and according to the pixel position information, the target distance, a UAV position and a UAV attitude.

[0029] In a second aspect, the present application discloses a UAV reconnaissance positioning method, comprising:

[0030] controlling the airborne photoelectric device to perform step-by-step gaze search on a to-be-scanned region in the priori coordinate information based on the received priori coordinate information, calibrating a reference point, and sending a first search video generated in a search process to an image recognition device;

[0031] performing frame-by-frame image recognition on the first search video by the image recognition device to obtain a first search image containing a plurality of positioning targets, calculating the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain coarse positioning coordinates of each of the positioning targets, and then sending the coarse positioning coordinates to a UAV task management device.

[0032] controlling the airborne photoelectric device to conduct gaze detection according to the coarse positioning coordinates, to obtain a second search video, and sending the second search video to the image recognition device;

[0033] conducting secondary image recognition on the second search video by the image recognition device, to obtain pixel position information of the positioning target in a display screen, and sending the pixel position information to the unmanned aerial vehicle task management device;

[0034] controlling the airborne photoelectric device to conduct tracking processing on the positioning target according to the pixel position information, and obtaining precise positioning coordinates of the positioning target by laser ranging and a preset positioning algorithm.

[0035] In a third aspect, the present application discloses a method for reconnaissance and positioning of an unmanned aerial vehicle, applied to an unmanned aerial vehicle task management device, comprising:

[0036] controlling the airborne photoelectric device to conduct step-by-step gaze search on a scanning area in the priori coordinate information, and calibrating a reference point, sending a first search video generated in the search process to an image recognition device, so that the image recognition device conducts frame-by-frame image recognition on the first search video, to obtain a first search image containing a plurality of positioning targets, and calculates the positioning targets in each of the first search images based on coordinate information of the reference point, to obtain coarse positioning coordinates of each of the positioning targets, and then sends the coarse positioning coordinates to the unmanned aerial vehicle task management device;

[0037] controlling the airborne photoelectric device to conduct gaze detection according to the coarse positioning coordinates, to obtain a second search video, and sending the second search video to the image recognition device, so that the image recognition device conducts secondary image recognition on the second search video, to obtain pixel position information of the positioning target in a display screen, and sends the pixel position information to the unmanned aerial vehicle task management device;

[0038] controlling the airborne photoelectric device to conduct tracking processing on the positioning target according to the pixel position information, and obtaining precise positioning coordinates of the positioning target by laser ranging and a preset positioning algorithm.

[0039] In a fourth aspect, the present application discloses an electronic device, comprising:

[0040] a memory for saving a computer program;

[0041] a processor for executing the computer program to realize steps of the above-mentioned method for reconnaissance and positioning of an unmanned aerial vehicle.

[0042] It can be seen that the application discloses a UAV reconnaissance positioning system, comprising: a UAV task management device, an airborne photoelectric device, and an image recognition device, wherein the UAV task management device is configured to control the airborne photoelectric device to perform step-by-step staring search on a scanning area in prior coordinate information based on the received prior coordinate information, calibrate a reference point, and send a first search video generated in the search process to the image recognition device; the image recognition device is configured to perform frame-by-frame image recognition on the first search video to obtain a first search image containing a plurality of positioning targets, and calculate the positioning targets in each of the first search images based on coordinate information of the reference point to obtain coarse positioning coordinates of the positioning targets, and then send the coarse positioning coordinates to the UAV task management device; the UAV task management device is configured to control the airborne photoelectric device to perform staring reconnaissance according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device; the image recognition device is further configured to perform secondary image recognition on the second search video to obtain pixel position information of the positioning targets in a display screen, and send the pixel position information to the UAV task management device; and the UAV task management device is further configured to control the airborne photoelectric device to perform tracking processing on the positioning targets according to the pixel position information, and obtain precise positioning coordinates of the positioning targets through laser ranging and a preset positioning algorithm. As can be seen, the system performs frame-by-frame image recognition on the first search video through the image recognition device to determine the positioning targets, and calculates the relative coordinates of the positioning targets based on the predefined reference point to obtain the coarse positioning coordinates of the positioning targets, and then performs secondary image recognition on the staring reconnaissance to obtain the pixel position information of the positioning targets in the display screen, and finally obtains the precise positioning coordinates by combining the laser ranging and the preset positioning algorithm. Through the multi-round image recognition and the fusion processing of the pixel position information and the laser ranging information, the determination of the target position can be gradually converged, and compared with the single recognition or single information positioning mode, the positioning precision is greatly improved. And a series of automatic processes from the prior coordinate information to the final precise positioning coordinate determination are realized. Compared with the traditional manual reconnaissance positioning mode, the artificial intervention link is greatly reduced, the target area can be searched, recognized and positioned more quickly, and the overall reconnaissance efficiency is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0044] Figure 1 A structure diagram of a UAV reconnaissance positioning system disclosed in the present application;

[0045] Figure 2 A net format step scanning schematic diagram disclosed in the present application;

[0046] Figure 3 A pixel tracking result diagram disclosed in the present application;

[0047] Figure 4 A flow chart of a UAV reconnaissance positioning method disclosed in the present application;

[0048] Figure 5 A specific flow chart of a UAV reconnaissance positioning method disclosed in the present application;

[0049] Figure 6 A structure diagram of an electronic device disclosed in the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0051] The UAV can cruise at high altitude for a long time, and the operator controls the airborne photoelectric device to continuously monitor a wide area. Specifically, the operator controls the airborne photoelectric device by operating the control lever at the ground station. The data flow is that the ground station remote control instruction is sent to the airborne measurement and control system of the UAV through a wireless link, the airborne measurement and control system transmits the control instruction to the UAV task management device, the UAV task management device schedules the airborne photoelectric device to execute the corresponding instruction to perform reconnaissance work. In this way, the operator manually controls the azimuth, pitch, focal length, field of view and other parameters of the airborne photoelectric device through the instruction panel and the control lever to realize detailed reconnaissance of the designated area and to make human judgment to confirm whether there is a suspicious target.

[0052] However, as the time for using the UAV to monitor and reconnaissance the ground is continuously increasing, the pressure on the personnel is gradually increasing. First, the current medium and large UAV reconnaissance time is generally twenty to forty hours, and the fatigue of the personnel working for a long time is seriously increased. Second, the control mode of the airborne photoelectric device based on the prior art is mostly the human-in-the-loop mode, that is, the personnel need to continuously operate the control lever to realize reconnaissance search. Third, the discrimination of suspicious targets is all human judgment, which may cause the personnel to be unable to concentrate due to fatigue caused by long-time reconnaissance, resulting in target misjudgment, target missed judgment and the like.

[0053] To this end, the application provides a UAV reconnaissance positioning scheme, which can realize automatic tracking and accurate positioning of a positioning target by a UAV.

[0054] Referring to Figure 1 As shown in the drawings, the application discloses a UAV reconnaissance positioning system, which comprises a UAV task management device 11, an airborne photoelectric device 12, and an image recognition device 13, wherein,

[0055] The UAV task management device 11 is configured to control the airborne photoelectric device 12 to perform step-by-step staring search on a scanning area in prior coordinate information based on the received prior coordinate information, calibrate a reference point, and send a first search video generated in the search process to the image recognition device 13.

[0056] The image recognition device 13 is configured to perform frame-by-frame image recognition on the first search video, obtain a first search image containing a plurality of positioning targets, calculate the positioning targets in each of the first search images based on coordinate information of the reference point to obtain coarse positioning coordinates of the positioning targets, and then send the coarse positioning coordinates to the UAV task management device 11.

[0057] The UAV task management device 11 is configured to control the airborne photoelectric device 12 to perform staring reconnaissance according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device 13.

[0058] The image recognition device 13 is further configured to perform secondary image recognition on the second search video to obtain pixel position information of the positioning targets in a display screen, and send the pixel position information to the UAV task management device 11.

[0059] The UAV task management device 11 is further configured to control the airborne photoelectric device 12 to perform tracking processing on the positioning targets according to the pixel position information, and obtain accurate positioning coordinates of the positioning targets through laser ranging and a preset positioning algorithm.

[0060] In the embodiment, the unmanned aerial vehicle reconnaissance positioning system comprises three parts: unmanned aerial vehicle task management device 11, airborne photoelectric device 12 and image recognition device 13. The unmanned aerial vehicle task management device 11 is responsible for condition judgment, logic execution, data conversion and processing and other comprehensive service management. The airborne photoelectric device 12 realizes network format step-by-step gaze search, field of view matching adjustment, target tracking and accurate positioning and other execution services. The image recognition device 13 completes real-time target detection and recognition of the video stream output by the airborne photoelectric device 12. It should be noted that the airborne photoelectric device 12 searches and detects the target based on visible light or infrared imaging mode. Traditionally, the airborne photoelectric device 12 itself does not have the ability of autonomous target detection and accurate positioning, and the operator needs to complete target discrimination, manual tracking and start laser ranging positioning in the loop. Therefore, the unmanned aerial vehicle reconnaissance positioning system constructed by the present application can realize autonomous accurate positioning of the airborne photoelectric device 12 through the cooperation of each part.

[0061] Specifically, in order to realize the intelligent target recognition and accurate positioning process, the unmanned aerial vehicle task management device 11 is taken as the control center. First, the target information is obtained through synthetic aperture radar (hereinafter taken as an example), communication reconnaissance device, electronic reconnaissance device and other long-range detection payloads. However, due to the limitation of hardware conditions, the accuracy has a large error. The synthetic aperture radar is used to obtain the prior coordinate information of the positioning target, and the prior coordinate information is sent to the unmanned aerial vehicle task management device 11. The unmanned aerial vehicle task management device 11 sends the obtained prior coordinate information to the airborne photoelectric device 12, and continuously step-by-step gaze searches the scanning area where the prior coordinate information is located. The image recognition device 13 combines the shape features of each positioning target in the preset model database to perform frame-by-frame target recognition on the video stream in the continuous search process, calculates the target pixel deviation based on the reference point information and the recognized positioning target in the scanning area, and outputs the target coordinates. The unmanned aerial vehicle task management device 11 guides the airborne photoelectric device 12 to the target coordinates output by the image recognition device 13 and keeps scene tracking. The image recognition device 13 performs secondary target recognition on the image frames in the scene tracking process, and outputs the target pixel deviation. The unmanned aerial vehicle task management device 11 controls the airborne photoelectric device 12 to track the pixels according to the target pixel deviation and start laser ranging, and calculates the accurate coordinates of the current positioning target.

[0062] The unmanned aerial vehicle task management device 11 comprises:

[0063] The instruction issuing module 111 is configured to issue a first search instruction and prior coordinate information to the airborne photoelectric device 12 when it is detected that the relative distance between the positioning target and the unmanned aerial vehicle where the airborne photoelectric device 12 is located meets the preset distance condition.

[0064] It can be understood that the unmanned aerial vehicle task management device 11 detects that there is a target to be positioned with a relative distance of ≤30km from the carrier, and issues a first search instruction to the airborne photoelectric device 12, and at the same time, sends the prior coordinate information received by itself to the airborne photoelectric device 12, wherein the prior coordinate information is detected and provided by the airborne radar SAR or other information sources, and the prior coordinate information is the rough position information including the position and possible range of the position of the target to be positioned.

[0065] Moreover, the sampling image obtained by sampling the airborne photoelectric device 12 satisfies the average value of the relative distance of the corresponding target to be positioned from the carrier under the requirement of identifying pixels, and the preset distance condition is determined, so that the image collected by the airborne photoelectric device 12 under the condition of meeting the preset distance condition can be identified at the pixel level.

[0066] The airborne photoelectric device 12 comprises:

[0067] The first search module 121 is configured to adjust the investigation direction according to the first search instruction and the prior coordinate information, to perform a net format spiral step scanning to the scanning area, and to use laser to position and calibrate the reference point in the video picture formed by scanning, to obtain a first search video carrying a time stamp of each frame of image; wherein the first search instruction comprises a reference point selection rule, so as to determine the reference point in the video picture based on the reference point selection rule.

[0068] It can be understood that after the first search module 121 in the airborne photoelectric device 12 receives the first search instruction and the prior coordinate information, the first search module 121 directly controls the airborne photoelectric device 12 to perform a range area scanning on the prior coordinate information, that is, a cell search+;

[0069] It should be noted that the first search module 121 sends the cell search+ instruction and the target rough coordinate to the airborne photoelectric device 12, and determines the reference point in the entire search process according to the reference point selection rule in the first search instruction. For example, at the beginning of the net format step staring search, it is stipulated that the upper left corner or other fixed corner position of the area to be searched is taken as the reference point. Such a rule can ensure the consistency and certainty of the selection of the reference point each time the search is performed, and then the step staring time is 80ms and the overlap rate is 20%; then the airborne photoelectric device 12 enters the cell search+ mode;

[0070] The first step is to track the position of the target to be positioned;

[0071] The second step is automatic focusing;

[0072] The third step is to complete laser ranging and positioning, and calculate the position coordinate information of the target to be positioned;

[0073] The fourth step is to perform a grid spiral step scanning with the location of the positioning target as the initial grid. The process of the grid spiral step scanning is shown in the arrow direction to perform the spiral step scanning on the grid divided scanning area. Figure 2 The current working status and cell search + status are broadcasted on the system bus, and the default initialization is N / A, ING during the process, END after completing the complete cycle, and FAILURE if the complete cycle is not completed.

[0074] The fifth step is that the airborne photoelectric device 12 sends the video data to the image recognition device 13 through the digital signal, and the timestamp and reference point of each frame of picture are attached in the video stream. In this way, the first search video carrying the timestamp and reference point of each frame of picture is obtained.

[0075] The image recognition device 13 comprises:

[0076] The first image recognition module 131 is configured to perform a frame-by-frame image recognition operation on the first search video according to the target shape features of each positioning target stored in the preset model database, to obtain a plurality of positioning targets in each first search image, and calculate the coarse positioning coordinates of each positioning target in the corresponding first search image based on the timestamp of each frame of picture and the coordinate information of the reference point. The coarse positioning coordinates include the timestamp, the positioning target ID, the positioning target longitude, the positioning target latitude, and the positioning target height.

[0077] The coordinate sending module 132 is configured to send the corresponding coarse positioning coordinates to the unmanned aerial vehicle task management device 11 according to the timestamp of each frame of picture.

[0078] It can be understood that the first image recognition module 131 of the image recognition device 13 performs image target matching recognition on the frame-by-frame picture of the first search video according to the target shape feature information of a plurality of positioning targets in the model database, to identify each positioning target (which can be 0, 1, or more, without specific limitation) existing in the first search image, and then calculate the coordinate information of each positioning target in each first search image through the image algorithm module and the coordinates of the reference point attached to the first search video, as the coarse positioning coordinates, and then the coordinate sending module 12 sends the coarse positioning coordinates to the bus according to the timestamp information. Specifically, the execution steps of the first image recognition module 131 and the coordinate sending module 132 are as follows:

[0079] a) receiving the 3G-SDI video stream data from the airborne photoelectric device 12, i.e., the first search video;

[0080] b) performing image target recognition on the frame-by-frame picture of the first search video, and receiving the attached information (coordinates of the reference point);

[0081] c) calculating the target position by the image algorithm module;

[0082] d) outputting the coarse positioning coordinates; wherein the output coarse positioning coordinates contain time stamp, target ID, target longitude, target latitude, target height, data validity, etc.

[0083] The airborne photoelectric device 12 comprises:

[0084] The direction adjustment module 122 is configured to adjust the detection direction of the airborne photoelectric device according to the coarse positioning coordinates.

[0085] The second search module 123 is configured to perform focusing operation and gaze detection on the detection direction to obtain the second search video.

[0086] The second search module 122 comprises:

[0087] The gaze detection unit is configured to perform focusing operation and gaze detection on the scene where the positioning target is located in the detection direction to track the target scene containing the positioning target.

[0088] It can be understood that after the image recognition device 13 calculates the coarse positioning coordinates, the image recognition device 13 follows the airborne photoelectric device 12 to the coarse positioning target position and maintains the scene tracking process. Specifically, after the unmanned aerial vehicle task management device 11 sends the gaze instruction and the coarse positioning coordinates to the airborne photoelectric device 12, the gaze detection step is as follows:

[0089] a) The airborne photoelectric device 12 performs the gaze process according to the gaze instruction and the coarse positioning coordinates;

[0090] b) Geographically tracking to the position where the coarse positioning coordinates are located;

[0091] c) Performing lens automatic focusing operation;

[0092] d) Performing scene tracking.

[0093] The image recognition device 13 comprises:

[0094] The second image recognition module 133 is configured to perform image recognition on each target scene by the target shape features of each positioning target stored in the preset model database, to determine the pixel position information of the positioning target in the display picture according to the difference between the current pixel position of the positioning target of each target scene and the coordinates of the reference point, and send the pixel position information to the unmanned aerial vehicle task management device.

[0095] It can be understood that, as the same as the identification process of the first image identification module 131, the image identification is performed on each target scene of the second search video generated under the coarse positioning, the pixel position information of the positioning target is determined, and the information is sent to the unmanned aerial vehicle task management device 11, and then the pixel position information is sent to the airborne optoelectronic device 12 by the unmanned aerial vehicle task management device 11.

[0096] Further, the airborne optoelectronic device 12 performs pixel tracking on the positioning target identified in the gaze process, and the steps are as follows:

[0097] a) The unmanned aerial vehicle task management device 11 sends a gaze instruction and a target pixel deviation value to the airborne optoelectronic device 12; wherein the target pixel deviation value is the pixel position information of the positioning target in the display picture determined by the difference between the current pixel position of the positioning coordinate and the coordinate of the reference point.

[0098] b) The airborne optoelectronic device 12 performs pixel tracking according to the target pixel deviation value, and maintains an automatic tracking state, as shown in Figure 3 .

[0099] The airborne optoelectronic device 12 comprises:

[0100] The laser ranging module 124 is configured to receive a laser ranging positioning instruction sent by the unmanned aerial vehicle task management device based on the pixel position information, emit a laser beam to the positioning target according to the laser ranging positioning instruction, and receive a laser signal reflected by the positioning target, so as to calculate a target distance between the positioning target and the unmanned aerial vehicle.

[0101] The fine positioning module 125 is configured to calculate a fine positioning coordinate of the positioning target by using a preset positioning algorithm and according to the pixel position information, the target distance, a position of the unmanned aerial vehicle, and an attitude of the unmanned aerial vehicle.

[0102] It can be understood that the airborne optoelectronic device 12 performs laser ranging positioning on the automatically tracked positioning target, and sends the fine positioning coordinate to the bus, and specifically:

[0103] a) The unmanned aerial vehicle task management device 11 sends a Mark instruction to the airborne optoelectronic device 12;

[0104] b) Laser ranging positioning is performed on the positioning target;

[0105] c) The fine positioning coordinate of the positioning target is calculated;

[0106] d) The fine positioning coordinate is output to the bus.

[0107] It can be seen that the unmanned aerial vehicle reconnaissance positioning system disclosed by the application comprises an unmanned aerial vehicle task management device, an airborne photoelectric device and an image recognition device. The unmanned aerial vehicle task management device is configured to control the airborne photoelectric device to perform step-by-step staring search on a to-be-scanned region in prior coordinate information based on the received prior coordinate information, calibrate a reference point, and send a first search video generated in the search process to the image recognition device. The image recognition device is configured to perform frame-by-frame image recognition on the first search video, obtain a first search image containing a plurality of positioning targets, calculate the positioning targets in each of the first search images based on coordinate information of the reference point to obtain coarse positioning coordinates of the positioning targets, and then send the coarse positioning coordinates to the unmanned aerial vehicle task management device. The unmanned aerial vehicle task management device is configured to control the airborne photoelectric device to perform staring reconnaissance according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device. The image recognition device is further configured to perform secondary image recognition on the second search video to obtain pixel position information of the positioning targets in a display screen, and send the pixel position information to the unmanned aerial vehicle task management device. The unmanned aerial vehicle task management device is further configured to control the airborne photoelectric device to perform tracking processing on the positioning targets according to the pixel position information, and obtain precise positioning coordinates of the positioning targets through laser ranging and a preset positioning algorithm. It can be seen that the system successively performs frame-by-frame image recognition on the first search video by the image recognition device to determine the positioning targets, calculates the coarse positioning coordinates of the positioning targets based on the pre-defined reference point, performs secondary image recognition on the staring reconnaissance to obtain the pixel position information of the positioning targets in the display screen, and finally obtains the precise positioning coordinates by combining the laser ranging and the preset positioning algorithm. Through the multi-round image recognition and the fusion processing of the pixel position information and the laser ranging information, the determination of the target position can be gradually converged, the positioning precision is greatly improved compared with the single recognition or single information positioning mode, and a series of automatic processes from the acquisition of the prior coordinate information to the determination of the final precise positioning coordinates are realized. Compared with the traditional manual reconnaissance positioning mode, the artificial intervention link is greatly reduced, the target region can be searched, recognized and positioned more quickly, and the overall reconnaissance efficiency is effectively improved.

[0108] Referring to Figure 4 The application also discloses an unmanned aerial vehicle reconnaissance positioning method, which comprises the following steps:

[0109] In step S11, the airborne photoelectric device is controlled to perform step-by-step staring search on a to-be-scanned region in prior coordinate information based on the received prior coordinate information, a reference point is calibrated, and a first search video generated in the search process is sent to the image recognition device.

[0110] Step S12: performing frame-by-frame image recognition on the first search video by the image recognition device to obtain first search images containing a plurality of positioning targets, and calculating the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain coarse positioning coordinates of each of the positioning targets, and then sending the coarse positioning coordinates to the UAV task management device;

[0111] Step S13: controlling the airborne photoelectric device to perform gaze detection according to the coarse positioning coordinates by the UAV task management device to obtain a second search video, and sending the second search video to the image recognition device;

[0112] Step S14: performing secondary image recognition on the second search video by the image recognition device to obtain pixel position information of the positioning targets in a display screen, and sending the pixel position information to the UAV task management device;

[0113] Step S15: controlling the airborne photoelectric device to perform tracking processing on the positioning targets according to the pixel position information by the UAV task management device, and obtaining precise positioning coordinates of the positioning targets by laser ranging and a preset positioning algorithm.

[0114] It can be seen that the application discloses a method for controlling an airborne photoelectric device to perform step-by-step staring search on a to-be-scanned area in prior coordinate information and calibrating a reference point, and transmitting first search video generated in the search process to an image recognition device; performing frame-by-frame image recognition on the first search video by the image recognition device to obtain first search images containing a plurality of positioning targets, and calculating the positioning targets in each of the first search images based on coordinate information of the reference point to obtain coarse positioning coordinates of each of the positioning targets, and then transmitting the coarse positioning coordinates to a UAV task management device; controlling the airborne photoelectric device to perform staring investigation by the UAV task management device and according to the coarse positioning coordinates to obtain second search video, and transmitting the second search video to the image recognition device; performing secondary image recognition on the second search video by the image recognition device to obtain pixel position information of the positioning targets in a display screen, and transmitting the pixel position information to the UAV task management device; and controlling the airborne photoelectric device to perform tracking processing on the positioning targets by the UAV task management device and according to the pixel position information, and obtaining fine positioning coordinates of the positioning targets by laser ranging and a preset positioning algorithm. It can be seen that the system performs frame-by-frame image recognition on the first search video by the image recognition device to determine each positioning target, and calculates relative coordinates based on the predefined reference point to obtain coarse positioning coordinates of each positioning target, and then performs secondary image recognition after staring investigation to obtain pixel position information of the positioning targets in the display screen, and finally obtains fine positioning coordinates in combination with laser ranging and the preset positioning algorithm. Through multi-round image recognition and fusion processing of pixel position information and laser ranging information, the determination of the target position can be gradually converged, the positioning accuracy is greatly improved compared with single recognition or single information positioning, and a series of automatic processes from obtaining prior coordinate information to determining final fine positioning coordinates are realized. Compared with the traditional manual investigation positioning mode, the artificial intervention link is greatly reduced, the target area can be searched, recognized and positioned more quickly, and the overall investigation efficiency is effectively improved.

[0115] Referring to Figure 5 The application further discloses a UAV investigation positioning method applied to a UAV task management device and comprising the following steps.

[0116] Step S21: based on the received priori coordinate information, control the airborne photoelectric device to perform step-by-step staring search on the to-be-scanned area in the priori coordinate information, and calibrate a reference point, send the first search video generated in the search process to the image recognition device, so that the image recognition device performs frame-by-frame image recognition on the first search video, obtains a first search image containing a plurality of positioning targets, and calculates the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the coarse positioning coordinates of each of the positioning targets, and then sends the coarse positioning coordinates to the UAV task management device.

[0117] Step S22: control the airborne photoelectric device to perform staring investigation according to the coarse positioning coordinates to obtain a second search video, and send the second search video to the image recognition device, so that the image recognition device performs secondary image recognition on the second search video to obtain pixel position information of the positioning targets in a display screen, and sends the pixel position information to the UAV task management device.

[0118] Step S23: control the airborne photoelectric device to perform tracking processing on the positioning targets according to the pixel position information, and obtain the precise positioning coordinates of the positioning targets through laser ranging and preset positioning algorithm.

[0119] Wherein, the detailed contents of the above steps please refer to the foregoing disclosure, which will not be repeated here.

[0120] It can be seen that the application discloses a method for controlling an airborne photoelectric device to perform step-by-step staring search on a scanning area in prior coordinate information, calibrating a reference point, sending first search video generated in the search process to an image recognition device, so that the image recognition device performs frame-by-frame image recognition on the first search video to obtain first search images containing positioning targets, and calculates the positioning targets in each of the first search images based on coordinate information of the reference point to obtain coarse positioning coordinates of each of the positioning targets, and then sends the coarse positioning coordinates to a UAV task management device; controls the airborne photoelectric device to perform staring detection according to the coarse positioning coordinates to obtain second search video, and sends the second search video to the image recognition device, so that the image recognition device performs secondary image recognition on the second search video to obtain pixel position information of the positioning targets in a display screen, and sends the pixel position information to the UAV task management device; controls the airborne photoelectric device to perform tracking processing on the positioning targets according to the pixel position information, and obtains precise positioning coordinates of the positioning targets through laser ranging and a preset positioning algorithm. It can be seen that the system successively determines each positioning target through frame-by-frame image recognition of the image recognition device on the first search video, calculates relative coordinates of each positioning target based on a predefined reference point to obtain coarse positioning coordinates of each positioning target, performs secondary image recognition on the staring detection to obtain pixel position information of the positioning targets in a display screen, and finally obtains precise positioning coordinates through laser ranging and a preset positioning algorithm. Through multi-round image recognition and fusion processing of pixel position information and laser ranging information, the determination of the target position can be gradually converged, compared with a single recognition or single information positioning mode, the positioning precision is greatly improved. And a series of automatic processes from obtaining prior coordinate information to determining final precise positioning coordinates are realized. Compared with a traditional method of relying on manual detection and positioning, the artificial intervention link is greatly reduced, the target area can be searched, recognized and positioned more quickly, and the overall detection efficiency is effectively improved.

[0121] Further, the embodiment of the application further discloses an electronic device, Figure 6 FIG. 1 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the contents in the figure cannot be considered as any limitation on the use range of the application.

[0122] Figure 6A structural schematic diagram of an electronic device 20 is provided in the embodiments of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the processor 21 is configured to load and execute the computer program to implement the related steps in the unmanned aerial vehicle reconnaissance positioning method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the embodiments of the present application can be specifically an electronic computer.

[0123] In the embodiments of the present application, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 is capable of creating a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solutions of the present application, which is not specifically limited herein; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not specifically limited herein.

[0124] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array) and a PLA (Programmable Logic Array). The processor 21 can also include a main processor and a coprocessor. The main processor is a processor for processing data in a wake-up state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 can further include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0125] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0126] The operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to realize the operation and processing of the processor 21 on the mass data 223 in the memory 22, which can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the unmanned aerial vehicle reconnaissance positioning method executed by the electronic device 20 disclosed in any one of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work. In addition to the data transmitted by the external device and received by the electronic device, the data 223 can also include the data collected by the self input and output interface 25, etc.

[0127] Further, the application further discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by the processor to realize the foregoing disclosed unmanned aerial vehicle reconnaissance positioning method. For the specific steps of the method, refer to the corresponding contents disclosed in the foregoing embodiments, which will not be described here.

[0128] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. For the same or similar parts between each embodiment, refer to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant part is described in the method part.

[0129] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, various components have been described above generally in terms of their functionality, without limitation. The handwiring and software implementations of the examples described herein could be accomplished using any number of microprocessors, microcontrollers, programmable consumption logic devices, application-specific integrated circuits, or general-purpose computers with interconnecting circuits that either run software programs or use opencircuit or other hardware components that are designed to perform the functions described herein. The embodiments described herein can be implemented along with software modules, and the software modules can be stored on any of a variety of non-transitory machine-readable media. A non-transitory machine-readable medium includes any medium that participates in providing instructions to a processor for execution. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks and other persistent memory. Volatile media includes dynamic memories, and physical registers. Transmission media includes coaxial cables, copper wires and fiber optic cables, including wires that comprise bus conductors. Transmission media also can also take the form of acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, solid-state drive, magnetic tape, or any other magnetic data storage medium, a Compact Disc - Read Only Memory (CD-ROM), any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a programmable ROM (PROM), an erasable PROM (EPROM), a FLASH-EPROM, any other memory chip or cartridge, a carrier wave, a

[0130] Finally, it should also be noted that, in the present text, relational terms such as first and second and the like can only be used to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0131] The above has carried on the detailed introduction to the scheme provided by the present application, the principle and implementation mode of the present application are described by applying the specific examples in the present text, the above example explanation is only for helping the understanding of the method and core idea of the present application; simultaneously, for the general technical personnel of the field, according to the idea of the present application, there will be the change in the specific implementation mode and application range, the above-mentioned content of the description should not be understood as the limitation of the present application.

Claims

1. A UAV reconnaissance and positioning system, characterized in that, include: Unmanned aerial vehicle (UAV) mission management equipment, airborne optoelectronic equipment, and image recognition equipment, among which, The UAV mission management device is used to control the airborne optoelectronic device to perform a step-by-step staring search towards the area to be scanned in the prior coordinate information based on the received prior coordinate information, and to calibrate the reference point, and to send the first search video generated during the search process to the image recognition device. The image recognition device is used to perform frame-by-frame image recognition on the first search video to obtain a first search image containing several positioning targets, and to calculate the positioning targets in each of the first search images based on the coordinate information of the reference point to obtain the coarse positioning coordinates of each positioning target, and then send the coarse positioning coordinates to the UAV mission management device. The UAV mission management device is used to control the airborne optoelectronic device to perform staring reconnaissance based on the coarse positioning coordinates, so as to obtain a second search video, and send the second search video to the image recognition device; The image recognition device is further configured to perform secondary image recognition on the second search video to obtain the pixel position information of the positioning target in the display screen, and send the pixel position information to the UAV mission management device; The UAV mission management device is also used to control the airborne optoelectronic device to track the positioning target based on the pixel position information, and to obtain the precise positioning coordinates of the positioning target through laser ranging and a preset positioning algorithm; The unmanned aerial vehicle (UAV) mission management device includes: The instruction issuing module is used to issue a first search instruction and prior coordinate information to the airborne optoelectronic device when the relative distance between the positioning target and the UAV where the airborne optoelectronic device is located meets the preset distance condition. The airborne optoelectronic equipment includes: The first search module is used to adjust the detection direction according to the first search instruction and the prior coordinate information to perform a grid-like spiral step scan on the area to be scanned, and to use a laser to locate and calibrate the reference points in the video frame formed by the scan, so as to obtain a first search video carrying the timestamp of each frame image; wherein, the first search instruction includes a reference point selection rule, so as to determine the reference points located in the video frame based on the reference point selection rule.

2. The UAV reconnaissance and positioning system according to claim 1, characterized in that, The image recognition device includes: The first image recognition module is used to perform frame-by-frame image recognition on the first search video using the target shape features of each positioning target stored in the preset model database, so as to obtain several positioning targets in each first search image, and calculate the coarse positioning coordinates of each positioning target in the corresponding first search image based on the timestamp of each frame image and the coordinate information of the reference point; wherein, the coarse positioning coordinates include timestamp, positioning target ID, positioning target longitude, positioning target latitude and positioning target height; The coordinate sending module is used to send the corresponding coarse positioning coordinates to the UAV mission management device according to the timestamp of each frame image.

3. The UAV reconnaissance and positioning system according to claim 1, characterized in that, The airborne optoelectronic equipment includes: A direction adjustment module is used to adjust the detection direction of the airborne optoelectronic device according to the coarse positioning coordinates; The second search module is used to perform a focusing operation and stare in the detection direction to obtain a second search video.

4. The UAV reconnaissance and positioning system according to claim 3, characterized in that, The second search module includes: A staring detection unit is used to perform a focusing operation and to perform staring detection on the scene in the detection direction where the positioning target is located, so as to track the target scene containing the positioning target.

5. The UAV reconnaissance and positioning system according to claim 4, characterized in that, The image recognition device includes: The second image recognition module is used to perform image recognition on each target scene by storing the target shape features of each positioning target in a preset model database, so as to determine the pixel position information of the positioning target in the display screen according to the difference between the current pixel position of the positioning target in each target scene and the coordinates of the reference point, and send the pixel position information to the UAV mission management device.

6. The UAV reconnaissance and positioning system according to claim 5, characterized in that, The airborne optoelectronic equipment includes: The laser ranging module is used to receive laser ranging and positioning instructions sent by the UAV mission management device based on the pixel position information, to emit a laser beam toward the positioning target according to the laser ranging and positioning instructions, and to receive the laser signal reflected by the positioning target in order to calculate the target distance between the positioning target and the UAV. The fine positioning module is used to calculate the fine positioning coordinates of the positioning target using a preset positioning algorithm and based on the pixel position information, the target distance, the UAV position, and the UAV attitude.

7. A method for reconnaissance and positioning of unmanned aerial vehicles (UAVs), characterized in that, include: Based on the received prior coordinate information, the airborne optoelectronic equipment is controlled to perform a step-by-step staring search of the area to be scanned in the prior coordinate information, and a reference point is calibrated. The first search video generated during the search process is sent to the image recognition device. The image recognition device performs frame-by-frame image recognition on the first search video to obtain a first search image containing several positioning targets. Based on the coordinate information of the reference point, the positioning targets in each of the first search images are calculated to obtain the coarse positioning coordinates of each positioning target. Then, the coarse positioning coordinates are sent to the UAV mission management device. The UAV mission management device controls the onboard optoelectronic equipment to perform staring reconnaissance based on the coarse positioning coordinates to obtain a second search video, and then sends the second search video to the image recognition device. The image recognition device performs secondary image recognition on the second search video to obtain the pixel position information of the positioning target in the display screen, and sends the pixel position information to the UAV mission management device. The UAV mission management device controls the onboard optoelectronic equipment to track the target based on the pixel position information, and obtains the precise positioning coordinates of the target through laser ranging and a preset positioning algorithm. The UAV reconnaissance and positioning method also includes: When the relative distance between the target and the UAV where the airborne optoelectronic device is located is detected to meet the preset distance condition, the first search command and prior coordinate information are sent to the airborne optoelectronic device. The method of controlling the airborne optoelectronic equipment to perform a step-by-step staring search towards the area to be scanned in the prior coordinate information based on the received prior coordinate information, and calibrating the reference point, includes: The detection direction is adjusted according to the first search instruction and the prior coordinate information to perform a grid-like spiral step scan on the area to be scanned, and the reference points in the video frame formed by the scan are located and calibrated using a laser to obtain a first search video carrying the timestamp of each frame image; wherein, the first search instruction includes a reference point selection rule so as to determine the reference points located in the video frame based on the reference point selection rule.

8. A method for reconnaissance and positioning of unmanned aerial vehicles (UAVs), characterized in that, Applications in drone mission management equipment include: Based on the received prior coordinate information, the airborne optoelectronic device is controlled to perform a step-by-step staring search towards the area to be scanned in the prior coordinate information, and a reference point is calibrated. The first search video generated during the search process is sent to the image recognition device, so that the image recognition device can perform frame-by-frame image recognition on the first search video to obtain a first search image containing several positioning targets. Based on the coordinate information of the reference point, the positioning targets in each of the first search images are calculated to obtain the coarse positioning coordinates of each positioning target. Then, the coarse positioning coordinates are sent to the UAV mission management device. The airborne optoelectronic device is controlled to perform staring reconnaissance based on the coarse positioning coordinates to obtain a second search video. The second search video is then sent to the image recognition device so that the image recognition device can perform secondary image recognition on the second search video to obtain the pixel position information of the positioning target in the display screen. The pixel position information is then sent to the UAV mission management device. The onboard optoelectronic equipment is controlled to track the positioning target based on the pixel position information, and the precise positioning coordinates of the positioning target are obtained through laser ranging and a preset positioning algorithm. The UAV reconnaissance and positioning method also includes: When the relative distance between the target and the UAV where the airborne optoelectronic device is located is detected to meet the preset distance condition, the first search command and prior coordinate information are sent to the airborne optoelectronic device. The method of controlling the airborne optoelectronic equipment to perform a step-by-step staring search towards the area to be scanned in the prior coordinate information based on the received prior coordinate information, and calibrating the reference point, includes: The detection direction is adjusted according to the first search instruction and the prior coordinate information to perform a grid-like spiral step scan on the area to be scanned, and the reference points in the video frame formed by the scan are located and calibrated using a laser to obtain a first search video carrying the timestamp of each frame image; wherein, the first search instruction includes a reference point selection rule so as to determine the reference points located in the video frame based on the reference point selection rule.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the UAV reconnaissance and positioning method as described in claim 8.

Images