Target identification method and device based on photoelectric pod, unmanned aerial vehicle and medium

Abstract

The invention provides a target identification method and device based on a photoelectric pod, an unmanned aerial vehicle and a medium, and relates to the technical field of unmanned aerial vehicle identification control. In the application, firstly, a first photoelectric image of a first sub-region is acquired; secondly, performing target object recognition on the first photoelectric image to obtain a first object recognition result; then, if the first object recognition result represents that the target object exists in the first photoelectric image, position information of the target object is determined based on position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region; and finally, if the first object recognition result represents that the target object does not exist in the first photoelectric image, acquiring a second photoelectric image of the second sub-region, and performing target object recognition on the second photoelectric image to obtain a second object recognition result. On the basis of the content, the problem that in the prior art, the reliability of target recognition is relatively low can be solved.

Description

Technical Field

[0001] The present application relates to the field of drone identification and control technology, and more specifically, to a target identification method, device, drone, and medium based on an optoelectronic pod. Background Art

[0002] Drones typically carry electro-optical pods, which capture images and video and transmit them via a link to a ground-based command and control station. The drone operator manipulates the pod's pitch and azimuth to change the scanning field of view. By observing the electro-optical images, the operator manually identifies targets in the scene, perceives the situation, and further controls the drone to execute subsequent missions. This method of acquiring intelligence, which relies on manually controlled electro-optical pods to search for targets and relies on manual visual search, can easily lead to visual fatigue, misidentification of important targets, and difficulty in effectively positioning. Consequently, the operator suffers from relatively low target recognition reliability. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a target recognition method, device, drone and medium based on an optoelectronic pod to improve the problem of relatively low reliability of target recognition in the prior art.

[0004] To achieve the above objectives, this application adopts the following technical solutions: A target recognition method based on an optoelectronic pod, comprising: Acquire a first photoelectric image obtained by an electro-optical pod of the UAV by capturing an image of a first sub-region among a plurality of sub-regions of a target region, wherein the first sub-region is a sub-region currently searched by the electro-optical pod among the plurality of sub-regions; performing target object recognition on the first photoelectric image to obtain a first object recognition result; If the first object recognition result indicates that a target object exists in the first photoelectric image, determining the position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-area; If the first object recognition result indicates that the target object does not exist in the first photoelectric image, a second photoelectric image obtained by the photoelectric pod through image acquisition of a second sub-area among the multiple sub-areas is obtained, and target object recognition is performed on the second photoelectric image to obtain a second object recognition result, wherein the second sub-area refers to the sub-area searched later.

[0005] In a preferred embodiment of the present application, in the above-mentioned target recognition method based on the optoelectronic pod, if the first object recognition result indicates that the target object exists in the first optoelectronic image, the step of determining the position information of the target object based on the position coordinates of the target object in the first optoelectronic image and the distance between the optoelectronic pod and the first sub-area includes: If the first object recognition result indicates that a target object exists in the first photoelectric image, determining the image position coordinates of the target object in the first photoelectric image, and determining a target distance between the photoelectric pod and the area center of the first sub-area; Mapping the image position coordinates based on a projection relationship between the first photoelectric image and the first sub-region to form the regional position coordinates of the target object in the first sub-region; The position information of the target object is determined based on the target distance and the area position coordinates.

[0006] In a preferred embodiment of the present application, in the above-mentioned target recognition method based on the optoelectronic pod, the step of mapping the image position coordinates based on the projection relationship between the first optoelectronic image and the first sub-area to form the regional position coordinates of the target object in the first sub-area includes: Obtaining the length and width of the first sub-region to obtain the region length and region width; Obtaining the scanning field of view unit size of the optoelectronic pod to obtain the field of view length and field of view width; determining a first projection relationship between the first photoelectric image and the first sub-region in a length direction based on the region length and the field of view length; determining, based on the region width and the field of view width, a second projection relationship between the first photoelectric image and the first sub-region in a width direction; Based on the first projection relationship and the second projection relationship, the coordinates in the length direction and the coordinates in the width direction of the image position coordinates are mapped respectively to form the regional position coordinates of the target object in the first sub-region.

[0007] In a preferred embodiment of the present application, in the above-mentioned target recognition method based on the optoelectronic pod, the step of obtaining the scanning field of view unit size of the optoelectronic pod and obtaining the field of view length and field of view width includes: Obtaining the pitch angle, azimuth angle, horizontal scanning angle, and vertical scanning angle of the optoelectronic pod; Acquiring a relative field height of the optoelectronic pod relative to the first sub-area; Determining a field of view length of the electro-optical pod based on the pitch angle, the azimuth angle, the horizontal scanning angle of view, and the relative field height; The field of view width of the optoelectronic pod is determined based on the pitch angle, the azimuth angle, the vertical scanning angle of view, and the relative field height.

[0008] In a preferred embodiment of the present application, in the above-mentioned target recognition method based on the optoelectronic pod, the step of determining the position information of the target object based on the target distance and the regional position coordinates includes: Determining first relative position coordinates between a center of the first sub-area and the optoelectronic pod based on the target distance; Determining a second relative position coordinate between the target object and the photoelectric cabin based on the regional position coordinate and the first relative position coordinate; Converting the second relative position coordinates belonging to the rectangular coordinate system into longitude and latitude coordinates to obtain third relative position coordinates; Based on the current longitude and latitude coordinates of the optoelectronic pod and the third relative position coordinates, the longitude and latitude coordinates of the target object are determined and used as the position information of the target object.

[0009] In a preferred embodiment of the present application, in the above-mentioned target recognition method based on the optoelectronic pod, the step of determining the first relative position coordinates between the center of the first sub-area and the optoelectronic pod based on the target distance includes: Obtaining the current pitch angle and azimuth angle of the optoelectronic pod; Determining a relative position coordinate value in a length direction based on the target distance, the sine value of the pitch angle, and the cosine value of the azimuth angle, and determining a relative position coordinate value in a width direction based on the target distance, the sine value of the pitch angle, and the sine value of the azimuth angle; Based on the relative position coordinate value in the length direction and the relative position coordinate value in the width direction, a first relative position coordinate between the center of the first sub-region and the optoelectronic pod is determined.

[0010] In a preferred embodiment of the present application, in the above-mentioned target recognition method based on the optoelectronic pod, the step of obtaining a first optoelectronic image obtained by the optoelectronic pod of the drone capturing an image of a first sub-region of a target area including multiple sub-regions includes: Determining the center position coordinates of an optoelectronic scanning center of an optoelectronic pod of the UAV for a target area including a first sub-area among a plurality of sub-areas, determining the center reference position coordinates of the first sub-area, and performing a comparative analysis on the center position coordinates and the center reference position coordinates; If the coordinate value of the length direction in the center position coordinate is greater than the coordinate value of the length direction in the center reference position coordinate, the azimuth angle of the optoelectronic pod is increased; if the coordinate value of the length direction in the center position coordinate is less than the coordinate value of the length direction in the center reference position coordinate, the azimuth angle of the optoelectronic pod is decreased; If the coordinate value of the width direction in the center position coordinate is greater than the coordinate value of the width direction in the center reference position coordinate, the pitch angle of the optoelectronic pod is increased; if the coordinate value of the width direction in the center position coordinate is less than the coordinate value of the width direction in the center reference position coordinate, the pitch angle of the optoelectronic pod is decreased; If the coordinate value in the length direction of the center position coordinate is equal to the coordinate value in the length direction of the center reference position coordinate, and the coordinate value in the width direction of the center position coordinate is equal to the coordinate value in the width direction of the center reference position coordinate, image acquisition is performed on the first sub-area through the optoelectronic pod to obtain a first optoelectronic image.

[0011] The present application also provides a target recognition device based on an optoelectronic pod, comprising: an optoelectronic image acquisition module, configured to acquire a first optoelectronic image obtained by an optoelectronic pod of the UAV capturing an image of a first sub-region of a target area including multiple sub-regions, wherein the first sub-region is a sub-region currently searched by the optoelectronic pod within the multiple sub-regions; a first object recognition module, configured to perform target object recognition on the first photoelectric image to obtain a first object recognition result; a position information determining module, configured to determine position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-area if the first object recognition result indicates that the target object exists in the first photoelectric image; A second object recognition module is configured to obtain a second photoelectric image obtained by the photoelectric pod performing image acquisition on a second sub-region among the multiple sub-regions, and perform target object recognition on the second photoelectric image to obtain a second object recognition result if the first object recognition result indicates that the target object does not exist in the first photoelectric image, wherein the second sub-region refers to the sub-region searched later.

[0012] Based on the above, this application also provides a drone, including: memory for storing computer programs; The processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned target recognition method based on the optoelectronic pod.

[0013] On the basis of the above, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is run, it executes the various steps of the above-mentioned target recognition method based on the optoelectronic pod.

[0014] The present application provides a target recognition method, device, drone, and medium based on an optoelectronic pod. First, a first optoelectronic image of a first sub-region is acquired. Second, target object recognition is performed on the first optoelectronic image to obtain a first object recognition result. Then, if the first object recognition result indicates the presence of a target object in the first optoelectronic image, the target object's location information is determined based on the target object's location coordinates in the first optoelectronic image and the distance between the optoelectronic pod and the first sub-region. Finally, if the first object recognition result indicates the absence of the target object in the first optoelectronic image, a second optoelectronic image of a second sub-region is acquired and target object recognition is performed on the second optoelectronic image to obtain a second object recognition result. Based on the above, on the one hand, because target objects can be automatically identified and searched sequentially across multiple sub-regions, the method has higher efficiency and reliability compared to manual search control. On the other hand, because the target object's location information can also be determined based on the target object's location coordinates in the first optoelectronic image and the distance between the optoelectronic pod and the first sub-region, the target object can be effectively located, thereby improving the relatively low reliability of target recognition in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings.

[0016] Figure 1 This is a structural block diagram of the drone provided in an embodiment of the present application.

[0017] Figure 2 A schematic diagram of a target recognition method based on an optoelectronic pod provided in an embodiment of the present application.

[0018] Figure 3 A diagram showing the composition of an optoelectronic pod-based identification and positioning system according to an embodiment of the present application.

[0019] Figure 4 Schematic diagram of target search for the optoelectronic pod provided in an embodiment of the present application.

[0020] Figure 5 A schematic diagram of a target recognition device based on an optoelectronic pod provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0023] like Figure 1 As shown, an embodiment of the present application provides a drone, wherein the drone may include a memory, a processor, and a target recognition device based on an optoelectronic pod.

[0024] In detail, the memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, the memory and the processor can be electrically connected via one or more communication buses or signal lines. The target recognition device based on the optoelectronic pod includes at least one software function module stored in the memory in the form of software or firmware. The processor is used to execute an executable computer program stored in the memory, such as the software function module and computer program included in the target recognition device based on the optoelectronic pod, to implement the target recognition method based on the optoelectronic pod provided in the embodiment of the present application.

[0025] Optionally, the memory may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0026] Furthermore, the processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0027] I understand. Figure 1 The structure shown is for illustration only, and the drone may also include Figure 1 More or fewer components than shown, or with Figure 1 The different configurations shown may, for example, further include a communication unit for exchanging information with other devices.

[0028] Combine Figure 2 The present application also provides an optoelectronic pod-based target recognition method applicable to the aforementioned UAV. The method steps defined in the process related to the optoelectronic pod-based target recognition method can be implemented by the UAV.

[0029] The following will Figure 2 The specific process shown is explained in detail.

[0030] Step S110 , obtaining a first photoelectric image obtained by the photoelectric pod of the UAV capturing an image of a first sub-region among a plurality of sub-regions in a target region.

[0031] In an embodiment of the present application, the drone may acquire a first photoelectric image obtained by the drone's photoelectric pod capturing an image of a first sub-region of a target area including multiple sub-regions. The first sub-region refers to a sub-region currently searched by the photoelectric pod within the multiple sub-regions, such as the first sub-region.

[0032] Step S120 : performing target object recognition on the first photoelectric image to obtain a first object recognition result.

[0033] In an embodiment of the present application, after acquiring the first photoelectric image, the electronic device may perform target object recognition on the first photoelectric image to obtain a first object recognition result. The first object recognition result is used to indicate whether the target object exists in the first photoelectric image. Exemplarily, target object recognition may be implemented based on a salient object detection (SOD) algorithm. The specific implementation process can be referenced to relevant existing technologies and will not be further described here.

[0034] Step S130: If the first object recognition result indicates that a target object exists in the first photoelectric image, the position information of the target object is determined based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-area.

[0035] In this embodiment of the present application, after obtaining the first object recognition result, if the first object recognition result indicates that a target object exists in the first photoelectric image, the electronic device can determine the location information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-area. In this way, reliable positioning of the target object can be achieved.

[0036] Step S140: If the first object recognition result indicates that the target object does not exist in the first photoelectric image, a second photoelectric image is obtained by the photoelectric pod performing image capture on a second sub-area of ​​the multiple sub-areas, and target object recognition is performed on the second photoelectric image to obtain a second object recognition result.

[0037] In an embodiment of the present application, after obtaining the first object recognition result, if the first object recognition result indicates that the target object does not exist in the first photoelectric image, the electronic device can obtain a second photoelectric image obtained by the photoelectric pod performing image acquisition on a second sub-area among the multiple sub-areas, and perform target object recognition on the second photoelectric image to obtain a second object recognition result. The second sub-area refers to the sub-area that is searched later. Exemplarily, for the target area including multiple sub-areas, the search can be started from the first sub-area until the target object is identified, or until the last sub-area is searched. Based on this, reliable identification and positioning of the target object in the target area can be achieved.

[0038] Based on the above, the automatic identification of the target object and the sequential search of multiple sub-areas provide greater efficiency and reliability compared to manual search control. Furthermore, the effective positioning of the target object is achieved by determining its positional information based on its coordinates in the first photoelectric image and the distance between the photoelectric pod and the first sub-area. This improves the relatively low reliability of target identification found in prior art technologies.

[0039] First, it should be noted that, with respect to step S110 , the specific method of acquiring the first photoelectric image of the first sub-region is not limited and can be selected according to actual needs.

[0040] For example, in an alternative embodiment, in order to ensure that the acquired first photoelectric image can effectively reflect the first sub-region, the above-mentioned step S110 may further include the following specific implementation contents: First, the center position coordinates of the optoelectronic scanning center of the optoelectronic pod of the UAV for a first sub-region of a target area including multiple sub-regions can be determined (for example, the center position coordinates can be determined by the optoelectronic pod scanning the first sub-region), and the center reference position coordinates of the first sub-region can be determined (for example, the center reference position coordinates can be determined during the process of dividing the target area into multiple sub-regions), and the center position coordinates and the center reference position coordinates can be compared and analyzed; Secondly, if the coordinate value of the length direction (such as the x-axis in the x, y, z coordinate system) of the center position coordinate is greater than the coordinate value of the length direction of the center reference position coordinate, the azimuth angle of the optoelectronic pod is increased; if the coordinate value of the length direction of the center position coordinate is less than the coordinate value of the length direction of the center reference position coordinate, the azimuth angle of the optoelectronic pod is decreased; Furthermore, if the coordinate value of the width direction (such as the y-axis in the x, y, z coordinate system) in the center position coordinate is greater than the coordinate value of the width direction in the center reference position coordinate, the pitch angle of the optoelectronic pod is increased; if the coordinate value of the width direction in the center position coordinate is less than the coordinate value of the width direction in the center reference position coordinate, the pitch angle of the optoelectronic pod is decreased; In addition, if the coordinate value in the length direction of the center position coordinate is equal to the coordinate value in the length direction of the center reference position coordinate, and the coordinate value in the width direction of the center position coordinate is equal to the coordinate value in the width direction of the center reference position coordinate, then the photoelectric pod is used to collect images of the first sub-area to obtain a first photoelectric image. That is, when the center position coordinate of the photoelectric scanning center of the first sub-area is the same as the center reference position coordinate, the first sub-area is scanned again to form a corresponding first photoelectric image.

[0041] It is understandable that in order to effectively scan the multiple sub-regions and obtain reliable photoelectric images, the sub-regions may be segmented based on the following method: First, the size of the scanning field of view unit of the optoelectronic pod can be calculated, where the field of view length is: length = ; Field of view width: width= ; H is the relative field height of the optoelectronic pod (e.g., the height in the z-axis direction in the x, y, z coordinate system), is the scanning pitch angle of the electro-optical pod, is the scanning azimuth angle of the optoelectronic pod, is the horizontal scanning angle of the optoelectronic pod, is the vertical scanning angle of the electro-optical pod; Secondly, the target area is divided into m rows and n columns of photoelectric image scanning unit array R = {{R 11 , R 12 , R 13 ...R 1n},{R 21 , R 22 , R 23 ...R 2n},……{R m1 , R m2 , R m3 ...R mn}}, the photoelectric image scanning unit array R maps the actual area S={{S 11 , S 12 , S 13 ……S 1n},{S 21 , S 22 , S 23 ……S 2n},……{S i1 , S i2 , S i3 ……S in}}, the corresponding reference point set (i.e., the point corresponding to the coordinates of the center reference position) is {{O 11 , O 12 , O 13 ...O 1n},{O 21 , O 22 , O 23 ...O 2n},……{O m1 , O m2 , O m3 ...O mn}}.

[0042] Secondly, it should be noted that for step S130 , the specific method of determining the location information of the target object is not limited and can be selected according to actual needs.

[0043] For example, in an alternative embodiment, in order to ensure that the determined location information has a higher accuracy, the above-mentioned step S130 may further include step S131, step S132 and step S133, and the specific content of each step is described as follows.

[0044] Step S131: If the first object recognition result indicates that a target object exists in the first photoelectric image, the image position coordinates of the target object in the first photoelectric image are determined, and a target distance between the photoelectric pod and the area center of the first sub-area is determined.

[0045] In this embodiment of the present application, if the first object recognition result indicates the presence of a target object in the first photoelectric image, the image position coordinates of the target object in the first photoelectric image are determined (for example, the corresponding coordinates may be determined in step S120 above), and the target distance between the photoelectric pod and the center of the first sub-area is determined. The target distance may be obtained by laser ranging using the photoelectric pod.

[0046] Step S132 : Mapping the image position coordinates based on the projection relationship between the first photoelectric image and the first sub-region to form the region position coordinates of the target object in the first sub-region.

[0047] In this embodiment of the present application, after obtaining the image position coordinates, the image position coordinates can be mapped based on the projection relationship between the first photoelectric image and the first sub-region to form the regional position coordinates of the target object in the first sub-region. In other words, the coordinates in the image are projected to the actual coordinates.

[0048] Step S133: Determine the location information of the target object based on the target distance and the area location coordinates.

[0049] In the embodiment of the present application, after the target distance and the area position coordinates are obtained, the position information of the target object may be determined based on the target distance and the area position coordinates.

[0050] It can be understood that in the above-mentioned step S132, the specific method of forming the regional position coordinates of the target object in the first sub-area is not limited. For example, in an alternative embodiment, in order to enable reliable mapping so that the formed regional position coordinates are more accurate, the above-mentioned step S132 can further include step S132a, step S132b, step S132c, step S132d and step S132e, and the specific content of each step is described below.

[0051] Step S132a: Obtain the length and width of the first sub-region to obtain the region length and region width.

[0052] In the embodiment of the present application, the length and width of the first sub-region may be acquired to obtain the region length and region width.

[0053] Step S132b: Obtain the scanning field of view unit size of the optoelectronic pod to obtain the field of view length and field of view width.

[0054] In the embodiment of the present application, the scanning field of view unit size of the optoelectronic pod can also be obtained to obtain the field of view length and field of view width.

[0055] Step S132c: determining a first projection relationship between the first photoelectric image and the first sub-region in the length direction based on the region length and the field of view length.

[0056] In an embodiment of the present application, after obtaining the region length and the field of view length, a first projection relationship between the first photoelectric image and the first sub-region in the length direction can be determined based on the region length and the field of view length. For example, the first projection relationship can be the ratio between the region length and the field of view length.

[0057] Step S132d: determining a second projection relationship between the first photoelectric image and the first sub-region in the width direction based on the region width and the field of view width.

[0058] In this embodiment of the present application, after obtaining the region width and the field of view width, a second projection relationship between the first photoelectric image and the first sub-region in the width direction can be determined based on the region width and the field of view width. For example, the second projection relationship can be the ratio between the region width and the field of view width.

[0059] Step S132e: Based on the first projection relationship and the second projection relationship, the coordinates in the length direction and the coordinates in the width direction of the image position coordinates are mapped respectively to form the regional position coordinates of the target object in the first sub-region.

[0060] In an embodiment of the present application, after obtaining the first projection relationship and the second projection relationship, the coordinates in the length direction and the coordinates in the width direction of the image position coordinates can be mapped respectively based on the first projection relationship and the second projection relationship to form the regional position coordinates of the target object in the first sub-area. Exemplarily, the value corresponding to the first projection relationship can be multiplied by the coordinates in the length direction of the image position coordinates to obtain the coordinates in the length direction of the regional position coordinates of the target object in the first sub-area. In addition, the value corresponding to the second projection relationship can be multiplied by the coordinates in the width direction of the image position coordinates to obtain the coordinates in the width direction of the regional position coordinates of the target object in the first sub-area.

[0061] It is understood that, in the above step S132b, the specific method of obtaining the scanning field of view unit size of the electro-optical pod is not limited. For example, in an alternative embodiment, in order to accurately calculate the field of view length and field of view width, the above step S132b may further include the following specific implementation content: First, the pitch angle, azimuth angle, horizontal scanning angle and vertical scanning angle of the optoelectronic pod can be obtained; Secondly, the relative field height of the optoelectronic pod relative to the first sub-area can be obtained; Then, the field of view length of the optoelectronic pod can be determined based on the pitch angle, the azimuth angle, the horizontal scanning angle and the relative field height, as described above; Finally, the field of view width of the optoelectronic pod may be determined based on the pitch angle, the azimuth angle, the vertical scanning angle of view, and the relative field height, as described above.

[0062] It can be understood that in the above-mentioned step S132, the specific method of determining the location information of the target object based on the target distance and the area location coordinates is not limited. In order to be able to reliably determine the location information of the target object, the above-mentioned step S132 can further include step S132a, step S132b, step S132c and step S132d, the specific contents of which are as follows.

[0063] Step S132a: determining first relative position coordinates between the center of the first sub-area and the optoelectronic pod based on the target distance.

[0064] In the embodiment of the present application, the first relative position coordinates between the center of the first sub-area and the optoelectronic pod can be determined based on the target distance, such as .

[0065] Step S132b: determining a second relative position coordinate between the target object and the photoelectric cabin based on the regional position coordinate and the first relative position coordinate.

[0066] In the embodiment of the present application, after obtaining the first relative position coordinates, the second relative position coordinates between the target object and the photoelectric cabin can be determined based on the regional position coordinates and the first relative position coordinates, such as ,in, , , the coordinates of the area position are (P xs , P ys ).

[0067] Step S132c: convert the second relative position coordinates belonging to the rectangular coordinate system into longitude and latitude coordinates to obtain third relative position coordinates.

[0068] In the embodiment of the present application, after obtaining the second relative position coordinates, the second relative position coordinates belonging to the rectangular coordinate system can be converted into the longitude and latitude coordinates to obtain the third relative position coordinates, such as Convert to , where lon is longitude and lat is latitude.

[0069] Step S132d: Based on the current longitude and latitude coordinates of the optoelectronic pod and the third relative position coordinates, the longitude and latitude coordinates of the target object are determined and used as the position information of the target object.

[0070] In an embodiment of the present application, after obtaining the third relative position coordinates, the longitude and latitude coordinates of the target object can be determined based on the current longitude and latitude coordinates of the optoelectronic pod and the third relative position coordinates, and used as the position information of the target object.

[0071] It is understood that, in the above step S132a, the specific method of determining the first relative position coordinates between the area center of the first sub-area and the optoelectronic pod is not limited. For example, in an alternative embodiment, in order to include the reliability of the determined first relative position coordinates, the above step S132a may further include the following: First, the current pitch angle and azimuth angle of the optoelectronic pod can be obtained, for example, To characterize the scanning pitch angle of the electro-optical pod, we can use To characterize the scanning azimuth angle of the optoelectronic pod; Secondly, the relative position coordinate value in the length direction can be determined based on the target distance, the sine value of the pitch angle and the cosine value of the azimuth angle, and the relative position coordinate value in the width direction can be determined based on the target distance, the sine value of the pitch angle and the sine value of the azimuth angle, such as the relative position coordinate value in the length direction , relative position coordinate value in the width direction , s is the target distance, is the pitch angle of the optoelectronic pod, is the azimuth of the optoelectronic pod; Then, based on the relative position coordinate value in the length direction and the relative position coordinate value in the width direction, the first relative position coordinate between the center of the first sub-region and the optoelectronic pod can be determined, that is, .

[0072] On the basis of the above embodiment, in order to facilitate understanding of the above target recognition method based on the optoelectronic pod, the following specific application example is also provided.

[0073] like Figure 3 As shown, an identification system is provided, which may include an electro-optical pod, a flight control computer, a ground command station, a mission search module, an image forwarding module and a target identification module.

[0074] Among them, the flight control computer and the ground command and control station are interconnected through link communication. The ground command and control personnel select the search area information on the ground command and control station and upload it to the flight control computer through remote control data. The flight control computer will transmit the identified target information to the ground command and control station through the link.

[0075] Among them, the flight control computer and the task search module communicate and interconnect, the flight control computer sends the location information of the UAV to the task search module, and receives task feedback information from the task search module.

[0076] Among them, the mission search module and the optoelectronic pod are interconnected in communication. The mission search module sends pitch angle and azimuth angle instructions to the optoelectronic pod and receives status information feedback from the optoelectronic pod.

[0077] Among them, the optoelectronic pod receives the pitch angle and azimuth angle instructions from the mission search module, can change the scanning field of view, collect regional images, and then send the original images collected by the optoelectronic pod to the image forwarding module.

[0078] Among them, the image forwarding module sends the original image to the target recognition module and the flight control computer respectively.

[0079] Among them, the target recognition module sends the image position information of the target object to the task search module and sends the recognized image to the flight control computer.

[0080] The working principle of the system is as follows: the ground command personnel input the optoelectronic search area at the ground command station, and send the data to the flight control computer through the remote control link. The task search module reads the flight control information, calculates the scanning field of view of a single frame image, and divides the search area into scanning array units of the optoelectronic pod with the scanning field size as a unit. The pitch and azimuth angles of the optoelectronic pod are controlled, and the scanning array units of the divided area are traversed to collect data. The image forwarding module sends the collected image of the optoelectronic pod to the flight control computer one way and to the target recognition module the other way. The target recognition module uses the salient target recognition (SOD) algorithm to recognize and detect pre-trained targets in the image. The recognized target objects are marked and processed and then sent to the flight control computer. According to the pixel position of the target object in the image, the relative position and absolute position coordinates of the target object in the actual area are calculated by the projection relationship. Finally, the position coordinates of the target object can be sent to the flight control computer and the ground command station. Specifically: S1: Collect the target vehicle data set to be searched and train the recognition model algorithm; S2: Select the search and identification rectangular area on the ground control station, and the ground control station sends the boundary coordinates A (x1, y1), B (x2, y2), C (x3, y3), and D (x4, y4) to the flight control computer; S3: The mission search module reads the UAV’s location information and search area information: (1) UAV longitude Lon; (2) latitude Lat; (3) relative field height H; (4) pitch angle of the optoelectronic pod ; (5) Azimuth of the optoelectronic pod ; (6) Horizontal scanning angle of the optoelectronic pod ; (7) Vertical scanning angle of the optoelectronic pod ; (8) The length of the search area L; (9) The width of the search area W; S4: Calculate the size of the scanning field of view unit of the electro-optical pod Field of view length: length= ; Field of view width: width= ; in, is the pitch angle of the optoelectronic pod, is the azimuth of the optoelectronic pod, is the horizontal scanning angle of the optoelectronic pod, is the vertical scanning angle of the electro-optical pod; S5: Taking the scanning field of view unit of the optoelectronic pod as the smallest unit, the target area is divided into an optoelectronic image scanning unit array R={{R 11 , R 12 , R 13 ...R 1n},{R 21 , R 22 , R 23 ...R 2n},……{R m1 , R m2 , R m3 ...R mn}}, the photoelectric image scanning unit array R maps the actual area S={{S 11 , S 12 , S 13 ……S 1n},{S 21 , S 22 , S 23 ……S 2n},……{S i1 , S i2 , S i3 ……Sin}}, the corresponding reference point set (i.e., the point corresponding to the coordinates of the center reference position) is {{O 11 , O 12 , O 13 ...O 1n},{O 21 , O 22 , O 23 ...O 2n},……{O m1 , O m2 , O m3 ...O mn}},in, (rounded), (rounded), S6: The mission search module controls the pitch angle and azimuth angle of the optoelectronic pod and collects the segmented area S11 (such as Figure 4 Medium R 11 The image of the corresponding area) is obtained based on the photoelectric scanning center O0 (x0, y0) and the regional center reference point O ij (x i ,y j ) Calculate the control angle of the optoelectronic pod. If x0>x i , the azimuth angle increases, if x0 <x i , the azimuth angle decreases; if y0>y j , the pitch angle increases, if y0 <y j , the pitch angle decreases; S7: The electro-optical pod sends the image to the image forwarding module through the network. The image forwarding module uses ffmepg and mediamtx software to forward the image to the target recognition module and the flight control computer. S8: The target recognition module uses a trained salient object detection (SOD) algorithm to detect and identify the segmented image, setting parameter thresholds. If a target is identified, the target vehicle is marked on the image and sent to the flight control computer, which forwards the information to the ground control station via a link for display and processing. If the target vehicle is not identified, S15 is executed to search the next area. S9: The target recognition module reads the target vehicle in the image set R ij Coordinate P in R (P xr , P yr ), the optoelectronic pod obtains the distance s to the center of the scanning area through laser ranging; S10: Calculate the target vehicle in the actual area S according to the projection relationship ij Coordinates P S (P xs , P ys ); Pxs =l / length×P xr , P ys =w / width×P yr ; Among them, l and w are the length and width of a scanned segmented area respectively; S11: Calculate the relative position coordinates from the center of the segmented area to the UAV (photoelectric pod) : , ; S12: Calculate the relative position coordinates of the target vehicle ; , ; S13: Cartesian coordinates Convert to longitude and latitude coordinates ; S14: Calculate the longitude and latitude position information of the target vehicle based on the current longitude and latitude of the UAV and send it to the flight control computer.

[0081] S15: The task search module controls the optoelectronic pod to scan and collect the next area, repeating steps S2 to S14, traversing the search area S, and completing the area search.

[0082] Combine Figure 5 The present application also provides an optoelectronic pod-based target recognition device applicable to the aforementioned drone. The optoelectronic pod-based target recognition device includes an optoelectronic image acquisition module, a first object recognition module, a position information determination module, and a second object recognition module.

[0083] The photoelectric image acquisition module is used to acquire a first photoelectric image obtained by the photoelectric pod of the drone for capturing an image of a first sub-region of a target area including multiple sub-regions, wherein the first sub-region refers to a sub-region currently searched by the photoelectric pod in the multiple sub-regions. In the embodiment of the present application, the photoelectric image acquisition module can be used to perform Figure 2 As shown in step S110 , for the relevant contents of the photoelectric image acquisition module, reference may be made to the above description of step S110 .

[0084] The first object recognition module is used to perform target object recognition on the first photoelectric image to obtain a first object recognition result. In the embodiment of the present application, the first object recognition module 0 can be used to perform Figure 2 As shown in step S120, for the relevant content of the first object recognition module, reference can be made to the above description of step S120.

[0085] The position information determination module is configured to determine the position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-area if the first object recognition result indicates that the target object exists in the first photoelectric image. In the embodiment of the present application, the position information determination module can be used to perform Figure 2 As shown in step S130, for the relevant content of the position information determination module, reference can be made to the above description of step S130.

[0086] The second object recognition module is configured to obtain a second photoelectric image obtained by the photoelectric pod performing image acquisition on a second sub-region of the multiple sub-regions if the first object recognition result indicates that the target object does not exist in the first photoelectric image, and perform target object recognition on the second photoelectric image to obtain a second object recognition result, wherein the second sub-region refers to the sub-region that was searched last. In this embodiment of the present application, the second object recognition module can be used to perform Figure 2 As shown in step S140, for the relevant content of the second object recognition module, reference can be made to the above description of step S140.

[0087] In an embodiment of the present application, corresponding to the above-mentioned target recognition method based on the optoelectronic pod applied to the drone, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is run, the various steps of the target recognition method based on the optoelectronic pod are executed.

[0088] Among them, the steps executed when the aforementioned computer program is running will not be described here one by one, and reference can be made to the above explanation of the target recognition method based on the optoelectronic pod.

[0089] In summary, the target recognition method, device, drone, and medium based on an optoelectronic pod provided in this application first acquire a first optoelectronic image of a first sub-region; second, perform target object recognition on the first optoelectronic image to obtain a first object recognition result; then, if the first object recognition result indicates the presence of a target object in the first optoelectronic image, determine the target object's location information based on the target object's position coordinates in the first optoelectronic image and the distance between the optoelectronic pod and the first sub-region; finally, if the first object recognition result indicates the absence of the target object in the first optoelectronic image, acquire a second optoelectronic image of a second sub-region, and perform target object recognition on the second optoelectronic image to obtain a second object recognition result. Based on the above, on the one hand, because the target object can be automatically identified and searched sequentially across multiple sub-regions, it has higher efficiency and reliability compared to manual search control. On the other hand, because the target object's location information can also be determined based on the target object's position coordinates in the first optoelectronic image and the distance between the optoelectronic pod and the first sub-region, effective positioning of the target object can be achieved, thereby improving the relatively low reliability of target recognition in the prior art.

[0090] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0091] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0092] If the functions are implemented in the form of software modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, electronic device, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. It should be noted that, in this document, the terms "comprise," "include," or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device. Without further constraints, an element defined by the phrase "comprises a..." does not preclude the existence of additional identical elements in the process, method, article or apparatus that includes the element.

[0093] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A target recognition method based on an optoelectronic pod, characterized in that: include: Acquire a first photoelectric image obtained by an electro-optical pod of the UAV by capturing an image of a first sub-region among a plurality of sub-regions of a target region, wherein the first sub-region is a sub-region currently searched by the electro-optical pod among the plurality of sub-regions; performing target object recognition on the first photoelectric image to obtain a first object recognition result; If the first object recognition result indicates that a target object exists in the first photoelectric image, determining the position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-area; If the first object recognition result indicates that the target object does not exist in the first photoelectric image, a second photoelectric image obtained by the photoelectric pod through image acquisition of a second sub-area among the multiple sub-areas is obtained, and target object recognition is performed on the second photoelectric image to obtain a second object recognition result, wherein the second sub-area refers to the sub-area searched later.

2. The target recognition method based on the optoelectronic pod according to claim 1 is characterized in that: If the first object recognition result indicates that a target object exists in the first photoelectric image, the step of determining the position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-area includes: If the first object recognition result indicates that a target object exists in the first photoelectric image, determining the image position coordinates of the target object in the first photoelectric image, and determining a target distance between the photoelectric pod and the area center of the first sub-area; Mapping the image position coordinates based on a projection relationship between the first photoelectric image and the first sub-region to form the regional position coordinates of the target object in the first sub-region; The position information of the target object is determined based on the target distance and the area position coordinates.

3. The target recognition method based on the optoelectronic pod according to claim 2, characterized in that: The step of mapping the image position coordinates based on the projection relationship between the first photoelectric image and the first sub-region to form the regional position coordinates of the target object in the first sub-region includes: Obtaining the length and width of the first sub-region to obtain the region length and region width; Obtaining the scanning field of view unit size of the optoelectronic pod to obtain the field of view length and field of view width; determining a first projection relationship between the first photoelectric image and the first sub-region in a length direction based on the region length and the field of view length; determining, based on the region width and the field of view width, a second projection relationship between the first photoelectric image and the first sub-region in a width direction; Based on the first projection relationship and the second projection relationship, the coordinates in the length direction and the coordinates in the width direction of the image position coordinates are mapped respectively to form the regional position coordinates of the target object in the first sub-region.

4. The target recognition method based on the optoelectronic pod according to claim 3 is characterized in that: The step of obtaining the scanning field of view unit size of the optoelectronic pod and obtaining the field of view length and field of view width includes: Obtaining the pitch angle, azimuth angle, horizontal scanning angle, and vertical scanning angle of the optoelectronic pod; Acquiring a relative field height of the optoelectronic pod relative to the first sub-area; Determining a field of view length of the electro-optical pod based on the pitch angle, the azimuth angle, the horizontal scanning angle of view, and the relative field height; The field of view width of the optoelectronic pod is determined based on the pitch angle, the azimuth angle, the vertical scanning angle of view, and the relative field height.

5. The target recognition method based on the optoelectronic pod according to claim 2, characterized in that: The step of determining the location information of the target object based on the target distance and the area location coordinates includes: Determining first relative position coordinates between a center of the first sub-area and the optoelectronic pod based on the target distance; Determining a second relative position coordinate between the target object and the photoelectric cabin based on the regional position coordinate and the first relative position coordinate; Converting the second relative position coordinates belonging to the rectangular coordinate system into longitude and latitude coordinates to obtain third relative position coordinates; Based on the current longitude and latitude coordinates of the optoelectronic pod and the third relative position coordinates, the longitude and latitude coordinates of the target object are determined and used as the position information of the target object.

6. The target recognition method based on the optoelectronic pod according to claim 5, characterized in that: The step of determining first relative position coordinates between the center of the first sub-area and the optoelectronic pod based on the target distance includes: Obtaining the current pitch angle and azimuth angle of the optoelectronic pod; Determining a relative position coordinate value in a length direction based on the target distance, the sine value of the pitch angle, and the cosine value of the azimuth angle, and determining a relative position coordinate value in a width direction based on the target distance, the sine value of the pitch angle, and the sine value of the azimuth angle; Based on the relative position coordinate value in the length direction and the relative position coordinate value in the width direction, a first relative position coordinate between the center of the first sub-region and the optoelectronic pod is determined.

7. The target recognition method based on an optoelectronic pod according to any one of claims 1 to 6, characterized in that: The step of obtaining a first photoelectric image obtained by the photoelectric pod of the drone capturing an image of a first sub-region of a target area including multiple sub-regions comprises: Determining the center position coordinates of an optoelectronic scanning center of an optoelectronic pod of the UAV for a target area including a first sub-area among a plurality of sub-areas, determining the center reference position coordinates of the first sub-area, and performing a comparative analysis on the center position coordinates and the center reference position coordinates; If the coordinate value of the length direction in the center position coordinate is greater than the coordinate value of the length direction in the center reference position coordinate, the azimuth angle of the optoelectronic pod is increased; if the coordinate value of the length direction in the center position coordinate is less than the coordinate value of the length direction in the center reference position coordinate, the azimuth angle of the optoelectronic pod is decreased; If the coordinate value of the width direction in the center position coordinate is greater than the coordinate value of the width direction in the center reference position coordinate, the pitch angle of the optoelectronic pod is increased; if the coordinate value of the width direction in the center position coordinate is less than the coordinate value of the width direction in the center reference position coordinate, the pitch angle of the optoelectronic pod is decreased; If the coordinate value in the length direction of the center position coordinate is equal to the coordinate value in the length direction of the center reference position coordinate, and the coordinate value in the width direction of the center position coordinate is equal to the coordinate value in the width direction of the center reference position coordinate, image acquisition is performed on the first sub-area through the optoelectronic pod to obtain a first optoelectronic image.

8. A target recognition device based on an optoelectronic pod, characterized in that: include: an optoelectronic image acquisition module, configured to acquire a first optoelectronic image obtained by an optoelectronic pod of the UAV capturing an image of a first sub-region of a target area including multiple sub-regions, wherein the first sub-region is a sub-region currently searched by the optoelectronic pod within the multiple sub-regions; a first object recognition module, configured to perform target object recognition on the first photoelectric image to obtain a first object recognition result; a position information determining module, configured to determine position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-area if the first object recognition result indicates that the target object exists in the first photoelectric image; A second object recognition module is configured to obtain a second photoelectric image obtained by the photoelectric pod performing image acquisition on a second sub-region among the multiple sub-regions, and perform target object recognition on the second photoelectric image to obtain a second object recognition result if the first object recognition result indicates that the target object does not exist in the first photoelectric image, wherein the second sub-region refers to the sub-region searched later.

9. A drone, characterized in that: include: memory for storing computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the target recognition method based on the optoelectronic pod according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when running, executes the target recognition method based on the optoelectronic pod according to any one of claims 1 to 7.

Images