Target identification method and device based on photoelectric pod, unmanned aerial vehicle and medium

Abstract

The application provides a target recognition method and device based on an optoelectronic pod, a UAV and a medium, and relates to the technical field of UAV recognition control. In the application, first, a first optoelectronic image of a first sub-region is acquired; second, target object recognition is performed on the first optoelectronic image to obtain a first object recognition result; then, if the first object recognition result indicates that there is a target object in the first optoelectronic image, the position information of the target object is determined based on the position coordinates of the target object in the first optoelectronic image and the distance between the optoelectronic pod and the first sub-region; finally, if the first object recognition result indicates that there is no target object in the first optoelectronic image, a second optoelectronic image of a second sub-region is acquired, and target object recognition is performed on the second optoelectronic image to obtain a second object recognition result. Based on the above, the problem that the reliability of target recognition is relatively low in the prior art can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle identification control, in particular to a target identification method and device based on an optoelectronic pod, an unmanned aerial vehicle and a medium. BACKGROUND

[0002] An unmanned aerial vehicle generally carries an optoelectronic pod, collects image videos through the optoelectronic pod, and transmits the image videos to a ground control station through a link. An operator of the unmanned aerial vehicle controls the pitch and azimuth angle of the optoelectronic pod to change the scanning field of view, manually identifies targets in the scene by observing the optoelectronic image, senses the scene situation, and further controls the unmanned aerial vehicle to perform subsequent tasks. This intelligence acquisition method that relies on manual control of the optoelectronic pod to search for targets relies on manual visual search for targets, which is prone to visual fatigue and misidentification of important targets, and it is difficult to effectively locate targets. Therefore, the reliability of target identification is relatively low. SUMMARY

[0003] In view of the above, the purpose of the present application is to provide a target identification method and device based on an optoelectronic pod, an unmanned aerial vehicle and a medium to improve the problem of relatively low reliability of target identification in the prior art.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0005] A target identification method based on an optoelectronic pod, comprising:

[0006] Obtaining a first optoelectronic image obtained by an optoelectronic pod of an unmanned aerial vehicle performing image collection on a first sub-region in a target region including a plurality of sub-regions, wherein the first sub-region refers to a sub-region currently searched by the optoelectronic pod in the plurality of sub-regions;

[0007] Performing target object identification on the first optoelectronic image to obtain a first object identification result;

[0008] If the first object identification result indicates that there is a target object in the first optoelectronic image, determining the position information of the target object based on the position coordinates of the target object in the first optoelectronic image and the distance between the optoelectronic pod and the first sub-region;

[0009] If the first object identification result indicates that there is no target object in the first optoelectronic image, obtaining a second optoelectronic image obtained by the optoelectronic pod performing image collection on a second sub-region in the plurality of sub-regions, and performing target object identification on the second optoelectronic image to obtain a second object identification result, wherein the second sub-region refers to a sub-region searched subsequently.

[0010] In the preferred selection of the present application, in the above-mentioned target recognition method based on the photoelectric pod, if the first object recognition result represents that there is a target object in the first photoelectric image, the step of determining the position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region, comprises:

[0011] If the first object recognition result represents that there is a target object in the first photoelectric image, the image position coordinates of the target object in the first photoelectric image are determined, and the target distance between the photoelectric pod and the center of the first sub-region is determined.

[0012] The image position coordinates are mapped based on the projection relationship between the first photoelectric image and the first sub-region, to form the region position coordinates of the target object in the first sub-region.

[0013] The position information of the target object is determined based on the target distance and the region position coordinates.

[0014] In the preferred selection of the present application, in the above-mentioned target recognition method based on the photoelectric pod, the step of mapping the image position coordinates based on the projection relationship between the first photoelectric image and the first sub-region to form the region position coordinates of the target object in the first sub-region, comprises:

[0015] The length and width of the first sub-region are obtained to obtain the region length and the region width.

[0016] The scanning field of view unit size of the photoelectric pod is obtained to obtain the field of view length and the field of view width.

[0017] Based on the region length and the field of view length, the first projection relationship between the first photoelectric image and the first sub-region in the length direction is determined.

[0018] Based on the region width and the field of view width, the second projection relationship between the first photoelectric image and the first sub-region in the width direction is determined.

[0019] Based on the first projection relationship and the second projection relationship, the coordinates in the length direction and the coordinates in the width direction in the image position coordinates are respectively mapped to form the region position coordinates of the target object in the first sub-region.

[0020] In the preferred selection of the present application, in the above-mentioned target recognition method based on the photoelectric pod, the step of obtaining the scanning field of view unit size of the photoelectric pod to obtain the field of view length and the field of view width, comprises:

[0021] obtaining a pitch angle, an azimuth angle, a horizontal scanning view angle and a vertical scanning view angle of the electro-optical pod;

[0022] obtaining a relative field height of the electro-optical pod relative to the first sub-region;

[0023] determining a field length of the electro-optical pod based on the pitch angle, the azimuth angle, the horizontal scanning view angle and the relative field height;

[0024] determining a field width of the electro-optical pod based on the pitch angle, the azimuth angle, the vertical scanning view angle and the relative field height.

[0025] In a preferred selection of the present application, in the above target identification method based on the electro-optical pod, the step of determining the position information of the target object based on the target distance and the region position coordinates comprises:

[0026] determining a first relative position coordinate between the region center of the first sub-region and the electro-optical pod based on the target distance;

[0027] determining a second relative position coordinate between the target object and the electro-optical pod based on the region position coordinates and the first relative position coordinate;

[0028] converting the second relative position coordinate belonging to the rectangular coordinate system into a geographic coordinate system to obtain a third relative position coordinate;

[0029] determining the geographic coordinates of the target object based on the current geographic coordinates of the electro-optical pod and the third relative position coordinate, and taking the geographic coordinates as the position information of the target object.

[0030] In a preferred selection of the present application, in the above target identification method based on the electro-optical pod, the step of determining the first relative position coordinate between the region center of the first sub-region and the electro-optical pod based on the target distance comprises:

[0031] obtaining a current pitch angle and an azimuth angle of the electro-optical pod;

[0032] determining a relative position coordinate value in the length direction based on the target distance, the sine value of the pitch angle and the cosine value of the azimuth angle, and determining a relative position coordinate value in the width direction based on the target distance, the sine value of the pitch angle and the sine value of the azimuth angle;

[0033] determining the first relative position coordinate between the region center of the first sub-region and the electro-optical pod based on the relative position coordinate value in the length direction and the relative position coordinate value in the width direction.

[0034] In the preferred selection of the present application, in the above-mentioned target recognition method based on the photoelectric pod, the step of acquiring the first photoelectric image obtained by the photoelectric pod of the unmanned aerial vehicle for image collection on the first sub-region in the target region including a plurality of sub-regions comprises:

[0035] determining the center position coordinates of the photoelectric scanning center of the photoelectric pod of the unmanned aerial vehicle for the first sub-region in the target region including a plurality of sub-regions, and determining the center reference position coordinates of the first sub-region, and comparing and analyzing the center position coordinates and the center reference position coordinates;

[0036] if the coordinate value in the length direction of the center position coordinates is greater than the coordinate value in the length direction of the center reference position coordinates, the azimuth of the photoelectric pod is increased, and if the coordinate value in the length direction of the center position coordinates is less than the coordinate value in the length direction of the center reference position coordinates, the azimuth of the photoelectric pod is decreased;

[0037] if the coordinate value in the width direction of the center position coordinates is greater than the coordinate value in the width direction of the center reference position coordinates, the pitch angle of the photoelectric pod is increased, and if the coordinate value in the width direction of the center position coordinates is less than the coordinate value in the width direction of the center reference position coordinates, the pitch angle of the photoelectric pod is decreased;

[0038] if the coordinate value in the length direction of the center position coordinates is equal to the coordinate value in the length direction of the center reference position coordinates, and the coordinate value in the width direction of the center position coordinates is equal to the coordinate value in the width direction of the center reference position coordinates, the photoelectric pod is used to collect images on the first sub-region to obtain a first photoelectric image.

[0039] The present application also provides a target recognition device based on a photoelectric pod, comprising:

[0040] a photoelectric image acquisition module for acquiring a first photoelectric image obtained by the photoelectric pod of the unmanned aerial vehicle for image collection on a first sub-region in a target region including a plurality of sub-regions, wherein the first sub-region refers to a sub-region currently searched by the photoelectric pod in the plurality of sub-regions;

[0041] a first object recognition module for performing target object recognition on the first photoelectric image to obtain a first object recognition result;

[0042] a position information determination module for determining the position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region if the first object recognition result represents that there is a target object in the first photoelectric image;

[0043] The second object recognition module is configured to, if the first object recognition result indicates that there is no target object in the first photoelectric image, acquire a second photoelectric image obtained by image collection of the photoelectric pod on a second sub-region of the plurality of sub-regions, and perform target object recognition on the second photoelectric image to obtain a second object recognition result, wherein the second sub-region refers to a searched sub-region.

[0044] Based on the above, the application further provides a UAV, comprising:

[0045] A memory configured to store a computer program;

[0046] A processor connected to the memory and configured to execute the computer program stored in the memory to implement the target recognition method based on the photoelectric pod.

[0047] Based on the above, the application further provides a computer readable storage medium, which stores a computer program, and the computer program performs each step of the target recognition method based on the photoelectric pod when running.

[0048] The target recognition method, device, UAV and medium based on the photoelectric pod provided by the application first acquire a first photoelectric image of a first sub-region, then perform target object recognition on the first photoelectric image to obtain a first object recognition result, then, if the first object recognition result indicates that there is a target object in the first photoelectric image, determine the position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region, and finally, if the first object recognition result indicates that there is no target object in the first photoelectric image, acquire a second photoelectric image of a second sub-region and perform target object recognition on the second photoelectric image to obtain a second object recognition result. Based on the above, on the one hand, the target object can be automatically recognized, and a plurality of sub-regions can be searched in turn, so that the efficiency and reliability are higher than those of the search control based on manual operation. On the other hand, the position information of the target object can be determined based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region, so that the target object can be effectively positioned, and thus the problem that the reliability of target recognition is relatively low in the prior art can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to make the above objectives, characteristics and advantages of the application more apparent and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are referred to.

[0050] Figure 1 The structure block diagram of the UAV provided by the embodiments of the application is shown.

[0051] Figure 2 A schematic diagram of a target identification method based on an optoelectronic pod is provided for the embodiments of the present application.

[0052] Figure 3 A composition diagram of a target identification and positioning system based on an optoelectronic pod is provided for the embodiments of the present application.

[0053] Figure 4 A target search schematic diagram of an optoelectronic pod is provided for the embodiments of the present application.

[0054] Figure 5 A schematic diagram of a target identification device based on an optoelectronic pod is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0057] As shown in Figure 1 The embodiments of the present application provide a UAV. The UAV can include a memory, a processor and a target identification device based on an optoelectronic pod.

[0058] In detail, the memory and the processor are directly or indirectly electrically connected to realize data transmission or interaction. For example, the memory and the processor can be electrically connected through one or more communication buses or signal lines. The target identification device based on an optoelectronic pod includes at least one software function module stored in the memory in the form of software or firmware. The processor is used to execute the executable computer programs stored in the memory, for example, the software function modules and computer programs included in the target identification device based on an optoelectronic pod, etc., to realize the target identification method based on an optoelectronic pod provided by the embodiments of the present application.

[0059] Optionally, the memory can be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electric Erasable Programmable Read-Only Memory (EEPROM) and the like.

[0060] In addition, the processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), a System on Chip (SoC) and the like; and can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0061] It can be understood that, Figure 1 The structure shown is only schematic, and the unmanned aerial vehicle can further include more or fewer components than those shown in the figures, or have a different configuration from that shown in the figures, for example, can further include a communication unit for information interaction with other devices. Figure 1 The structure shown is only schematic, and the unmanned aerial vehicle can further include more or fewer components than those shown in the figures, or have a different configuration from that shown in the figures, for example, can further include a communication unit for information interaction with other devices. Figure 1 The structure shown is only schematic, and the unmanned aerial vehicle can further include more or fewer components than those shown in the figures, or have a different configuration from that shown in the figures, for example, can further include a communication unit for information interaction with other devices.

[0062] In combination with Figure 2 The embodiment of the present application also provides a target identification method based on an optoelectronic pod applicable to the unmanned aerial vehicle. The method steps defined by the flow of the target identification method based on the optoelectronic pod can be realized by the unmanned aerial vehicle.

[0063] The specific flow shown in Figure 2 will be described in detail below.

[0064] In step S110, a first optoelectronic image obtained by the optoelectronic pod of the unmanned aerial vehicle performing image acquisition on a first sub-region included in a plurality of sub-regions of a target region is acquired.

[0065] In the embodiment of the present application, the unmanned aerial vehicle can acquire a first optoelectronic image obtained by the optoelectronic pod of the unmanned aerial vehicle performing image acquisition on a first sub-region included in a plurality of sub-regions of a target region. The first sub-region refers to a sub-region currently searched by the optoelectronic pod in the plurality of sub-regions, such as the first sub-region.

[0066] Step S120, target object recognition is performed on the first photoelectric image to obtain a first object recognition result.

[0067] In the embodiment of the present application, after the first photoelectric image is acquired, the electronic device can perform target object recognition on the first photoelectric image to obtain a first object recognition result. The first object recognition result is used to represent whether there is a target object in the first photoelectric image. The target object recognition can be implemented based on a salient object detection (SOD) algorithm. The specific implementation process can refer to related prior art, and will not be described here.

[0068] Step S130, if the first object recognition result represents that there is a target object in the first photoelectric image, the position information of the target object is determined based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region.

[0069] In the embodiment of the present application, after the first object recognition result is obtained, if the first object recognition result represents that there is a target object in the first photoelectric image, the electronic device can determine the position information of the target object based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region. In this way, reliable positioning of the target object can be achieved.

[0070] Step S140, if the first object recognition result represents that there is no target object in the first photoelectric image, a second photoelectric image obtained by the photoelectric pod for image acquisition on a second sub-region in the plurality of sub-regions is acquired, and target object recognition is performed on the second photoelectric image to obtain a second object recognition result.

[0071] In the embodiment of the present application, after the first object recognition result is obtained, if the first object recognition result represents that there is no target object in the first photoelectric image, the electronic device can acquire a second photoelectric image obtained by the photoelectric pod for image acquisition on a second sub-region in the plurality of sub-regions, and perform target object recognition on the second photoelectric image to obtain a second object recognition result. The second sub-region refers to a searched sub-region. For example, for a target region including a plurality of sub-regions, the search can start from the first sub-region until the target object is recognized, or until the last sub-region is searched. Based on this, reliable identification and positioning of the target object in the target region can be achieved.

[0072] Based on the above, on the one hand, since the target object can be automatically identified, and a plurality of sub-regions are searched in turn, compared with the search control based on manual, it has higher efficiency and reliability. On the other hand, since the position information of the target object can be determined based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region, the effective positioning of the target object can be realized, and thus the problem that the reliability of target identification is relatively low in the prior art can be improved.

[0073] In the first aspect, it needs to be explained that the specific way of obtaining the first photoelectric image of the first sub-region in step S110 is not limited, and can be selected according to actual needs.

[0074] For example, in an alternative embodiment, in order to ensure that the obtained first photoelectric image can effectively reflect the first sub-region, the above step S110 can further include the following specific implementation contents:

[0075] Firstly, the center position coordinates of the photoelectric scanning center of the photoelectric pod of the unmanned aerial vehicle aiming at the first sub-region in the plurality of sub-regions of the target region can be determined (for example, it can be determined by scanning the first sub-region by the photoelectric pod), the center reference position coordinates of the first sub-region can be determined (for example, it can be determined in the process of dividing the target region into a plurality of sub-regions), and the center position coordinates and the center reference position coordinates are compared and analyzed;

[0076] Secondly, if the coordinate value of the length direction (such as the x-axis in the x, y, z coordinate system) in the center position coordinates is greater than the coordinate value of the length direction in the center reference position coordinates, the azimuth angle of the photoelectric pod is increased, and if the coordinate value of the length direction in the center position coordinates is less than the coordinate value of the length direction in the center reference position coordinates, the azimuth angle of the photoelectric pod is decreased;

[0077] And if the coordinate value of the width direction (such as the y-axis in the x, y, z coordinate system) in the center position coordinates is greater than the coordinate value of the width direction in the center reference position coordinates, the pitch angle of the photoelectric pod is increased, and if the coordinate value of the width direction in the center position coordinates is less than the coordinate value of the width direction in the center reference position coordinates, the pitch angle of the photoelectric pod is decreased;

[0078] In addition, if the length direction coordinate value of the center position coordinate is equal to the length direction coordinate value of the center reference position coordinate, and the width direction coordinate value of the center position coordinate is equal to the width direction coordinate value of the center reference position coordinate, the image acquisition is performed on the first sub-region by the photoelectric pod to obtain a first photoelectric image, that is, when the center position coordinate of the photoelectric scanning center of the first sub-region is equal to the center reference position coordinate, the first sub-region is scanned again to form a corresponding first photoelectric image.

[0079] It can be understood that, in order to effectively scan the plurality of sub-regions to obtain reliable photoelectric images, the sub-regions can also be divided based on the following manner:

[0080] Firstly, the size of the scanning field of view unit of the photoelectric pod can be calculated, wherein the length of the field of view: length= ; the width of the field of view: width= ;

[0081] H is the relative field height of the photoelectric pod (such as the height in the z-axis direction of the x, y, z coordinate system), is the scanning pitch angle of the photoelectric pod, is the scanning azimuth angle of the photoelectric pod, is the horizontal scanning view angle of the photoelectric pod, is the vertical scanning view angle of the photoelectric pod;

[0082] Secondly, the target region is divided into m rows and n columns of photoelectric image scanning unit arrays R={{R 11 , R 12 , R 13 ……R 1n}, {R 21 , R 22 , R 23 ……R 2n}, …… {R m1 , R m2 , R m3 ……R mn}} with the scanning field of view unit of the photoelectric pod as the minimum unit, and the photoelectric image scanning unit array R maps the actual region S={{S 11 , S 12 , S 13 ……S 1n}, {S 21 , S 22 , S 23 ……S 2n}, …… {S i1 , S i2 , S i3 ……Sin}, the corresponding set of reference points (i.e. points corresponding to the central reference position coordinates) is {O 11 , O 12 , O 13 ... O 1n}, {O 21 , O 22 , O 23 ... O 2n},... {O m1 , O m2 , O m3 ... O mn}.

[0083] In the second aspect, it needs to be explained that the specific manner of determining the position information of the target object is not limited and can be selected according to actual requirements.

[0084] For example, in an alternative embodiment, in order to ensure that the determined position information has high precision, the above step S130 can further include steps S131, S132 and S133, and the specific contents of each step are as follows.

[0085] Step S131: If the first object recognition result indicates that there is a target object in the first photoelectric image, the image position coordinates of the target object in the first photoelectric image are determined, and the target distance between the photoelectric pod and the center of the first sub-region is determined.

[0086] In the embodiment of the application, if the first object recognition result indicates that there is a target object in the first photoelectric image, the image position coordinates of the target object in the first photoelectric image are determined (for example, the corresponding coordinates can be determined in the foregoing step S120), and the target distance between the photoelectric pod and the center of the first sub-region is determined. The target distance can be obtained by laser ranging of the photoelectric pod.

[0087] Step S132: mapping the image position coordinates based on the projection relationship between the first photoelectric image and the first sub-region to form the region position coordinates of the target object in the first sub-region.

[0088] In the embodiment of the application, after obtaining the image position coordinates, the image position coordinates can be mapped based on the projection relationship between the first photoelectric image and the first sub-region to form the region position coordinates of the target object in the first sub-region. That is, the coordinates in the image are projected to the actual coordinates.

[0089] Step S133, determining the position information of the target object based on the target distance and the region position coordinate.

[0090] In the embodiments of the present application, after the target distance and the region position coordinate are obtained, the position information of the target object can be determined based on the target distance and the region position coordinate.

[0091] It can be understood that, in the step S132, the specific manner of forming the region position coordinate of the target object in the first sub-region is not limited, for example, in an alternative embodiment, in order to be able to perform reliable mapping, so that the accuracy of the formed region position coordinate is higher, the above-mentioned step S132 can further include steps S132a, S132b, S132c, S132d and S132e, and the specific contents of each step are as follows.

[0092] Step S132a, obtaining the length and width of the first sub-region to obtain a region length and a region width.

[0093] In the embodiments of the present application, the length and width of the first sub-region can be obtained to obtain a region length and a region width.

[0094] Step S132b, obtaining the scanning field of view unit size of the photoelectric pod to obtain a field of view length and a field of view width.

[0095] In the embodiments of the present application, the scanning field of view unit size of the photoelectric pod can also be obtained to obtain a field of view length and a field of view width.

[0096] Step S132c, determining a first projection relationship between the first photoelectric image and the first sub-region in the length direction based on the region length and the field of view length.

[0097] In the embodiments of the present application, after the region length and the field of view length are obtained, the first projection relationship between the first photoelectric image and the first sub-region in the length direction can be determined based on the region length and the field of view length. For example, the first projection relationship can be the ratio between the region length and the field of view length.

[0098] Step S132d, determining a second projection relationship between the first photoelectric image and the first sub-region in the width direction based on the region width and the field of view width.

[0099] In the embodiments of the present application, after the region width and the field width are obtained, the second projection relationship between the first photoelectric image and the first sub-region in the width direction can be determined based on the region width and the field width. For example, the second projection relationship can be the ratio between the region width and the field width.

[0100] In the embodiments of the present application, after the first projection relationship and the second projection relationship are obtained, the length direction coordinate and the width direction coordinate in the image position coordinate can be mapped respectively based on the first projection relationship and the second projection relationship, to form the region position coordinate of the target object in the first sub-region. For example, the value corresponding to the first projection relationship can be multiplied by the length direction coordinate in the image position coordinate, to obtain the length direction coordinate of the region position coordinate of the target object in the first sub-region. In addition, the value corresponding to the second projection relationship can be multiplied by the width direction coordinate in the image position coordinate, to obtain the width direction coordinate of the region position coordinate of the target object in the first sub-region.

[0101] In the embodiments of the present application, after the first projection relationship and the second projection relationship are obtained, the length direction coordinate and the width direction coordinate in the image position coordinate can be mapped respectively based on the first projection relationship and the second projection relationship, to form the region position coordinate of the target object in the first sub-region. For example, the value corresponding to the first projection relationship can be multiplied by the length direction coordinate in the image position coordinate, to obtain the length direction coordinate of the region position coordinate of the target object in the first sub-region. In addition, the value corresponding to the second projection relationship can be multiplied by the width direction coordinate in the image position coordinate, to obtain the width direction coordinate of the region position coordinate of the target object in the first sub-region.

[0102] It can be understood that, in the above step S132b, the specific manner of obtaining the scanning field unit size of the photoelectric pod is not limited, for example, in an alternative embodiment, in order to accurately calculate the field length and the field width, the above step S132b can further include the following specific implementation contents:

[0103] Firstly, the pitch angle, the azimuth angle, the horizontal scanning view angle and the vertical scanning view angle of the photoelectric pod can be obtained;

[0104] Secondly, the relative field height of the photoelectric pod relative to the first sub-region can be obtained;

[0105] Then, the field length of the photoelectric pod can be determined based on the pitch angle, the azimuth angle, the horizontal scanning view angle and the relative field height, as described above.

[0106] Finally, the field width of the photoelectric pod can be determined based on the pitch angle, the azimuth angle, the vertical scanning view angle and the relative field height, as described above.

[0107] It can be understood that in the step S132, the specific manner of determining the position information of the target object based on the target distance and the region position coordinate is not limited, in order to be able to reliably determine the position information of the target object, the step S132 can further include steps S132a, S132b, S132c and S132d, and the specific contents are as follows.

[0108] In step S132a, based on the target distance, the first relative position coordinate between the region center of the first sub-region and the electro-optical pod is determined.

[0109] In the embodiment of the present application, based on the target distance, the first relative position coordinate between the region center of the first sub-region and the electro-optical pod can be determined, for example, .

[0110] In step S132b, based on the region position coordinate and the first relative position coordinate, the second relative position coordinate between the target object and the electro-optical pod is determined.

[0111] In the embodiment of the present application, after obtaining the first relative position coordinate, based on the region position coordinate and the first relative position coordinate, the second relative position coordinate between the target object and the electro-optical pod can be determined, for example, , wherein, , , the region position coordinate is (P xs , P ys ).

[0112] In step S132c, the second relative position coordinate belonging to the rectangular coordinate system is converted into the longitude and latitude coordinate system to obtain the third relative position coordinate.

[0113] In the embodiment of the present application, after obtaining the second relative position coordinate, the second relative position coordinate belonging to the rectangular coordinate system can be converted into the longitude and latitude coordinate system to obtain the third relative position coordinate, for example, is converted into , wherein lon is the longitude and lat is the latitude.

[0114] In step S132d, based on the current longitude and latitude coordinate of the electro-optical pod and the third relative position coordinate, the longitude and latitude coordinate of the target object is determined as the position information of the target object.

[0115] In the embodiments of the present application, after the third relative position coordinate is obtained, the latitude and longitude coordinates of the target object can be determined based on the current latitude and longitude coordinates of the electro-optical pod and the third relative position coordinate, and used as the position information of the target object.

[0116] It can be understood that, in the step S132a, the specific manner of determining the first relative position coordinate between the region center of the first sub-region and the electro-optical pod is not limited, for example, in an alternative embodiment, in order to include the reliability of the determined first relative position coordinate, the above-mentioned step S132a can further include the following contents:

[0117] Firstly, the current pitch angle and azimuth angle of the electro-optical pod can be obtained, for example, the scanning pitch angle of the electro-optical pod can be represented by , and the scanning azimuth angle of the electro-optical pod can be represented by .

[0118] Secondly, the relative position coordinate value in the length direction can be determined based on the target distance, the sine value of the pitch angle and the cosine value of the azimuth angle, and the relative position coordinate value in the width direction can be determined based on the target distance, the sine value of the pitch angle and the sine value of the azimuth angle, such as the relative position coordinate value in the length direction , the relative position coordinate value in the width direction , s is the target distance, is the pitch angle of the electro-optical pod, is the azimuth angle of the electro-optical pod.

[0119] Then, the first relative position coordinate between the region center of the first sub-region and the electro-optical pod can be determined based on the relative position coordinate value in the length direction and the relative position coordinate value in the width direction, that is, .

[0120] On the basis of the above-mentioned embodiments, in order to facilitate the understanding of the above-mentioned target recognition method based on the electro-optical pod, the following specific application example is further provided.

[0121] As shown in Figure 3 , a recognition system is provided, which can include an electro-optical pod, a flight control computer, a ground control station, a task search module, an image forwarding module and a target recognition module.

[0122] Among them, the flight control computer and the ground control station are cross-linked through link communication, the ground control personnel selects the search area information on the ground control station, uploads it to the flight control computer through remote control data, and the flight control computer downloads the recognized target information to the ground control station through the link.

[0123] Wherein, the flight control computer and the task search module are in communication crosslinking, the flight control computer sends the position information of the unmanned aerial vehicle to the task search module, and receives the task feedback information of the task search module.

[0124] Wherein, the task search module and the photoelectric pod are in communication crosslinking, the task search module sends the pitch angle and the azimuth angle instruction to the photoelectric pod, and receives the state information feedback of the photoelectric pod.

[0125] Wherein, the photoelectric pod receives the pitch angle and the azimuth angle instruction of the task search module, can change the scanning field of view, and sends the original image collected by the photoelectric pod to the image forwarding module after collecting the area image.

[0126] Wherein, the image forwarding module sends the original image to the target identification module and the flight control computer respectively.

[0127] Wherein, the target identification module sends the image position information of the target object to the task search module, and sends the image after identification processing to the flight control computer.

[0128] Wherein, the working principle of the system is that: the ground control personnel inputs the photoelectric search area on the ground control station, sends the data to the flight control computer through the remote control link, the task search module reads the flight control information, calculates the scanning field of view size of a single image, divides the search area into the scanning array unit of the photoelectric pod with the scanning field of view size as the unit, controls the pitch and azimuth angle of the photoelectric pod, and traverses to collect the scanning array unit of the divided area; the image forwarding module sends the image collected by the photoelectric pod to the flight control computer in one way and to the target identification module in the other way, the target identification module uses the salient object detection (SOD) algorithm to detect the pre-trained target in the image, and sends the target object after the marking processing to the flight control computer; according to the pixel position of the target object in the image, the relative position and the absolute position coordinates of the target object in the actual area are calculated by the projection relationship; finally, the position coordinates of the target object can be sent to the flight control computer and the ground control station. Specifically:

[0129] S1: collect the target vehicle data set to be searched, and train the algorithm of the identification model;

[0130] S2: select the rectangular area for search and identification on the ground control station, and send the boundary coordinates A (x1, y1), B (x2, y2), C (x3, y3), and D (x4, y4) to the flight control computer;

[0131] S3: the task search module reads the position information and the search area information of the unmanned aerial vehicle: (1) the longitude Lon of the unmanned aerial vehicle; (2) the latitude Lat; (3) the relative field height H; (4) the pitch angle of the photoelectric pod ; (5) the azimuth angle of the photoelectric pod (6) horizontal scanning view angle of the photoelectric pod (7) vertical scanning view angle of the photoelectric pod (8) length L of the search area; (9) width W of the search area

[0132] S4: calculate the size of the scanning field of view unit of the photoelectric pod

[0133] Field of view length: length= ;

[0134] Field of view width: width= ;

[0135] wherein, is the pitch angle of the photoelectric pod, is the azimuth angle of the photoelectric pod, is the horizontal scanning view angle of the photoelectric pod, is the vertical scanning view angle of the photoelectric pod

[0136] S5: divide the target area into the scanning field of view unit of the photoelectric pod as the minimum unit to form an m-row n-column photoelectric image scanning unit array R={{R 11 , R 12 , R 13 ……R 1n}, {R 21 , R 22 , R 23 ……R 2n}, ……{R m1 , R m2 , R m3 ……R mn}}, the photoelectric image scanning unit array R maps the actual area S={{S 11 , S 12 , S 13 ……S 1n}, {S 21 , S 22 , S 23 ……S 2n}, ……{S i1 , S i2 , S i3 ……S in}}, and the corresponding reference point (i.e. the point corresponding to the center reference position coordinate) set is {{O 11 , O 12 , O 13 ……O 1n}, {O 21 , O 22 , O 23 ……O 2n}, ……{Om1 O m2 O m3 ...O mn}},in, (round down) (round down)

[0137] S6: The task search module controls the pitch and azimuth angles of the electro-optical pod and collects data from segmented region S11 (e.g., ...). Figure 4 Chinese R 11 The image of the corresponding region is obtained based on the photoelectric scanning center O0(x0, y0) and the region center reference point O. ij (x) i y j Calculate the control angle of the photoelectric pod; if x0 > x i As the azimuth increases, if x0 <x i The azimuth angle decreases; if y0 > y j As the pitch angle increases, if y0 <y j The pitch angle decreases;

[0138] S7: The electro-optical pod sends images to the image forwarding module via the network. The image forwarding module uses ffmepg and mediamtx software to forward the images to the target recognition module and the flight control computer.

[0139] S8: The target recognition module uses the trained Satisfactory Target Detection (SOD) algorithm to detect and recognize the segmented region of the image, setting parameter thresholds. If a target is detected, the target vehicle is marked on the image and sent to the flight control computer. The flight control computer forwards the data to the ground command and control station for display and processing via a link. If no target vehicle is detected, S15 is executed to search the next region.

[0140] S9: The target recognition module reads the target vehicle in the image set R. ij coordinates P in R (P) xr P yr The photoelectric pod obtains the distance s from the center of the scanned area through laser ranging;

[0141] S10: Calculate the target vehicle's position in the actual area S based on the projection relationship. ij coordinates P S (P) xs P ys );

[0142] P xs =l / length×P xr P ys =w / width×P yr ;

[0143] Wherein, l and w are length and width of a scanned partition region respectively;

[0144] S11: Calculate the relative position coordinates of the center of the partition region to the unmanned aerial vehicle (optoelectronic pod) :

[0145] , ;

[0146] S12: Calculate the relative position coordinates of the target vehicle ;

[0147] , ;

[0148] S13: Rectangular coordinate is converted into longitude and latitude coordinate ;

[0149] S14: According to the current longitude and latitude of the unmanned aerial vehicle, the longitude and latitude position information of the target vehicle is calculated and sent to the flight control computer.

[0150] S15: The task search module controls the optoelectronic pod to scan and collect the next region, and repeats steps S2 to S14 to traverse the search region S and complete the region search.

[0151] In combination Figure 5 , the application embodiment also provides a target identification device based on an optoelectronic pod applicable to the above unmanned aerial vehicle. Wherein, the target identification device based on the optoelectronic pod includes an optoelectronic image acquisition module, a first object identification module, a position information determination module and a second object identification module.

[0152] The optoelectronic image acquisition module is used to acquire a first optoelectronic image obtained by the optoelectronic pod of the unmanned aerial vehicle for image collection on a first sub-region in a plurality of sub-regions of a target region, wherein the first sub-region refers to a sub-region currently searched by the optoelectronic pod in the plurality of sub-regions. In the application embodiment, the optoelectronic image acquisition module can be used to execute step S110 as shown in the drawing, and the related content of the optoelectronic image acquisition module can be referred to the description of step S110 in the foregoing. Figure 2

[0153] The first object identification module is used to identify the target object of the first optoelectronic image to obtain a first object identification result. In the application embodiment, the first object identification module can be used to execute step S120 as shown in the drawing, and the related content of the first object identification module can be referred to the description of step S120 in the foregoing. Figure 2

[0154] ​​The position information determining module is configured to, if the first object recognition result indicates that there is a target object in the first photoelectric image, determine position information of the target object based on a position coordinate of the target object in the first photoelectric image and a distance between the photoelectric pod and the first sub-region. In the embodiments of the present application, the position information determining module can be configured to execute the step S130 shown in FIG. 1. Figure 2 The step S130 shown in FIG. 1, and the related content of the position information determining module can be referred to the foregoing description of the step S130.

[0155] The second object recognition module is configured to, if the first object recognition result indicates that there is no target object in the first photoelectric image, acquire a second photoelectric image obtained by the photoelectric pod performing image collection on a second sub-region in the plurality of sub-regions, and perform target object recognition on the second photoelectric image to obtain a second object recognition result, wherein the second sub-region refers to a searched sub-region in sequence. In the embodiments of the present application, the second object recognition module can be configured to execute the step S140 shown in FIG. 1. Figure 2 The step S140 shown in FIG. 1, and the related content of the second object recognition module can be referred to the foregoing description of the step S140.

[0156] In the embodiments of the present application, corresponding to the photoelectric pod based target recognition method applied to the unmanned aerial vehicle, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. The computer program is configured to execute each step of the photoelectric pod based target recognition method when running.

[0157] The steps executed by the computer program when running are not described herein again, and can be referred to the foregoing explanation and description of the photoelectric pod based target recognition method.

[0158] To sum up, the target identification method and device based on the photoelectric pod, the unmanned aerial vehicle and the medium provided in the application are provided, first, the first photoelectric image of the first sub-region is acquired; second, the target object identification is performed on the first photoelectric image to obtain the first object identification result; then, if the first object identification result represents that there is a target object in the first photoelectric image, the position information of the target object is determined based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region; finally, if the first object identification result represents that there is no target object in the first photoelectric image, the second photoelectric image of the second sub-region is acquired, and the target object identification is performed on the second photoelectric image to obtain the second object identification result. Based on the above content, on the one hand, since the target object can be automatically identified, and a plurality of sub-regions are searched in turn, compared with the search control based on manual operation, the efficiency and reliability are higher. On the other hand, since the position information of the target object can be determined based on the position coordinates of the target object in the first photoelectric image and the distance between the photoelectric pod and the first sub-region, the effective positioning of the target object can be realized, and therefore, the problem that the reliability of target identification is relatively low in the prior art can be improved.

[0159] In several embodiments provided in the application, it should be understood that the disclosed apparatus and method can also be implemented by other manners. The apparatus and method embodiments described above are only illustrative, for example, the flowchart and block diagram in the drawings show the possible implementation architecture, function and operation of the apparatus, method and computer program product according to the embodiments of the application. In this regard, each block in the flowchart or block diagram can represent a module, program segment or part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementation manners, the functions annotated in the blocks can also occur in different order from that annotated in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can also be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0160] In addition, the function modules in each embodiment of the application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0161] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or in the form of a part of the prior art or part of the technical solutions of the present application. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media. It should be noted that in this paper, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0162] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for target recognition based on an optoelectronic pod, characterized in that, The method comprises the following steps: acquiring a first photoelectric image of a target area acquired by a photoelectric pod of a UAV, the target area comprising a first sub-area of a plurality of sub-areas, the first sub-area being a sub-area currently searched by the photoelectric pod; performing target object recognition on the first photoelectric image to obtain a first object recognition result; if the first object recognition result indicates that there is a target object in the first photoelectric image, determining position information of the target object based on a position coordinate of the target object in the first photoelectric image and a distance between the photoelectric pod and a center of the first sub-area; mapping the image position coordinate based on a projection relationship between the first photoelectric image and the first sub-area to form a region position coordinate of the target object in the first sub-area; and determining the position information of the target object based on the target distance and the region position coordinate; if the first object recognition result indicates that there is no target object in the first photoelectric image, acquiring a second photoelectric image of the target area acquired by the photoelectric pod, the target area comprising a second sub-area of the plurality of sub-areas, the second sub-area being a sub-area searched subsequently; and performing target object recognition on the second photoelectric image to obtain a second object recognition result.

2. The optoelectronic pod-based target identification method of claim 1, wherein, The step of mapping the image position coordinate based on a projection relationship between the first photoelectric image and the first sub-area to form a region position coordinate of the target object in the first sub-area comprises the following steps: acquiring a length and a width of the first sub-area to obtain a region length and a region width; acquiring a scanning field of view unit size of the photoelectric pod to obtain a field of view length and a field of view width; determining a first projection relationship in a length direction between the first photoelectric image and the first sub-area based on the region length and the field of view length; determining a second projection relationship in a width direction between the first photoelectric image and the first sub-area based on the region width and the field of view width; mapping a coordinate in the length direction and a coordinate in the width direction in the image position coordinate based on the first projection relationship and the second projection relationship to form the region position coordinate of the target object in the first sub-area.

3. The optoelectronic pod-based target identification method of claim 2, wherein, The step of acquiring a scanning field of view unit size of the photoelectric pod to obtain a field of view length and a field of view width comprises the following steps: acquiring a pitch angle, an azimuth angle, a horizontal scanning view angle and a vertical scanning view angle of the photoelectric pod; acquiring a relative field height of the photoelectric pod relative to the first sub-area; determining a field of view length of the photoelectric pod based on the pitch angle, the azimuth angle, the horizontal scanning view angle and the relative field height; and Determine a field of view width of the electro-optical pod based on the pitch angle, the azimuth angle, the vertical scanning view angle, and the relative field height. 4.The optoelectronic pod based target identification method of claim 1, wherein, The step of determining the position information of the target object based on the target distance and the area position coordinates comprises: Determine a first relative position coordinate between the area center of the first sub-area and the electro-optical pod based on the target distance; Determine a second relative position coordinate between the target object and the electro-optical pod based on the area position coordinates and the first relative position coordinate; Convert the second relative position coordinate belonging to the rectangular coordinate system into a longitude-latitude coordinate system to obtain a third relative position coordinate; Determine the longitude-latitude coordinate of the target object based on the current longitude-latitude coordinate of the electro-optical pod and the third relative position coordinate, and take the longitude-latitude coordinate as the position information of the target object.

5. The optoelectronic pod-based target identification method of claim 4, wherein, The step of determining the first relative position coordinate between the area center of the first sub-area and the electro-optical pod based on the target distance comprises: Obtain the current pitch angle and azimuth angle of the electro-optical pod; Determine a relative position coordinate value in the length direction based on the target distance, the sine value of the pitch angle, and the cosine value of the azimuth angle, and determine a relative position coordinate value in the width direction based on the target distance, the sine value of the pitch angle, and the sine value of the azimuth angle; Determine the first relative position coordinate between the area center of the first sub-area and the electro-optical pod based on the relative position coordinate value in the length direction and the relative position coordinate value in the width direction.

6. The optoelectronic pod based target recognition method according to any one of claims 1-5, characterized in that, The step of obtaining the first electro-optical image collected by the electro-optical pod of the unmanned aerial vehicle for the first sub-area in the target area comprising a plurality of sub-areas comprises: Determine the center position coordinate of the electro-optical scanning center of the electro-optical pod of the unmanned aerial vehicle for the first sub-area in the target area comprising a plurality of sub-areas, determine the center reference position coordinate of the first sub-area, and compare and analyze the center position coordinate and the center reference position coordinate; If the coordinate value in the length direction of the center position coordinate is greater than the coordinate value in the length direction of the center reference position coordinate, increase the azimuth angle of the electro-optical pod, and if the coordinate value in the length direction of the center position coordinate is less than the coordinate value in the length direction of the center reference position coordinate, decrease the azimuth angle of the electro-optical pod; If the coordinate value in the width direction of the center position coordinate is greater than the coordinate value in the width direction of the center reference position coordinate, increase the pitch angle of the electro-optical pod, and if the coordinate value in the width direction of the center position coordinate is less than the coordinate value in the width direction of the center reference position coordinate, decrease the pitch angle of the electro-optical pod; If the coordinate value in the length direction of the center position coordinate is equal to the coordinate value in the length direction of the center reference position coordinate, and the coordinate value in the width direction of the center position coordinate is equal to the coordinate value in the width direction of the center reference position coordinate, collect the first electro-optical image by the electro-optical pod for the first sub-area.

7. An opto-electronic pod based target recognition apparatus, characterized in that, Comprise: An optoelectronic image acquisition module is configured to acquire a first optoelectronic image obtained by an optoelectronic pod of a UAV when the optoelectronic pod collects images of a first sub-region of a target region, wherein the first sub-region refers to a sub-region currently searched by the optoelectronic pod. A first object identification module is configured to identify a target object in the first optoelectronic image to obtain a first object identification result. A position information determination module is configured to determine position information of the target object based on a position coordinate of the target object in the first optoelectronic image and a distance between the optoelectronic pod and the first sub-region if the first object identification result indicates that the target object exists in the first optoelectronic image, including: determining an image position coordinate of the target object in the first optoelectronic image and a target distance between the optoelectronic pod and a region center of the first sub-region if the first object identification result indicates that the target object exists in the first optoelectronic image; mapping the image position coordinate based on a projection relationship between the first optoelectronic image and the first sub-region to form a region position coordinate of the target object in the first sub-region; and determining the position information of the target object based on the target distance and the region position coordinate. A second object identification module is configured to acquire a second optoelectronic image obtained by the optoelectronic pod when the optoelectronic pod collects images of a second sub-region of the target region if the first object identification result indicates that the target object does not exist in the first optoelectronic image, and identify a target object in the second optoelectronic image to obtain a second object identification result, wherein the second sub-region refers to a sub-region searched later.

8. A drone, characterized in that, The memory is configured to store a computer program. The processor connected with the memory is configured to execute the computer program stored in the memory to implement the target identification method based on the optoelectronic pod according to any one of claims 1-6. The computer readable storage medium stores a computer program, and the computer program is configured to execute the target identification method based on the optoelectronic pod according to any one of claims 1-6 when running.

9. A computer-readable storage medium, characterized in that, ​

Images