Target identification system based on unmanned aerial vehicle

Abstract

The invention discloses a target recognition system based on an unmanned aerial vehicle, and belongs to the technical field of image recognition. The target recognition system based on the unmanned aerial vehicle comprises a camera control module, an unmanned aerial vehicle camera, a target recognition module and a target tracking module which are connected with an unmanned aerial vehicle system, and further comprises a directional sound box, the directional sound box and the unmanned aerial vehicle camera are installed on the camera control module, and the directional sound box and the unmanned aerial vehicle camera face the same direction. The unmanned aerial vehicle camera comprises a high-definition camera and an infrared camera, and each module has the following functions: the high-definition camera is used for acquiring visible light high-definition images of a target and an environment, the infrared camera is used for acquiring infrared images of the target and the environment, and the directional sound box is used for sending audios to the target; the method has the advantages that the method can be used for recognizing the target under the conditions that light around the target is insufficient, an obstacle exists between the target and the unmanned aerial vehicle, and effective visible light image information of the target is insufficient.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a target recognition system based on an unmanned aerial vehicle. Background Art

[0002] UAV is an unmanned aerial equipment that can plan and fly autonomously, and can complete various tasks such as environmental information perception and target detection in an autonomous or human-intervention manner. It is mainly used to perform dangerous tasks that are not suitable for manned ships. Once equipped with advanced control systems, sensor systems and communication systems, it can be widely used in various civil fields, such as traffic monitoring, resource exploration and mapping, environmental monitoring and public security system monitoring. After properly installing other sensors or even weapon systems, it can perform a variety of war and non-war military tasks, such as counter-terrorism, target monitoring, military intelligence reconnaissance, electronic information warfare and communication relay.

[0003] Automatic target detection and recognition technology is one of the main technical challenges faced by the development of UAV systems. The existing recognition methods include: 1. Feature extraction method: Identify the target by extracting the features of the target, such as color, texture, shape, etc. 2. Template matching method: Match the template of the target with the input image one by one, and find the most similar part as the target. 3. Visual bag of words method: Divide the image or video into multiple small blocks, extract features from each small block, and use these features to build a bag of words model. By comparing the bag of words models in different images or videos, the target can be identified. 4. Deep learning method: Use deep neural networks to learn a large amount of data, automatically extract the features of the target and classify them to achieve target recognition. 5. Behavior detection method: Identify the target by analyzing the behavior of the target, such as movement trajectory, posture changes, etc.

[0004] Different recognition methods have their own advantages and disadvantages. Therefore, in actual application, different methods are often combined to improve recognition accuracy. However, in any case, this type of recognition method is mainly used on the basis of being able to obtain target image information more clearly. In scenarios where less target image information is obtained, there is a lack of effective recognition methods in the existing technology. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that the UAV system in the prior art lacks an effective recognition method in scenes where target image information is missing, and to propose a target recognition system based on a UAV.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: The target recognition system based on the UAV includes a camera control module connected to the UAV system, a UAV camera, a target recognition module, a target tracking module, and a directional speaker. The directional speaker and the UAV camera are installed on the camera control module. The directional speaker and the UAV camera have the same direction. The UAV camera includes a high-definition camera and an infrared camera, and each module includes the following functions: High-definition camera, used to obtain visible light high-definition images of the target and environment; Infrared camera, used to obtain infrared images of the target and environment; Directional speakers, used to send audio to a target; The camera control module is used to adjust the directional speaker and the drone camera toward the target; The target recognition module pre-processes the target image; The target tracking module tracks the target in the image in real time and records the movement trajectory to form a log; The shooting mode is that when the high-definition camera cannot obtain visible light high-definition images of the target and the environment, the infrared camera is enabled to obtain infrared images of the target and the environment, and audio is sent to the target through the directional speaker to obtain the image of the target after hearing the audio.

[0007] Furthermore, it also includes a meteorological detection module connected to the drone system, which is connected to the camera control module and the directional speaker. The weather information of the drone and the target environment is obtained through the meteorological detection module to adjust the direction and output power of the directional speaker.

[0008] Furthermore, it also includes a cloud database connected to the drone system, the cloud database including a meteorological data module, an image data module, a motion data module, and an audio data module; A meteorological data module is used to match the data detected by the meteorological detection module; Image data module, storing target contour image information for initial recognition; Action data module, which stores target action image information for secondary recognition; The audio data module stores the audio information that needs to be transmitted to the target through the directional speaker.

[0009] Furthermore, it also includes a manual control terminal connected to the drone system, the manual control terminal includes an operation module and a display module, and the display module is connected to the drone camera.

[0010] Furthermore, the manual control terminal also includes a monitoring module, the monitoring module is connected to an audio device, and the audio device is connected to a directional speaker.

[0011] Furthermore, the directional speaker is an ultrasonic directional speaker.

[0012] Furthermore, the infrared camera is a thermal imaging dual-spectrum camera with a specification of 25mm or more.

[0013] Furthermore, it also includes a communication module, which is connected to the drone system, the manual control terminal, and the cloud database.

[0014] Compared with the prior art, the present invention provides a target recognition system based on drones, which has the following beneficial effects: The target recognition system of the present invention is based on an unmanned aerial vehicle. When in use, the unmanned aerial vehicle carries the system to a designated location, obtains infrared images of the target and the environment through an infrared camera, then sends audio to the target through a directional speaker, obtains an image of the target after hearing the audio, and a target recognition module compares the images of the target before and after hearing the audio to determine whether the target is a correct target and completes the recognition. Compared with the prior art, the system can be used to identify the target when there is insufficient light around the target, there are obstacles between the target and the unmanned aerial vehicle, and there is insufficient effective visible light image information of the target.

[0015] Other advantages, objectives and features of the present invention will be described in part in the following description; and in part, will be apparent to those skilled in the art based on an examination of the following; or, may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the installation effect of the directional speaker of the present invention; Figure 2 It is the overall structural block diagram of the present invention.

[0017] In the figure: 1. Camera control module; 2. Directional speaker; 3. Drone camera; 4. Monitoring module; 5. Operation module; 6. Display module; 7. Weather data module; 8. Image data module; 9. Action data module; 10. Audio data module; 11. Target recognition module; 12. Target tracking module; 13. Weather detection module. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] Reference Figure 1-2The target recognition system based on the UAV of the present invention comprises a camera control module 1 connected to the UAV system, a UAV camera 3, a target recognition module 11, and a target tracking module 12. The system also comprises a directional speaker 2, which is an ultrasonic directional speaker. The ultrasonic directional speaker has the characteristics of good directional effect and long propagation distance, which reduces the impact on non-target objects, thereby improving the recognition accuracy; The directional speaker 2 and the drone camera 3 are installed on the camera control module 1. The directional speaker 2 and the drone camera 3 have the same orientation. When the drone camera 3 faces the target object to shoot and obtain image information, the directional speaker 2 also faces the target object to send specific audio to the target object and perform external stimulation on the target object. The drone camera 3 includes a high-definition camera and an infrared camera. The high-definition camera is mainly used for daily and normal environment shooting and target recognition, while the infrared camera is mainly used for obtaining infrared thermal imaging images of the target object at night, in dim light, with barriers, etc., when the high-definition camera cannot obtain the target visible light image information, so as to obtain the position of the target object and obtain the position data for the next step of sending specific audio to the directional speaker 2; The modules in this system include the following functions: High-definition camera, used to obtain high-definition visible light images of the target and environment, used in conventional environments, and can be used to capture visible light image information of the target; Infrared cameras are used to obtain infrared images of targets and environments. They are mainly used in environments where visible light image information that can be used for identification cannot be obtained. In addition to night, haze, dust, and obstructions, infrared cameras can also directly identify target objects with temperature differences through color temperature information, such as dummies and real people, animal specimens, and live animals. Directional speaker 2 is used to send audio to the target. Compared with several commonly used recognition methods in the prior art, this system sends audio to the target object through the directional speaker 2, performs external stimulation on the target object, and judges whether the target object is the correct target or what kind of target it is according to the reaction of the target object, thereby improving the recognition rate; The camera control module 1 is used to adjust the directional speaker 2 and the drone camera 3 toward the target. The camera control module 1 can be equipped with a rotating base, which can be rotated in the horizontal and vertical directions, so that the directional speaker 2 and the drone camera 3 can be directed toward the target object from different angles; The target recognition module 11 pre-processes the target image, mainly used for real-time recording of target image information and comparing it with the image information in the database, and determining the target's motion information based on the comparison information; Specifically, for example, after the target hears the audio information sent by the directional speaker 2, will it produce a predetermined action response, such as sending a certain language or a certain dialect voice warning to the target group or individual, and observing whether the target group or individual will turn their heads, raise their heads, stand up, turn around, suddenly run, etc. If so, such a group or individual can be identified as a target using such language or dialect. Of course, the target object is not necessarily a human being, but can also be an animal, such as a wolf and a dog, which have a high similarity and cannot be identified only by infrared images. Specifically, the barking of a dog or a wolf is used to observe the reaction of the target object; The target tracking module 12 tracks the target in the image in real time and records the movement trajectory to form a log. Since the infrared image cannot obtain the appearance image information of the target object, after the target is identified, a signal can be sent to the drone system through the target tracking module 12 according to the identification result to track the identified target object to avoid target loss and the need for re-identification; The shooting mode is that when the high-definition camera cannot obtain visible light high-definition images of the target and the environment, the infrared camera is enabled to obtain infrared images of the target and the environment, and audio is sent to the target through the directional speaker 2 to obtain the image of the target after hearing the audio. The images before and after the target reacts when hearing the audio are compared to determine and identify the target; The system further includes a weather detection module 13 connected to the drone system, the weather detection module 13 is connected to the camera control module 1 and the directional speaker 2, and the weather information of the drone and the target environment is obtained through the weather detection module 13, which is used to adjust the direction and output power of the directional speaker 2. Since sound waves are mechanical waves, their propagation characteristics are significantly affected by the external environment. Therefore, after the weather conditions of the drone and the target environment are detected by the weather detection module 13, the direction and output power of the directional speaker 2 are adjusted according to the weather conditions to ensure that the target location can hear the audio emitted by the directional speaker 2, thereby generating a reaction; The system also includes a cloud database connected to the drone system, the cloud database includes a meteorological data module 7, an image data module 8, a motion data module 9, and an audio data module 10. The cloud database is mainly used to store existing data, data temporarily uploaded from the main body, and data updated from the Internet, for the drone system to call to the target recognition module 11 for target image information comparison and thus recognition; The meteorological data module 7 stores normal and abnormal historical meteorological data of different regions and seasons, such as sunrise and sunset times, surface and air temperature change trends, air humidity, haze levels, etc., for matching the data detected by the meteorological detection module 13; The image data module 8 stores the target contour image information, including the target's sitting, lying, lying, standing, height, body shape and other information. When the infrared camera captures the infrared image of the target object, the same type of data is retrieved from the image data module 8 and compared with the acquired image information for judgment and initial recognition; Action data module 9 stores target action image information, including target walking, running, jumping, raising head, turning head, turning around and other action information. After the target hears the audio information and makes corresponding actions, the UAV system retrieves the same type of data, compares it with the acquired action information, makes a judgment, and performs secondary recognition; Of course, it is also possible that the target does not take corresponding actions after hearing the audio information. In this case, you can change the type of audio and identify it again; The audio data module 10 stores audio information to be transmitted to the target through the directional speaker 2, mainly storing different types of audio, such as warnings, shock, shouts, etc., and can also be simulated animal sounds, such as dog barking, wolf howling, or other animal sounds, and can also be other types of sounds, such as car horns, engine sounds, or other sounds that may attract attention to humans and animals; The system also includes a manual control terminal connected to the drone system, the manual control terminal includes an operation module 5 and a display module 6, the display module 6 is connected to the drone camera 3, and the manual control terminal is used for local operation. The staff can directly view the target image information screen captured by the drone camera 3 through the display module 6, and input commands to the drone system through the operation module 5 as needed to adjust the drone, the camera control module 1, the directional speaker 2, and the drone camera 3; When the system is controlled by the operating module 5, as needed, if a group target needs to be identified, the directional speaker 2 sends audio toward the group target; if an individual target needs to be identified, the directional speaker 2 only sends audio to the individual target to avoid affecting other target objects; at the same time, if there is an obstruction between the individual target and the directional speaker 2, or other targets, the drone and camera control module 1 is controlled by the operating module 5 to adjust the position of the drone, and adjust the orientation of the directional speaker 2 and the drone camera 3, so that the directional speaker 2 and the drone camera 3 can shoot the target and send audio information from an unobstructed position; The manual control end also includes a monitoring module 4, which is connected to an audio device, which is connected to the directional speaker 2. The staff wears the audio device, and the manual control end controls the audio device to simulate the audio emitted by the directional speaker 2, and according to the meteorological data detected by the meteorological detection module 13, the impact on the emitted audio makes the audio device emit an audio effect heard by the target object. The staff judges whether the target object can hear the audio emitted by the directional speaker 2, as well as the sound volume, pitch, timbre change and other sound information. According to this feedback, the emission position and power of the directional speaker 2 are adjusted to ensure the audio emission effect and further improve the recognition rate. The infrared camera is a thermal imaging dual-spectrum camera with a specification of 25mm or more. The effective shooting distance of a thermal imaging dual-spectrum camera with a specification of 25mm or more is more than 100 meters. The noise of the drone during flight is generally 60-80 decibels. For the recognition task with people as the target, when the drone shoots the target at a distance of more than 100 meters, the human ear cannot directly hear the 80-decibel ambient sound at a distance of 100 meters, which is convenient for the drone to perform hidden flight. Hidden flight allows the drone to stay away from the target object and prevent the target object from discovering the drone, thereby reducing external stimulation to the target object, making the action response made by the target object after hearing the audio emitted by the directional speaker 2 more realistic, further improving the recognition rate; The system also includes a communication module, which is connected to the drone system, the manual control terminal, and the cloud database. The communication module is wireless communication and can use 5G signal transmission equipment to transmit data between the system and the drone system and the manual control terminal and the cloud database.

[0020] Working principle: When in use, according to the noise level of the drone itself during flight, the drone is controlled to fly to a position where the target (mainly human target) cannot hear the drone noise, the drone camera 3 is adjusted to obtain the infrared image of the target, and the target recognition module 11 compares and recognizes the captured infrared image, determines the target type, and performs initial recognition; When the initial recognition similarity is too low, the target is changed; When the initial recognition similarity reaches the predetermined requirement, a specific audio is sent through the directional speaker 2 item list to observe the target reaction; If the target produces a reaction action corresponding to the audio type, such as being frightened when a scary audio is played, and looking for the source of the sound when a shout or greeting sound is played, then the target can be judged as an object that can understand the language of the audio content. Of course, more specific recognition operations can be performed, such as directly hearing the target's name and observing whether the target has a reaction and how intense the reaction is, so as to determine whether the target is the object of that name; If the target does not produce a reaction action corresponding to the audio type, the audio type is changed and recognized again.

[0021] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0022] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0023] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A target recognition system based on a drone, comprising a camera control module (1) connected to a drone system, a drone camera (3), a target recognition module (11), and a target tracking module (12), characterized in that: It also includes a directional speaker (2), the directional speaker (2) and the drone camera (3) are mounted on the camera control module (1), the directional speaker (2) and the drone camera (3) are facing the same direction, the drone camera (3) includes a high-definition camera and an infrared camera, and each module includes the following functions: High-definition camera, used to obtain visible light high-definition images of the target and environment; Infrared camera, used to obtain infrared images of the target and environment; A directional speaker (2) for sending audio to a target; A camera control module (1) is used to adjust the directional speaker (2) and the drone camera (3) toward a target; A target recognition module (11) pre-processes the target image; The target tracking module (12) tracks the target in the image in real time and records the movement trajectory to form a log; The shooting mode is that when the high-definition camera cannot obtain visible light high-definition images of the target and the environment, the infrared camera is enabled to obtain infrared images of the target and the environment, and audio is sent to the target through the directional speaker (2) to obtain the image of the target after hearing the audio.

2. The target recognition system based on drone according to claim 1, characterized in that: It also includes a meteorological detection module (13) connected to the drone system, the meteorological detection module (13) being connected to the camera control module (1) and the directional speaker (2), and obtaining weather information of the environment where the drone and the target are located through the meteorological detection module (13), which is used to adjust the direction and output power of the directional speaker (2).

3. The target recognition system based on drone according to claim 2 is characterized in that: It also includes a cloud database connected to the drone system, the cloud database including a meteorological data module (7), an image data module (8), a motion data module (9), and an audio data module (10); A meteorological data module (7), used for matching data detected by the meteorological detection module (13); An image data module (8) stores target contour image information for initial recognition; An action data module (9) stores target action image information for secondary recognition; The audio data module (10) stores audio information that needs to be transmitted to a target through the directional speaker (2).

4. The target recognition system based on drone according to claim 1, characterized in that: It also comprises a manual control terminal connected to the drone system, the manual control terminal comprising an operation module (5) and a display module (6), and the display module (6) is connected to the drone camera (3).

5. The target recognition system based on drone according to claim 4, characterized in that: The manual control terminal further comprises a monitoring module (4), the monitoring module (4) being connected to an audio device, and the audio device being connected to a directional speaker (2).

6. The target recognition system based on drone according to claim 1, characterized in that: The directional speaker (2) is an ultrasonic directional speaker.

7. The target recognition system based on drone according to claim 1, characterized in that: The infrared camera is a thermal imaging dual-spectrum camera with a specification of 25mm or more.

8. The target recognition system based on drone according to claim 1, characterized in that: It also includes a communication module, which is connected to the drone system, the manual control terminal, and the cloud database.

Images