Unmanned aerial vehicle public security inspection method, system and device

Abstract

The invention belongs to the technical field of public security management, and relates to an unmanned aerial vehicle public security inspection method, system and device. In public security inspection, a traditional method is limited by the environment and is difficult to accurately extract figure targets and process dynamic distance relations among figures. A target inspection area image is collected through an unmanned aerial vehicle, a grid image is constructed by using a feature extraction method, and a character target is extracted by combining a character target feature image set; establishing a character vector field by means of a capsule neural network, analyzing a distance relationship, tracking the change of the distance relationship, and analyzing and processing through a long-short-term memory network to obtain a constant distance interval; and finally, judging whether the person is normal according to the distance relationship among the persons and prompting. According to the invention, people can be accurately identified, the distance relation is dynamically analyzed, abnormity is judged, and public security inspection efficiency and prevention and control capability are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of public security management, and relates to a method, system and device for unmanned aerial vehicle (UAV) public security patrol inspection. Background Art

[0002] Compared with traditional patrol vehicles, helicopters and other equipment, the procurement and maintenance costs of UAVs are relatively low, and the service life of UAVs is relatively long, and the operation costs are also low. In urban security patrol, UAVs can be combined with existing monitoring systems to reduce the investment in additional equipment.

[0003] In a complex environment and background, it is difficult for the traditional method of UAV patrol inspection to accurately extract and locate human targets from images.

[0004] In a dynamic environment, the relative positions of multiple moving people are constantly changing. The traditional method of analyzing static images cannot effectively handle this dynamic change and is prone to ignoring abnormal states such as human disturbances. In actual situations, it is necessary to be able to detect and report abnormal situations such as abnormal human gatherings or abnormal distances between people in a timely manner to avoid potential safety hazards. Summary of the Invention

[0005] To solve the problems in the background art, the present invention proposes a method, system and device for UAV public security patrol inspection.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows: On the one hand, the present invention provides a method for UAV public security patrol inspection, including: The UAV collects images of the target patrol inspection area, and extracts human targets from the images of the target patrol inspection area based on a feature extraction method; The capsule neural network is used to analyze the distance relationship between people in the images of the target patrol inspection area, and the long short-term memory network is used to analyze the distance relationship between any two human targets to obtain a constant distance interval between the corresponding two human targets; Based on the distance relationship between any human targets, it is judged whether the distance between people belongs to the normal range, and a prompt or non-prompt is given based on the distance between the corresponding two human targets.

[0007] Further, the specific method for extracting human targets from the images of the target patrol inspection area based on the feature extraction method is: The images of the target patrol inspection area are constructed into a first grid image, the side length of the grid is a preset pixel value, the feature gradient change relationship between each grid and its 8-neighborhood grids in the first grid image is extracted, and the feature gradient change relationship of each grid is extracted to generate a second grid image; Set a collection of pre-trained target feature images of people, where each target feature image of a person corresponds to a different action of the target person; For each target feature image of a person in the collection of target feature images of people, a window is generated; Set the window in the second grid image and displace it. Compare the similarity between the image in the window and the corresponding target feature image of the person in the window through the Euclidean distance, and set the corresponding image with a similarity higher than the similarity threshold in the window as the target person.

[0008] Further, the specific method for analyzing the distance relationship between people in the target inspection area image by using the capsule neural network is as follows: The capsule neural network establishes a vector field based on all the target people in the target inspection area image, and each vector is the distance between a certain target person and other target people; The capsule network tracks the change data of each vector in the vector field and records the distance relationship between all the target people in the target inspection area image.

[0009] Further, the specific method for analyzing the distance relationship between any two target people by using the long short-term memory network is as follows: The long short-term memory network receives the distance relationship and changes between any two target people analyzed by the capsule neural network; The long short-term memory network determines the calculated value of the forgetting gate for each distance relationship according to the probability of each distance relationship occurring; add each distance relationship not processed by the forgetting gate to the input gate; Based on the processing results of the forgetting gate and the input gate, calculate the unit state of each current distance relationship. The specific formula is: ; where is the unit state at the previous moment, is the output of the forgetting gate, is the output of the input gate, is the candidate memory generated by the current input; Generate the current output according to the current unit state and the output gate in each moment state. The current output contains the unit state at the previous moment; Based on all possible outputs of the unit state of each distance relationship, obtain the constant distance interval corresponding to the distance relationship.

[0010] On the other hand, the present invention provides a drone public security inspection system that executes the above drone public security inspection method, including: A data acquisition module that acquires target inspection area images through a drone; A feature extraction module that extracts human targets from the images of the target patrol area based on a feature extraction method; A data processing module that analyzes the distance relationship between humans in the images of the target patrol area using a capsule neural network, and analyzes the distance relationship between any two human targets through a long short-term memory network to obtain a constant distance interval between the corresponding two human targets; A prompt module that determines whether the distance between humans belongs to the normal range based on the distance relationship between any human targets, and gives a prompt or no prompt based on the distance between the corresponding two human targets.

[0011] On the other hand, the present invention provides a drone public security patrol device, including a memory and a processor. The above-mentioned drone public security patrol system is stored on the memory, and the processor calls the drone public security patrol system in the memory when working.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The present invention constructs a grid image, uses the feature gradient change relationship to enhance the salient features in the image, and combines the pre-trained collection of human target feature images to more accurately identify the humans in the image.

[0013] The present invention uses a capsule neural network to establish a vector field between humans, tracks the change data of these vectors, so as to realize real-time and dynamic analysis of the distance relationship between humans. And uses a long short-term memory network to perform serial analysis on the human distance relationship, determines whether it belongs to the normal range, and gives necessary prompts or warnings according to the analysis results. And a probability factor is added to the long short-term memory network to reduce the error rate of the activation function judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flowchart of the method of the present invention; Figure 2 is a system structure diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0016] As Figure 1 - Figure 2 shown, the technical solution adopted by the present invention is as follows: A drone public security patrol method includes: The drone collects images of the target inspection area and extracts human targets from the images of the target inspection area based on the feature extraction method; The capsule neural network is used to analyze the distance relationship between people in the images of the target inspection area. The long short-term memory network is used to analyze the distance relationship between any two human targets, and the constant distance interval between the corresponding two human targets is obtained; Based on the distance relationship between any human targets, it is judged whether the distance between people belongs to the normal range, and prompts or non-prompts are given based on the distance between the corresponding two human targets.

[0017] The drone collects images of the target inspection area and extracts human targets from the images of the target inspection area based on the feature extraction method. By collecting images of the target area, it provides basic data for subsequent human recognition and distance analysis. Extracting human targets from images is the first step in security inspection, which helps to monitor the personnel situation and can detect potential safety hazards or abnormal situations, such as identifying dangerous or missing people in public places.

[0018] The image of the target inspection area is constructed into a first grid image, the side length of the grid is a preset pixel value, the feature gradient change relationship between each grid in the first grid image and its 8-neighborhood grids is extracted, and the feature gradient change relationship of each grid is extracted to generate a second grid image. By gridifying the image and analyzing the feature gradient change relationship between grids, it can better capture the local features in the image, provide more refined information for the extraction of human targets, and help to accurately distinguish the features represented by people in complex scenarios.

[0019] A pre-trained collection of human target feature images is set, and each human target feature image corresponds to a different action of the human target. The pre-trained collection of human target feature images covers different actions, enabling the system to recognize people in various states, enhancing the diversity and accuracy of human target recognition, and avoiding missed detections or misjudgments caused by different human actions.

[0020] Each human target feature image in the collection of human target feature images generates a window.

[0021] Each human target feature image corresponds to the action state of a human target. This can achieve a detailed matching of human features at different positions and angles, improve the search and matching ability for human targets, and increase the flexibility of human recognition.

[0022] A corresponding window is generated for each person target feature image. The window is set in the second grid image and displaced. By comparing the similarity between the image in the window and the person target feature image corresponding to the window using the Euclidean distance, the image with a similarity higher than the similarity threshold in the window is set as the person target. Through the above operations and using the Euclidean distance comparison, the area with a high similarity to the person target feature can be accurately found, and the recognition result of this area is marked as the person target, ensuring that the extracted person target is more accurate and improving the accuracy of person recognition.

[0023] Use a capsule neural network to analyze the distance relationships of people in the target inspection area image. Analyzing the distance relationships of people helps to grasp the spatial distribution of person targets and the relative positions between people, providing an important basis for judging whether there are abnormal people. For example, if the distance between two people is affected by the sudden acceleration of one of them and the distance is shortened, these two people are prone to collision risks. Another example is that in a crowd, if someone decelerates and moves in the opposite direction away from the crowd, this person target is prone to risks such as getting lost. This can effectively detect potential risks in scenarios such as large-scale events or crowded streets.

[0024] The capsule neural network establishes a vector field based on all person targets in the target inspection area image. Each vector value is the distance between a certain person target and other person targets. The vector relationship between a certain person target and other person targets is the change relationship of the distance between this person target and other person targets, that is, the relationship changes such as the distance between this person target and other person targets remains unchanged, increases, or decreases. Through the vector field, the distance relationships between people can be clearly and intuitively represented. By using vectors, the distance relationship between any two target people can be locked, facilitating subsequent distance analysis and abnormal judgment. Quantifying the distance relationships of people into vectors provides structured data for subsequent calculations and analyses.

[0025] The capsule network tracks the change data of each vector in the vector field and records the distance relationships between all person targets in the target inspection area image. Tracking the change data of vectors can grasp the dynamic changes in the distances between people, which is very important for identifying the flow, aggregation, or dispersion of people and can timely detect possible abnormal situations, such as people suddenly gathering or suddenly separating.

[0026] Analyze the distance relationship between any two character targets through a long short-term memory network to obtain a constant distance interval between any two or more characters. Using the long short-term memory network can consider the characteristics of the distance relationship in the time series, not only focus on the current distance relationship, but also integrate historical information, which is more scientific for judging the long-term stability and normal range of the distance between any two personnel, and improves the accuracy of judging abnormal situations. The relationship between two character targets can be determined only by the distance between any two character targets, which is sufficient to meet the requirements of the inspection task and reduces the calculation amount in subsequent analysis.

[0027] The long short-term memory network receives the distance relationship and changes between the distances of any two character targets analyzed by the capsule neural network. Integrate the data from the capsule neural network to make the distance analysis more comprehensive, use the characteristics of the long short-term memory network to process time series information, and make full use of historical data for more in-depth analysis.

[0028] The long short-term memory network determines the calculated value of the forget gate for each distance relationship through the probability of each distance relationship occurring; add each distance relationship not processed by the forget gate to the input gate. The mechanisms of the forget gate and the input gate can filter and screen information, focus on important distance relationships, improve the efficiency and accuracy of analysis, and avoid interference from irrelevant information. And the forget gate can be more in line with the actual situation through the probability method, avoiding the situation where the activation function passes through in the process but does not conform to the actual situation.

[0029] Based on the processing results of the forget gate and the input gate, calculate the unit state of each current distance relationship. The specific formula is: ; where is the unit state of the previous moment, is the output of the forget gate, is the output of the input gate, is the candidate memory generated by the current input. By calculating the unit state with this formula, the information of different moments can be comprehensively considered to obtain a more representative and reliable distance relationship state, providing an accurate basis for the subsequent calculation of the constant distance interval.

[0030] Generate the current output according to the current unit state and the output gate in each moment state. The current output contains the unit state of the previous moment. The output containing the unit state of the previous moment can more comprehensively reflect the time series characteristics of the distance relationship, which helps to more accurately judge whether the distance relationship is normal.

[0031] Based on all possible outputs of the unit state for each distance relationship, a constant distance interval corresponding to the distance relationship is obtained. Obtaining the constant distance interval provides a clear quantitative standard for judging whether the distance between people is normal. When the actual distance exceeds this interval, it can be regarded as abnormal, which is convenient for the system to take corresponding measures such as making prompts.

[0032] Based on the distance relationship between any two person targets, judge whether the distance between people is within the normal range, and give a prompt or no prompt based on the distance between the corresponding two person targets. By judging whether the distance between people is normal, situations of people gathering or abnormal spacing can be discovered in time, and corresponding prompt or non-prompt operations can be performed, realizing automated public security patrol monitoring, improving the patrol efficiency, and enhancing the public security prevention and control ability. For example, potential conflict hazards or illegal gathering behaviors can be discovered in time in crowded areas.

[0033] On the other hand, the present invention provides a drone public security patrol system that executes the above-mentioned drone public security patrol method, including: A data acquisition module that acquires images of the target patrol area through a drone.

[0034] A feature extraction module that extracts person targets from the images of the target patrol area based on a feature extraction method.

[0035] A data processing module that uses a capsule neural network to analyze the distance relationship between people in the images of the target patrol area, and analyzes the distance relationship between any two person targets through a long short-term memory network to obtain a constant distance interval between the corresponding two person targets.

[0036] A prompt module that judges whether the distance between people is within the normal range based on the distance relationship between any two person targets, and gives a prompt or no prompt based on the distance between the corresponding two person targets.

[0037] On the other hand, the present invention provides a drone public security patrol device, including a memory and a processor. The above-mentioned drone public security patrol system is stored on the memory, and when the processor works, it calls the drone public security patrol system in the memory.

[0038] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A drone security inspection method, characterized in that: Included are: The drone collects images of the target inspection area, and extracts human targets from the images of the target inspection area based on a feature extraction method; The capsule neural network is used to analyze the distance relationship between people in the target inspection area image, and the distance relationship between any two human targets is analyzed through the long short-term memory network to obtain the constant distance interval between the two human targets. Based on the distance relationship between any character targets, it is determined whether the distance between the characters is within a normal range, and a prompt or no prompt is given based on the distance between the corresponding two character targets.

2. The method for public security inspection using an unmanned aerial vehicle according to claim 1, characterized in that: The specific method for extracting human targets from the target inspection area image based on the feature extraction method is: The target inspection area image is constructed as a first grid image, the side length of the grid is a preset pixel value, the characteristic gradient change relationship between each grid in the first grid image and its 8 neighboring grids is extracted, and the characteristic gradient change relationship of each grid is extracted to generate a second grid image; A collection of pre-trained character target feature images is set, each character target feature image corresponds to a different action of the character target; Each person target feature image in the person target feature image collection generates a window; The window is set in the second grid image and shifted, and the similarity between the image in the window and the human target feature image corresponding to the window is compared by Euclidean distance, and the corresponding image in the window whose image similarity is higher than the similarity threshold is set as the human target.

3. The method for public security inspection using an unmanned aerial vehicle according to claim 1, characterized in that: The specific method of using capsule neural network to analyze the distance relationship between people in the target inspection area image is: The capsule neural network establishes a vector field based on all the human targets in the target inspection area image, and each vector is the distance between a certain human target and other human targets; The capsule network tracks the change data of each vector in the vector field and records the distance relationship between all human targets in the target inspection area image.

4. The method for public security inspection using an unmanned aerial vehicle according to claim 1, characterized in that: The specific method for analyzing the distance relationship between any two character targets through the long short-term memory network is: The long short-term memory network receives the distance relationship and change between any two human targets analyzed by the capsule neural network; The long short-term memory network determines the calculated value of each distance relationship by the forget gate through the probability of each distance relationship appearing; each distance relationship not processed by the forget gate is added to the input gate; Based on the processing results of the forget gate and the input gate, the unit state of each current distance relationship is calculated. The specific formula is: ； in is the unit state at the previous moment, is the output of the forget gate, is the output of the input gate, is the candidate memory generated by the current input; At each moment, the current output is generated according to the current cell state and the output gate. The current output contains the cell state at the previous moment. Based on all possible outputs of the cell state of each distance relation, a constant distance interval corresponding to the distance relation is obtained.

5. A drone security inspection system, characterized in that: The method for performing the unmanned aerial vehicle security inspection described in claim 1 comprises: Data acquisition module, which collects images of the target inspection area through drones; A feature extraction module extracts human targets from the target inspection area image based on a feature extraction method; The data processing module uses capsule neural network to analyze the distance relationship between people in the target inspection area image, and uses long short-term memory network to analyze the distance relationship between any two human targets, and obtains the constant distance interval between the two human targets; The prompt module determines whether the distance between characters is within a normal range based on the distance relationship between any character targets, and gives a prompt or no prompt based on the distance between the corresponding two character targets.

6. A drone security inspection device, comprising a memory and a processor, characterized in that: The memory stores the UAV security inspection system according to claim 5, and the processor calls the UAV security inspection system in the memory when working.

Images