Area monitoring method and device, electronic equipment and storage medium

Abstract

The application provides a region monitoring method and device, electronic equipment and storage medium, and the region monitoring comprises the following steps: acquiring at least two first detection images collected by a camera in the flight process of a UAV; performing target object detection on the at least two first detection images to obtain a target object detection result; and when the target object detection result indicates that a target object is detected from the first detection image, generating first alarm information. The application can simultaneously monitor indoor and outdoor monitoring objects by the monitoring mode of the UAV carrying the camera, can improve the frequency of the patrol, in addition, the collected images are detected, the outdoor monitoring objects can also be better monitored, and relevant managers can be prompted in time, and the safety is improved.

Description

Technical Field

[0001] This application relates to the field of remote monitoring and management of equipment, and more particularly to a regional monitoring method, device, electronic device, and storage medium. Background Technology

[0002] Food security is one of the most pressing global issues today. Only with sufficient food can a nation achieve stable and healthy development in industry, defense, services, science, and culture. Food security is not only an economic and livelihood issue, but also a matter of development and security. Furthermore, the development of modern information technology necessitates intelligent monitoring of grain warehouses and storage facilities.

[0003] Currently, the main way to monitor grain warehouses or grain piles is through manual inspections, supplemented by devices such as alarms.

[0004] However, due to the low frequency of manual inspections and the fact that alarms can only be installed indoors, they cannot effectively monitor grain piles placed outdoors. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for regional monitoring to solve or alleviate technical problems in the prior art.

[0006] The technical solution adopted in this invention is as follows:

[0007] In a first aspect, embodiments of this application provide a regional monitoring method, which includes:

[0008] Acquire at least two first detection images captured by a camera during the flight of the UAV; perform target object detection on the at least two first detection images to obtain target object detection results; when the target object detection results indicate that a target object has been detected in the first detection images, generate a first alarm message.

[0009] Optionally, in one embodiment, the area monitoring method further includes: controlling the drone to perform at least one flight according to preset monitoring points and monitoring routes, and acquiring images of the monitoring points through a camera during the flight of the drone along the monitoring route to obtain at least two second detection images; for each second detection image, calculating the volume of the object stored at the monitoring point based on the second detection image to obtain corresponding calculation data; obtaining volume change data of the object stored at the monitoring point by comparing the calculation data; and issuing a second alarm message when the volume change data meets a preset second alarm condition.

[0010] Optionally, in one embodiment, the step of performing target object detection on the at least two first detection images to obtain target object detection results includes: locating the target object in each of the first detection images and extracting the target object image based on the located target object; extracting key features from each of the target object images based on preset key points, and obtaining the target object detection result by comparing the key features.

[0011] Optionally, in one embodiment, the area monitoring method further includes: performing angle calibration on at least two images captured by the camera based on pre-acquired standard images; and performing data enhancement on the angle-calibrated images to obtain a first detection image.

[0012] Optionally, in one embodiment, the area monitoring method further includes: scaling the pixel values ​​of pixels in the image acquired by the camera proportionally to the range of 0-1, and determining the image with scaled pixel values ​​as the first detection image.

[0013] Optionally, in one embodiment, the area monitoring method further includes: acquiring video data collected by a camera during the flight of the drone, and uploading the video data to a user terminal, wherein the video data is used for display by the user terminal;

[0014] In response to voice information sent by the user, the voice information is played through the drone.

[0015] Secondly, embodiments of this application provide an area monitoring device, comprising: an acquisition module for acquiring at least two first detection images captured by a camera during the flight of a drone; a detection module for performing target object detection on the at least two first detection images to obtain a target object detection result; and an alarm module for generating first alarm information when the target object detection result indicates that a target object has been detected in the first detection images.

[0016] Thirdly, embodiments of this application provide an electronic device having a computer program stored thereon, which, when executed by a processor, implements the area monitoring method as described in any one of the first aspects of the embodiments.

[0017] The system includes a processor, a communication interface, a memory, and a communication bus. The processor, memory, and communication interface communicate with each other via the communication bus. The memory stores at least one executable instruction, which causes the processor to perform the operation corresponding to the area monitoring method described in any of the first aspects of the embodiments.

[0018] Fourthly, embodiments of this application provide a computer storage medium storing a computer program that, when executed by a processor, implements the area monitoring method as described in any one of the first aspects of the embodiments.

[0019] Fifthly, embodiments of this application provide a computer program product including computer instructions that instruct a computing device to perform operations corresponding to the area monitoring method as described in any one of the first aspects of the embodiments.

[0020] As can be seen from the above scheme, monitoring using drones equipped with cameras can simultaneously monitor both indoor and outdoor objects, increasing the frequency of patrols. In addition, by detecting the collected images, outdoor objects can also be effectively monitored, and timely alerts can be issued to relevant management personnel, thus improving security. Attached Figure Description

[0021] Figure 1 This is a flowchart of a regional monitoring method according to an embodiment of this application;

[0022] Figure 2 This is a flowchart of a regional monitoring method according to another embodiment of this application;

[0023] Figure 3 This is a flowchart of a target object detection method according to an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of a regional monitoring device according to an embodiment of this application;

[0025] Figure 5 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0026] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0027] Area monitoring methods

[0028] Figure 1 This is a flowchart of a regional monitoring method according to an embodiment of this application, such as... Figure 1 As shown, the monitoring method for this area includes the following steps:

[0029] Step 101: Acquire at least two first detection images captured by the camera during the flight of the UAV.

[0030] To achieve area monitoring, at least two initial detection images are first acquired by the camera during the drone's flight.

[0031] For example, drones can acquire images by carrying dual-optical payload pods.

[0032] Step 102: Perform target object detection on at least two first detection images to obtain target object detection results.

[0033] After acquiring the first detection image using a camera mounted on a drone, target object detection is performed on all the acquired first detection images to obtain the target object detection results.

[0034] For example, the target object could be a source of fire, a suspicious person, or a suspicious vehicle, or any other object that needs to be monitored.

[0035] Step 103: When the target object detection result indicates that a target object has been detected in the first detection image, generate the first alarm information.

[0036] After obtaining the target object detection result, the target object detection result is analyzed. If the detection result indicates that there is a target object in the first detection image, a first alarm message is generated. The first alarm message is used to prompt relevant management personnel to handle the target object.

[0037] For example, if the target object detection result indicates that there is a fire source in the first detection image, then a first alarm message indicating a fire will be issued.

[0038] In this embodiment of the application, monitoring is carried out by using a drone equipped with a camera, which can monitor both indoor and outdoor objects simultaneously, thereby increasing the frequency of patrols. In addition, by detecting the collected images, outdoor objects can also be monitored effectively, and timely alerts can be issued to relevant management personnel, thus improving security.

[0039] In one possible implementation, video data collected by a camera during the drone's flight can be acquired and uploaded to the user's device. Then, in response to voice messages sent by the user's device, the voice messages can be played through the drone.

[0040] Real-time monitoring can be achieved through drones. First, video data collected by the camera during the drone's flight is acquired. This video data is then displayed on the user's device. After the video data is uploaded to the user's device, the user can play it on a screen or other device. If relevant management personnel discover any abnormalities, such as staff members performing incorrect or dangerous operations, they can send voice messages from the relevant management personnel to the drone through the user's device, and the drone will then play them.

[0041] In this embodiment, by acquiring video data collected by a camera during the drone's flight, managers can conduct real-time monitoring. Furthermore, by playing managers' voice messages through the drone, timely reminders can be given to staff, and the drone-based playback ensures accurate delivery of voice messages to the appropriate staff, improving communication efficiency.

[0042] Figure 2 This is a flowchart illustrating a regional monitoring method according to another embodiment of this application. Figure 2 As shown, the monitoring method for this area includes the following steps:

[0043] Step 201: Based on the preset monitoring points and monitoring routes, control the UAV to fly at least once, and collect images of the monitoring points through the camera during the flight of the UAV along the monitoring route to obtain at least two second detection images.

[0044] To calculate the volume of objects stored at designated monitoring points, the monitoring points to be photographed are first preset. A detection route can be set based on these preset points, and the drone can be controlled to fly at least once. Alternatively, the drone can be manually controlled to take pictures at each monitoring point. Then, a second detection image is acquired as the drone flies along the monitoring route.

[0045] Step 202: For each second detection image, calculate the volume of the object stored at the monitoring point based on the second detection image to obtain the corresponding calculation data.

[0046] After obtaining the second detection image, the volume is calculated based on at least two second detection images taken at the same detection point to obtain the volume calculation data of the object stored at the monitoring point.

[0047] Step 203: By comparing the various measured data, obtain the volume change data of the objects stored at the monitoring points.

[0048] Based on the volume measurement data of objects stored at at least two monitoring points, obtain the volume change data of the objects stored at that monitoring point.

[0049] Step 204: When the volume change data meets the preset second alarm condition, issue the second alarm message.

[0050] When the volume change data of an object stored at a certain monitoring point meets the preset second alarm condition, a second alarm message is issued. The second alarm message is used to prompt management personnel to pay attention to objects with abnormal volume change data.

[0051] For example, if the object stored at a monitoring point is a grain pile, the second alarm information can be generated based on the operation information of the grain pile. The operation of the grain pile can be that 200KG is removed from the warehouse. Then, the volume of the removed grain is calculated based on 200KG and the density of the grain pile. The second alarm information is issued when the volume change data does not match the volume of the removed grain.

[0052] In this embodiment of the application, the drone collects images of the monitoring points through its camera while flying along the monitoring route. The volume of the objects stored at the monitoring points is calculated through the images, thereby understanding the volume change data of the stored objects. This method is not limited to indoor or outdoor scenes and can be applied to monitoring various environments.

[0053] Figure 3 This is a flowchart illustrating a target object detection method according to an embodiment of this application. Figure 3 As shown, the target object detection method includes the following steps:

[0054] Step 301: Locate the target object in each first detection image and extract the target object image based on the located target object.

[0055] After obtaining the first detection object, the target object is first located in each first detection image to locate the location of the target object in the image, and then the target object image is extracted based on the located target object.

[0056] For example, if the target object is a person, the person's position in the image is located from the first detection image, and then the image of the person is extracted.

[0057] Step 302: Extract key features from each target object image according to the preset key points, and obtain the target object detection results by comparing each key feature.

[0058] After obtaining the image of the target object, key features are extracted from the target object according to preset key points. Then, the obtained key features are compared to obtain the target object detection results.

[0059] For example, if the target object is a person, the preset key points could be the eyes, nose, and mouth of the face. Images of the eyes, nose, and mouth are extracted from the person's image and compared with pre-stored data in the database to confirm whether the person in the image has been entered into the database. If the result is negative, the person is identified as a suspicious individual and included in the target object detection result. Preset key points could also be the knees and feet of a person. Images of the knees and feet are extracted from each person's image and compared to determine whether the knees are bent and whether the feet are moving, thus determining whether the person is in motion and whether they are a living person, and this is also included in the target object detection result.

[0060] In this embodiment of the application, by extracting key features from the target object image, comparison can be made to determine the detection result of the target object, thus ensuring the security of the monitored area.

[0061] In one possible implementation, during the detection of the target object, at least two images captured by the camera can be angle-calibrated based on a pre-acquired standard image, and then data augmentation can be performed on the angle-calibrated image to obtain the first detection image.

[0062] Based on the image angle of the pre-acquired standard image, the angle of the image captured by the camera is used as the standard angle, and then the angle-corrected image is subjected to data augmentation. The data augmentation methods include, but are not limited to, data flipping, data rotation, image scaling, image cropping, image translation, and adding noise.

[0063] In this embodiment, angle calibration of the camera-captured images using standard images can further improve volume detection accuracy. Data augmentation of the angle-calibrated images can enrich the sample size and further enhance volume detection accuracy.

[0064] In one possible implementation, during the detection of the target object, the pixel values ​​of the pixels in the image captured by the camera can be scaled proportionally to the range of 0-1, and the image with scaled pixel values ​​can be determined as the first detection image.

[0065] Images captured by a camera can be normalized by scaling the pixel values ​​of the pixels in the image proportionally to the range of 0-1.

[0066] For example, the pixel values ​​of a typical image are concentrated between 0 and 255. By scaling proportionally, the pixel values ​​can be controlled to the range of 0 to 1.

[0067] In this embodiment of the application, by scaling the pixel values ​​of the pixels in the image captured by the camera proportionally within the range of 0-1, it is possible to facilitate the corresponding measurement model to perform volume measurement and improve the measurement efficiency of the model.

[0068] Area monitoring device

[0069] Figure 4 This is a schematic diagram of a regional monitoring device according to an embodiment of this application, as shown below. Figure 4 As shown, the area monitoring device 400 includes: an acquisition module 401, a detection module 402, and an alarm module 403.

[0070] The acquisition module 401 is used to acquire at least two first detection images captured by the camera during the flight of the UAV.

[0071] To achieve area monitoring, at least two first detection images captured by the camera during the flight of the UAV are first obtained through the acquisition module 401.

[0072] For example, drones can acquire images by carrying dual-optical payload pods.

[0073] The detection module 402 is used to perform target object detection on at least two first detection images and obtain target object detection results.

[0074] After the acquisition module 401 acquires the first detection image captured by the camera on the UAV, the detection module 402 performs target object detection on all the acquired first detection images to obtain the target object detection result.

[0075] For example, the target object could be a source of fire, a suspicious person, or a suspicious vehicle, or any other object that needs to be monitored.

[0076] The alarm module 403 is used to generate a first alarm message when the target object detection result indicates that a target object has been detected in the first detection image.

[0077] After the detection module 402 obtains the target object detection result, the alarm module 403 analyzes the target object detection result. If the detection result indicates that there is a target object in the first detection image, a first alarm message is generated. The first alarm message is used to prompt relevant management personnel to handle the target object.

[0078] For example, if the target object detection result indicates that there is a fire source in the first detection image, then a first alarm message indicating a fire will be issued.

[0079] In this embodiment of the application, monitoring is carried out by using a drone equipped with a camera, which can monitor both indoor and outdoor objects simultaneously, thereby increasing the frequency of patrols. In addition, by detecting the collected images, outdoor objects can also be monitored effectively, and timely alerts can be issued to relevant management personnel, thus improving security.

[0080] electronic devices

[0081] Figure 5 This is a schematic diagram of an electronic device according to one embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device. Figure 5 As shown, the electronic device 500 may include: a processor 501, a communications interface 502, a memory 503, and a communications bus 504. Wherein:

[0082] The processor 501, communication interface 502, and memory 503 communicate with each other through the communication bus 504.

[0083] Communication interface 502 is used for communication with other electronic devices or servers.

[0084] The processor 501 is used to execute program 505, which can specifically execute the relevant steps in any of the aforementioned method embodiments.

[0085] Specifically, program 505 may include program code that includes computer operation instructions.

[0086] The processor 501 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0087] Memory 503 is used to store program 505. Memory 503 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0088] Specifically, program 505 can be used to cause processor 501 to execute any of the methods in the foregoing embodiments.

[0089] The specific implementation of each step in procedure 505 can be found in the corresponding steps and units described in the aforementioned region detection method embodiments, and will not be repeated here. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned method embodiments, and will not be repeated here.

[0090] The electronic device in this application embodiment uses a camera mounted on a drone for monitoring, which can simultaneously monitor indoor and outdoor objects, increasing the frequency of patrols. In addition, by detecting the collected images, it can also achieve good monitoring results for outdoor objects and can promptly issue prompts to relevant management personnel, thereby improving security.

[0091] Computer storage media

[0092] This application also provides a computer-readable storage medium storing instructions for causing a machine to perform any of the methods described in the plurality of method embodiments herein. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0093] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of this application.

[0094] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0095] Computer program products

[0096] This application also provides a computer program product, including computer instructions that instruct a computing device to perform any corresponding operation in the above-described plurality of method embodiments.

[0097] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0098] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0099] It should be noted that not all steps and modules in the above processes and system structure diagrams are necessary; some steps or modules can be omitted as needed. The execution order of each step is not fixed and can be adjusted as required. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0100] In the above embodiments, the hardware modules can be implemented mechanically or electrically. For example, a hardware module may include permanent dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operation. The hardware module may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operation. The specific implementation method (mechanical, dedicated permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0101] The present application has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present application is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art will know that more embodiments of the present application can be obtained by combining the code review methods in the different embodiments above. These embodiments are also within the protection scope of the present application.

Claims

1. A method for regional monitoring, characterized in that, include: Acquire at least two first detection images captured by the camera during the flight of the drone; Target object detection is performed on at least two first detection images to obtain target object detection results; this step specifically includes: locating target objects in each of the first detection images and extracting target object images based on the located target objects; extracting knee and foot images from each of the target object images based on preset knee and foot as key points, and determining whether the person is moving by comparing whether the knee is bent and whether the foot is moving in each image, and using this as a basis to determine whether the person is a living body, and using this as the target object detection result; When the target object detection result indicates that a target object has been detected in the first detection image, a first alarm message is generated; The method further includes: controlling the drone to perform at least one flight according to preset monitoring points and monitoring routes, and acquiring images of the monitoring points through a camera during the flight of the drone along the monitoring route to obtain at least two second detection images; for each second detection image, calculating the volume of the object stored at the monitoring point based on the second detection image to obtain corresponding calculation data; obtaining volume change data of the object stored at the monitoring point by comparing the calculation data; and issuing a second alarm message when the volume change data meets a preset second alarm condition. Specifically, by comparing various measured data, the volume change data of the objects stored at the monitoring point is obtained; when the volume change data meets the preset second alarm condition, a second alarm message is issued, including: obtaining operation information of the objects stored at the monitoring point, the operation information including at least the weight of the objects entering and leaving the warehouse; calculating the expected volume change based on the operation information and the physical properties of the objects; comparing the actual volume change data measured by the UAV image with the expected volume change; when the two do not match, a second alarm message is generated. The method further includes: performing angle calibration on at least two images captured by the camera based on a pre-acquired standard image; performing data enhancement on the angle-calibrated image to obtain a first detection image; In addition, the system acquires video data collected by the camera during the drone's flight and uploads the video data to the user terminal, wherein the video data is used for display on the user terminal; and in response to voice information sent by the user terminal, the system plays the voice information through the drone.

2. The method according to claim 1, characterized in that, The step of performing target object detection on the at least two first detection images to obtain target object detection results includes: The target object is located in each of the first detected images, and the target object image is extracted based on the located target object; Based on preset key points, key features are extracted from each of the target object images, and the target object detection results are obtained by comparing each key feature.

3. The method according to claim 2, characterized in that, The method further includes: The pixel values ​​of the pixels in the image acquired by the camera are scaled proportionally to the range of 0-1, and the image with scaled pixel values ​​is determined as the first detection image.

4. A regional monitoring device, characterized in that, include: The acquisition module is used to acquire at least two first detection images captured by the camera during the flight of the UAV; The detection module is used to perform target object detection on at least two first detection images to obtain target object detection results; specifically, it is used to locate the target object in each of the first detection images and extract the target object image based on the located target object; based on preset knees and feet as key points, it extracts knee and foot images from each of the target object images respectively, and determines whether the person is moving by comparing whether the knee is bent and whether the foot is moving in each image, and uses this as a basis to determine whether the person is a living body, and uses this as the target object detection result; An alarm module is configured to generate a first alarm message when the target object detection result indicates that a target object has been detected in the first detection image. The area monitoring device is also used to control the drone to fly at least once according to preset monitoring points and monitoring routes, and to acquire images of the monitoring points through a camera during the drone's flight along the monitoring route, obtaining at least two second detection images; for each second detection image, to calculate the volume of the object stored at the monitoring point based on the second detection image, and to obtain corresponding calculation data; to obtain volume change data of the object stored at the monitoring point by comparing the calculation data; and to issue a second alarm message when the volume change data meets a preset second alarm condition. Specifically, by comparing various measured data, the volume change data of the objects stored at the monitoring point is obtained; when the volume change data meets the preset second alarm condition, a second alarm message is issued, including: obtaining operation information of the objects stored at the monitoring point, the operation information including at least the weight of the objects entering and leaving the warehouse; calculating the expected volume change based on the operation information and the physical properties of the objects; comparing the actual volume change data measured by the UAV image with the expected volume change; when the two do not match, a second alarm message is generated. The acquisition module is also used to perform angle calibration on at least two images captured by the camera based on pre-acquired standard images; and to perform data enhancement on the angle-calibrated images to obtain a first detection image. The area monitoring device is also used to acquire video data collected by the camera during the flight of the drone and upload the video data to the user terminal, wherein the video data is used for display on the user terminal; in response to voice information sent by the user terminal, the voice information is played through the drone.

5. An electronic device, comprising: The processor, communication interface, memory, and communication bus communicate with each other through the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the area monitoring method as described in any one of claims 1-3.

6. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the area monitoring method as described in any one of claims 1-3.

7. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to the area monitoring method as described in any one of claims 1-3.

Images