A method, system, electronic device and storage medium for detecting objects thrown from a high altitude

Abstract

The embodiment of the present invention provides a method, system, electronic device and storage medium for detecting objects dropped from a high altitude. The method includes: obtaining a target image sequence to be detected for objects dropped from a high altitude, and determining a detection frame of a target object in each frame of the target image sequence; when the motion of the detection frame of the target object meets the conditions for objects dropped from a high altitude, determining the target position information of the stop motion detection frame when the detection frame stops moving, a first timestamp, and a playback image sequence including the target frame image and a preset number of images before the target frame image; determining a target area within a predetermined range of each frame image in the playback image sequence based on the target position information, performing reverse behavior detection with the target frame image as the starting frame, and determining a second timestamp of the reverse stop motion detection frame; and determining the video segment between the second timestamp and the first timestamp as the detection result of the target image sequence. The embodiment of the present invention can improve the accuracy of detecting objects dropped from a high altitude.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, in particular to the technical fields of artificial intelligence, video processing, etc., and specifically to a method, system, electronic device and storage medium for detecting objects thrown from a high altitude. Background Art

[0002] With the development and construction of cities, more and more high-rise buildings are spread all over the city, and high-altitude object dropping incidents have followed. High-altitude object dropping incidents can cause huge social harm, and there are problems such as difficulty in tracing the source and obtaining evidence. At present, the high-altitude object dropping video surveillance system has been listed as one of the contents of urban construction in projects such as smart communities and smart cities.

[0003] In the related art, the following method is used to detect objects thrown from high places: video surveillance equipment is pre-installed facing the target high-rise building, and real-time monitoring and motion detection are performed on the target monitoring area. When the movement of an object is detected in the video screen, an alarm message is generated, and the video clip corresponding to the alarm message is manually reviewed to find the evidence information corresponding to the objects thrown from high places, so as to realize the detection of objects thrown from high places.

[0004] However, in the above-mentioned process of detecting objects thrown from high places, when flying birds, swaying leaves, moving people, etc. are detected in the video screen, false alarms of motion detection other than objects thrown from high places are easily generated, which affects the accuracy of detection of objects thrown from high places and requires manual secondary review and screening, consuming manpower, material resources and other resources. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method, system, electronic device and storage medium for detecting objects thrown from a high altitude, so as to improve the accuracy of detecting objects thrown from a high altitude. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting objects thrown from a high altitude, the method comprising:

[0007] Acquire a target image sequence to be subjected to high-altitude object throwing detection, wherein the target image sequence includes multiple frames of images arranged in time sequence;

[0008] For each frame image in the target image sequence, determining a detection frame corresponding to the target object in the frame image;

[0009] In the case where the motion of the detection frame of the target object meets the high-altitude throwing condition, determining, according to the detection frame of the target object, target position information of the stop motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence including a target frame image and a preset number of images before the target frame image; wherein the target frame image is a frame image corresponding to the stop motion detection frame;

[0010] Based on the target position information, determining a target area within a predetermined range of each frame image in the playback image sequence;

[0011] Based on the target area within the predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame to determine a second timestamp of a reverse stop motion detection frame, wherein the reverse stop motion detection frame is a detection frame when the target object starts to move;

[0012] The video segment between the second timestamp and the first timestamp is determined as the detection result of the target image sequence.

[0013] Optionally, the step of acquiring a target image sequence to be subjected to high-altitude object detection includes:

[0014] Obtain the video stream to be detected for high-altitude object throwing detection;

[0015] Performing frame extraction processing on the video stream to be detected to obtain a frame-extracted image;

[0016] The image after frame extraction is blurred to obtain a target image sequence.

[0017] Optionally, for each frame image in the target image sequence, determining a detection box corresponding to the target object in the frame image includes:

[0018] For each frame of the target image sequence, performing foreground extraction on the target object in the frame of the image to obtain a foreground area of ​​the target object;

[0019] The main body frame of the foreground area is determined as the detection frame of the target object.

[0020] Optionally, the method further includes:

[0021] Determine a motion speed of the detection frame of the target object of the current frame image based on the center position of the detection frame of the target object of the current frame image in the target image sequence and the center position of the detection frame of the target object of the previous frame image adjacent to the current frame image, wherein the motion speed includes a horizontal speed and a vertical speed;

[0022] When the longitudinal speed of the detection frame of the target object in the current frame image is greater than the lateral speed, and the longitudinal speed is greater than a first speed threshold, it is determined that the movement of the detection frame of the target object meets the high-altitude throwing condition.

[0023] Optionally, the determining, based on the detection frame of the target object, target position information of the stopped motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence including a target frame image and a preset number of images before the target frame image, comprises:

[0024] Tracking the detection frame of the target object and calculating the movement speed of the detection frame of the target object in adjacent frame images;

[0025] When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive frame images are both zero, the first frame image among the multiple frame images is determined as the target frame image, and the position information of the detection frame of the target object of the target frame image is determined as the target position information of the stopped motion detection frame when the detection frame stops moving, and the timestamp corresponding to the target frame image is determined as the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image is determined.

[0026] Optionally, the performing reverse behavior detection based on a target area within a predetermined range of each frame image in the playback image sequence and taking the target frame image as a starting frame to determine a second timestamp of the reverse stop motion detection frame includes:

[0027] Based on the target area within a predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame, and the movement speed of the detection frame of the target object in the adjacent frame images is calculated;

[0028] When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive reverse frame images are both less than the second speed threshold, the first reverse frame image among the multiple reverse frame images is determined as the starting frame image, and the timestamp of the starting frame image is determined as the second timestamp of the reverse stop motion detection frame.

[0029] Optionally, the method further includes:

[0030] At least one of the second timestamp and the first timestamp, the video segment between the second timestamp and the first timestamp, the start frame image, the target frame image, and the target key frame in the video segment between the second timestamp and the first timestamp is displayed.

[0031] In a second aspect, an embodiment of the present invention provides a high-altitude object throwing detection system, comprising:

[0032] An image acquisition module is used to acquire a target image sequence to be detected for high-altitude projectiles, wherein the target image sequence includes multiple frames of images arranged in time sequence;

[0033] A detection frame determination module, used to determine, for each frame image in the target image sequence, a detection frame corresponding to the target object in the frame image;

[0034] A behavior detection module is used to determine, based on the detection frame of the target object, target position information of the stop motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence including a target frame image and a preset number of images before the target frame image, when the motion of the detection frame of the target object meets the high-altitude throwing condition; wherein the target frame image is a frame image corresponding to the stop motion detection frame;

[0035] A region determination module, used for determining a target region within a predetermined range of each frame image in the playback image sequence based on the target position information;

[0036] A trajectory backtracking module is used to perform reverse behavior detection based on a target area within a predetermined range of each frame image in the playback image sequence, taking the target frame image as a starting frame, and determine a second timestamp of a reverse stop motion detection frame, wherein the reverse stop motion detection frame is a detection frame when the target object starts to move;

[0037] The result detection module is used to determine the video segment between the second timestamp and the first timestamp as the detection result of the target image sequence.

[0038] Optionally, the image acquisition module is specifically used to:

[0039] Obtain the video stream to be detected for high-altitude object throwing detection;

[0040] Performing frame extraction processing on the video stream to be detected to obtain a frame-extracted image;

[0041] The image after frame extraction is blurred to obtain a target image sequence.

[0042] Optionally, the detection frame determination module is specifically used to:

[0043] For each frame of the target image sequence, performing foreground extraction on the target object in the frame of the image to obtain a foreground area of ​​the target object;

[0044] The main body frame of the foreground area is determined as the detection frame of the target object.

[0045] Optionally, the system further includes:

[0046] a speed determination module, configured to determine a motion speed of a detection frame of a target object in a current frame image based on a center position of a detection frame of a target object in a current frame image in the target image sequence and a center position of a detection frame of a target object in a previous frame image adjacent to the current frame image, wherein the motion speed includes a horizontal speed and a vertical speed;

[0047] The high-altitude object dropping determination module is used to determine that the movement of the detection frame of the target object in the current frame image meets the high-altitude object dropping condition when the longitudinal speed of the detection frame of the target object is greater than the lateral speed and the longitudinal speed is greater than a first speed threshold.

[0048] Optionally, the behavior detection module is specifically used to:

[0049] Tracking the detection frame of the target object and calculating the movement speed of the detection frame of the target object in adjacent frame images;

[0050] When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive frame images are both zero, the first frame image among the multiple frame images is determined as the target frame image, and the position information of the detection frame of the target object of the target frame image is determined as the target position information of the stopped motion detection frame when the detection frame stops moving, and the timestamp corresponding to the target frame image is determined as the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image is determined.

[0051] Optionally, the trajectory backtracking module is specifically used to:

[0052] Based on the target area within a predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame, and the movement speed of the detection frame of the target object in the adjacent frame images is calculated;

[0053] When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive reverse frame images are both less than the second speed threshold, the first reverse frame image among the multiple reverse frame images is determined as the starting frame image, and the timestamp of the starting frame image is determined as the second timestamp of the reverse stop motion detection frame.

[0054] Optionally, the system further includes:

[0055] A result display module is used to display at least one of the second timestamp and the first timestamp, the video segment between the second timestamp and the first timestamp, the starting frame image, the target frame image, and the target key frame in the video segment between the second timestamp and the first timestamp.

[0056] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0057] Memory, used to store computer programs;

[0058] The processor is used to implement any method step described in the first aspect when executing a program stored in the memory.

[0059] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0060] An embodiment of the present invention further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any one of the method steps described in the first aspect above.

[0061] Beneficial effects of the embodiments of the present invention:

[0062] A method, system, electronic device and storage medium for detecting objects dropped from a high altitude provided by an embodiment of the present invention obtain a target image sequence to be detected for objects dropped from a high altitude, the target image sequence comprising a plurality of frames of images arranged in a time sequence, and for each frame of the target image sequence, determining a detection frame corresponding to a target object in the frame of the target image, and when the motion of the detection frame of the target object meets a condition for objects dropped from a high altitude, determining target position information of a stop motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence comprising a target frame image and a preset number of images before the target frame image, and determining a target area within a predetermined range of each frame of the playback image sequence based on the target position information, and then performing reverse behavior detection based on the target frame image as a starting frame based on the target area within the predetermined range of each frame of the playback image sequence, determining a second timestamp of the reverse stop motion detection frame, and determining a video segment between the second timestamp and the first timestamp as a detection result of the target image sequence. The embodiments of the present invention can determine the target position information of the stopped motion detection frame when the detection frame stops moving, the first timestamp corresponding to the target position information, and the second timestamp of the reverse stopped motion detection frame based on the movement of the detection frame of the target object, and thus can determine the trajectory of high-altitude thrown objects, avoid the false alarm of non-high-altitude thrown objects caused by motion detection, improve the accuracy of high-altitude thrown object detection, and do not require manual secondary review, saving manpower and material resources.

[0063] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0065] Figure 1 A schematic diagram of a flow chart of a method for detecting objects thrown from a high altitude provided in an embodiment of the present invention;

[0066] Figure 2 A schematic diagram of a process flow for obtaining a target image sequence provided by an embodiment of the present invention;

[0067] Figure 3 A schematic diagram of an image sequence provided by an embodiment of the present invention;

[0068] Figure 4 A schematic diagram of a method for detecting objects thrown from a high altitude provided in an embodiment of the present invention;

[0069] Figure 5 A schematic diagram of reverse behavior detection provided by an embodiment of the present invention;

[0070] Figure 6 A schematic diagram of detecting objects thrown from a high altitude provided by an embodiment of the present invention;

[0071] Figure 7 Another schematic diagram of detecting objects thrown from a high altitude provided by an embodiment of the present invention;

[0072] Figure 8 A schematic diagram of another high-altitude object throwing detection method provided in an embodiment of the present invention;

[0073] Fig. 9 A schematic diagram of the structure of a high-altitude object throwing detection system provided in an embodiment of the present invention;

[0074] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0076] In order to solve the problem that in the process of detecting high-altitude objects in the related art, when flying birds, swinging leaves, movement of people, etc. are detected in the video screen, it is easy to generate false alarms of motion detection that is not high-altitude objects, which affects the accuracy of high-altitude object detection, and requires manual secondary review and screening, which consumes manpower, material resources and other resources, the embodiment of the present invention provides a method for detecting high-altitude objects, including:

[0077] Obtain a target image sequence to be detected for high-altitude objects, wherein the target image sequence includes multiple frames of images arranged in time sequence;

[0078] For each frame image in the target image sequence, determine a detection box corresponding to the target object in the frame image;

[0079] In the case where the motion of the detection frame of the target object meets the high-altitude throwing condition, determining, according to the detection frame of the target object, target position information of the stop motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence including a target frame image and a preset number of images before the target frame image; wherein the target frame image is a frame image corresponding to the stop motion detection frame;

[0080] Based on the target position information, determine the target area within a predetermined range of each frame image in the playback image sequence;

[0081] Based on the target area within the predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame to determine a second timestamp of a reverse stop motion detection frame, wherein the reverse stop motion detection frame is a detection frame when the target object starts to move;

[0082] The video segment between the second timestamp and the first timestamp is determined as a detection result of the target image sequence.

[0083] The embodiments of the present invention can determine the target position information of the stopped motion detection frame when the detection frame stops moving, the first timestamp corresponding to the target position information, and the second timestamp of the reverse stopped motion detection frame based on the movement of the detection frame of the target object, and thus can determine the trajectory of high-altitude thrown objects, avoid the false alarm of non-high-altitude thrown objects caused by motion detection, improve the accuracy of high-altitude thrown object detection, and do not require manual secondary review, saving manpower and material resources.

[0084] The method for detecting objects thrown from a high altitude provided by the present invention is described in detail below through specific embodiments.

[0085] The method for detecting objects dropped from a high altitude provided by the embodiment of the present invention can be applied to application scenarios such as smart community and smart city construction. The method for detecting objects dropped from a high altitude provided by the embodiment of the present invention can be applied to electronic devices such as terminals, servers and other devices.

[0086] See also Figure 1 , Figure 1 A method for detecting objects thrown from a high altitude provided in an embodiment of the present invention may include:

[0087] S101, obtaining a target image sequence for high-altitude object detection.

[0088] In an embodiment of the present invention, a video acquisition device can be installed for a building to be detected for high-altitude objects, such as a building, and the video acquisition device includes a camera. The camera of the video acquisition device can then be used to obtain a video stream to be detected of the building to be detected for high-altitude objects, and the obtained video stream to be detected can be further processed to obtain a target image sequence to be detected for high-altitude objects. The target image sequence can include multiple frames of images arranged in time sequence.

[0089] In an optional implementation, the video acquisition device may be a surveillance camera, and the resolution of the surveillance camera may be no less than 1080P.

[0090] In an optional implementation, each frame of the video stream to be detected or the target image sequence to be detected should include the complete building to be detected. For example, when a 20-story building is detected for high-altitude object dropping, each frame of the video stream to be detected or the target image sequence obtained by the video acquisition device covers all 20 floors of the building.

[0091] S102: for each frame image in the target image sequence, determine a detection box corresponding to the target object in the frame image.

[0092] After acquiring the target image sequence to be detected for high-altitude objects, a detection frame corresponding to the target object can be determined in each frame of the target image sequence. For example, the target object can be a moving object other than a building in each frame of the image, such as a high-altitude object.

[0093] To determine the detection frame corresponding to the target object in the image, the background model of the image can be firstly performed, and the foreground part of the image can be extracted. The foreground part can be the foreground corresponding to the target object, and then the detection frame corresponding to the target object can be obtained by threshold dichotomy.

[0094] S103, when the motion of the detection frame of the target object meets the high-altitude throwing condition, determine, based on the detection frame of the target object, the target position information of the stopped motion detection frame when the detection frame stops moving, the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image.

[0095] After determining the detection frame corresponding to the target object in each frame of the target image sequence, it is possible to determine whether the operation of the detection frame of the target object meets the high-altitude object dropping condition based on the movement of the detection frame of the target object in each image. Specifically, it is possible to determine whether the movement direction and movement speed of the detection frame of the target object meet the high-altitude object dropping condition to filter out the interference of irrelevant object movement. Exemplarily, the high-altitude object dropping condition may be that the detection frame of the target object moves downward, and the longitudinal movement speed is greater than a set speed threshold, etc.

[0096] When the motion of the detection frame of the target object meets the high-altitude throwing condition, the detection frame of the target object is tracked, and then the motion of the detection frame of the target object is determined according to the position information of the detection frame of the target object in the adjacent frame image. The motion condition may be the motion speed, etc., and it is further judged whether the motion speed meets the motion stop condition, such as the motion speed is zero, etc., and when it is determined that the detection frame of the target object stops moving, the target position information corresponding to the stopped motion detection frame when the detection frame stops moving, the first timestamp corresponding to the target position information, and the playback image sequence including the target frame image and a preset number of images before the target frame image are determined to realize the behavior detection of the detection frame of the target object.

[0097] Among them, the target frame image is the frame image corresponding to the stop motion detection frame, and the numerical value of the preset number can be set by those skilled in the art according to actual needs, for example, according to the time of falling of the high-altitude parabolic object. The target position information can be coordinate information represented by the pixel points in the image as coordinate points. The first timestamp corresponding to the target position information can be the timestamp corresponding to the frame image corresponding to the stop motion detection frame in the video stream to be detected, that is, the timestamp corresponding to the stop motion of the target object.

[0098] S104, determining a target area within a predetermined range of each frame image in the playback image sequence based on the target position information.

[0099] After determining the target position information corresponding to the stop motion detection frame when the detection frame stops moving, that is, the target position information corresponding to the detection frame when the high-altitude parabolic target object stops moving, the target area within the predetermined range of each frame image in the playback image sequence can be further determined based on the target position information and the position information of each detection frame in the corresponding image. The target position information can be the position information of the four vertices of the stop motion detection frame when the detection frame stops moving, or the position information of the center of the stop motion detection frame when the detection frame stops moving, etc.

[0100] In an embodiment of the present invention, the target position information is taken as the position information of the center of the stop motion detection frame when the detection frame stops moving. The position coordinates represented by the target position information are (X, Y). Correspondingly, the target area within the predetermined range in the above-mentioned target frame image can be expressed as: the area composed of (N, M) pixels around the target position information, which can be specifically expressed as: the area composed of [(XN, YM), (XN, Y+M), (X+N, YM), (X+N, Y+M)].

[0101] Correspondingly, the position coordinates of the detection box of the target object in any frame before the target frame image in the playback image sequence are (Xn, Ym). Then, the target area within the predetermined range in the frame image is: the area composed of [(X2-nN, Y2-mM), (X2-nN, Y2-m+M), (X2-n+N, Y2-mM), (X2-n+N, Y2-m+M)].

[0102] Among them, X, Y, N, M, n, and m are all positive integers. Optionally, the target area within the above predetermined range should be smaller than the original pixel range of the complete image frame, that is, 2*N and 2*M are smaller than the number of pixels corresponding to the length or height of the complete image frame.

[0103] S105 , based on the target area within a predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as a starting frame to determine a second timestamp of the reverse stop motion detection frame.

[0104] After determining the target area within the predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed in the target area within the predetermined range of each frame image in the playback image sequence with the target frame image as the starting frame.

[0105] Exemplarily, N and M are both 50, that is, reverse behavior detection is performed in a square area consisting of 50 pixels around the target position (X, Y), that is, a square area consisting of [(X-50, Y-50), (X-50, Y+50), (X+50, Y-50), (X+50, Y+50)]. When the target position (X, Y) is updated in the playback image sequence, such as when it is updated to (X-0.7, Y-1) in the penultimate frame of the playback image sequence, reverse behavior detection is performed accordingly in a square area consisting of [(X-0.7-50, Y-1-50), (X-0.7-50, Y-1+50), (X-0.7+50, Y-1-50), (X-0.7+50, Y-1+50)].

[0106] Similarly, the motion of the detection frame of the target object can be determined based on the position information of the target area within a predetermined range of the reverse adjacent frame images in the playback image sequence. The motion condition can be the motion speed, etc., and it is further determined whether the motion speed meets the motion stop condition, such as the motion speed is zero, etc., and when it is determined that the detection frame of the target object stops moving, the second timestamp of the reverse stop motion detection frame is determined.

[0107] Among them, the reverse stop motion detection frame is the detection frame when the target object starts to move. Correspondingly, the second timestamp can be the timestamp corresponding to the frame image corresponding to the reverse stop motion detection frame in the video stream to be detected, that is, the timestamp corresponding to the start of movement of the target object.

[0108] In an embodiment of the present invention, after determining the target area within a predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed within the target area within each predetermined range with the target frame image as the starting frame, thereby narrowing the detection range and being able to achieve the purpose of reducing the impact of global image noise while tracking the trajectory of the target object.

[0109] S106: Determine the video segment between the second timestamp and the first timestamp as the detection result of the target image sequence.

[0110] The second timestamp is the timestamp corresponding to the start of movement of the target object, the first timestamp is the timestamp corresponding to the stop of movement of the target object, and the video segment between the second timestamp and the first timestamp is the video segment corresponding to the start of movement to the stop of movement of the target object, which fully shows the movement trajectory of high-altitude projectiles.

[0111] A method for detecting objects dropped from a high altitude provided by an embodiment of the present invention can determine the target position information of a stopped motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a second timestamp of the reverse stopped motion detection frame based on the movement of a detection frame of a target object, thereby determining the trajectory of objects dropped from a high altitude, avoiding false alarms of non-objects dropped from a high altitude caused by motion detection, improving the accuracy of detection of objects dropped from a high altitude, and eliminating the need for manual secondary review, saving manpower and material resources.

[0112] In an optional embodiment, if Figure 2 As shown, the above step S101 obtains the target image sequence to be detected for high-altitude objects, which may include:

[0113] S201, obtaining a video stream to be detected for high-altitude object throwing detection.

[0114] In an embodiment of the present invention, a video acquisition device can be installed for a building where high-altitude object dropping detection is to be performed, such as a building, and the video acquisition device includes a camera. The video stream to be detected for high-altitude object dropping detection can be obtained through the camera of the video acquisition device.

[0115] In an optional implementation, the video stream to be detected for high-altitude object throwing detection obtained above may be a real-time video stream.

[0116] S202, performing frame extraction processing on the video stream to be detected to obtain a frame-extracted image.

[0117] In an embodiment of the present invention, the acquired video stream to be detected may be subjected to frame extraction processing to obtain an image in frames, i.e., an extracted image, which may be a plurality of frames arranged in time sequence. Exemplarily, for the acquired video stream to be detected, a certain number of frame images may be extracted per second to obtain an extracted image, and the certain number may be, for example, 20, 25 or 30 frames.

[0118] Taking the example of extracting 30 frames of images per second, the video stream to be detected can be a real-time video stream. Usually, the complete time of a high-altitude object falling does not exceed 5 seconds. Then, after the video sequence corresponding to the video stream to be detected obtained above is processed by frame extraction, an image sequence with a length of 150 frames can be maintained. For example, when processing the current frame in the video stream to be detected, a 150-frame image sequence including the current frame and the frame image before the current frame can be maintained, and the 150-frame image sequence can be continuously updated during the process of obtaining the video stream to be detected. The video sequence corresponding to the video stream to be detected is processed by frame extraction, and the maintained image sequence is as follows: Figure 3 shown.

[0119] In an embodiment of the present invention, an image sequence is maintained so that when the motion of the detection frame of the target object meets the high-altitude throwing condition, a playback image sequence including the target frame image and a preset number of images before the target frame image can be quickly determined.

[0120] S203, performing blur processing on the extracted images to obtain a target image sequence.

[0121] Gaussian blur processing can be performed on each frame of the image after frame extraction to reduce noise in the image and obtain a target image sequence corresponding to the video stream to be detected for high-altitude object detection.

[0122] In an optional implementation, the above step S102 determines the detection box corresponding to the target object in each frame image in the target image sequence, which may include:

[0123] For each frame of the target image sequence, the foreground of the target object in the frame is extracted to obtain the foreground area of ​​the target object; the main frame of the foreground area is determined as the detection frame of the target object.

[0124] After obtaining the target image sequence, background modeling can be performed for each frame in the target image sequence, and the foreground part in each frame can be detected and extracted. When the target object appears in the foreground part of the image, the foreground area of ​​the target object is obtained, and the main frame of the foreground area is further obtained through the threshold binary method, and the main frame is determined as the detection frame of the target object.

[0125] In an optional implementation, whether the motion of the detection frame of the target object meets the high-altitude throwing condition can be determined in the following manner:

[0126] Determine the motion speed of the detection frame of the target object in the current frame image based on the center position of the detection frame of the target object in the current frame image in the target image sequence and the center position of the detection frame of the target object in the previous frame image adjacent to the current frame image, wherein the motion speed includes a horizontal speed and a vertical speed;

[0127] When the longitudinal speed of the detection frame of the target object in the current frame image is greater than the lateral speed, and the longitudinal speed is greater than the first speed threshold, it is determined that the movement of the detection frame of the target object meets the high-altitude throwing condition.

[0128] In an embodiment of the present invention, the speed of the object movement is expressed in pixels / frame. The movement speed of the detection frame of the target object of the current frame image can be determined based on the center position of the detection frame of the target object of the current frame image in the target image sequence and the center position of the detection frame of the target object of the previous frame image adjacent to the current frame image. Exemplarily, the current frame image is represented as Pt, and the previous frame image adjacent to the current frame image is represented as Pt-1, then, the horizontal speed of the detection frame of the target object of the current frame image = (the horizontal coordinate of the center position of the detection frame of the target object of Pt-the horizontal coordinate of the center position of the detection frame of the target object of Pt-1) / frame; the vertical speed of the detection frame of the target object of the current frame image = (the vertical coordinate of the center position of the detection frame of the target object of Pt-the vertical coordinate of the center position of the detection frame of the target object of Pt-1) / frame.

[0129] Further, it is determined whether the moving direction and moving speed of the detection frame of the target object meet the high-altitude dropping condition to filter out the interference of irrelevant object movement. It can be determined whether the longitudinal speed of the detection frame of the target object in the current frame image is greater than the lateral speed, and whether the longitudinal speed is greater than a first speed threshold. If the longitudinal speed of the detection frame of the target object in the current frame image is greater than the lateral speed, and the longitudinal speed is greater than the first speed threshold, it is determined that the movement of the detection frame of the target object meets the high-altitude dropping condition.

[0130] For example, Figure 4 As shown, when V2 is greater than V1 and V2 is greater than the first speed threshold V0, it is determined that the motion of the detection frame of the target object meets the high-altitude throwing condition. Wherein, V2 represents the longitudinal speed, V1 represents the lateral speed, and the first speed threshold V0 represents the longitudinal speed threshold. The first speed threshold V0 can be set according to actual needs, for example, it can be set to 3 pixels / frame, or 2 pixels / frame, etc.

[0131] In an optional implementation, the maintained image sequence may include motion speed information of each frame of image.

[0132] In an optional implementation, determining, according to the detection frame of the target object, the target position information of the stopped motion detection frame when the detection frame stops moving, the first timestamp corresponding to the target position information, and the playback image sequence including the target frame image and a preset number of images before the target frame image in the above step S103 may include:

[0133] Track the detection frame of the target object and calculate the movement speed of the detection frame of the target object in adjacent frame images;

[0134] When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive frame images are both zero, the first frame image of the multiple frame images is determined as the target frame image, and the position information of the detection frame of the target object of the target frame image is determined as the target position information of the stopped motion detection frame when the detection frame stops moving, and the timestamp corresponding to the target frame image is determined as the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image is determined.

[0135] In an embodiment of the present invention, the detection frame of the target object in each frame image in the target image sequence corresponding to the received video stream to be detected can be tracked, and the motion speed of the detection frame of the target object in adjacent frame images can be calculated using the motion speed calculation method of the detection frame of the target object.

[0136] When the lateral speed and the longitudinal speed of the detection frame of the target object in multiple consecutive frame images are both zero, it indicates that the detection frame of the target object stops moving. At this time, the first frame image in the multiple frame images is determined as the target frame image, and the position information of the detection frame of the target object of the target frame image is determined as the target position information of the stopped motion detection frame when the detection frame stops moving, and the timestamp corresponding to the target frame image is determined as the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image is determined.

[0137] Among them, the continuous multiple frame images can be 3 frames, 4 frames or 5 frames, etc. Usually, the complete time of the high-altitude parabolic fall does not exceed 5 seconds, and the preset number can be determined according to the frame extraction situation. Exemplarily, if 30 frames are extracted per second, the corresponding preset number can be set to 150 frames, and if 20 frames are extracted per second, the preset number can be set to 100 frames, etc. In the case of the above-mentioned maintenance image sequence, the target frame image corresponding to the stop of the motion of the detection frame of the target object can be directly determined as the last frame of the maintained image sequence, and a playback image sequence containing the target frame image and a preset number of images before the target frame image is obtained.

[0138] In an optional implementation, each frame of the image in the playback image sequence may include: the movement speed of the detection frame of the target object, the position information of the detection frame of the target object, and the timestamp corresponding to the frame of the image.

[0139] In an optional implementation, the above step S105 performs reverse behavior detection based on the target area within a predetermined range of each frame image in the playback image sequence, taking the target frame image as the starting frame, and determines the second timestamp of the reverse stop motion detection frame, which may include:

[0140] Based on the target area within a predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame, and the movement speed of the detection frame of the target object in the adjacent frame image is calculated;

[0141] When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive reverse frame images are both less than the second speed threshold, the first reverse frame image among the multiple reverse frame images is determined as the starting frame image, and the timestamp of the starting frame image is determined as the second timestamp of the reverse stop motion detection frame.

[0142] After determining the target frame image and the target area within the predetermined range of each frame image in the playback image sequence, the target frame image is the frame image corresponding to the stop motion detection frame, and reverse behavior detection is performed with the target frame image as the starting frame within the target area within the predetermined range of each frame image in the playback image sequence.

[0143] During the reverse behavior detection process, when the lateral speed and longitudinal speed of the detection frame of the target object in a plurality of consecutive reverse frame images are both less than the second speed threshold, it is determined that the detection frame of the target object stops moving. At this time, the first reverse frame image in the plurality of reverse frame images is determined as the starting frame image, which is the frame image corresponding to the start of the movement of the target object, and the timestamp of the starting frame image is determined as the second timestamp of the reverse stop motion detection frame. The plurality of consecutive reverse frame images may be 3 frames, 4 frames, or 5 frames, etc., and the second speed threshold may be less than the first speed threshold, for example, it may be set to 2 pixels / frame, or 1 pixel / frame, etc.

[0144] like Figure 5 As shown, Figure 5 It is shown in the figure that the playback image sequence is determined from the video sequence corresponding to the video stream to be detected, and then the target frame image corresponding to the stop motion detection frame when the detection frame stops moving is determined, and the starting frame image corresponding to the reverse stop motion detection frame is determined through reverse behavior detection, and then the detection result is obtained.

[0145] The second timestamp determined above is the timestamp corresponding to the start of movement of the target object, and the first timestamp is the timestamp corresponding to the stop of movement of the target object. Furthermore, the movement trajectory of the target object can be determined based on the second timestamp and the first timestamp to realize the detection of objects thrown from high altitudes.

[0146] In an optional embodiment, at least one of the second timestamp and the first timestamp, the video segment between the second timestamp and the first timestamp, the starting frame image, the target frame image, and the target key frame in the video segment between the second timestamp and the first timestamp can also be displayed.

[0147] In an embodiment of the present invention, after determining a second timestamp corresponding to the start of movement of the target object and a first timestamp corresponding to the stop of movement of the target object, a video segment between the second timestamp and the first timestamp can be determined, that is, a video segment corresponding to the target object from the start of movement to the stop of movement, as well as a starting frame image (that is, the first frame image), a target frame image (that is, the last frame image) in the video segment, and a target key frame image between the starting frame image and the target frame image. The target key frame image can be a frame image between the starting frame image and the target frame image that can completely and clearly display the entire frame image to be detected for high-altitude throwing.

[0148] In an optional embodiment, at least one of the target key frames in the second timestamp and the first timestamp, the video clip between the second timestamp and the first timestamp, the starting frame image, the target frame image, and the video clip between the second timestamp and the first timestamp can be displayed through a mobile terminal APP, a WEB interface, an application program interface, an SMS gateway, etc., so as to achieve traceability of objects thrown from high places.

[0149] In an optional embodiment, if Figure 6 and Figure 7 As shown, the above-mentioned high-altitude object dropping detection method can be implemented by a high-altitude object dropping detection device, which may include a video acquisition and transmission module, a behavior detection module, a trajectory backtracking module, and a result display and alarm module.

[0150] The video acquisition and transmission module acquires the video stream to be detected for high-altitude object throwing detection, and transmits the acquired video stream to be detected to the behavior detection module. The resolution of the camera used for video acquisition is not less than 1080P, and the transmission of the video stream to be detected supports protocols such as H.264, H.265, RSTP (Rapid Spanning Tree Protocol) or RTMP (Real Time Messaging Protocol).

[0151] The video acquisition and transmission module can transmit the video stream to be detected to the behavior detection module through a network transmission medium. The network transmission medium can be an Internet dedicated line, a digital circuit, FTTH (Fiber To The Home) and other network media with network signal transmission function.

[0152] The behavior detection module detects each frame and adjacent frames in the video stream to be detected, determines the playback image sequence, and transmits the playback image sequence to the trajectory tracing module. Specifically, the video stream to be detected is subjected to frame extraction processing to obtain the extracted image, and the extracted image is blurred to obtain the target image sequence. For each frame image in the target image sequence, the detection frame corresponding to the target object is determined in the frame image. When the motion of the detection frame of the target object meets the high-altitude throwing condition, the target position information of the stop motion detection frame when the detection frame stops moving is determined according to the detection frame of the target object, the first timestamp corresponding to the target position information, and the playback image sequence including the target frame image and a preset number of images before the target frame image, wherein the target frame image is the frame image corresponding to the stop motion detection frame.

[0153] The trajectory backtracking module performs reverse behavior detection based on the playback image sequence to obtain the detection result. Specifically, based on the target position information, the target area within the predetermined range of each frame image in the playback image sequence is determined, and based on the target area within the predetermined range of each frame image in the playback image sequence, the reverse behavior detection is performed with the target frame image as the starting frame, and the second timestamp of the reverse stop motion detection frame is determined, wherein the reverse stop motion detection frame is the detection frame when the target object starts to move, and the second timestamp corresponding to the start of the target object movement, the first timestamp corresponding to the stop of the target object movement, the video segment between the second timestamp and the first timestamp, the starting frame image, the target frame image, and the target key frame in the video segment between the second timestamp and the first timestamp (i.e., the detection result, which represents the trajectory information of the target object) are obtained, and the detection result information is transmitted to the result display and alarm module.

[0154] The result display and alarm module displays at least one of the target key frames in the second timestamp and the first timestamp, the video clip between the second timestamp and the first timestamp, the starting frame image, the target frame image, and the video clip between the second timestamp and the first timestamp through a mobile terminal APP, a WEB interface, an application program interface, an SMS gateway, etc., so as to realize the tracing and alarm of objects thrown from high places.

[0155] Embodiments of the present invention Figure 6 and 7 In the method, the video acquisition and transmission module acquires the video stream to be detected for high-altitude object dropping detection, and transmits the video stream to be detected to the behavior detection module. The behavior detection module performs Gaussian blur processing on each frame image in the video sequence corresponding to the video stream to be detected to reduce noise, obtains the detection frame corresponding to the target object in each frame image by threshold dichotomy, and then compares the motion of the detection frame of the target object in adjacent frames. When the motion of the detection frame of the target object meets the high-altitude object dropping condition, a playback image sequence including the target frame image and a preset number of images before the target frame image is determined, and the target frame image is the frame image corresponding to the stop motion detection frame, and the playback image sequence is transmitted to the trajectory tracing module. The trajectory tracing module performs reverse behavior detection based on the target area within the predetermined range of each frame image in the determined playback image sequence, taking the target frame image as the starting frame, that is, performing reverse behavior detection with a smaller pixel range, determining the second timestamp of the reverse stop motion detection frame, and obtaining the detection result.

[0156] The above process can also be Figure 8 As shown, from the video sequence corresponding to the video stream to be detected, when the movement of the detection frame of the target object meets the high-altitude throwing condition, the playback image sequence is determined, and then the target frame image corresponding to the stop motion detection frame when the detection frame stops moving is determined, and through reverse behavior detection, the starting frame image corresponding to the reverse stop motion detection frame is determined, and then the detection result is obtained.

[0157] In a related technology, a neural network model for target tracking is trained through a large amount of sample data, and then when detecting objects thrown from a high altitude, the acquired surveillance video is input into the trained neural network model to determine whether the motion trajectory of the target object in the surveillance video is an object thrown from a high altitude. The detection of objects thrown from a high altitude achieved by this method requires a large amount of sample data of objects thrown from a high altitude to support the training of the neural network model. At the same time, deep learning models such as neural networks have high requirements for computing resources and are difficult to use in practice.

[0158] Compared with the detection method of high-altitude objects in the related art, in the embodiments of the present invention, there is no need to train a complex neural network model, which reduces the dependence of high-altitude objects detection on deep learning computing resources and is easy to deploy.

[0159] Corresponding to the method for detecting objects thrown from a high altitude provided in the above embodiment, the embodiment of the present invention further provides a system for detecting objects thrown from a high altitude, such as Fig. 9 As shown, including:

[0160] The image acquisition module 901 is used to acquire a target image sequence to be detected for high-altitude projectiles, wherein the target image sequence includes multiple frames of images arranged in time sequence;

[0161] A detection frame determination module 902 is used to determine, for each frame image in the target image sequence, a detection frame corresponding to the target object in the frame image;

[0162] The behavior detection module 903 is used to determine, according to the detection frame of the target object, the target position information of the stop motion detection frame when the detection frame stops moving, the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image, when the motion of the detection frame of the target object meets the high-altitude throwing condition; wherein the target frame image is the frame image corresponding to the stop motion detection frame;

[0163] A region determination module 904 is used to determine a target region within a predetermined range of each frame of the playback image sequence based on the target position information;

[0164] The trajectory backtracking module 905 is used to perform reverse behavior detection based on the target area within the predetermined range of each frame image in the playback image sequence, taking the target frame image as the starting frame, and determine a second timestamp of a reverse stop motion detection frame, wherein the reverse stop motion detection frame is a detection frame when the target object starts to move;

[0165] The result detection module 906 is configured to determine the video segment between the second timestamp and the first timestamp as the detection result of the target image sequence.

[0166] A system for detecting objects dropped from a high altitude provided by an embodiment of the present invention can determine the target position information of a stopped motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a second timestamp of the reverse stopped motion detection frame based on the movement of a detection frame of a target object, thereby determining the trajectory of objects dropped from a high altitude, avoiding false alarms of non-objects dropped from a high altitude caused by motion detection, improving the accuracy of detection of objects dropped from a high altitude, and eliminating the need for manual secondary review, saving manpower and material resources.

[0167] Optionally, the image acquisition module 901 is specifically used for:

[0168] Obtain the video stream to be detected for high-altitude object throwing detection;

[0169] Perform frame extraction on the video stream to be detected to obtain a frame-extracted image;

[0170] The extracted images are blurred to obtain the target image sequence.

[0171] Optionally, the detection frame determination module 902 is specifically configured to:

[0172] For each frame image in the target image sequence, foreground extraction is performed on the target object in the frame image to obtain a foreground area of ​​the target object;

[0173] The main frame of the foreground area is determined as the detection frame of the target object.

[0174] Optionally, the above system further includes:

[0175] A speed determination module, used to determine the movement speed of the detection frame of the target object of the current frame image based on the center position of the detection frame of the target object of the current frame image in the target image sequence and the center position of the detection frame of the target object of the previous frame image adjacent to the current frame image, wherein the movement speed includes a horizontal speed and a vertical speed;

[0176] The high-altitude object dropping determination module is used to determine that the movement of the target object detection frame meets the high-altitude object dropping condition when the longitudinal speed of the target object detection frame in the current frame image is greater than the lateral speed and the longitudinal speed is greater than the first speed threshold.

[0177] Optionally, the behavior detection module 903 is specifically used to:

[0178] Track the detection frame of the target object and calculate the movement speed of the detection frame of the target object in adjacent frame images;

[0179] When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive frame images are both zero, the first frame image of the multiple frame images is determined as the target frame image, and the position information of the detection frame of the target object of the target frame image is determined as the target position information of the stopped motion detection frame when the detection frame stops moving, and the timestamp corresponding to the target frame image is determined as the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image is determined.

[0180] Optionally, the trajectory backtracking module 904 is specifically used to:

[0181] Based on the target area within a predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame, and the movement speed of the detection frame of the target object in the adjacent frame image is calculated;

[0182] When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive reverse frame images are both less than the second speed threshold, the first reverse frame image among the multiple reverse frame images is determined as the starting frame image, and the timestamp of the starting frame image is determined as the second timestamp of the reverse stop motion detection frame.

[0183] Optionally, the above system further includes:

[0184] The result display module is used to display at least one of the second timestamp and the first timestamp, the video segment between the second timestamp and the first timestamp, the starting frame image, the target frame image, and the target key frame in the video segment between the second timestamp and the first timestamp.

[0185] The embodiment of the present invention further provides an electronic device, such as Fig.10 As shown, it includes a processor 101, a communication interface 102, a memory 103 and a communication bus 104, wherein the processor 101, the communication interface 102, and the memory 103 communicate with each other through the communication bus 104.

[0186] Memory 103, used for storing computer programs;

[0187] The processor 101 is used to execute the program stored in the memory 103, and implements the following steps:

[0188] Acquire a target image sequence to be detected for high-altitude projectiles, wherein the target image sequence includes multiple frames of images arranged in time sequence;

[0189] For each frame image in the target image sequence, determine a detection box corresponding to the target object in the frame image;

[0190] In the case where the motion of the detection frame of the target object meets the high-altitude throwing condition, determining, according to the detection frame of the target object, target position information of the stop motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence including a target frame image and a preset number of images before the target frame image; wherein the target frame image is a frame image corresponding to the stop motion detection frame;

[0191] Based on the target position information, determine the target area within a predetermined range of each frame image in the playback image sequence;

[0192] Based on the target area within the predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame to determine a second timestamp of a reverse stop motion detection frame, wherein the reverse stop motion detection frame is a detection frame when the target object starts to move;

[0193] The video segment between the second timestamp and the first timestamp is determined as a detection result of the target image sequence.

[0194] An electronic device provided by an embodiment of the present invention can determine the target position information of a stop motion detection frame when the detection frame stops moving, the first timestamp corresponding to the target position information, and the second timestamp of the reverse stop motion detection frame based on the movement of the detection frame of the target object, and thus can determine the trajectory of high-altitude thrown objects, avoid the false alarm of non-high-altitude thrown objects caused by motion detection, improve the accuracy of high-altitude thrown object detection, and do not require manual secondary review, saving manpower and material resources.

[0195] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0196] The communication interface is used for communication between the above electronic device and other devices.

[0197] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0198] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0199] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for detecting high-altitude objects are implemented.

[0200] In another embodiment provided by the present invention, a computer program product containing instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of any of the methods for detecting objects thrown from a high altitude in the above-mentioned embodiments.

[0201] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0202] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0203] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system / electronic device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0204] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for detecting objects thrown from a high altitude, characterized in that: The method comprises: Acquire a target image sequence to be subjected to high-altitude object throwing detection, wherein the target image sequence includes multiple frames of images arranged in time sequence; For each frame image in the target image sequence, determining a detection frame corresponding to the target object in the frame image; In the case where the motion of the detection frame of the target object meets the high-altitude throwing condition, determining, according to the detection frame of the target object, target position information of the stop motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence including a target frame image and a preset number of images before the target frame image; wherein the target frame image is a frame image corresponding to the stop motion detection frame; Based on the target position information, determining a target area within a predetermined range of each frame image in the playback image sequence; Based on the target area within the predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame to determine a second timestamp of a reverse stop motion detection frame, wherein the reverse stop motion detection frame is a detection frame when the target object starts to move; The video segment between the second timestamp and the first timestamp is determined as the detection result of the target image sequence.

2. The method according to claim 1, characterized in that The step of obtaining a target image sequence to be subjected to high-altitude object detection comprises: Obtain the video stream to be detected for high-altitude object throwing detection; Performing frame extraction processing on the video stream to be detected to obtain a frame-extracted image; The image after frame extraction is blurred to obtain a target image sequence.

3. The method according to claim 1, characterized in that The step of determining, for each frame image in the target image sequence, a detection frame corresponding to the target object in the frame image comprises: For each frame of the target image sequence, performing foreground extraction on the target object in the frame of the image to obtain a foreground area of ​​the target object; The main body frame of the foreground area is determined as the detection frame of the target object.

4. The method according to claim 1, characterized in that: The method further comprises: Determine a motion speed of the detection frame of the target object of the current frame image based on a center position of a detection frame of the target object of the current frame image in the target image sequence and a center position of a detection frame of the target object of the previous frame image adjacent to the current frame image, wherein the motion speed includes a horizontal speed and a vertical speed; When the longitudinal speed of the detection frame of the target object in the current frame image is greater than the lateral speed, and the longitudinal speed is greater than a first speed threshold, it is determined that the movement of the detection frame of the target object meets the high-altitude throwing condition.

5. The method according to claim 4, characterized in that The method comprises: determining, based on the detection frame of the target object, target position information of the stop motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence including a target frame image and a preset number of images before the target frame image, including: Tracking the detection frame of the target object and calculating the movement speed of the detection frame of the target object in adjacent frame images; When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive frame images are both zero, the first frame image among the multiple frame images is determined as the target frame image, and the position information of the detection frame of the target object of the target frame image is determined as the target position information of the stopped motion detection frame when the detection frame stops moving, and the timestamp corresponding to the target frame image is determined as the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image is determined.

6. The method according to claim 4, characterized in that The method of performing reverse behavior detection based on a target area within a predetermined range of each frame image in the playback image sequence and taking the target frame image as a starting frame to determine a second timestamp of a reverse stop motion detection frame comprises: Based on the target area within a predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame, and the movement speed of the detection frame of the target object in the adjacent frame images is calculated; When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive reverse frame images are both less than the second speed threshold, the first reverse frame image among the multiple reverse frame images is determined as the starting frame image, and the timestamp of the starting frame image is determined as the second timestamp of the reverse stop motion detection frame.

7. The method according to claim 6, characterized in that The method further comprises: At least one of the second timestamp and the first timestamp, the video segment between the second timestamp and the first timestamp, the start frame image, the target frame image, and the target key frame in the video segment between the second timestamp and the first timestamp is displayed.

8. A high-altitude object detection system, characterized in that: The system comprises: An image acquisition module is used to acquire a target image sequence to be detected for high-altitude projectiles, wherein the target image sequence includes multiple frames of images arranged in time sequence; A detection frame determination module, used to determine, for each frame image in the target image sequence, a detection frame corresponding to the target object in the frame image; A behavior detection module is used to determine, based on the detection frame of the target object, target position information of the stop motion detection frame when the detection frame stops moving, a first timestamp corresponding to the target position information, and a playback image sequence including a target frame image and a preset number of images before the target frame image, when the motion of the detection frame of the target object meets the high-altitude throwing condition; wherein the target frame image is a frame image corresponding to the stop motion detection frame; A region determination module, used for determining a target region within a predetermined range of each frame image in the playback image sequence based on the target position information; A trajectory backtracking module is used to perform reverse behavior detection based on a target area within a predetermined range of each frame image in the playback image sequence, taking the target frame image as a starting frame, and determine a second timestamp of a reverse stop motion detection frame, wherein the reverse stop motion detection frame is a detection frame when the target object starts to move; The result detection module is used to determine the video segment between the second timestamp and the first timestamp as the detection result of the target image sequence.

9. The system according to claim 8, characterized in that The image acquisition module is specifically used for: Obtain the video stream to be detected for high-altitude object throwing detection; Performing frame extraction processing on the video stream to be detected to obtain a frame-extracted image; The image after frame extraction is blurred to obtain a target image sequence.

10. The system according to claim 8, characterized in that The detection frame determination module is specifically used to: For each frame of the target image sequence, performing foreground extraction on the target object in the frame of the image to obtain a foreground area of ​​the target object; The main body frame of the foreground area is determined as the detection frame of the target object.

11. The system according to claim 8, characterized in that The system further comprises: a speed determination module, configured to determine a motion speed of a detection frame of a target object in a current frame image based on a center position of a detection frame of a target object in a current frame image in the target image sequence and a center position of a detection frame of a target object in a previous frame image adjacent to the current frame image, wherein the motion speed includes a horizontal speed and a vertical speed; The high-altitude object dropping determination module is used to determine that the movement of the detection frame of the target object in the current frame image meets the high-altitude object dropping condition when the longitudinal speed of the detection frame of the target object is greater than the lateral speed and the longitudinal speed is greater than a first speed threshold.

12. The system according to claim 11, characterized in that The behavior detection module is specifically used for: Tracking the detection frame of the target object and calculating the movement speed of the detection frame of the target object in adjacent frame images; When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive frame images are both zero, the first frame image among the multiple frame images is determined as the target frame image, and the position information of the detection frame of the target object of the target frame image is determined as the target position information of the stopped motion detection frame when the detection frame stops moving, and the timestamp corresponding to the target frame image is determined as the first timestamp corresponding to the target position information, and a playback image sequence including the target frame image and a preset number of images before the target frame image is determined.

13. The system according to claim 11, characterized in that The trajectory backtracking module is specifically used for: Based on the target area within a predetermined range of each frame image in the playback image sequence, reverse behavior detection is performed with the target frame image as the starting frame, and the movement speed of the detection frame of the target object in the adjacent frame images is calculated; When the lateral speed and the longitudinal speed of the detection frame of the target object of multiple consecutive reverse frame images are both less than the second speed threshold, the first reverse frame image among the multiple reverse frame images is determined as the starting frame image, and the timestamp of the starting frame image is determined as the second timestamp of the reverse stop motion detection frame.

14. The system according to claim 8, characterized in that The system further comprises: A result display module is used to display at least one of the second timestamp and the first timestamp, the video segment between the second timestamp and the first timestamp, the starting frame image, the target frame image, and the target key frame in the video segment between the second timestamp and the first timestamp.

15. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images