A high-altitude projectile tracing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a high-altitude projectile tracing method and device, electronic equipment and storage medium. The method comprises: searching, according to an image of a target object, a first motion detection image and a second motion detection image containing the target object from motion detection images; obtaining video data of a home camera from a first time point before a shooting time point of the first motion detection image to a second time point after the shooting time point according to the shooting time point and the home camera; determining a plurality of motion detection positions coinciding with the projectile plane according to video frames contained in the video data, and determining a projectile trajectory of the target object according to the plurality of motion detection positions. The high-altitude projectile tracing scheme provided by the embodiments of the present application simplifies the method for determining the projectile trajectory of the target object, and improves the intelligent level of the high-altitude projectile video monitoring system.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to, but is not limited to, the field of video monitoring, and in particular, to a high-altitude throwing backtracking method and device, an electronic device, and a storage medium. BACKGROUND

[0002] High-altitude throwing is harmful, and its management and rectification work is related to the safety of everyone's life. Although relevant regulations have been issued, and high-altitude throwing monitoring cameras in the community have basically been in place, there are still some high-altitude throwing phenomena. The technical solutions for high-altitude throwing on the market are mostly solutions for full video coverage of the monitoring area. When a safety accident occurs, video materials are provided to relevant personnel, and then the backtracking is performed through manual investigation and search, which is low in overall efficiency. Another implementable solution traverses all video frames through a picture search algorithm to perform trajectory fitting. This way has very high requirements for video clarity and algorithm accuracy, which affects the promotion and application of such solutions.

[0003] How to improve the backtracking efficiency for high-altitude throwing is the direction of exploration for continuously improving and perfecting the video monitoring solution in this field. SUMMARY

[0004] The embodiments of the present disclosure provide a high-altitude throwing backtracking method, device, electronic device, and storage medium, a throwing plane of a target object is constructed according to a dynamic detection picture, and a trajectory point of the target object in a video frame is determined based on the constructed throwing plane, which simplifies the method for determining the throwing trajectory of the target object and improves the intelligent level of the high-altitude throwing video monitoring system.

[0005] The embodiments of the present disclosure provide a high-altitude throwing backtracking method, which includes:

[0006] According to the image of the target object, a first dynamic detection picture and a second dynamic detection picture containing the target object are searched and matched from the dynamic detection picture;

[0007] According to the depth information of the target object in the first dynamic detection picture and the depth information of the target object in the second dynamic detection picture, a throwing plane of the target object is constructed;

[0008] According to the shooting time point of the first dynamic detection picture and the attribution camera, video data of the attribution camera from a first time point before the shooting time point to a second time point after the shooting time point is acquired;

[0009] According to the video frames contained in the video data, a plurality of dynamic detection positions coinciding with the throwing plane are determined, and a throwing trajectory of the target object is determined according to the plurality of dynamic detection positions;

[0010] The shooting time difference between the first motion detection picture and the second motion detection picture is less than a first set time length.

[0011] The present disclosure also provides a high-altitude object throwing tracing device, comprising:

[0012] A motion detection picture acquisition module is configured to search for a first motion detection picture and a second motion detection picture containing the target object from the motion detection pictures according to the image of the target object.

[0013] A parabolic plane construction module is configured to construct a parabolic plane of the target object according to the depth information of the target object in the first motion detection picture and the depth information of the target object in the second motion detection picture.

[0014] A video acquisition module is configured to acquire video data of a home camera from a first time point before a shooting time point of the first motion detection picture to a second time point after the shooting time point according to the shooting time point and the home camera.

[0015] A parabolic trajectory determination module is configured to determine a plurality of motion detection positions coinciding with the parabolic plane according to video frames contained in the video data, and determine a parabolic trajectory of the target object according to the plurality of motion detection positions.

[0016] The shooting time difference between the first motion detection picture and the second motion detection picture is less than a first set time length.

[0017] The present disclosure also provides an electronic device, comprising:

[0018] One or more processors;

[0019] A storage device for storing one or more programs,

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the high-altitude object throwing tracing method as described in any embodiment of the present disclosure.

[0021] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the high-altitude object throwing tracing method as described in any embodiment of the present disclosure.

[0022] Other aspects can become apparent after reading and understanding the accompanying drawings and detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only represent some of the embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art based on these drawings without creative effort are within the protection scope of the present application.

[0024] Figure 1 is a flow chart of a high-altitude projectile tracing method provided by an embodiment of the present application;

[0025] Figure 2 is a flow chart of a first and second moving detection picture determination method provided by an embodiment of the present application;

[0026] Figure 3 is a flow chart of a first or second moving detection picture determination method provided by an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a projectile plane determination principle provided by an embodiment of the present application;

[0028] Figure 5 is a flow chart of a moving detection position determination method provided by an embodiment of the present application;

[0029] Figure 6 is a structural schematic diagram of a high-altitude projectile tracing device provided by an embodiment of the present application.

[0030] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present application.

[0032] It should be noted that all directionality indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings). If the specific posture changes, the directionality indications will also change accordingly.

[0033] In addition, the descriptions such as "first", "second", etc. in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the technical features indicated, or implicitly indicating the number of the technical features indicated. Therefore, the features defined as "first", "second" can be explicitly or implicitly included at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0034] In the present application, unless otherwise explicitly specified and limited, the terms "connection", "fixing", etc. should be understood broadly, for example, "fixing" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be direct connection, or indirect connection through intermediate medium; can be internal communication of two elements or interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0035] In addition, the technical solutions of various embodiments of the present application can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection required by the present application.

[0036] High-altitude throwing is harmful, and its management and regulation work is related to the safety of everyone's life. In the high-altitude throwing video monitoring system of the related scheme, video data is mainly provided for manual tracing, resulting in large manual input of management and monitoring and low work efficiency. At the same time, the serious lack of automation and intelligence also hinders the timely discovery of hidden dangers and early warning.

[0037] The present application provides an intelligent high-altitude throwing tracing scheme, which can automatically trace the throwing trajectory based on the recorded video and / or the captured picture according to the image of the thrown target object. The throwing trajectory formed based on the tracing can further realize more business functions such as predicting the room where the target object is thrown, estimating the related attributes of the target, and evaluating the danger degree.

[0038] Before detailing the embodiments of the present application, first, the related aspects are described:

[0039] The high-altitude projectile tracing scheme provided by the embodiments of the present disclosure is realized based on a video monitoring manner, a camera monitors a building that may produce a projectile by video, and according to the performance of the camera, one or more cameras can be used to monitor a region for one building. The first / second motion detection picture and video data in the scheme provided by the embodiments of the present disclosure are all from the same camera without special instructions. For the case of building a virtual camera with multiple physical cameras to cover a larger shooting range, if the virtual camera outputs pictures and videos after fusion processing of multiple cameras, the virtual camera is also regarded as a camera. The fusion scheme of specific pictures or videos is not discussed in detail in the scheme of the present application.

[0040] It should be noted that the camera used in the scheme of the embodiments of the present disclosure has basic video shooting function and picture shooting function; has motion detection snapshot function and can generate motion detection macro block information; has depth information acquisition function and can generate depth information corresponding to the shot picture / image, the depth information describes the depth corresponding to each pixel or block in the picture / image. Among them, motion detection snapshot, that is, motion object detection and generation of corresponding motion detection picture, the specific implementation scheme is not discussed in detail in the embodiments of the present application, which can be implemented according to related technical schemes.

[0041] The camera used in the embodiments of the present disclosure shoots video images during monitoring, and motion detection macro block information and depth information are synchronously acquired. When a moving object is detected, a motion detection picture is generated, which corresponds to a group of information: image information, motion detection information and depth information, and the three have a corresponding relationship. In some exemplary embodiments, the camera is an RGB-D (Red-Green-Blue-Deep) camera, which shoots RGB (Red-Green-Blue-Deep) video images. Therefore, when a moving object is detected, each motion detection RGB picture corresponds to a group of information: RGB image information, motion detection information and depth information.

[0042] The embodiments of the present disclosure provide a high-altitude projectile tracing method, as shown in Figure 1 The method comprises the following steps:

[0043] Step 110, searching for a first motion detection picture and a second motion detection picture containing the target object from the motion detection pictures according to the image of the target object;

[0044] Step 120, constructing a projectile plane of the target object according to the depth information of the target object in the first motion detection picture and the depth information of the target object in the second motion detection picture;

[0045] In step 130, video data of the home camera from a first time point before the shooting time point of the first motion detection picture to a second time point after the shooting time point is obtained according to the shooting time point of the first motion detection picture and the home camera.

[0046] In step 140, a plurality of motion detection positions coinciding with the parabolic plane are determined according to video frames contained in the video data, and a parabolic trajectory of the target object is determined according to the plurality of motion detection positions.

[0047] In some example embodiments, the shooting time difference between the first motion detection picture and the second motion detection picture is less than a first set time length.

[0048] In some example embodiments, the time difference between the shooting time point of the first motion detection picture and the first time point is less than a first set time length, and the time difference between the shooting time point of the first motion detection picture and the second time point is less than a first set time length. In some example embodiments, the shooting time point of the first motion detection picture is recorded as T, the first set time length is t, the first time point is T-t, and the second time point is T+t, that is, the video data is a piece of video data of the home camera from the first time point T-t to the second time point T+t.

[0049] In some example embodiments, the image of the target object can be a shooting image of the target object, or an image of the target object drawn according to related information, or an image of the target object obtained in other ways, or a shooting image or an existing image of a similar target object, and is not limited to a specific aspect.

[0050] In some example embodiments, when a user reports, or a security guard discovers, or a security accident occurs, people will find high-altitude parabolas near buildings, that is, the target object in the embodiment of the present disclosure. People can collect the image of the target object to be traced by shooting a photo, that is, obtain the image of the target object. The image will be used as the input of the picture search in step 110.

[0051] In some example embodiments, an operator uploads the image of the target object to a video monitoring device. The image of the target object can be one or more, that is, one or more shooting pictures can be uploaded. Multiple images can correspond to multiple shooting angles, such as front, side, top, bottom, and tilt.

[0052] In some example embodiments, the image can be an image obtained by cropping an original photo mainly including the target object, or can be an uncropped image.

[0053] In some example embodiments, in step 110, a search algorithm is used to search and match the moving detection pictures according to the image of the target object.

[0054] In some example embodiments, when the corresponding first moving detection picture and / or the second moving detection picture cannot be searched and matched according to one image of the target object due to the change of the aerial posture of the target object, another image is used again to perform step 110. The other image is an image from another angle.

[0055] In some example embodiments, step 110 includes:

[0056] searching and matching the first moving detection picture and the second moving detection picture containing the target object from the moving detection pictures according to the image of the target object and the set search range;

[0057] The search range includes one or more of the following: a time range, a camera range.

[0058] In some example embodiments, the time range corresponds to a start time and an end time. For example, if the security guard did not find the target object during the last patrol, the target object is most likely to be thrown after the last patrol, so the start time of the time range is the time of the last patrol, and the end time is the current time. For example, a resident reported that he heard a sound of throwing around 7:00 p.m., the property personnel went downstairs to find the suspicious target object, took pictures, and different target objects were submitted to the monitoring system for tracing. In this case, the start time of the time range can be set to 6:45 p.m., and the end time can be set to 7:15 p.m.

[0059] In some example embodiments, according to the monitoring area corresponding to the location where the target object falls to the ground, it can be determined which one or more cameras correspond to the monitoring area, and accordingly, one or more cameras included in the search range can be set.

[0060] It should be noted that in the embodiments of the present disclosure, the first moving detection picture and the second moving detection picture determined in step 110 are both from the same camera. This is not contradictory to searching and matching from the moving detection pictures of multiple cameras. Those skilled in the art can understand that the first moving detection picture is searched and matched from the moving detection pictures of multiple cameras first, and then the second moving detection picture is searched and matched in the moving detection pictures of the camera to which the first moving detection picture belongs.

[0061] It can be seen that based on the correlation analysis clue, the search range is set, which can effectively improve the execution efficiency of the search and matching, and improve the execution efficiency of the overall tracing scheme.

[0062] In some exemplary embodiments, the first set duration is 10 seconds, 20 seconds, or other values. The first set duration is set according to the building height. Generally, it is the time it takes for a target object of a certain weight to fall from the top of the monitored building to the ground.

[0063] For example, a first set duration of 10 seconds means that the time required for a target object to fall naturally from the rooftop is approximately within 10 seconds. It can be understood that determining in step 110 that the time difference between the first and second motion detection images is within 10 seconds preliminarily constrains these two motion detection images to be motion detection images of the target object during a single throwing process.

[0064] Accordingly, the video data obtained in step 130 corresponds to the video data within 10 seconds (20 seconds in total) before and after the shooting time of the first motion detection image, and it can be preliminarily determined that the video data can contain the video data of the target object from the throwing start point to the end point.

[0065] In some exemplary embodiments, such as Figure 2 As shown, step 110 includes:

[0066] Step 1110: Based on the image of the target object, search and match the first motion detection image containing the target object from the motion detection images;

[0067] Step 1120: Based on the shooting time of the first motion detection image and the camera to which it belongs, search and match a second motion detection image containing the target object from the motion detection images generated by the camera to be shot between the first time point Tt and the second time point T+t; where t is the first set time duration.

[0068] That is, a second motion detection image containing the target object is searched and matched from the motion detection images between a first time point before the shooting time point of the associated camera and a second time point after the shooting time point.

[0069] It should be noted that the time point at which the second motion detection image is captured in step 110 can be later or earlier than the time point at which the first motion detection image is captured, as long as the time difference is less than the first set duration. The specific time point can be determined according to the execution order of the search and matching, and is not limited to a specific situation.

[0070] In some exemplary embodiments, step 110 can also be replaced by: searching and matching N motion detection images containing the target object from the motion detection images based on the image of the target object, where N is an integer greater than 2;

[0071] Accordingly, step 120 is replaced by: fitting and constructing the parabolic plane of the target object based on the depth information of the target object in the N motion detection images.

[0072] The time difference between the earliest and latest capture times of the N motion detection images is less than the first set duration.

[0073] It is understandable that when N motion detection images are obtained, the parabolic plane can be fitted. The larger N is, the longer the time required to perform image search and matching. Therefore, the scheme of determining two motion detection images is the most efficient in comparison.

[0074] It is understandable that, regardless of whether it is later or earlier, the shooting time range corresponding to the video data obtained in step 130 includes the shooting time of the first motion detection image and the shooting time of the second motion detection image.

[0075] In some exemplary embodiments, such as Figure 3 As shown, the first motion detection image or the second motion detection image is obtained by searching and matching from motion detection images within the corresponding search range according to the following method:

[0076] Step 310: Based on the image of the target object, search and match from the motion-detected images within the search range to obtain a target image containing the target object;

[0077] Step 320: Obtain the location information of the target object in the target image;

[0078] Step 330: Obtain the motion detection macroblock information corresponding to the target image;

[0079] Step 340: If the location information of the target object and the motion detection macroblock information meet the set location overlap standard, the target image is determined to be either the first motion detection image or the second motion detection image.

[0080] It should be noted that when determining the first motion detection image in steps 310-340, the corresponding search range is recorded as search range 1; when determining the second motion detection image in steps 310-340, the corresponding search range is recorded as search range 2. It can be understood that search range 1 is greater than or equal to search range 2, and search range 1 includes search range 2. Compared to search range 1, search range 2 corresponds to a smaller or the same camera range, only corresponding to the camera to which the first motion detection image belongs. For example, if the first motion detection image is generated by camera 1, then when searching for a matching second motion detection image in steps 310-340, search range 2 is further narrowed based on search range 1, only searching and matching within the motion detection images generated by camera 1.

[0081] In some exemplary embodiments, the location information of the target object and the motion detection macroblock information are determined to meet a set location overlap standard if one of the following conditions is met:

[0082] The distance difference between the center point of the target object and the center point of the dynamic detection macroblock is less than a first distance threshold.

[0083] The ratio of the overlapping range of the target object contour range and the dynamic detection macroblock range to the total contour range of the target object is greater than a first proportional threshold.

[0084] It is understandable that determining the positional overlap standard in step 340 indicates that one of the moving objects captured in the target image (motion detection image) is likely the target object to be traced obtained in step 110, and can be identified as the first motion detection image or the second motion detection image.

[0085] It should be noted that step 310 employs an image search scheme to search and match target images containing the target object. Based on the search and matching results, the position of the target object within its assigned motion detection image can be determined. For example, if the target object is a cola bottle, and the search and matching result shows that the captured image 1 contains the cola bottle, then the position information of the cola bottle within captured image 1 can be obtained according to the search and matching algorithm. The target image itself is a motion detection image captured using the motion detection function, and its image contains at least one moving object. The motion detection macroblock information corresponding to this image records the information corresponding to the moving object, and the motion detection macroblock information includes its own position information.

[0086] Taking the first motion-detected image as an example, if there is a window in one location within the camera's field of view, and a Coca-Cola billboard (containing a Coca-Cola bottle) in another location, motion detection image 1 is generated after motion is detected when someone opens the window. This motion detection image 1 includes both the motion-detected location where the window-opening action occurred and the location of the Coca-Cola bottle, which is not moving. Performing the image search in step 310, motion detection image 1 is found to be the target image. However, further performing step 320, the location of the Coca-Cola bottle in the target image is identified as location 1. Performing step 330, the location indicated by the obtained motion detection macroblock information is identified as location 2. This motion detection macroblock is the macroblock corresponding to the window-opening action in the target image. Further performing step 340, it is determined that location 1 and location 2 are too far apart and do not meet the set location overlap standard; therefore, motion detection image 1 cannot be identified as the first motion-detected image.

[0087] The determination of the second motion detection image is carried out in a similar manner, and the specific details will not be repeated here.

[0088] It is understood that in some exemplary embodiments, the implementation of steps 310-340 can further improve the accuracy of selecting the first motion detection image and the second motion detection image, so that the first motion detection image and the second motion detection image are motion detection images containing the target object, thereby improving the accuracy of the parabolic plane.

[0089] In some exemplary embodiments, in step 120, the parabolic plane of the target object is constructed based on the depth information of the target object in the first motion detection image and the depth information of the target object in the second motion detection image.

[0090] As described above, when the camera generates motion-detected images, it also generates motion-detected macroblock information and depth information. For example... Figure 4 As shown, based on the depth information of the target object in these two figures, a parabolic plane for throwing the target object can be constructed. Since the object is moving downwards, the parabolic plane is perpendicular to the ground plane; since the target object can be thrown from any angle, the horizontal direction of the parabolic plane is arbitrary.

[0091] It should be noted that the high-altitude object tracking scheme provided in this disclosure is applicable when the trajectory of the object lies within a parabolic plane, i.e., it is applicable to targets with a density greater than a certain density threshold. However, for targets with lower density, such as paper scraps and leaves, their trajectories in the air are often not within a parabolic plane due to the significant influence of the environment (such as wind). Using the scheme provided in this disclosure for such targets may result in some deviation and unsatisfactory tracking performance. To achieve a more ideal tracking effect, further research is needed on corresponding solutions, but specific aspects are not within the scope of this disclosure.

[0092] It is understandable that after determining the parabolic plane, further execution of steps 130 and 140 will determine more trajectory points of the target object in order to finally determine the parabolic trajectory.

[0093] In step 130, based on a first set duration determined by considering the total time from the target object's throw to its landing, video data covering twice the first set duration is acquired. For example, if the first set duration is 10 seconds, then whether acquiring video data for 10 seconds before and after the capture time of the first motion detection image (a total of 20 seconds), or acquiring video data for 10 seconds before and after the capture time of the second motion detection image (a total of 20 seconds), the acquired video data will cover the process from the start point to the end point of the target object's appearance in the monitoring screen.

[0094] It should be noted that if the camera's shooting range covers the thrower's position and the ground, the acquired video data can cover the entire process of the target object from the throwing point to its landing. If the camera's shooting range is limited, it only covers the process from the starting point of its own shooting range to the ending point. For example, if the target object is thrown from the 16th floor, and camera 1 captures the window range from the 1st to the 20th floor, the acquired video data can cover the entire process from the throwing point to the landing. Similarly, if camera 2 only captures the window range from the 1st to the 10th floor, the acquired video data can cover the process from the target object's first appearance on the 10th floor to its landing; if camera 3 only captures the window range from the 20th to the 11th floor, the acquired video data can cover the process from the throwing point on the 16th floor to the 11th floor. Likewise, if camera 4 only captures the window range from the 1st to the 12th floor, the acquired video data can cover the process from the target object's first appearance on the 12th floor to its landing; and if camera 5 only captures the window range from the 20th to the 10th floor, the acquired video data can cover the process from the throwing point on the 16th floor to the 10th floor.

[0095] In some embodiments, if the throwing process of the target object spans two or more cameras, video data is acquired from multiple cameras separately. The trajectory of the object can be determined separately based on each video data point, and then the trajectories can be merged using a relevant scheme. Alternatively, the video data can be merged before determining the trajectory. The video data captured by the multiple cameras may have overlapping portions, and correspondingly, the determined trajectories may also have overlapping portions. More detailed trajectory merging or video merging schemes, as well as deduplication schemes during trajectory merging or video merging, are not discussed in detail in the embodiments of this application.

[0096] In some exemplary embodiments, the motion detection information is motion detection macroblock information, and correspondingly, the motion detection location is also called the motion detection macroblock location.

[0097] In some exemplary embodiments, such as Figure 5 In step 140, determining multiple motion detection positions coinciding with the parabolic plane based on the video frames contained in the video data includes:

[0098] Step 1410: Obtain multiple video frames contained in the video data according to preset rules;

[0099] Step 1420: For each video frame, perform the following steps to determine a motion detection location:

[0100] Step 1421: Based on the depth information of the parabolic plane and the depth information corresponding to the motion detection macroblock in the video frame, determine at least one motion detection macroblock in the video frame that meets the set depth overlap standard.

[0101] Step 1422: Determine the position of at least one of the motion detection macroblocks as the motion detection position.

[0102] In some exemplary embodiments, the step of obtaining multiple video frames contained in the video data according to preset rules includes one of the following:

[0103] Obtain each video frame contained in the video data;

[0104] Obtain each keyframe contained in the video data;

[0105] The video data, comprising multiple video frames, is acquired at preset time or quantity intervals.

[0106] As described above, each motion-detected image and video frame includes (RGB) image information, depth information, and motion-detection information, constituting a set of information. In some exemplary embodiments, the motion-detection information is motion-detection macroblock information, and correspondingly, each motion-detection macroblock can be assigned its (RGB) image information and depth information.

[0107] It is understandable that once the parabolic plane is determined, the depth information of the parabolic plane can be determined according to the position of the camera and relevant mathematical methods, that is, the depth of each point on the parabolic plane can be determined.

[0108] It should be noted that the video data determined in step 130 may include video footage from a period before the object is thrown, or video footage from a period after the object lands. Therefore, not every video frame determined in step 1410 can determine the motion detection location; only the video frames corresponding to the throwing process can determine the motion detection location. It should also be noted that due to configuration or detection anomalies in the relevant motion detection functions, one or more video frames determined in step 1410 may have missing motion detection information, making it impossible to determine their corresponding motion detection locations.

[0109] In some exemplary embodiments, determining at least one motion-detected macroblock in the video frame that meets a set depth overlap criterion based on the depth information of the parabolic plane and the depth information corresponding to the motion-detected macroblock in the video frame includes:

[0110] The position of the motion detection macroblock in the video frame is mapped to the parabolic plane and denoted as the parabolic plane position;

[0111] Based on the depth information of the parabolic plane position and the depth information corresponding to the dynamic detection macroblock, determine whether it meets the set depth overlap standard;

[0112] It should be noted that the size of the corresponding motion detection macroblock may vary depending on the parameters of the relevant camera. Correspondingly, a target object in a captured image may correspond to one or more motion detection macroblocks. Therefore, the motion detection macroblocks for the depth overlap standard in step 1421 can be one or more. However, it is understood that for the same target object, if a moving target object corresponds to multiple motion detection macroblocks in a motion detection image, these multiple motion detection macroblocks are in adjacent positions. Therefore, in step 1422, a motion detection position can be determined based on the position of at least one of the motion detection macroblocks.

[0113] In some exemplary embodiments, a condition is determined to meet a set depth overlap criterion if one of the following conditions is satisfied:

[0114] The difference between the depth of the parabolic plane position of the dynamic detection macroblock and the depth of the dynamic detection macroblock is less than the first depth difference threshold.

[0115] The ratio of the difference between the depth of the parabolic plane position of the dynamic detection macroblock and the depth of the dynamic detection macroblock to the depth of the parabolic plane position of the dynamic detection macroblock is less than the first depth ratio threshold.

[0116] The ratio of the difference between the depth of the parabolic plane position of the motion detection macroblock and the depth of the motion detection macroblock to the depth of the motion detection macroblock is less than the second depth ratio threshold.

[0117] It can be understood that motion detection macroblocks that meet the set depth overlap criteria are motion detection macroblocks that overlap with the parabolic plane in depth. That is, after the parabolic plane is established, the determination of more trajectory points involves searching for motion detection macroblocks that meet the requirements within the parabolic plane. This is different from using the image search and matching method described in step 110 to search and match a large number of video frames. In comparison, this significantly speeds up the matching process and improves the overall execution efficiency of the tracing scheme. Simultaneously, based on the depth overlap criteria, motion detection macroblocks that are not the target object can be eliminated, making the determined motion detection location more likely to be the target object's motion detection location, thereby ensuring the accuracy of the finally determined parabolic trajectory. For example, in a certain video frame, in addition to the target object's motion detection macroblock, there is also a motion detection macroblock caused by the swaying of leaves. However, based on the depth overlap criteria, it can be determined that the motion detection macroblock caused by the swaying leaves does not meet the depth overlap criteria; therefore, the motion detection macroblock caused by the swaying leaves will not be determined as a motion detection location.

[0118] In some exemplary embodiments, determining the parabolic trajectory of the target object based on the plurality of motion detection positions includes:

[0119] For each of the motion detection positions, a corresponding trajectory point is marked on the parabolic plane, and the parabolic trajectory of the target object is obtained by fitting multiple trajectory points.

[0120] In some exemplary embodiments, the step of fitting the target object based on a plurality of trajectory points to obtain the parabolic trajectory includes:

[0121] Multiple trajectory points are filtered to remove abnormal and boundary points, and the parabolic trajectory of the target object is obtained by fitting.

[0122] It is understood that, according to the tracing scheme provided in this disclosure, in addition to using image search for matching when determining the first and second motion detection images, step 140 is based on matching the depth information of the parabolic plane and the depth information of the motion detection macroblock. Besides effectively improving retrieval and matching efficiency, this also significantly reduces the reliance on the accuracy of image search during the determination of a large number of trajectory points. After all, the attitude of the target object in the air changes; if image search is relied upon for matching, the matching results will result in a large number of missing trajectory points, and the final fitted parabolic trajectory will be severely distorted. Therefore, the scheme provided in this disclosure can also effectively improve the accuracy of the parabolic trajectory.

[0123] In some exemplary embodiments, the method further includes:

[0124] Step 150: Using the shooting time of the video frame that determines the starting point of the parabolic trajectory as the starting point and the shooting time of the video frame that determines the ending point of the parabolic trajectory as the ending point, extract the video of the target object being thrown from the video data.

[0125] Based on the starting point of the determined parabolic trajectory and the capture time of the corresponding motion detection macroblock image, the complete video of the object being thrown, captured by the camera, is extracted. It can be understood that, compared to the traditional method of manually confirming and extracting video, this embodiment of the scheme can automatically generate the parabolic trajectory and locate and extract relevant video based on one or more images of the target object. This greatly improves the convenience of the high-altitude object throwing tracing scheme, making it easier for people to trace various potentially hazardous high-altitude object throwing behaviors, promptly hold those responsible accountable or provide criticism and education, and better warn relevant personnel.

[0126] In some exemplary embodiments, the method further includes adding static or dynamic markers to the thrown video to represent the trajectory of the parabola.

[0127] In some exemplary embodiments, the method further includes:

[0128] Step 160: Based on the pre-marked room information in the captured image, match the starting position of the parabolic trajectory to determine the room information in which the target object is thrown.

[0129] It is understandable that after the camera is calibrated, the image of the building corresponding to the captured image is basically stable, and room information can be pre-marked. This room information includes at least one of the following: floor, room number, etc. Then, after determining the parabolic trajectory, based on the position of the trajectory's starting point, a specific room can be matched according to the marked room information. This information can be provided to relevant management personnel for confirmation, criticism, education, or warning.

[0130] In some exemplary embodiments, the method further includes:

[0131] Step 170: Estimate the properties of the target object based on the parabolic trajectory;

[0132] The attributes include one or more of the following:

[0133] Falling speed, acceleration, and density.

[0134] It is understandable that, based on the parabolic trajectory determined in step 140 and combined with the time information of each point on the trajectory, the falling speed, acceleration, and / or density of the target object can be preliminarily estimated.

[0135] Furthermore, based on the attributes of these targets, the corresponding risk level can be determined according to established rules. Punitive measures can then be determined accordingly.

[0136] It should be noted that after determining the parabolic trajectory, more business functions can be expanded according to management needs, and are not limited to the aspects of this disclosure example.

[0137] This disclosure also provides a high-altitude object tracking device, such as... Figure 6 As shown, it includes:

[0138] The motion detection image acquisition module 610 is configured to search and match a first motion detection image and a second motion detection image containing the target object from the motion detection images based on the image of the target object.

[0139] The parabolic plane construction module 620 is configured to construct the parabolic plane of the target object based on the depth information of the target object in the first motion detection image and the depth information of the target object in the second motion detection image;

[0140] The video acquisition module 630 is configured to acquire video data of the home camera from a first time point before the first time point to a second time point after the first time point, based on the shooting time point and the home camera of the first motion detection image.

[0141] The parabolic trajectory determination module 640 is configured to determine multiple motion detection positions that coincide with the parabolic plane based on the video frames contained in the video data, and determine the parabolic trajectory of the target object based on the multiple motion detection positions.

[0142] The time difference between capturing the first motion detection image and the second motion detection image is less than a first set time.

[0143] This disclosure also provides an electronic device, including:

[0144] One or more processors;

[0145] Storage device for storing one or more programs.

[0146] When the one or more programs are executed by the one or more processors, the one or more processors implement the high-altitude object tracing method as described in any embodiment of this disclosure.

[0147] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, the program being implemented by a processor as the high-altitude object tracing method described in any embodiment of this disclosure.

[0148] As can be seen, the high-altitude object tracking scheme provided in this disclosure searches for at least two motion detection images in historical motion detection images. Based on the relevant information of these at least two motion detection images, the parabolic plane is determined. Furthermore, motion detection information from relevant video frames is matched within the parabolic plane to determine more motion detection locations (trajectory point locations) to fit and determine the parabolic trajectory. This significantly improves search and matching efficiency and overcomes trajectory tracking distortion that may be caused by the changing attitude of the target object in the air. Compared to manual verification, it greatly improves work efficiency and accuracy. It can also be seen that the automated parabolic trajectory determination scheme provided in this disclosure can be further expanded to include more intelligent functions, such as prediction of the room from which the object is thrown and assessment of the risk factor of the object being thrown. This makes tracking more convenient, faster, and more accurate, and can help management departments carry out management work more effectively.

[0149] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0150] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made under the concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for tracing objects thrown from heights, characterized in that, include: Based on the image of the target object, search and match the first and second motion detection images containing the target object from the motion detection images; Based on the depth information of the target object in the first motion detection image and the depth information of the target object in the second motion detection image, a parabolic plane of the target object is constructed; Based on the shooting time and the camera to which the first motion-detected image is located, video data of the camera to which the image is located is obtained from a first time point before the shooting time point to a second time point after the shooting time point. Based on the video frames contained in the video data, determine multiple motion detection positions that coincide with the parabolic plane, and determine the parabolic trajectory of the target object based on the multiple motion detection positions; The time difference between capturing the first motion detection image and the second motion detection image is less than a first set time; the parabolic plane is perpendicular to the ground.

2. The method as described in claim 1, characterized in that, The step of searching and matching a first motion detection image and a second motion detection image containing the target object from the motion detection images, based on the image of the target object, includes: Based on the image of the target object, a first motion detection image containing the target object is searched and matched from the motion detection images; Based on the shooting time T of the first motion detection image and the assigned camera, a second motion detection image containing the target object is searched and matched from the motion detection images generated by the assigned camera between the first time point Tt and the second time point T+t; where t is the first set time duration.

3. The method as described in claim 2, characterized in that, The first motion detection image or the second motion detection image is obtained by searching and matching from motion detection images within the corresponding search range according to the following method: Based on the image of the target object, a target image containing the target object is obtained by searching and matching from the motion-detected images within the search range; Obtain the location information of the target object in the target image; Obtain the motion detection macroblock information corresponding to the target image; If the location information of the target object and the motion detection macroblock information meet the set location overlap standard, the target image is determined to be either the first motion detection image or the second motion detection image.

4. The method as described in claim 1, characterized in that, The step of determining multiple motion detection positions coinciding with the parabolic plane based on the video frames contained in the video data includes: The video data contains multiple video frames according to preset rules; For each video frame, perform the following steps to determine a motion detection location: Based on the depth information of the parabolic plane and the depth information corresponding to the motion detection macroblock in the video frame, at least one motion detection macroblock in the video frame that meets the set depth overlap standard is determined. The position of at least one of the motion detection macroblocks is determined as the motion detection position.

5. The method as described in claim 1 or 4, characterized in that, Determining the parabolic trajectory of the target object based on the plurality of motion detection positions includes: marking corresponding trajectory points on the parabolic plane for each motion detection position, and fitting the parabolic trajectory of the target object based on the plurality of trajectory points.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: Using the shooting time of the video frame that determines the starting point of the parabolic trajectory as the starting point and the shooting time of the video frame that determines the ending point of the parabolic trajectory as the ending point, the video of the target object being thrown is extracted from the video data.

7. The method according to any one of claims 1-4, characterized in that, The method further includes: Based on the pre-marked room information in the captured image, the starting position of the parabolic trajectory is matched to determine the room information in which the target object is thrown.

8. The method according to any one of claims 1-4, characterized in that, The method further includes: Estimate the properties of the target object based on the parabolic trajectory; The attributes include one or more of the following: Falling speed, acceleration, and density.

9. A high-altitude object tracking device, characterized in that, include: The motion detection image acquisition module is configured to search and match a first motion detection image and a second motion detection image containing the target object from the motion detection images based on the image of the target object. The parabolic plane construction module is configured to construct the parabolic plane of the target object based on the depth information of the target object in the first motion detection image and the depth information of the target object in the second motion detection image; The video acquisition module is configured to acquire video data of the home camera from a first time point before the first time point to a second time point after the first time point, based on the shooting time point and the home camera of the first motion detection image. The parabolic trajectory determination module is configured to determine multiple motion detection positions that coincide with the parabolic plane based on the video frames contained in the video data, and determine the parabolic trajectory of the target object based on the multiple motion detection positions; The time difference between capturing the first motion detection image and the second motion detection image is less than a first set time; the parabolic plane is perpendicular to the ground.

10. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the high-altitude object tracing method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the high-altitude object tracing method as described in any one of claims 1-8.

Images