A target detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a target detection method and device, electronic equipment and storage medium. The method comprises: obtaining a depth image by image acquisition of a building through a depth camera, and determining position information of a target in the depth image; determining a position relationship between the target and a predicted falling area according to the position information of the target; wherein the predicted falling area is an area passed through by a predicted object when the predicted object is thrown from the building; and if the target is located in the predicted falling area, it is determined that the target is a target thrown from the building. In the above technical solution, the interference of interfering objects in the environment on target detection can be reduced, and the problem of misjudgment of high-terror throwing objects can be avoided, thereby improving the accuracy and real-time performance of detecting the behavior of throwing objects from the building.

Description

Technical Field

[0001] This application relates to the field of security technology, and in particular to a target detection method, device, electronic device, and storage medium. Background Technology

[0002] Throwing objects from high-rise buildings is known as a "pain hanging over the city," and this behavior has always been a major concern due to the significant social harm it causes. Managing and addressing this issue is crucial to social stability and the safety of the public.

[0003] Currently, detection methods for objects thrown from heights are significantly affected by the environment. For example, if the target in the monitoring footage is a person or object moving inside a building, it will affect the detection accuracy. Furthermore, current detection methods require real-time comparison of multiple image frames, which places a significant strain on the processor. Summary of the Invention

[0004] This application provides a target detection method, apparatus, electronic device, and storage medium to improve the accuracy of detecting targets thrown from high altitudes.

[0005] In one embodiment, this application provides a target detection method, including:

[0006] Depth images are obtained by capturing images of buildings using a depth camera, and the location information of targets in the depth images is determined.

[0007] Based on the target's location information, the positional relationship between the target and the predicted landing area is determined; wherein, the predicted landing area is the area that the predicted object will pass through when it is thrown from the building;

[0008] If the target is located within the predicted landing area, then the target is determined to be a target thrown from the building.

[0009] In one embodiment, this application provides a target detection device, including:

[0010] The location information determination module is used to acquire depth images of buildings through a depth camera and determine the location information of targets in the depth images;

[0011] The positional relationship determination module is used to determine the positional relationship between the target and the predicted landing area based on the target's positional information; wherein, the predicted landing area is the area that the predicted object will pass through when it is thrown from the building;

[0012] The target determination module is used to determine that if the target is located within the predicted landing area, the target is a target thrown from the building.

[0013] In one embodiment, this application provides an electronic device, including:

[0014] One or more processors;

[0015] Storage device for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the target detection method as described in any of the above embodiments.

[0017] In one embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the target detection method as described in any of the above embodiments.

[0018] This application provides a target detection method, apparatus, electronic device, and storage medium. The implementation involves acquiring depth images of a building using a depth camera and determining the location information of a target within the depth image. Based on the target's location information, the positional relationship between the target and a predicted landing area is determined. The predicted landing area is the area predicted to pass through when an object is thrown from the building. If the target is located within the predicted landing area, it is determined to be a target thrown from the building. This technical solution, by determining the positional relationship between the target and the predicted landing area, enables target detection in three-dimensional space, reducing interference from other objects in the environment and thus improving the accuracy and real-time performance of target detection. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A flowchart of a target detection method provided in one embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the predicted landing area provided in one embodiment of this application;

[0022] Figure 3 A flowchart of another target detection method provided in the embodiments of this application;

[0023] Figure 4 This is a schematic diagram of the image area of ​​a building provided in an embodiment of this application;

[0024] Figure 5A flowchart of yet another target detection method provided in the embodiments of this application;

[0025] Figure 6 This is a schematic diagram of the floor plan of a building provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of a high-altitude object being thrown, provided as an embodiment of this application.

[0027] Figure 8 This is a schematic diagram of the structure of a target detection device provided in one embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0029] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, the embodiments and features described herein can be combined with each other unless otherwise specified. It should also be noted that, for ease of description, only the parts relevant to the present application are shown in the accompanying drawings, not the entire structure.

[0030] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0031] To better understand the embodiments of this application, the relevant technologies are described below.

[0032] Figure 1 This is a flowchart illustrating a target detection method provided in an embodiment of this application. This embodiment is applicable to situations involving target detection. Typically, this embodiment is applicable to determining whether a target in a depth image is a projectile thrown from a height. Specifically, this target detection method can be executed by a target detection device, which can be implemented through software and / or hardware and integrated into an electronic device.

[0033] like Figure 1 As shown, the method specifically includes the following steps:

[0034] S110. Obtain a depth image of the building by using a depth camera, and determine the location information of the target in the depth image.

[0035] The target can be any object appearing in the depth image. The target can be obtained by filtering based on the depth information of each pixel in the depth image. For example, preliminary filtering can be performed based on the depth information of each pixel in the depth image to determine the depth values ​​of pixels belonging to stationary objects. Specifically, the depth values ​​of each pixel in at least two frames of depth images captured by the depth camera are analyzed. If the depth values ​​of pixels do not change in at least two frames of depth images, or in depth images captured within a preset time period, the area containing the pixels whose depth values ​​have not changed is determined to be the area where the stationary object is located, and the depth value of the corresponding pixel in the area where the stationary object is located is recorded. In the actual detection process, the object corresponding to the pixel whose depth value is less than the recorded depth value is determined as the target. For example, for a pixel (20, 10) in the depth image, if the depth value of the pixel does not change in at least two frames of depth images, or in depth images captured within a preset time period, and remains at 20, the depth value of the pixel in the area where the stationary object is located is recorded as 20. In the currently acquired depth image, if the depth value of this pixel becomes 15, then this pixel can be identified as a target appearing in front of the building. The region composed of pixels that meet the above conditions is identified as the target. Specifically, for multiple pixels in the depth image that meet the above conditions, a clustering algorithm can be used to cluster them, treating each cluster as a target, thereby distinguishing multiple targets. This scheme can detect situations where a target is thrown from a building. The depth camera can be placed around the building, for example, in front of, behind, or to the side of the building. In this embodiment, the depth camera can be placed on the ground or close to the ground to make the difference between the depth value of the building and the depth value of the ground in the depth image larger, facilitating the identification of the building. One relative position of the depth camera and the building is as follows: Figure 2 As shown, Figure 2 This illustrates a scenario where a depth camera is positioned on the front of a building, capturing an image of the building from the front. If an object is thrown from the building, for example... Figure 2 When a bottle is thrown from a building, a depth camera can capture a depth image containing the thrown object, and then determine whether the object was thrown from the building based on the depth image.

[0036] A depth camera captures images of a building, obtaining a depth image. The depth image can include the building and objects in the background area outside the building. The depth image can determine the coordinates of a target in a three-dimensional coordinate system with the depth camera as the origin. The depth image in the xoy plane is a two-dimensional image, with the z-axis value being the depth value. In this embodiment, for ease of understanding, a coordinate transformation can be performed, converting the coordinates with the depth camera as the origin to coordinates with a point on the building as the origin. For example, the foot of the perpendicular line drawn from the depth camera to the building is used as the origin, and the point on the foot of the perpendicular closest to the depth camera is used as the origin to establish the coordinate system. Figure 2 As shown. The target's position information can be determined based on its coordinates in a three-dimensional coordinate system. Methods that acquire planar images using an image acquisition device can only determine the target's planar coordinates, making it difficult to determine the target's specific position in three-dimensional space. In this embodiment, a depth camera acquires depth images, and then the target's position information in three-dimensional space within the depth images is determined, thereby more accurately determining the target's location and improving the accuracy of subsequent target motion detection.

[0037] S120. Determine the positional relationship between the target and the predicted landing area based on the target's position information; wherein, the predicted landing area is the area that the predicted object will pass through when it is thrown from the building.

[0038] For example, the area that an object would pass through if thrown from a building can be predicted in advance, and that area can be used as the predicted landing area. Figure 2 As shown, if an object is thrown from a building and falls along a parabolic trajectory, the region encompassing the entire parabolic trajectory of the object's fall is defined as the predicted landing area. Since the object can be thrown from any position on the building, the area in front of the depth camera on the side of the building perpendicular to the ground can also be considered as the predicted landing area. Figure 2 The area enclosed by the dashed line in the diagram. The predicted landing area in the x and y directions can be determined based on the plane of the building perpendicular to the ground, while the area in the z direction can be determined based on the actual situation, such as the value of the landing point of the object when it is thrown from the building in the z direction.

[0039] Specifically, the three-dimensional coordinate range of the predicted landing area in the coordinate system is determined, namely the coordinate range in the x-direction, y-direction, and z-direction. Based on the target's position information, i.e., the target's three-dimensional coordinates in the coordinate system, it is determined whether the target's three-dimensional coordinates are within the three-dimensional coordinate range of the predicted landing area, thereby determining the positional relationship between the target and the predicted landing area.

[0040] S130. If the target is located within the predicted landing area, then the target is determined to be a target thrown from the building.

[0041] For example, if there are suspended or moving objects between the building and the depth camera, or if there are moving objects inside the building, these objects may affect the accuracy of target detection. For instance, if an object falls inside the building, its trajectory may be from top to bottom, easily misidentified as a high-altitude object thrown from a height. In this embodiment, based on the positional relationship between the target and the predicted landing area, it is more accurately determined in three-dimensional space whether the target was thrown from the building, avoiding interference from moving objects inside the building or other moving objects in the external environment. In this embodiment, if the target is located within the predicted landing area, the target is determined to be a target thrown from the building.

[0042] Specifically, if the target is located within the predicted landing area, for example... Figure 2 In this example, if the target is a bottle and it is located within the predicted landing area enclosed by dashed lines, then the target is determined to have been thrown from a building. If the target is located outside the predicted landing area, then the target is determined not to have been thrown from a building. Compared to methods that only detect changes in target height to determine if it was thrown from a building, the solution in this application can eliminate interference from other objects with varying heights that were not actually thrown from a building, thus improving the accuracy of target detection. Furthermore, methods that detect changes in target height in adjacent frames in real time to determine if the target was thrown from a building require a large amount of real-time image recognition processing and calculation, resulting in a large processing load and reduced detection efficiency. In contrast, this solution only needs to determine the positional relationship between the target and the predicted landing area based on the target's positional information. Determining the positional relationship requires less processing, and only one frame of depth image needs to be processed, thus improving the efficiency of target detection.

[0043] In this embodiment of the application, S130 is further optimized. If the target is located within the predicted landing area, the target is determined to be a target thrown from the building. This includes: if the target is located within the predicted landing area, and the height value of the target is determined to decrease based on at least two frames of depth images of the target, then the target is determined to be a target thrown from the building.

[0044] Specifically, based on the above scheme, to further verify that the target was thrown from the building, at least two depth images containing the target can be acquired, and the target's height change can be determined based on these two depth images. For example, in the previous depth image, targets with depth values ​​less than the pixel depth value corresponding to stationary objects are identified, and their specific positions in the depth image are determined. In the next depth image, targets with depth values ​​less than the pixel depth value corresponding to stationary objects are again identified, and their specific positions in the depth image are determined. If the target's specific position in the next depth image is below the target's specific position in the previous depth image, or if the target's specific position in the next depth image is below the target's specific position in the previous depth image and is on the same vertical line, and after analysis of multiple depth images, the target's specific position consistently exhibits the above relationship, then the target's height value is determined to be gradually decreasing. If the target is located within the predicted landing area and the target's altitude value gradually decreases, it is determined that the target is falling within the predicted landing area, and the target is identified as a target thrown from the building. This eliminates interference from objects that are within the predicted landing area but are not falling, such as birds flying near the building within the predicted landing area, thereby improving the accuracy of target detection.

[0045] This application provides a target detection method, apparatus, electronic device, and storage medium. The implementation involves acquiring depth images of a building using a depth camera and determining the location information of a target within the depth image. Based on the target's location information, the positional relationship between the target and a predicted landing area is determined. The predicted landing area is the area predicted to pass through when an object is thrown from the building. If the target is located within the predicted landing area, it is determined that the target was thrown from the building. This technical solution, by determining the positional relationship between the target and the predicted landing area, enables target detection in three-dimensional space, reducing interference from other objects in the environment and thus improving the accuracy and real-time performance of target detection.

[0046] Figure 3 This is a flowchart of another target detection method provided in this application embodiment. This embodiment is an optimization based on the above embodiments. It should be noted that technical details not described in detail in this embodiment can be found in any of the above embodiments.

[0047] Specifically, such as Figure 3 As shown, the method specifically includes the following steps:

[0048] S210. Obtain a depth image of the building by using a depth camera, and determine the location information of the target in the depth image.

[0049] S220. If the pixel coordinates of the target are located within the image area of ​​the building in the depth image, and the depth value of the target is within the depth value range of the predicted landing area, then the target is determined to be located within the predicted landing area.

[0050] The location information includes the pixel coordinates and depth value of the target. The pixel coordinates and depth value of the target can be the pixel coordinates and depth value of the target's center pixel. The image region of the building is the region composed of the pixels corresponding to the building in the depth image, such as... Figure 4 As shown, Figure 4 The closed region in the image represents the building's image region. This region can be determined by filtering based on the depth values ​​of each pixel in the depth image. The predicted depth range for the landing area can be determined based on the specific circumstances. For example, assuming an object is thrown from the building, the predicted landing depths are calculated for different objects thrown, and / or for the same object thrown with different forces. The predicted landing depth range is then determined based on the maximum and minimum landing depth values. If the target's pixel coordinates are located within the building's image region, for example... Figure 4 If the target's depth value falls within the predicted landing area's depth range, then the target is determined to be within the predicted landing area. If the target's depth value does not fall within the predicted landing area's depth range, then the target is determined to be located elsewhere and be visible to the depth camera.

[0051] Specifically, in this embodiment, if a target is identified based on the depth image, it can first be determined whether the pixel coordinates of the target in the depth image are located within the image area of ​​the building. If so, it is further determined whether the depth value of the target is within the depth value range of the predicted landing area to exclude interference from objects inside the building or objects suspended or flying outside the building. If not, it can be determined that the target is not a target thrown from the building, and the subsequent depth value comparison operation is no longer performed, thereby reducing the processing load and improving the target detection efficiency.

[0052] S230. If the target is located within the predicted landing area, then the target is determined to be a target thrown from the building.

[0053] The target detection method provided in this application is an optimization based on the above embodiments. If the pixel coordinates of the target are located within the image area of ​​the building in the depth image, and the depth value of the target is within the depth value range of the predicted landing area, then the target is determined to be located within the predicted landing area. Thus, by combining the target's position on the image plane and its depth value, the method can more comprehensively determine whether the target is a target thrown from the building, thereby improving the accuracy and efficiency of target detection.

[0054] Figure 5 This is a flowchart illustrating another target detection method provided in this application embodiment. This embodiment is an optimization based on the above embodiments. It should be noted that technical details not described in detail in this embodiment can be found in any of the above embodiments.

[0055] Specifically, such as Figure 5 As shown, the method specifically includes the following steps:

[0056] S310. Obtain depth images by acquiring images of buildings using a depth camera.

[0057] S320. Determine the image region of the building in the depth image based on the gradient value of the depth values ​​between adjacent pixels in the depth image.

[0058] For example, in addition to the area where the building is located, the depth image may also include background areas, such as the sky, trees, roads, etc. In this embodiment, the edges of the building are detected based on the depth image, and the area enclosed by the edges is taken as the image area of ​​the building, so as to facilitate subsequent determination of whether the target is a target thrown from the building.

[0059] Specifically, the distance from the building to the depth camera differs from the distance from the background to the depth camera; that is, the depth value of the building differs from the depth value of the background. Therefore, the edges of the building can be determined based on the gradient values ​​of the depth values, thereby defining the image region of the building. For example, in a depth image, the depth value of the edge of the building facing the camera exhibits a significant abrupt change compared to the depth value of the surrounding background. Therefore, the gradient values ​​of the depth values ​​between adjacent pixels in the depth image are determined, and pixels belonging to the building's edge are selected based on these gradient values. The area enclosed by these edge pixels is then taken as the image region of the building. Figure 4 As shown, Figure 4 The closed region in the image is the image area of ​​the building.

[0060] Specifically, the depth image can be filtered to improve its smoothness, remove noise points, and eliminate pixels with large abrupt changes in depth value. For example, if a pixel in the depth image has a depth value greater than or less than the depth values ​​of its surrounding pixels, and the difference between its depth value and the surrounding pixels is significant, then this pixel is identified as a noise point and will not be analyzed further. For instance, if a pixel has a depth value of 40, and within a 9×9 area centered on this pixel, all other pixels have a depth value of 10, or a depth value within the range of [10-2, 10+2], then this pixel is identified as a noise point and will not be analyzed further. For each pixel in the depth image, the gradient value of the depth values ​​of its neighboring pixels is determined and compared with a pre-set gradient value threshold. In this embodiment, adjacent pixels are defined as pixel pairs. A small gradient threshold is preset. The gradient values ​​of the depth values ​​of adjacent pixels in the depth image are compared with the preset gradient threshold. Pixel pairs with gradient values ​​greater than the threshold are selected. The gradient threshold is then increased, and the gradient values ​​of the selected pixel pairs are compared with the increased gradient threshold. Pixel pairs with gradient values ​​greater than the increased gradient threshold are selected again. This process continues until the selected pixel pairs can form polygons, and the ratio of the polygon's area to the depth image area is greater than a preset area threshold. Specifically, when setting up the depth camera, the position and angle of the depth camera are adjusted so that the proportion of the building image area in the depth image captured by the depth camera is greater than the preset area threshold. This allows for comprehensive monitoring of objects thrown from buildings. When identifying the building image area based on the pixel depth values, if the selected pixels can form polygons, and the ratio of the polygon's area to the depth image area is greater than the preset area threshold, then the area corresponding to the polygon is determined to be the building image area, thus eliminating interference from polygonal areas such as balconies and windows. In another feasible approach, for each pixel in the depth image, the gradient values ​​of the depth values ​​of adjacent pixels are determined. The pixel pair with the largest gradient value is selected, and it is determined whether this pair can form a polygon. If not, the pixel pair with the second largest gradient value is selected, and it is determined whether this pair can form a polygon. This process is repeated until a selected pixel pair can form a polygon. In this approach, although theoretically the pixel pair with the largest gradient value can be considered the pixel pair at the edge of the building, in reality, due to acquisition errors in the depth image, the depth values ​​of pixels may be inaccurate. Therefore, the pixel pair with the largest gradient value alone may not be sufficient to form a polygon. Thus, in the above approach, the second largest gradient value is combined with the selected pixel pair, or the pixel pair with a further reduced gradient value, to form a polygon, thereby determining the image region of the building.The gradient values ​​are used to filter adjacent pixels, which are pixel pairs. Pixels closer to the center of the polygon can be used as edge pixels of the building, and the area surrounded by the edge pixels can be used as the image area of ​​the building.

[0061] S330. Based on the gradient values ​​of the depth values ​​of adjacent pixels in the area to be detected within the image region of the building, and the average gradient of the depth values ​​of adjacent pixels belonging to the same plane on the building in the depth image, determine the planar regions in the image region of the building that correspond to different planes on the building.

[0062] The region to be detected is the area in the image that has not yet been identified as a planar region corresponding to a different plane on the building. For example, such as... Figure 4 As shown, if no planar region has been identified yet, the entire image area of ​​the building is used as the region to be detected. Figure 6 As shown, if planar region A has been identified, then the region in the image area other than planar region A is taken as the region to be detected.

[0063] For example, such as Figure 2 As shown, not all walls of a building may be parallel to the xoy plane; some may be perpendicular to the xoz plane and have an angle with the xoy plane. The predicted landing area differs for walls facing different directions within the building. In this embodiment, based on the depth values ​​of pixels in the area to be detected within the building's range and the average gradient value of those depth values, planar regions corresponding to different planes on the building are identified within the building's image region. For example, in... Figure 4 The building image regions obtained from the image are identified to obtain various planar regions, such as... Figure 6 As shown, three planar regions, A, B, and C, are identified. Specifically, the gradient values ​​of the depth values ​​of adjacent pixels within a plane are the same. The average gradient of the depth values ​​of pixels belonging to the same plane can be determined. Based on the average gradient, pixels within the plane are filtered to obtain pixels belonging to the same plane, thereby identifying the planar regions in the image region of the building that correspond to different planes on the building.

[0064] In this embodiment, the process of determining the average gradient in S330 is optimized. The process of determining the average gradient includes: determining the vertical average gradient of the area to be detected based on the depth values ​​of the midpoints of the upper and lower edges of the area to be detected within the image region of the building, and the distance between the midpoints of the upper and lower edges; determining the horizontal average gradient based on the average gradient values ​​of the right and left adjacent pixels of the pixels on the middle line of the area to be detected; wherein the middle line is the line connecting the midpoints of the upper and lower edges of the area to be detected; and determining the average gradient based on the vertical and horizontal average gradients. For example, as shown... Figure 4 As shown, the entire area of ​​the building is taken as the detection region. Point A is the midpoint of the upper edge of the detection region, and point B is the midpoint of the lower edge of the detection region. The ratio of the absolute value of the difference between the depth values ​​of point A and point B to the pixel distance between points A and B is used as the vertical average gradient. Here, the pixel distance is the distance between the pixel corresponding to point A and the pixel corresponding to point B. For example, the pixel corresponding to point A is (x... a y a The pixel corresponding to point B is (x b y b If ), then the pixel distance between point A and point B is For the line connecting points A and B, calculate the difference between the depth values ​​of the right-hand and left-hand adjacent pixels of each pixel along the AB line. Use this difference as the gradient value for each pixel along the AB line. Then, calculate the average of these gradient values ​​as the horizontal average gradient. Finally, sum the vertical average gradient and the horizontal average gradient to obtain the average gradient.

[0065] In this embodiment, S330 is further optimized. Based on the gradient values ​​of the depth values ​​of adjacent pixels in the area to be detected within the image region of the building, and the average gradient of the depth values ​​of adjacent pixels belonging to the same plane on the building in the depth image, the planar regions in the image region of the building corresponding to different planes on the building are determined. This includes: subtracting a preset adjustment parameter from the average gradient to obtain the first endpoint of the gradient range, and adding the preset adjustment parameter to the average gradient to obtain the second endpoint of the gradient range, thereby determining the gradient range; taking the area composed of adjacent pixels whose gradient values ​​are within the gradient range as the planar regions in the image region of the building corresponding to the same plane on the building; taking other areas in the image region as new areas to be detected, determining the average gradient and gradient range in the new areas to be detected, and determining the planar regions in the areas to be detected corresponding to the same plane on the building, until the image region of the building is divided into planar regions corresponding to different planes on the building.

[0066] The preset adjustment parameter can be determined based on actual conditions, representing the acceptable error range. Specifically, assuming the average gradient is T, the preset adjustment parameter is k, and the gradient range is determined to be (Tk, T+k). If the gradient value is within the gradient range (Tk, T+k), then adjacent pixels are determined to be on the same plane of the building. Therefore, the region consisting of adjacent pixels whose gradient values ​​are within the gradient range is taken as the planar region corresponding to the same plane on the building in the image region of the building. This scheme is used to determine the planar regions belonging to the same plane on the building in the area to be detected. The areas in the image region of the building that have not yet been determined as planar regions on the same plane on the building are taken as new areas to be detected, and the above planar recognition steps are continued until the image region of the building is divided into planar regions of different planes, such as... Figure 6 As shown, the image region is divided into planar region A, planar region B, and planar region C.

[0067] S340. Determine the predicted landing area based on the planar region and the depth value range.

[0068] The depth range is determined based on the distance between the landing point of the predicted target and the building when the target is thrown from the building.

[0069] For example, an object thrown from a building falls along a parabolic trajectory. When the depth camera captures the target being thrown from the building, the target's pixel coordinates should be within the building's image area. There is a certain distance between the landing point and the building, which can be used to determine a preset depth range. Specifically, experiments can be conducted beforehand with different objects. Under a safe environment where no people or objects are in front of the building, different objects are thrown from the building, and the landing point positions are recorded. The depth value of the landing point relative to the depth camera is determined based on the landing point positions. Alternatively, under a safe environment where no people or objects are in front of the building, different objects or the same object are thrown from the building with different forces, and the landing point positions are recorded. The depth value of the landing point relative to the depth camera is determined based on the landing point positions. The minimum depth value is used as the lower limit of the depth range, and the maximum depth value is used as the upper limit of the depth range, thus obtaining the depth range. Therefore, based on the planar area, the predicted landing area can be determined in the x-direction and y-direction, and the depth range can be determined in the z-direction, thereby determining the predicted landing area.

[0070] In this embodiment, S340 is further optimized by determining the predicted landing area based on the planar region and the depth value range, including: taking the region whose contour from the depth camera's viewpoint coincides with the contour of the at least one planar region and whose depth value meets the depth value range as the predicted landing area.

[0071] For example, such as Figure 6 As shown, assuming the planar regions from the depth camera's perspective are planar regions A, B, and C, the contour of at least one of these planar regions can be determined as the contour of the predicted landing area from the depth camera's perspective. Combined with the depth value range, the predicted landing area can then be determined. Figure 7 As shown, the outline of the closed region pointed to by the arrow corresponding to the maximum depth value of the predicted landing area or the outline of the closed region pointed to by the arrow corresponding to the minimum depth value of the predicted landing area is the outline of the predicted landing area. The area between the closed region pointed to by the arrow corresponding to the maximum depth value of the predicted landing area and the closed region pointed to by the arrow corresponding to the minimum depth value of the predicted landing area is the predicted landing area.

[0072] S350, Determine the location information of the target in the depth image.

[0073] S360. Determine the positional relationship between the target and the predicted landing area based on the target's position information; wherein, the predicted landing area is the area that the predicted object will pass through when it is thrown from the building.

[0074] S370. If the target is located within the predicted landing area, then the target is determined to be a target thrown from the building.

[0075] For example, such as Figure 7 As shown, if a target appears within the predicted landing area and its altitude decreases, indicating a downward trend, then the target is determined to be a target thrown from a building. In this embodiment, to improve the accuracy of target detection, the target's position information in each depth image can be determined based on at least two depth images, thereby determining the target's descent trajectory. If the descent trajectory satisfies a parabolic trajectory, then the target is determined to be a target thrown from a building.

[0076] This application provides a target detection method that optimizes upon the above embodiments. It determines the image region of a building in the depth image based on the gradient values ​​of depth values ​​between adjacent pixels in the depth image. It then determines planar regions in the image region of the building corresponding to different planes on the building based on the gradient values ​​of depth values ​​of adjacent pixels in the region to be detected within the building's image region, and the average gradient of depth values ​​of adjacent pixels belonging to the same plane on the building in the depth image. The region to be detected is the region in the image region that has not yet been determined to correspond to a planar region on the building. Based on the planar regions and the depth value range, a predicted landing region is determined. This allows for accurate and efficient automatic determination of planes within the building based on the depth image, and further determination of the predicted landing region to detect whether the target is a target thrown from the building. The entire process is automated, improving detection efficiency. By establishing a predicted landing region to determine whether the target is a target thrown from the building, interference from moving objects within the building is eliminated, improving the accuracy of target detection.

[0077] Figure 8 This is a schematic diagram of a target detection device provided in one embodiment of this application. The target detection device provided in this embodiment includes:

[0078] The location information determination module 410 is used to acquire a depth image of a building by using a depth camera, and to determine the location information of a target in the depth image;

[0079] The position relationship determination module 420 is used to determine the position relationship between the target and the predicted landing area based on the target's position information; wherein, the predicted landing area is the area that the predicted object will pass through when it is thrown from the building;

[0080] The target determination module 430 is used to determine that the target is a target thrown from the building if the target is located within the predicted landing area.

[0081] In this embodiment of the application, the location information includes the pixel coordinates and depth value of the target;

[0082] Position relationship determination module 420 includes:

[0083] The comparison unit is configured to determine that the target is located within the predicted landing area if the pixel coordinates of the target are located within the image area of ​​a building in the depth image and the depth value of the target is within the depth value range of the predicted landing area.

[0084] In this embodiment of the application, the device further includes:

[0085] The image region determination module is used to determine the image region of a building in the depth image based on the gradient value of the depth values ​​between adjacent pixels in the depth image;

[0086] The planar region determination module is used to determine the planar regions in the image region of a building that correspond to different planes on the building, based on the gradient values ​​of the depth values ​​of adjacent pixels in the region to be detected within the image region of the building, and the average gradient of the depth values ​​of adjacent pixels belonging to the same plane on the building in the depth image; wherein, the region to be detected is the region in the image region that has not yet been determined as a planar region corresponding to a different plane on the building.

[0087] The landing area determination module is used to determine the predicted landing area based on the planar area and the depth value range; wherein, the depth value range is determined based on the distance between the landing point of the predicted target and the building when it is thrown from the building.

[0088] In this embodiment of the application, the device further includes:

[0089] The vertical average gradient determination module is used to determine the vertical average gradient of the area to be detected based on the depth values ​​of the midpoint of the upper edge and the midpoint of the lower edge of the area to be detected within the image area of ​​the building, as well as the distance between the midpoints of the upper and lower edges.

[0090] The horizontal average gradient determination module is used to determine the horizontal average gradient based on the average gradient values ​​of the right-side adjacent pixels and the left-side adjacent pixels of the pixels on the middle line of the region to be detected; wherein, the middle line is the line connecting the midpoint of the upper edge and the midpoint of the lower edge of the region to be detected.

[0091] The average gradient determination module is used to determine the average gradient based on the vertical average gradient and the horizontal average gradient.

[0092] In this embodiment of the application, the building plan determination module includes:

[0093] The gradient range determination unit is used to subtract a preset adjustment parameter from the average gradient to obtain the first endpoint of the gradient range, and add the preset adjustment parameter to the average gradient to obtain the second endpoint of the gradient range, thereby determining the gradient range.

[0094] A plane determination unit is used to define the region consisting of adjacent pixels whose gradient values ​​are within the gradient range as a planar region in the image region of a building that corresponds to the same plane on the building.

[0095] The loop execution unit is used to take other regions in the image region as new regions to be detected, determine the average gradient and gradient range in the new regions to be detected, and determine the planar regions in the regions to be detected that correspond to the same plane on the building, until the image region of the building is divided into planar regions corresponding to different planes on the building.

[0096] In this embodiment of the application, the landing area determination module is specifically used for:

[0097] The region whose contour from the depth camera's view coincides with the contour of at least one planar region, and whose depth value meets the depth value range, is designated as the predicted landing area.

[0098] In this embodiment of the application, the target determination module 430 is specifically used to determine that the target is a target thrown from the building if the target is located within the predicted landing area and the height value of the target is determined to decrease based on at least two frames of depth images of the target.

[0099] The target detection device provided in this application embodiment can be used to execute the target detection method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0100] Figure 9 This is a schematic diagram of an electronic device provided according to one embodiment of this application. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, user equipment, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0101] like Figure 9 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks and wireless networks.

[0103] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as object detection methods.

[0104] In some embodiments, the target detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the target detection method by any other suitable means (e.g., by means of firmware).

[0105] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0107] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 10, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device 10. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0109] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0110] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0111] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0112] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A target detection method, characterized in that, The method includes: Depth images are obtained by capturing images of buildings using a depth camera, and the location information of targets in the depth images is determined. Based on the target's location information, the positional relationship between the target and the predicted landing area is determined; wherein, the predicted landing area is the area that the predicted object will pass through when it is thrown from the building; If the target is located within the predicted landing area, then the target is determined to be a target thrown from the building; The process of determining the predicted landing area includes: The image region of the building in the depth image is determined based on the gradient value of the depth values ​​between adjacent pixels in the depth image; Based on the gradient values ​​of the depth values ​​of adjacent pixels in the area to be detected within the image region of the building, and the average gradient of the depth values ​​of adjacent pixels belonging to the same plane on the building in the depth image, the planar regions in the image region of the building corresponding to different planes on the building are determined; where the area to be detected is the area in the image region that has not yet been determined to be a planar region corresponding to a different plane on the building. The predicted landing area is determined based on the planar region and the depth range; wherein the depth range is determined based on the distance between the predicted target's landing point and the building when it is thrown from the building.

2. The method according to claim 1, characterized in that, The location information includes the pixel coordinates and depth value of the target; Determining the positional relationship between the target and the predicted landing area based on the target's location information includes: If the pixel coordinates of the target are located within the image area of ​​the building in the depth image, and the depth value of the target is within the depth value range of the predicted landing area, then the target is determined to be located within the predicted landing area.

3. The method according to claim 1, characterized in that, The process of determining the average gradient includes: The vertical average gradient of the area to be detected is determined based on the depth values ​​of the midpoints of the upper and lower edges of the area to be detected within the image region of the building, as well as the distance between the midpoints of the upper and lower edges. The horizontal average gradient is determined based on the average gradient values ​​of the right and left adjacent pixels of the pixel point on the middle line of the region to be detected; wherein, the middle line is the line connecting the midpoint of the upper edge and the midpoint of the lower edge of the region to be detected. The average gradient is determined based on the vertical average gradient and the horizontal average gradient.

4. The method according to claim 1, characterized in that, Based on the gradient values ​​of depth values ​​of adjacent pixels in the area to be detected within the image region of the building, and the average gradient of depth values ​​of adjacent pixels belonging to the same plane on the building in the depth image, planar regions corresponding to different planes on the building are determined within the image region of the building, including: The gradient range is determined by subtracting a preset adjustment parameter from the average gradient to obtain the first endpoint of the gradient range, and by adding the preset adjustment parameter to the average gradient to obtain the second endpoint of the gradient range. The region consisting of adjacent pixels whose gradient values ​​are within the gradient range is taken as the planar region in the image region of the building, corresponding to the same plane on the building. Other regions in the image area are used as new regions to be detected. The average gradient and gradient range in the new regions to be detected are determined, and the planar regions in the regions to be detected that correspond to the same plane on the building are determined, until the image area of ​​the building is divided into planar regions that correspond to different planes on the building.

5. The method according to claim 1, characterized in that, Based on the aforementioned planar region and depth range, the predicted landing area is determined, including: The region whose contour from the depth camera's viewpoint coincides with the contour of at least one planar region, and whose depth value meets the depth value range, is designated as the predicted landing area.

6. The method according to claim 1, characterized in that, If the target is located within the predicted landing area, then determining the target as a target thrown from the building includes: If the target is located within the predicted landing area, and the target's height value is determined to decrease based on at least two frames of depth images of the target, then the target is determined to be a target thrown from the building.

7. A target detection device, characterized in that, The device includes: The location information determination module is used to acquire depth images of buildings through a depth camera and determine the location information of targets in the depth images; The positional relationship determination module is used to determine the positional relationship between the target and the predicted landing area based on the target's positional information; wherein, the predicted landing area is the area that the predicted object will pass through when it is thrown from the building; The target determination module is used to determine that if the target is located within the predicted landing area, the target is a target thrown from the building. The device further includes: The image region determination module is used to determine the image region of a building in the depth image based on the gradient value of the depth values ​​between adjacent pixels in the depth image; The planar region determination module is used to determine the planar regions in the image region of a building that correspond to different planes on the building, based on the gradient values ​​of the depth values ​​of adjacent pixels in the region to be detected within the image region of the building, and the average gradient of the depth values ​​of adjacent pixels belonging to the same plane on the building in the depth image; wherein, the region to be detected is the region in the image region that has not yet been determined as a planar region corresponding to a different plane on the building. The landing area determination module is used to determine the predicted landing area based on the planar area and the depth value range; wherein, the depth value range is determined based on the distance between the landing point of the predicted target and the building when it is thrown from the building.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the target detection method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the target detection method as described in any one of claims 1-6.

Images