Target aggregation checking method and apparatus, electronic device, and storage medium

Abstract

The application discloses a target aggregation checking method and device, electronic equipment and a storage medium, and belongs to the technical field of computer data. The method comprises the following steps: acquiring a target image frame and at least one boundary box contained in the target image frame; acquiring ground projection parameters and height parameters of corresponding targets based on pixel coordinates of the boundary box in the target image frame; for any two targets contained in the target image frame, if the height parameters satisfy a preset similarity constraint and the distance between the ground projection positions satisfies a threshold value, it is determined that the two targets are associated with each other, and an association relationship between the two targets is established; and based on the association relationship between the targets, the targets with the association relationship are connected and merged to obtain at least one aggregation set. The application improves the reliability and universality of target aggregation detection, so that the target aggregation detection scheme can be used in various scenes, and the accuracy and reliability of target aggregation detection are improved.

Description

Technical Field

[0001] This application relates to the field of computer data processing technology, specifically to target aggregation inspection methods, apparatus, electronic devices, and storage media. Background Technology

[0002] In managing crowd flow in scenic areas, monitoring safety at transportation hubs, and ensuring the safety of large gatherings, real-time detection of crowd gatherings plays a crucial role in preventing stampedes and other safety accidents.

[0003] Currently, in related technologies, cameras capture images of the scene, and a fixed pixel distance threshold is used to determine the proximity of people. However, due to the perspective projection effect of monocular cameras, people closer to the camera have larger pixel sizes in the image, while people farther away have extremely small pixel sizes. The fixed threshold is prone to missed detections at close range and false alarms at distant ranges. Furthermore, to eliminate perspective distortion, inverse perspective mapping (IPM) is used to transform the coordinates of the scene image from the image coordinate system to the world coordinate system. Then, the crowd aggregation is detected in the transformed world coordinate system.

[0004] However, in the aforementioned related technologies, coordinate transformation requires precise intrinsic parameters of the camera (such as focal length and pixel size) and real-time extrinsic parameters (such as gimbal elevation angle and installation height). For non-fixed cameras (such as airborne cameras in drone aerial photography scenarios), since the camera's attitude and altitude are constantly changing, the calibration parameters are difficult to obtain and update in real time, making it difficult to effectively deploy the aggregation detection scheme in dynamic scenarios. Summary of the Invention

[0005] In view of this, this application provides a target aggregation inspection method, apparatus, electronic device and storage medium to solve the problem that aggregation detection schemes are difficult to effectively deploy in dynamic scenarios.

[0006] In a first aspect, this application provides a method for target aggregation inspection, the method comprising: Obtain a target image frame and at least one bounding box contained in the target image frame; wherein the bounding box is used to characterize the position of the target in the target image frame; Based on the pixel coordinates of the bounding box in the target image frame, the ground projection parameters and height parameters of the corresponding target are obtained; wherein, the ground projection parameters are used to characterize the ground projection position of the target in the target image frame; For any two targets contained in the target image frame, if the height parameter satisfies the preset similarity constraint and the distance between the ground projection positions satisfies the threshold value, it is determined that there is a clustering association between the two targets, and the association relationship between the two targets is established. Based on the relationships between the various targets, the targets with relationships are connected and merged to obtain at least one cluster set; wherein the cluster set includes at least two of the targets.

[0007] Secondly, this application provides a target aggregation inspection device, the device comprising: An image acquisition module is used to acquire a target image frame and at least one bounding box contained in the target image frame; wherein the bounding box is used to characterize the position of the target in the target image frame; The parameter acquisition module is used to acquire the ground projection parameters and height parameters of the corresponding target based on the pixel coordinates of the bounding box in the target image frame; wherein, the ground projection parameters are used to characterize the ground projection position of the target in the target image frame; The relationship establishment module is used to determine that there is a clustering association between any two targets contained in the target image frame, provided that the height parameter satisfies a preset similarity constraint and the distance between the ground projection positions satisfies a threshold value; and to establish the association relationship between the two targets. The connectivity merging module is used to connect and merge targets that have relationships based on the relationships between the targets, to obtain at least one cluster set; wherein the cluster set includes at least two targets.

[0008] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the target aggregation inspection method described in the first aspect or any corresponding embodiment.

[0009] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the target aggregation inspection method described in the first aspect or any corresponding embodiment.

[0010] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the target aggregation inspection method described in the first aspect or any corresponding embodiment thereof.

[0011] The target clustering detection method provided in this application obtains the ground projection position and height parameters of the target by using a bounding box to characterize the position of the target in the target image frame. Then, it clusters and associates the targets based on the height parameters and ground projection positions, and connects and merges the targets with the relationship to obtain the cluster set of targets in the target image frame. On the one hand, it does not require obtaining the precise intrinsic and real-time extrinsic parameters of the image acquisition device corresponding to the target image frame, making the target clustering detection scheme applicable to dynamic scenarios such as UAV aerial photography, improving the reliability and versatility of target clustering detection, and enabling the target clustering detection scheme to be used in various scenarios. On the other hand, by simultaneously using the similarity constraint of the height parameter and the threshold value judgment of the ground projection position to determine the clustering relationship between targets, it helps to avoid false alarms caused by single distance judgment, improving the accuracy and reliability of target clustering detection. Moreover, compared with the O(N²) complexity problem of traditional clustering algorithms (such as DBSCAN) that require the calculation of the full distance matrix, by establishing the relationship between targets and connecting and merging them, the computational overhead is reduced, making it suitable for running the target clustering detection scheme in real time on edge embedded devices with limited computing power. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this application, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0013] Figure 1 This is a schematic flowchart of a target aggregation inspection method according to an embodiment of this application; Figure 2 This is another flowchart illustrating the target aggregation inspection method according to an embodiment of this application; Figure 3 This is another schematic flowchart of the target aggregation inspection method according to the embodiments of this application; Figure 4 This is a structural block diagram of a target aggregation inspection device according to an embodiment of this application; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] It should be noted that the information (including but not limited to user input information, such as information entered by the user into input boxes), data (including but not limited to data used for analysis, stored data, and displayed data, such as context code, all code of the current project, the service pressure corresponding to operations performed on all code of the current project, and the code development status of the current project), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. For example, the context code, operations performed on all code of the current project, the corresponding service pressure, and the code development status involved in this application were all obtained with full authorization.

[0016] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0017] According to an embodiment of this application, a target aggregation inspection method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0018] This embodiment provides a target aggregation inspection method, which can be used in electronic devices, such as standalone physical servers, server clusters or distributed systems, cloud servers providing cloud services, and airborne processors of drones. Figure 1 This is a flowchart of a target aggregation inspection method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the target image frame and at least one bounding box contained in the target image frame.

[0019] In this embodiment of the application, when performing aggregation inspection on a target, the electronic device acquires a target image frame and at least one bounding box contained in the target image frame. The bounding box is used to characterize the position of the target within the target image frame.

[0020] For example, an electronic device acquires a target image frame through a built-in or external image acquisition device.

[0021] In one possible implementation, the electronic device directly acquires the target image frame through an image acquisition device, and then performs target detection on the target image frame to obtain at least one bounding box contained in the target image frame.

[0022] In another possible implementation, the image acquisition device acquires video to obtain a video stream. Then, the electronic device extracts a single target image frame from the video stream and performs target detection on that target image frame to obtain at least one bounding box contained within it. In one possible implementation, the target image frame is any image frame in the video stream. In another possible implementation, to improve inspection efficiency, after acquiring the video stream, the electronic device performs frame segmentation processing on the video stream to obtain multiple single-frame image frames. Further, it performs target quantity detection on these single-frame image frames, identifying single-frame image frames with a target quantity greater than a first target value as target image frames. Exemplarily, the first target value can be any value and can be flexibly set and adjusted according to actual conditions. Optionally, in this embodiment, the first target value is greater than or equal to a first preset threshold (described below).

[0023] For example, the image acquisition device can be a fixed device, such as a fixed surveillance camera; or, the image acquisition device can be a non-fixed device, such as an airborne camera on a drone.

[0024] In one possible implementation, the bounding box is directly displayed in the target image frame. In another possible implementation, to improve inspection efficiency, the bounding box display process is skipped. After performing target detection on the target image frame, the electronic device obtains the coordinate information of the bounding box, which represents the pixel coordinates of the bounding box in the target image frame. For example, the bounding box is rectangular, and the coordinate information includes the pixel coordinates of the four vertices of the bounding box.

[0025] For example, the target is any species or object capable of generating aggregation behavior, such as people, birds, vehicles, etc.

[0026] Step S102: Based on the pixel coordinates of the bounding box in the target image frame, obtain the ground projection parameters and height parameters of the corresponding target.

[0027] In this embodiment, after acquiring the target image frame and its corresponding bounding box, the electronic device obtains the ground projection parameters and height parameters of the corresponding target based on the pixel coordinates of the bounding box in the target image frame. The ground projection parameters characterize the ground projection position of the target in the target image frame, and the height parameters refer to the pixel height of the target in the target image frame.

[0028] Step S103: For any two targets contained in the target image frame, if the height parameter satisfies the preset similarity constraint and the distance between the ground projection positions satisfies the threshold value, it is determined that there is an aggregation association between the two targets, and the association relationship between the two targets is established.

[0029] In this embodiment of the application, after obtaining the aforementioned height parameters and ground projection positions, the electronic device, for any two targets contained in the target image frame, determines that there is a clustering association between the two targets, and establishes an association relationship between the two targets, provided that the height parameters satisfy a preset similarity constraint and the distance between the ground projection positions meets a threshold value.

[0030] For example, for two targets that are clustered together, the electronic device abstracts the target into a node and establishes edges in the node to represent the relationship between the two targets.

[0031] Step S104: Based on the relationship between each target, connect and merge the targets that have a relationship to obtain at least one aggregate set.

[0032] In this embodiment, after obtaining the relationships between various targets, the electronic device connects and merges the related targets to obtain at least one aggregate set. Each aggregate set includes at least two targets. For example, the electronic device uses the Disjoint Set Union (DSU) algorithm to connect and merge related targets to obtain at least one aggregate set.

[0033] For example, an electronic device may group targets that have direct or indirect relationships into the same set of aggregates. For instance, if there is a relationship between a first target and a second target, and a relationship between the second target and a third target, the electronic device may group the first target, the second target, and the third target into the same set of aggregates.

[0034] The target clustering detection method provided in this embodiment obtains the ground projection position and height parameters of the target by using bounding boxes to characterize the position of the target in the target image frame. Then, it clusters and associates the targets based on the height parameters and ground projection positions, and connects and merges the targets with the same relationship to obtain the cluster set of targets in the target image frame. On the one hand, it does not require obtaining the precise intrinsic and real-time extrinsic parameters of the image acquisition device corresponding to the target image frame, making the target clustering detection scheme applicable to dynamic scenarios such as UAV aerial photography, improving the reliability and versatility of target clustering detection, and enabling the target clustering detection scheme to be used in various scenarios. On the other hand, by simultaneously using similarity constraints of height parameters and threshold values ​​of ground projection positions to determine the clustering relationship between targets, it helps to avoid false alarms caused by single distance determination, improving the accuracy and reliability of target clustering detection. Moreover, compared with the O(N²) complexity problem of traditional clustering algorithms (such as DBSCAN) that require the calculation of the full distance matrix, by establishing the relationship between targets and connecting and merging them, the computational overhead is reduced, making it suitable for real-time operation of the target clustering detection scheme on edge embedded devices with limited computing power.

[0035] In an exemplary embodiment, step S103 includes: Step S1031: Obtain the ratio between the larger and smaller values ​​of the height parameters of the two targets.

[0036] In this embodiment of the application, after obtaining the above-mentioned height parameters, the electronic device obtains the ratio between the larger and smaller values ​​of the height parameters of the two targets.

[0037] Step S1032: If the ratio is less than or equal to a preset threshold, determine that the height parameter satisfies the similarity constraint.

[0038] In this embodiment of the application, after obtaining the above ratio, the electronic device determines that the height parameter satisfies the similarity constraint if the ratio is less than or equal to a preset threshold.

[0039] For example, the preset threshold can be any value, and can be flexibly set and adjusted according to the actual situation. For instance, if the target is a person, and the height ratio between adults and children is met, so that adults and children in the same cluster can be successfully clustered, the preset threshold can range from 1.8 to 2.5.

[0040] Step S1033: Generate threshold values ​​based on the height parameters of the two targets.

[0041] In this embodiment, after obtaining the aforementioned height parameters, the electronic device generates a threshold value based on the height parameters of the two targets. The threshold value is positively correlated with the smaller value.

[0042] In one possible implementation, the threshold value is positively correlated with the height parameters of the two targets (including the larger and smaller values). For example, the electronic device averages the larger and smaller values ​​to obtain an average value; further, a preset proportional constant is multiplied by the average value to obtain the threshold value.

[0043] In another possible implementation, the threshold value is only positively correlated with the smaller value. For example, the electronic device multiplies a preset proportional constant by the smaller value to obtain the threshold value.

[0044] For example, the preset scaling constant can be any value, and can be flexibly set and adjusted according to the actual situation. For instance, taking a person as the target, the preset scaling constant can range from 1.5 to 2.5.

[0045] For example, the threshold value is related to the height parameter in order to use the bounding box height to approximate the perspective scale of the target's location. For instance, when the target is close to the image, the pixel height is larger, and the same real ground distance will appear as a larger pixel distance. In this case, the threshold value increases accordingly with the height parameter. When the target is far from the image, the height parameter is smaller, and the pixel distance corresponding to the real distance is also smaller. In this case, the threshold value decreases accordingly with the height parameter.

[0046] Step S1034: If the distance between the ground projection positions of two targets is less than or equal to a threshold value, it is determined that there is an aggregation association between the two targets.

[0047] In this embodiment of the application, after obtaining the threshold value, if the distance between the ground projection positions of the two targets is less than or equal to the threshold value, the electronic device determines that there is an aggregation association between the two targets.

[0048] The target clustering detection method provided in this embodiment uses the ratio of height parameters to determine similarity between two targets, and then uses the distance between their ground projection positions to determine a threshold. This process of coarse screening followed by fine-tuning helps reduce unnecessary distance calculations and improves the computational efficiency of clustering detection. Furthermore, the threshold value is positively correlated with the smaller of the height parameters of the two targets, allowing the clustering determination to adapt to the differences in target imaging scale at different perspective distances. This helps to solve the problem of false alarms and missed alarms when using a fixed pixel distance threshold in near and far scenes, thereby improving the reliability and accuracy of clustering detection.

[0049] In an exemplary embodiment, step S102 includes: Step S1021: Using the bottom of the target as a reference, obtain the pixel coordinates of the center point of the bottom edge of the bounding box based on the pixel coordinates of the two bottom edge endpoints, and use the pixel coordinates of the center point of the bottom edge as the ground projection parameter.

[0050] In this embodiment of the application, after obtaining the bounding box, the electronic device uses the bottom of the target as a reference and obtains the pixel coordinates of the center point of the bottom edge of the bounding box based on the pixel coordinates of the two bottom edge endpoints, and uses the pixel coordinates of the center point of the bottom edge as the ground projection parameter.

[0051] Step S1022: Use the absolute value of the difference between the top-left ordinate and the bottom-right ordinate of the bounding box as the height parameter.

[0052] In this embodiment, after obtaining the bounding box, the electronic device uses the absolute value of the difference between the top-left ordinate and the bottom-right ordinate of the bounding box as the height parameter. The direction represented by the vertical axis is perpendicular to the bottom of the bounding box.

[0053] The target aggregation inspection method provided in this embodiment uses the pixel coordinates of the center point of the bottom edge of the bounding box as the ground projection parameter. This center point of the bottom edge is closest to the actual contact position between the target and the ground. Compared with using the geometric center point or centroid of the target, the center point of the bottom edge can more accurately reflect the true spatial position of the target on the ground, thus improving the accuracy of ground proximity judgment. Using the pixel height of the bounding box as the height parameter simplifies the operation and eliminates the need for additional depth sensors or camera calibration information, thereby reducing the overall deployment cost and maintenance difficulty of the solution.

[0054] In an exemplary embodiment, step S104 includes: Step S1041: Merge targets that are directly or indirectly related into the same set to obtain at least one candidate set.

[0055] In this embodiment of the application, after obtaining the correlation between targets, the electronic device merges targets that are directly or indirectly related into the same set to obtain at least one candidate set. The candidate set includes at least two targets.

[0056] Step S1042: The candidate set containing more than the first threshold is determined as the aggregate set.

[0057] In this embodiment of the application, after obtaining the above-mentioned candidate set, the electronic device determines the candidate set containing a target number greater than a first threshold as an aggregate set.

[0058] For example, the first threshold can be any value, and can be flexibly set and adjusted according to the actual situation, such as 5, 6, 7, etc.

[0059] The target clustering detection method provided in this embodiment can accurately identify clusters connected by multi-hop associations by merging targets that are directly or indirectly related into the same set. For example, if target A is related to target B, and target B is related to target C, then target A, target B, and target C belong to the same set. Moreover, by determining the candidate set containing more than a first threshold as the final cluster set, it is beneficial to exclude sets with fewer targets, making the final result of the target clustering detection more reflective of the actual clustering events.

[0060] This embodiment provides a target aggregation inspection method, which can be used in the aforementioned electronic device. Figure 2 This is a flowchart of a target aggregation inspection method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the target image frame and at least one bounding box contained in the target image frame.

[0061] Step S202: Based on the pixel coordinates of the bounding box in the target image frame, obtain the ground projection parameters and height parameters of the corresponding target.

[0062] The above steps S201-202 and Figure 1 Steps S101-102 in the embodiment are similar; see details below. Figure 1 Examples are not described in detail here.

[0063] Step S203: Divide the target image frame into multiple grids.

[0064] In this embodiment of the application, after acquiring the target image frame, the electronic device divides the target image frame into multiple grids. The ground projection positions of any two targets belong to the same grid or adjacent grids.

[0065] For example, the electronic device divides the target image frame into multiple grids based on a preset grid size. Optionally, to improve inspection efficiency, the electronic device divides the target image frame into multiple grids in response to the total number of targets contained in the target image frame being greater than a preset second target value. The second target value can be any value and can be flexibly set and adjusted according to actual conditions. Optionally, the second target value is greater than or equal to the aforementioned first target value.

[0066] Step S204: For any two targets contained in the target image frame, if the height parameter satisfies the preset similarity constraint and the distance between the ground projection positions satisfies the threshold value, it is determined that there is an aggregation association between the two targets, and the association relationship between the two targets is established.

[0067] Step S205: Based on the relationship between each target, connect and merge the targets that have a relationship to obtain at least one aggregate set.

[0068] The above steps S204-205 and Figure 1 Steps S103-204 in the embodiment are similar; see details below. Figure 1 Examples are not described in detail here.

[0069] The target clustering inspection method provided in this embodiment divides the target image frame into multiple grids and only performs clustering judgment on targets belonging to the same grid or adjacent grids, which helps to reduce the number of target pairs that need to be judged and further improves the computational efficiency of target clustering inspection.

[0070] This embodiment provides a target aggregation inspection method, which can be used in the aforementioned electronic device. Figure 3 This is a flowchart of a target aggregation inspection method according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain the target image frame and at least one bounding box contained in the target image frame.

[0071] Step S302: Based on the pixel coordinates of the bounding box in the target image frame, obtain the ground projection parameters and height parameters of the corresponding target.

[0072] Step S303: For any two targets contained in the target image frame, if the height parameter satisfies the preset similarity constraint and the distance between the ground projection positions satisfies the threshold value, it is determined that there is an aggregation association between the two targets, and the association relationship between the two targets is established.

[0073] Step S304: Based on the relationship between each target, connect and merge the targets that have a relationship to obtain at least one aggregate set.

[0074] The above steps S301-304 and Figure 1 Steps S101-104 in the embodiment are similar; see details below. Figure 1 Examples are not described in detail here.

[0075] Step S305: Obtain the vertices of each bounding box contained in the aggregation set, and construct a vertex set.

[0076] In this embodiment of the application, after obtaining the aforementioned aggregate set, the electronic device obtains the vertices of each bounding box contained in the aggregate set and constructs a vertex set. Each aggregate set corresponds to one vertex set.

[0077] Step S306: Obtain the smallest convex polygon used to wrap the vertices contained in the vertex set.

[0078] In this embodiment of the application, after obtaining the vertex set, the electronic device obtains the minimum convex polygon used to wrap each vertex contained in the vertex set. Each aggregate set corresponds to one minimum convex polygon.

[0079] Step S307: Display the aggregated image frame based on the target image frame and the minimum convex polygon.

[0080] In this embodiment, after obtaining the minimum convex polygon, the electronic device displays a clustered image frame based on the target image frame and the minimum convex polygon. The clustered image frame represents the position of the cluster set within the target image frame. For example, a target image frame may include one or more minimum convex polygons.

[0081] For example, when acquiring the above-mentioned clustered image frame, the electronic device creates a first layer with the same size as the target image frame; further, in the first layer, target information is filled in the area indicated by the smallest convex polygon to obtain a second layer; further, the second layer and the target image frame are fused to obtain a first image frame; further, in the first image frame, an outline is drawn along the smallest convex polygon to obtain a second image frame; further, in the second image frame, prompt information indicating the number of clustered targets is marked in the area around the smallest convex polygon, and the clustered image frame is displayed.

[0082] For example, the target information includes, but is not limited to, at least one of the following: semi-transparent color information, pattern information, etc.

[0083] The target clustering inspection method provided in this embodiment calculates the minimum convex polygon for the vertices of all target bounding boxes within the cluster set. This allows for an accurate reflection of the distribution of irregularly shaped (such as L-shaped, arc-shaped, etc.) target clusters with a compact envelope region. This enables the minimum convex polygon to better fit the actual spatial distribution of the target clusters, which helps reduce the blank area within the envelope region and thus helps avoid visual clutter and detail occlusion caused by a large amount of irrelevant blank ground.

[0084] In addition, by drawing the outline along the convex hull and marking the number of clustered targets above the convex hull, the location and scale information of the clustering event can be intuitively reflected.

[0085] This embodiment also provides a target aggregation inspection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0086] This embodiment provides a target aggregation inspection device, such as... Figure 4 As shown, it includes: The image acquisition module 401 is used to acquire a target image frame and at least one bounding box contained in the target image frame; wherein the bounding box is used to characterize the position of the target in the target image frame; The parameter acquisition module 402 is used to acquire the ground projection parameters and height parameters of the corresponding target based on the pixel coordinates of the bounding box in the target image frame; wherein, the ground projection parameters are used to characterize the ground projection position of the target in the target image frame. The relationship establishment module 403 is used to determine the clustering relationship between two targets and establish the association relationship between them if the height parameter satisfies the preset similarity constraint and the distance between the ground projection positions satisfies the threshold value for any two targets contained in the target image frame. The connectivity merging module 404 is used to connect and merge targets with relationships based on the relationships between them, to obtain at least one cluster set; wherein the cluster set includes at least two targets.

[0087] In an exemplary embodiment, the relationship establishment module 403 is used for: Obtain the ratio between the larger and smaller values ​​of the height parameters of two targets; If the ratio is less than or equal to a preset threshold, the height parameter is determined to satisfy the similarity constraint. Threshold values ​​are generated based on the height parameters of the two targets; the threshold values ​​are positively correlated with the smaller value. If the distance between the ground projection locations of two targets is less than or equal to a threshold value, then an aggregation association is determined to exist between the two targets.

[0088] In an exemplary embodiment, the bounding box is rectangular; the parameter acquisition module 402 is used for: Using the bottom of the target as a reference, obtain the pixel coordinates of the center point of the bottom edge of the bounding box based on the pixel coordinates of the two bottom edge endpoints, and use the pixel coordinates of the center point of the bottom edge as the ground projection parameter. The absolute value of the difference between the top-left corner y-coordinate and the bottom-right corner y-coordinate of the bounding box is used as the height parameter; where the direction represented by the y-axis is perpendicular to the bottom of the bounding box.

[0089] In an exemplary embodiment, the connection merging module 404 is used for: Targets that are directly or indirectly related are merged into the same set to obtain at least one candidate set; Candidate sets containing more than a first threshold are identified as aggregate sets.

[0090] In an exemplary embodiment, the apparatus further includes: The grid division module is used to divide the target image frame into multiple grids; In this context, the ground projection positions of any two targets belong to the same grid or adjacent grids.

[0091] In an exemplary embodiment, the apparatus further includes: The vertex acquisition module is used to acquire the vertices of each bounding box contained in the aggregate set and construct the vertex set; The vertex wrapping module is used to obtain the minimum convex polygon used to wrap the individual vertices contained in a vertex set; The result generation module is used to display aggregated image frames based on the target image frame and the minimum convex polygon; wherein, the aggregated image frame is used to represent the position of the aggregate set in the target image frame.

[0092] In an exemplary embodiment, the result generation module is configured to: Create a first layer with the same dimensions as the target image frame; In the first layer, fill the area indicated by the smallest convex polygon with the target information to obtain the second layer; The second layer and the target image frame are fused to obtain the first image frame; In the first image frame, draw the outline along the smallest convex polygon to obtain the second image frame; In the second image frame, a prompt indicating the number of clustered targets is marked in the area surrounding the smallest convex polygon, and the clustered image frame is displayed.

[0093] The target aggregation inspection apparatus provided in this application can execute the target aggregation inspection method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0094] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0095] The following is a detailed reference. Figure 5The diagram illustrates a structural schematic suitable for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0096] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0097] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the target aggregation checking method of embodiments of this application.

[0098] Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0099] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the target aggregation inspection method shown in the above embodiments is implemented.

[0100] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0101] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for checking target aggregation, characterized in that, The method includes: Obtain a target image frame and at least one bounding box contained in the target image frame; wherein the bounding box is used to characterize the position of the target in the target image frame; Based on the pixel coordinates of the bounding box in the target image frame, the ground projection parameters and height parameters of the corresponding target are obtained; wherein, the ground projection parameters are used to characterize the ground projection position of the target in the target image frame; For any two targets contained in the target image frame, if the height parameter satisfies the preset similarity constraint and the distance between the ground projection positions satisfies the threshold value, it is determined that there is a clustering association between the two targets, and the association relationship between the two targets is established. Based on the relationships between the various targets, the targets with relationships are connected and merged to obtain at least one cluster set; wherein the cluster set includes at least two of the targets.

2. The method according to claim 1, characterized in that, The step of determining that there is a clustering association between the two targets when the height parameter satisfies a preset similarity constraint and the distance between the ground projection positions satisfies a threshold value includes: Obtain the ratio between the larger and smaller values ​​of the height parameters of the two targets; If the ratio is less than or equal to a preset threshold, it is determined that the height parameter satisfies the similarity constraint; The threshold value is generated based on the height parameters of the two targets; wherein the threshold value is positively correlated with the smaller value. If the distance between the ground projection locations of the two targets is less than or equal to the threshold value, it is determined that there is an aggregation association between the two targets.

3. The method according to claim 1, characterized in that, The bounding box is rectangular; The step of obtaining the ground projection parameters and height parameters of the corresponding target based on the pixel coordinates of the bounding box in the target image frame includes: Using the bottom of the target as a reference, and based on the pixel coordinates of the two bottom edge endpoints, the pixel coordinates of the bottom edge center point of the bounding box are obtained, and the pixel coordinates of the bottom edge center point are used as the ground projection parameter; The absolute value of the difference between the top-left ordinate and the bottom-right ordinate of the bounding box is used as the height parameter; wherein the direction represented by the ordinate is perpendicular to the bottom of the bounding box.

4. The method according to claim 1, characterized in that, Based on the relationships between the various targets, the targets with relationships are connected and merged to obtain at least one aggregate set, including: Targets that have direct or indirect relationships are merged into the same set to obtain at least one candidate set; The candidate set containing more than a first threshold is determined as the aggregate set.

5. The method according to claim 1, characterized in that, The method further includes: The target image frame is divided into multiple grids; Wherein, the ground projection positions of any two targets belong to the same grid or adjacent grids.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the vertices of each bounding box contained in the aggregate set, and construct a vertex set; Obtain the smallest convex polygon used to wrap all the vertices contained in the vertex set; Display aggregated image frames based on the target image frame and the minimum convex polygon; wherein, the aggregated image frames are used to characterize the position of the aggregate set in the target image frame.

7. The method according to claim 6, characterized in that, The method of displaying aggregated image frames based on the target image frame and the minimum convex polygon includes: Create a first layer with the same size as the target image frame; In the first layer, target information is filled into the area indicated by the smallest convex polygon to obtain the second layer; The second layer and the target image frame are fused to obtain the first image frame; In the first image frame, a contour line is drawn along the smallest convex polygon to obtain the second image frame; In the second image frame, a prompt message indicating the number of clustered targets is marked in the area surrounding the smallest convex polygon, and the clustered image frame is displayed.

8. A target aggregation inspection device, characterized in that, The device includes: An image acquisition module is used to acquire a target image frame and at least one bounding box contained in the target image frame; wherein the bounding box is used to characterize the position of the target in the target image frame; The parameter acquisition module is used to acquire the ground projection parameters and height parameters of the corresponding target based on the pixel coordinates of the bounding box in the target image frame; wherein, the ground projection parameters are used to characterize the ground projection position of the target in the target image frame; The relationship establishment module is used to determine that there is a clustering association between any two targets contained in the target image frame, provided that the height parameter satisfies a preset similarity constraint and the distance between the ground projection positions satisfies a threshold value; and to establish the association relationship between the two targets. The connectivity merging module is used to connect and merge targets that have relationships based on the relationships between the targets, to obtain at least one cluster set; wherein the cluster set includes at least two targets.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the target aggregation inspection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the target aggregation inspection method according to any one of claims 1 to 7.

Images