Radar-based behavior warning method, electronic device, and storage medium

Abstract

This application discloses a radar-based behavior warning method, electronic device, and storage medium. The method includes: obtaining a radar measurement map acquired by radar; performing rasterization processing on the radar measurement map to obtain multiple sub-maps corresponding to the radar measurement map; wherein a preset warning line is provided in the radar measurement map; clustering all targets in the radar measurement map to obtain at least one target cluster area; determining a critical sub-map block among all sub-map blocks based on the preset warning line and the target cluster area; dividing the critical sub-map block and the sub-map blocks on both sides to obtain at least two warning areas bounded by the critical sub-map block; wherein each warning area corresponds to a preset safety value; and for each target, issuing a warning for the target's behavior based on the preset safety value corresponding to the warning area where the target is located, the distance of the target relative to the target cluster area, and the direction of movement of the target relative to the critical sub-map block. The above scheme can improve the accuracy of behavior warning.

Description

Technical Field

[0001] This application relates to the field of target recognition technology, and in particular to a radar-based behavior warning method, electronic device, and storage medium. Background Technology

[0002] With the continuous improvement of security levels, in some high-risk scenarios, target recognition requirements go beyond simply identifying targets; they also need to provide early warnings about the targets' behavior. Current technologies typically rely on a single type of camera to capture images of the monitored area, and then use staff or intelligent algorithms to identify and issue warnings about target behavior. However, this approach is highly dependent on the quality of the video footage captured by the camera. When the image quality or environmental conditions are poor, it is difficult to provide timely and accurate warnings about target behavior. Therefore, improving the accuracy of behavioral warnings has become an urgent problem to be solved. Summary of the Invention

[0003] The main technical problem addressed in this application is to provide a radar-based behavior warning method, electronic device, and storage medium that can improve the accuracy of behavior warnings.

[0004] To address the aforementioned technical problems, a first aspect of this application provides a radar-based behavior warning method, comprising: obtaining a radar measurement map acquired by radar; performing rasterization processing on the radar measurement map to obtain multiple sub-maps corresponding to the radar measurement map; wherein, the radar measurement map is provided with a preset warning line; clustering all targets in the radar measurement map to obtain at least one target clustering area; wherein, the target clustering area corresponds to at least one of the sub-maps; determining a critical sub-map based on the preset warning line and the target clustering area; dividing the critical sub-map and the sub-maps on both sides of the critical sub-map to obtain at least two warning areas bounded by the critical sub-map; wherein, each warning area corresponds to a preset safety value; and, for each target, issuing a warning regarding the target's behavior based on the preset safety value corresponding to the warning area where the target is located, the distance of the target relative to the target clustering area, and the direction of movement of the target relative to the critical sub-map.

[0005] To address the aforementioned technical problems, a second aspect of this application provides an electronic device comprising: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor invokes the program data to execute the method described in the first aspect.

[0006] To address the aforementioned technical problems, a third aspect of this application provides a computer-readable storage medium storing program data thereon, wherein the program data, when executed by a processor, implements the method described in the first aspect.

[0007] The above scheme obtains a radar measurement map acquired by radar, performs rasterization processing on the radar measurement map, dividing it into multiple sub-map tiles. The radar measurement map includes a preset warning line. All targets in the radar measurement map are clustered, with targets that are close together grouped together. The sub-map tiles where targets are clustered are designated as target clustering areas, resulting in at least one target clustering area. Each target clustering area corresponds to at least one sub-map tile. The preset warning line is pre-set and does not change. However, in specific application environments, the scenario requiring warning may change, and targets may form target clustering areas based on their own judgment and instincts. Therefore, based on the preset warning line and target clustering areas, targets are identified from all sub-map tiles. The critical sub-block divides the critical sub-block and the sub-blocks on both sides into at least two warning areas bounded by the critical sub-block. Therefore, the critical sub-block can be regarded as a correction to the preset warning line. The warning areas divided based on the critical sub-block are more suitable and reasonable for the application scenario. Each warning area has its own preset safety value. Then, based on the preset safety value corresponding to the warning area where the target is located, the distance of the target from the target gathering area, and the direction of movement of the target relative to the critical sub-block, the target's behavior is warned. In order to comprehensively consider the warning area where the target is located, the distance of the target from the target gathering area, and the direction of movement of the target, the target's behavior is warned, thus improving the accuracy of behavior warning. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0009] Figure 1 This is a flowchart illustrating one implementation method of the radar-based behavioral early warning method of this application;

[0010] Figure 2 This is a schematic diagram illustrating an application scenario of one implementation method of the radar-based behavioral early warning method of this application;

[0011] Figure 3 This is a flowchart illustrating another implementation of the radar-based behavioral early warning method of this application;

[0012] Figure 4 yes Figure 3 A flowchart of one embodiment corresponding to step S307;

[0013] Figure 5 This is a flowchart illustrating one implementation method of the behavior warning method based on radar-visual fusion in this application;

[0014] Figure 6 This is a flowchart illustrating another implementation of the behavior warning method based on radar-visual fusion in this application;

[0015] Figure 7 This is a schematic diagram illustrating an application scenario of one implementation method of the behavior warning method based on radar-visual fusion in this application;

[0016] Figure 8 This is a schematic diagram of the structure of one embodiment of the electronic device of this application;

[0017] Figure 9 This is a schematic diagram of one embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0019] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper means two or more.

[0020] The radar-based behavior warning method provided in this application is applied to scenarios requiring warning of target behavior, and its corresponding execution entity is a processor or processing module capable of calling radar. The radar-visual fusion-based behavior warning method provided in this application is also applied to scenarios requiring warning of target behavior, and its corresponding execution entity is a processor or processing module capable of calling radar and camera. Scenarios requiring warning of target behavior include, but are not limited to, areas near oceans, rivers, lakes, and other bodies of water. Furthermore, the radar includes millimeter-wave radar and lidar, preferably millimeter-wave radar.

[0021] Please see Figure 1 , Figure 1 This is a flowchart illustrating one embodiment of the radar-based behavioral early warning method of this application. The method includes:

[0022] S101: Obtain the radar measurement map collected by the radar, perform rasterization processing on the radar measurement map to obtain multiple sub-map blocks corresponding to the radar measurement map, wherein the radar measurement map is equipped with a preset warning line.

[0023] Specifically, please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of one embodiment of the radar-based behavioral early warning method of this application. It obtains a radar measurement map collected by radar, performs rasterization processing on the radar measurement map, dividing it into multiple sub-maps. The radar measurement map includes preset early warning lines, corresponding to... Figure 2 In the image, the radar measurement map is a sector plot, containing multiple targets represented by black dots. The rectangular boxes correspond to the rasterized sub-plots. Figure 2 The thick line in the image is the preset warning line.

[0024] In one application method, a radar measurement map acquired by radar is obtained. Based on the measurement range of the radar measurement map, the radar measurement map is rasterized to obtain multiple sub-maps corresponding to the radar measurement map.

[0025] In another application, a radar measurement map acquired by radar is obtained. Based on the resolution of the radar measurement map, the radar measurement map is rasterized to obtain multiple sub-maps corresponding to the radar measurement map.

[0026] In one application scenario, a radar is positioned on one side of the area to be monitored. A pre-set warning line for the area to be monitored is pre-defined. The radar collects radar measurement data based on a preset frequency. The radar measurement data includes the target's position, speed, and direction of movement. Based on all radar measurement data collected per second, a radar measurement map including the target and the pre-set warning line is generated. After obtaining the radar measurement map, it is rasterized to obtain multiple sub-maps corresponding to the radar measurement map, thereby making the radar measurement map more refined.

[0027] S102: Cluster all targets in the radar measurement map to obtain at least one target cluster area, wherein the target cluster area corresponds to at least one sub-map tile.

[0028] Specifically, please refer to [the relevant document] again. Figure 2 Clustering is performed on all targets in the radar measurement map, grouping targets that are close together into one class. The sub-patterns of the target clusters are taken as target cluster regions, resulting in at least one target cluster region. Each target cluster region corresponds to at least one sub-pattern. Figure 2 The target cluster area is a sub-block filled with diagonal lines.

[0029] In one application method, a clustering distance threshold is obtained, wherein the clustering distance threshold is a preset fixed value. Based on the clustering distance threshold, all targets in the radar measurement map are clustered. Targets whose distance is within the clustering distance threshold are clustered into one class. The sub-map tile where the targets in the same class are located is taken as the target clustering area, thus obtaining at least one target clustering area. Each target clustering area corresponds to at least one sub-map tile, thereby improving the efficiency of obtaining the target clustering area.

[0030] In another application, a clustering distance threshold is determined based on the distance between the target and the radar. The clustering distance threshold is proportional to the distance between the target and the radar to reduce the impact of measurement errors caused by the target being far from the radar on clustering. Based on the clustering distance threshold, all targets in the radar measurement map are clustered. Targets within the clustering distance threshold are grouped into one class, and the sub-plots where the clustered targets are located are taken as target clustering areas, resulting in at least one target clustering area. Each target clustering area corresponds to at least one sub-plot.

[0031] S103: Based on the preset warning line and target aggregation area, determine the critical sub-block among all sub-blocks, divide the critical sub-block and the sub-blocks on both sides to obtain at least two warning areas with the critical sub-block as the boundary, wherein each warning area corresponds to a preset safety value.

[0032] Specifically, please refer to [the relevant document] again. Figure 2 In specific application environments, the scenarios requiring early warning may change, and targets will form target clusters based on their own judgment and instincts. Then, based on preset warning lines and target clusters, critical sub-tiles are determined among all sub-tiles. This corresponds to... Figure 2 In the middle, the critical sub-tiles are filled with a grid, and at least two warning areas are obtained with the critical sub-tiles as the dividing lines. Figure 2 The fill color is the same for the same warning area.

[0033] In one application method, the sub-block containing the preset warning line is determined, and the target clustering area closest to the preset warning line is identified. The distances of the sub-block corresponding to the target clustering area and the sub-block containing the preset warning line in the same column are compared with the distances relative to the radar. When the sub-block corresponding to the target clustering area is closer to the radar, the sub-block corresponding to the target clustering area is used as the benchmark to determine the critical sub-block. When the sub-block corresponding to the target clustering area is farther from the radar, the sub-block containing the preset warning line is used as the critical sub-block. Thus, the warning position is corrected based on the target clustering area and the preset warning line to obtain the critical sub-block, thereby improving the rationality of the critical sub-block.

[0034] In another application, the sub-plot containing the preset warning line is determined. In the direction from the preset warning line towards the radar, it is determined whether the sub-plots in the same column and adjacent to the sub-plot containing the preset warning line constitute a target aggregation area. If so, the sub-plot corresponding to the preset warning line is designated as a critical sub-plot; otherwise, the sub-plots in the same column and adjacent to the sub-plot containing the preset warning line are also designated as critical sub-plots. Thus, when targets are clustered close to the preset warning line, the critical sub-plots are appropriately moved closer to the preset warning line; when targets are clustered away from the preset warning line, the warning line is appropriately moved closer to the radar. For example, in... Figure 2 In this application method, it is possible to find a critical sub-plot between two target clusters that protrudes in the opposite direction to the preset warning line. This critical sub-plot is the area that the target has not entered based on the actual scene. Therefore, this critical sub-plot is likely to be a sub-plot with a low safety value, so as to improve the rationality of the critical sub-plot.

[0035] Furthermore, the critical sub-block and its adjacent sub-blocks are divided into at least two warning zones bounded by the critical sub-block. Therefore, the critical sub-block can be seen as a correction to the preset warning line. The warning zones defined by the critical sub-block are more suitable and reasonable for the application scenario. Each warning zone has its own preset safety value, corresponding to... Figure 2 In the diagram, the darker the warning area, the smaller the preset safety value.

[0036] In one application scenario, critical sub-patterns and sub-patterns that are relatively far from the radar relative to the critical sub-image are designated as high-risk warning areas. Figure 2 Darker areas will be designated as low-risk warning zones, with sub-patterns within a preset radar distance. Figure 2 The white area represents the sub-tile between the low-risk and high-risk warning areas, designated as the medium-risk warning area. Figure 2 The light-colored area has three different warning areas, each with its own preset safety value. For example, the preset safety value for the high-risk warning area is 0, the preset safety value for the medium-risk warning area is 5, and the preset safety value for the low-risk warning area is 9. In other application scenarios, the preset safety value can also be customized based on the application scenario. This application does not impose specific restrictions on this.

[0037] S104: For each target, based on the preset safety value corresponding to the warning area where the target is located, the distance of the target relative to the target cluster area, and the movement direction of the target relative to the critical sub-block, the target behavior is given a warning.

[0038] Specifically, for each target, based on the preset safety value corresponding to the warning area where the target is located, the distance of the target relative to the target cluster area, and the movement direction of the target relative to the critical sub-block, the target's behavior is warned. In order to comprehensively consider the warning area where the target is located, the distance of the target from the target cluster area, and the direction of the target's movement, the target's behavior is warned, thereby improving the accuracy of behavior warning.

[0039] In one application method, the preset safety value corresponding to the warning area where the target is located is used as the base value. The distance of the target relative to the nearest target cluster area is used as the first deduction factor, and the movement direction of the target relative to the critical sub-plot is used as the second deduction factor. The first deduction factor is positively correlated with the distance of the target relative to the nearest target cluster area, and the second deduction factor is negatively correlated with the angle between the target and the critical sub-plot in the direction away from the radar. Therefore, when the target moves away from the target cluster area and / or the target moves in the direction away from the radar, the safety value of the target is appropriately reduced from the base value, so as to provide early warning of the target's behavior based on the final obtained safety value.

[0040] In another application, a preset safety value corresponding to the warning area where the target is located is used as the base value. The distance of the target relative to the nearest target cluster area is used as the first bonus factor, and the movement direction of the target relative to the critical sub-plot is used as the second deduction factor. The first bonus factor is negatively correlated with the distance of the target relative to the nearest target cluster area. Therefore, when the target is close to the nearest target cluster area, it is easier for targets in the target cluster area to discover and seek help from nearby targets, so a bonus is appropriately added to targets that are close to the target cluster area. The second deduction factor is negatively correlated with the angle between the target and the critical sub-plot in the direction away from the radar. Therefore, when the target moves away from the radar, the target's destination is towards a dangerous area, so a deduction is appropriately added to targets that are moving away from the radar. A safety value is obtained on the base value corresponding to the target, and a warning is given for the target's behavior based on the final safety value.

[0041] The above scheme obtains a radar measurement map acquired by radar, performs rasterization processing on the radar measurement map, dividing it into multiple sub-map tiles. The radar measurement map includes a preset warning line. All targets in the radar measurement map are clustered, with targets that are close together grouped together. The sub-map tiles where targets are clustered are designated as target clustering areas, resulting in at least one target clustering area. Each target clustering area corresponds to at least one sub-map tile. The preset warning line is pre-set and does not change. However, in specific application environments, the scenario requiring warning may change, and targets may form target clustering areas based on their own judgment and instincts. Therefore, based on the preset warning line and target clustering areas, targets are identified from all sub-map tiles. The critical sub-block divides the critical sub-block and the sub-blocks on both sides into at least two warning areas bounded by the critical sub-block. Therefore, the critical sub-block can be regarded as a correction to the preset warning line. The warning areas divided based on the critical sub-block are more suitable and reasonable for the application scenario. Each warning area has its own preset safety value. Then, based on the preset safety value corresponding to the warning area where the target is located, the distance of the target from the target gathering area, and the direction of movement of the target relative to the critical sub-block, the target's behavior is warned. In order to comprehensively consider the warning area where the target is located, the distance of the target from the target gathering area, and the direction of movement of the target, the target's behavior is warned, thus improving the accuracy of behavior warning.

[0042] Please see Figure 3 , Figure 3 This is a flowchart illustrating another embodiment of the radar-based behavioral early warning method of this application, the method comprising:

[0043] S301: Obtain the radar measurement map collected by the radar, perform rasterization processing on the radar measurement map to obtain multiple sub-maps corresponding to the radar measurement map, wherein the radar measurement map is equipped with a preset warning line.

[0044] Specifically, the radar is set up on one side of a preset water body, and the preset warning line is related to the water level of the preset water body. The preset water body includes, but is not limited to, oceans, rivers, and lakes. When the preset water body is ocean, the radar is set up on a beach or shore. When the preset water body is a river or lake, the radar is set up on a dam or embankment to facilitate the installation of the radar and to monitor targets near the preset water body.

[0045] Furthermore, the preset warning line is pre-set based on the water level of a preset water body, specifically based on historical averages or historical extreme values, to provide a reference for the location of the target.

[0046] In one application scenario, the preset water body is the ocean, and the radar is set on the beach. The preset warning line can be set based on the lowest water level after the tide recedes to reduce the probability of frequent false alarms, or it can be set based on the highest water level after the tide rises to improve the timeliness of behavioral warnings and ensure the safety of the target.

[0047] In another application scenario, the water body is preset to be a river, and the radar is set on the embankment. The preset warning line can be set at a fixed distance from the historical average water level in order to improve the timeliness of behavioral warnings and ensure the safety of the target.

[0048] In one application method, a radar measurement map acquired by radar is obtained, and the radar measurement map is rasterized to obtain multiple sub-maps corresponding to the radar measurement map. This includes: obtaining multiple frames of radar measurement data acquired by radar within a preset period; generating a radar measurement map based on the multiple frames of radar measurement data within the preset period; and rasterizing the radar measurement map based on its measurement range and resolution to obtain multiple sub-maps corresponding to the radar measurement map.

[0049] Specifically, the radar has a corresponding sampling frequency. Within a preset period corresponding to the sampling frequency, the radar collects multiple frames of radar measurement data. The radar measurement data includes the acquisition time, the target's position, speed, and direction of motion. Based on all the radar measurement data within the preset period, the points corresponding to each target are generated in the radar coordinate system to obtain the radar measurement map.

[0050] Furthermore, based on the measurement range and resolution of the radar measurement map, the radar measurement map is rasterized and divided into multiple sub-maps that match the measurement range and resolution to improve the rationality of radar measurement map segmentation. The information corresponding to each sub-map includes the sub-map index and the number of targets in the sub-map. Each sub-map is a single-channel image / grayscale image. After obtaining the sub-maps corresponding to the radar measurement map, it is convenient to obtain the target cluster area and correct the preset warning line.

[0051] S302: Based on the distance of each target relative to the radar and the distance between each target, cluster all targets to obtain at least one target cluster.

[0052] Specifically, radar has higher accuracy in identifying targets that are closer to it when measuring targets. Therefore, the closer the targets are to the radar, the more accurate the distance between the targets is.

[0053] Furthermore, based on the distance of each target relative to the radar and the distance between each target, all targets are clustered, thereby integrating the radar's identification accuracy into the target clustering process, improving the accuracy of target clustering, and obtaining at least one target cluster, wherein each target cluster includes multiple targets.

[0054] In one application method, based on the distance of each target relative to the radar and the distance between each target, all targets are clustered to obtain at least one target cluster. This includes: determining the clustering distance threshold corresponding to each target based on the distance of each target relative to the radar; wherein the distance of the target relative to the radar corresponds to multiple distance ranges, and each distance range is matched with its own corresponding clustering distance threshold; and clustering all targets based on the relationship between the distance between each target and the clustering distance threshold corresponding to each target to obtain at least one target cluster.

[0055] Specifically, the distance between the target and the radar is set to multiple distance ranges, and each distance range has its own corresponding clustering distance threshold. After obtaining the distance between the target and the radar, the distance range in which the distance between the target and the radar is located is determined, thereby obtaining the clustering distance threshold that matches the distance range.

[0056] Furthermore, for each target, it is determined whether the distance between each target and other targets is less than the corresponding clustering distance threshold. Targets whose distances are less than the clustering distance threshold are clustered into the same target cluster. The above process is expressed by the following formula:

[0057]

[0058]

[0059] N(C k )>N t (3)

[0060] Among them, e i To represent a target, its spatial location is (x i y i );e j This indicates that the spatial location of a target in a cluster is... dis(e i e j ) represents the distance between two targets; κ ik It is the clustering distance threshold; N(C) k ) represents clustering C k The number of targets in the array; N t This represents the threshold number of clusters that meet the target criteria.

[0061] S303: Based on the sub-map tiles where the targets in each target cluster are located, determine the target cluster corresponding to each target cluster, and set a cluster safety value for each target cluster. The cluster safety value is positively correlated with the number of targets in the target cluster.

[0062] Specifically, the sub-plots where the targets are located in each target cluster are determined, and the sub-plots where the targets are located in the same target cluster are taken as the target clusters corresponding to that target cluster, and a clustering safety value is set for each target cluster.

[0063] Understandably, the areas where targets cluster are usually chosen spontaneously by the targets themselves. When there are risk factors within the target cluster area, most targets will instinctively leave the risky area. Therefore, the more targets included in a target cluster area, the safer the corresponding target cluster area tends to be. Moreover, when there are more targets, they can also help each other. Thus, the cluster safety value corresponding to each target cluster area is positively correlated with the number of targets in the target cluster area, which can improve the rationality of the cluster safety value corresponding to the target cluster area.

[0064] S304: Based on the positional relationship between the preset warning line and all target cluster areas, determine the critical sub-block in the sub-block corresponding to the radar measurement map.

[0065] Specifically, the positional relationship between the preset warning line and all target clustering areas is determined in the radar measurement map, and the critical sub-blocks are determined in the corresponding sub-blocks of the radar measurement map based on the target clustering areas.

[0066] In one application scenario, the preset warning line is set based on the historical minimum value corresponding to the water level of a preset water body. When determining the critical sub-map, the critical sub-map is determined based on the location of the target cluster area in the sub-map on the side of the preset warning line closer to the radar. Thus, when the preset warning line is located deep into the preset water body, the sub-map containing the target cluster area that has crossed the preset warning line is removed, and the obtained critical sub-map is used as the corrected warning line, thereby improving the security of the critical sub-map.

[0067] In another application scenario, the preset warning line is set based on the historical maximum value corresponding to the water level of a preset water body. When determining the critical sub-block, if there is a target cluster area on the side of the sub-block that is far from the radar relative to the preset warning line, the critical sub-block is determined based on the position of the target cluster area in the sub-block that is far from the radar relative to the preset warning line. Thus, when the position of the preset warning line is relatively close to the radar, the critical sub-block is determined based on the sub-block corresponding to the target cluster area that crosses the preset warning line. The obtained critical sub-block is used as the corrected warning line, reducing the probability of frequent alarms.

[0068] In another application scenario, the preset warning line is set based on the historical average water level of a preset water body. When determining the critical sub-map, if there is a target cluster area on the side of the preset warning line that is far from the radar, the critical sub-map is determined based on the location of the target cluster area in the sub-map that is far from the radar. If there is no target cluster area on the side of the preset warning line that is far from the radar, the critical sub-map is determined based on the location of the target cluster area in the sub-map that is close to the radar. The obtained critical sub-map is used as the corrected warning line, thereby improving the accuracy of the corrected warning line.

[0069] S305: The sub-maps on the side furthest from the radar relative to the critical sub-maps, as well as the critical sub-maps, are designated as the first warning zone.

[0070] Specifically, please refer to [the relevant document] again. Figure 2 The sub-patterns furthest from the radar on the relatively critical sub-pattern side, as well as the critical sub-patterns themselves, are all designated as the first warning zone, that is... Figure 2 The darkest sub-tiles in the middle all belong to the first warning area.

[0071] S306: Based on the distance between the sub-map tiles and the critical sub-map tiles, the sub-map tiles closer to the radar side of the relatively critical sub-map tiles are divided to obtain at least one second warning area.

[0072] Specifically, in the sub-maps of the relatively critical sub-maps that are closer to the radar, the sub-maps are divided according to their distance from the relatively critical sub-maps to obtain at least one second warning area. This allows for a more detailed division of the sub-maps of the relatively critical sub-maps that are closer to the radar, thereby improving the accuracy of early warning of target behavior.

[0073] Furthermore, based on the distance between the sub-map tiles and the critical sub-map tiles, the sub-map tiles closer to the radar side of the relative critical sub-map tiles are divided to obtain at least one second warning area. The method further includes: setting preset safety values ​​for the first warning area and the second warning area; wherein, the preset safety value of the first warning area is less than the preset safety value of the second warning area, and when the number of second warning areas exceeds one, the preset safety value of the second warning area is positively correlated with the distance between the second warning area and the critical sub-map tile.

[0074] Specifically, please refer to [the relevant document] again. Figure 2 ,by Figure 2 For example, when setting preset safety values ​​for the first warning area and the second warning area, the first warning area corresponds to... Figure 2In the darkest area, the preset safety value of the first warning area is less than that of the second warning area. When a sub-map that is closer to the radar than the critical sub-map is divided into at least two second warning areas, the preset safety value of the second warning area is positively correlated with the distance of the second warning area from the critical sub-map. That is, the greater the distance of the sub-map within the second warning area from the critical sub-map, the greater the corresponding preset safety value. This is to ensure that the second warning areas that are far from the critical sub-map obtain a larger preset safety value, thereby improving the rationality of the preset safety value and the rationality of the target safety value obtained at the end, and improving the accuracy of behavioral warnings.

[0075] S307: For each target, based on the preset safety value corresponding to the warning area where the target is located, the distance of the target relative to the target cluster area, and the movement direction of the target relative to the critical sub-block, the target behavior is given a warning.

[0076] Specifically, for each target, the system takes into account the warning zone where the target is located, the distance of the target from the target gathering area, and the direction of the target's movement to issue early warnings about the target's behavior, thereby improving the accuracy of behavior warnings.

[0077] In one application method, please refer to Figure 4 , Figure 4 yes Figure 3 A flowchart of one embodiment corresponding to step S307 is shown. Step S307 specifically includes:

[0078] S401: Use the preset safety value corresponding to the warning area where the target is located as the first safety value corresponding to the target.

[0079] Specifically, the warning zone where the target is located is determined, and the preset safety value corresponding to the warning zone where the target is located is used as the first safety value corresponding to the target.

[0080] S402: Based on the cluster safety value corresponding to the target cluster area closest to the target, and the distance of the target relative to the nearest target cluster area, determine the second safety value corresponding to the target, wherein the second safety value is positively correlated with the cluster safety value and negatively correlated with the distance of the target relative to the nearest target cluster area.

[0081] Specifically, the target cluster area closest to the target is determined, and a second safety value corresponding to the target is determined based on the cluster safety value corresponding to the target cluster area closest to the target and the distance of the target relative to the target cluster area.

[0082] The above process can be expressed by the following formula:

[0083]

[0084]

[0085]

[0086] Where, d imin Let N represent the nearest distance between target i and its cluster, and N represent the number of clusters. If N is 0, then target d... imin The value is the default maximum value. i Represents the spatial location (x) of target i i y i ), P j d(p) represents the spatial location of cluster j. i P j ) represents the spatial distance between target i and cluster j. Represents the spatial location of the k-th point in cluster j. This represents the distance between target i and the k-th point in cluster j.

[0087] Furthermore, the second safety value is positively correlated with the cluster safety value r, and negatively correlated with the distance of the target relative to the nearest target cluster area. The second safety value is... exp represents an exponential function with the natural constant e as its base, thus using the cluster safety value as the basis for the second safety value. The second safety value is smaller for targets farther away from the target cluster area compared to those closer to the target cluster area.

[0088] S403: Based on the critical sub-block, the preset warning line is corrected to obtain the target warning line. Based on the angle between the target's movement direction and the normal of the target warning line, the third safety value corresponding to the target is determined. The normal direction of the target warning line is towards the preset water body, and the third safety value is negatively correlated with the angle.

[0089] Specifically, based on the critical sub-blocks, the preset warning lines are corrected, and new warning lines are drawn as the target warning lines. The angle between the target's movement direction and the normal to the target warning line is then determined. The above process is expressed by the following formula:

[0090] dβ i =|β i -β L | (7)

[0091] Wherein, β i β represents the direction of motion of target i. L Indicates the direction of the normal to the target warning line.

[0092] Furthermore, based on this included angle, the third safety value corresponding to the target is determined. The normal direction of the target warning line is towards the preset water body. That is, when the target's movement direction coincides with the normal direction, the included angle is the smallest and the third safety value is the largest.

[0093] S404: The first safety value, the second safety value, and the third safety value are weighted and summed to obtain the target safety value corresponding to the target. The weights corresponding to the first safety value and the second safety value are positive, and the weight corresponding to the third safety value is negative.

[0094] Specifically, the first, second, and third safety values ​​are weighted and summed to obtain the target safety value. This value, combined with the target's location within the warning zone, its distance from the target's concentration area, and its direction of movement, provides an early warning of the target's behavior, improving the accuracy of behavioral warnings. The above process is expressed by the following formula:

[0095]

[0096] Among them, r1 i r2 represents the preset safety value of the warning area where target i is located. i The cluster safety value represents the distance of target i from the nearest target cluster area. α1, α2, and α3 respectively identify the weights of the first, second, and third safety values, where α3 is negative and α1 and α2 are positive. The preset safety value corresponding to the warning area where the target is located is the base value of the target safety value, hence α1 is positive. When the target is close to the target cluster area, the target is relatively safer, hence α2 is positive. The base value is added based on the distance of the target from the nearest target cluster area. When the target moves in the direction of the preset water body, the target's behavior is relatively dangerous, hence α3 is negative. The base value is subtracted based on the angle between the target and the normal, resulting in the final target safety value, thus improving the rationality of the target safety value.

[0097] S405: Based on the relationship between the target's safety value and the safety threshold, issue an early warning for the target's behavior.

[0098] Specifically, the target safety value is compared with the safety threshold to determine the relationship between the two. When the target safety value is less than the safety threshold, an early warning is issued for the target's behavior to improve the target's safety.

[0099] In this embodiment, the radar is positioned on one side of a preset water body, and the preset warning line is related to the water level of the preset water body. After obtaining the radar measurement map, based on the measurement range and resolution of the radar measurement map, the radar measurement map is rasterized, dividing it into multiple sub-maps that match the measurement range and resolution to improve the rationality of radar measurement map segmentation. Based on the distance of each target relative to the radar and the distance between each target, all targets are clustered, thereby integrating the radar's recognition accuracy into the target clustering process, improving the accuracy of target clustering, and obtaining at least one target cluster. The sub-maps where targets in the same target cluster are located are taken as the target aggregation area corresponding to that target cluster, and an aggregation safety value is set for the target aggregation area. The pre-warning line is determined in the radar measurement map. The positional relationship between the warning line and all target clustering areas is established. Using the target clustering areas as a reference, critical sub-plots are determined in the corresponding sub-plots of the radar measurement map. The obtained critical sub-plots are used as the corrected warning line. A first safety value is determined based on the preset safety value corresponding to the warning area where the target is located. A second safety value is determined based on the distance of the target from the target clustering area and the clustering safety value corresponding to the target clustering area. A third safety value is determined based on the angle between the target's movement direction and the normal of the target warning line. The first, second, and third safety values ​​are weighted and summed to obtain the target safety value corresponding to the target. By comprehensively considering the warning area where the target is located, the distance of the target from the target clustering area, and the target's movement direction, warnings are issued for the target's behavior, thereby improving the accuracy of behavior warnings.

[0100] It should be noted that in scenarios where early warning of target behavior is required, cameras can be set up in addition to radar. Therefore, cameras can be added to collect video based on any of the above embodiments, and the radar measurement map and video frames in the video can be combined to provide early warning of target behavior.

[0101] Please see Figure 5 , Figure 5 This is a flowchart illustrating one embodiment of the behavior warning method based on radar-visual fusion of this application. The method includes:

[0102] S501: Obtain radar measurement maps acquired by radar and video frames acquired by camera, and determine the preset warning areas in the radar measurement maps and video frames, wherein there is a corresponding mapping relationship between the radar measurement maps and video frames.

[0103] Specifically, the radar and camera are positioned on the same side of the area to be monitored, obtaining radar measurement maps collected by the radar and video frames collected by the camera. The video frames are extracted from the video captured by the camera. The timestamps of the radar measurement maps and the video frames correspond, and there is a mapping relationship between the radar measurement maps and the video frames. Based on this mapping relationship, the position in the radar measurement map can be mapped to the video frame, and the position in the video frame can also be mapped to the radar measurement map.

[0104] Furthermore, a pre-defined warning zone is determined in the radar measurement map and video frame to provide a reference for the area where the target is located.

[0105] In one application, after the camera is set on one side of the area to be monitored, a preset warning area is pre-set in the video captured by the camera. This allows the preset warning area to be set in the intuitive video scene, thereby improving the accuracy of setting the preset warning area. The preset warning area is bound to the camera. Then, when the camera captures video, the video frames extracted from the video include the preset warning area. Based on the mapping relationship, the preset warning area is mapped onto the radar measurement map to determine the preset warning area in the radar measurement map.

[0106] In another application, after the camera is positioned on one side of the area to be monitored, a preset warning area is pre-set in the video captured by the camera. This allows the preset warning area to be set in the intuitive video scene, improving the accuracy of setting the preset warning area. Based on the mapping relationship, the preset warning area is mapped from the video frames of the video to the radar measurement map acquired by the radar. The preset warning area is bound to the radar and the camera. Thus, when the radar acquires the radar measurement map, the radar measurement map includes the preset warning area, and when the camera acquires the video, the video frames extracted from the video include the preset warning area.

[0107] S502: Cluster all targets in the radar measurement map to obtain at least one target cluster area, and map the target cluster area to the video frame based on the mapping relationship.

[0108] Specifically, all targets in the radar measurement map are clustered, with targets that are close to each other grouped together. Based on the area enclosed by the targets that are clustered together, at least one target cluster area is determined.

[0109] In one application method, a clustering distance threshold is obtained, wherein the clustering distance threshold is a preset fixed value. Based on the clustering distance threshold, all targets in the radar measurement map are clustered, and targets within the clustering distance threshold are clustered into one class. The area enclosed by the targets clustered into one class is taken as the target clustering area, thereby obtaining at least one target clustering area, thereby improving the efficiency of obtaining the target clustering area.

[0110] In another application, a clustering distance threshold is determined based on the distance between the target and the radar. The clustering distance threshold is proportional to the distance between the target and the radar to reduce the impact of measurement errors caused by the target being far from the radar on clustering. Based on the clustering distance threshold, all targets in the radar measurement map are clustered. Targets within the clustering distance threshold are grouped into one class, and the area enclosed by the clustered targets is taken as the target clustering area, thus obtaining at least one target clustering area.

[0111] Furthermore, based on the mapping relationship, the target cluster area is mapped from the radar measurement map to the corresponding position in the video frame, so that the position of the target cluster area can also be determined in the video frame.

[0112] S503: For each target, based on the target's position relative to the preset warning area and the target's distance relative to the target gathering area, determine the corresponding safety value of the target in the radar measurement map and video frame.

[0113] Specifically, for each target, based on the target's position relative to the preset warning area and the target's distance relative to the target cluster area, the corresponding safety value of the target in the radar measurement map and video frame is determined. In the specific application environment, the target will form a target cluster area based on its own judgment and instinct. After the target cluster area is formed, the target closer to the target cluster area is safer than the target farther away from the target cluster area. Therefore, by combining the target's position relative to the preset warning area and the target's distance relative to the target cluster area, the safety value of the target is determined in the radar measurement map and video frame respectively, thereby improving the accuracy of the safety value.

[0114] In one application method, based on the target's position relative to a preset warning area, the base value corresponding to the target's safety value is determined in both the radar measurement map and the video frame. The distance of the target relative to the nearest target cluster area is used as the first deduction item. The first deduction item is positively correlated with the distance of the target relative to the nearest target cluster area. Thus, when the target moves away from the target cluster area, the base value is deducted to obtain the target's corresponding safety value.

[0115] In another application, based on the target's position relative to a preset warning area, the base value corresponding to the target's safety value is determined in both the radar measurement map and the video frame. The distance of the target relative to the nearest target cluster area is used as the first bonus item. The first bonus item is positively correlated with the distance of the target relative to the nearest target cluster area. Thus, when the target is close to the nearest target cluster area, it is easier for targets in the target cluster area to discover and seek help from nearby targets. Appropriate bonuses are added to targets that are close to the target cluster area, and the base value corresponding to the target is added to obtain the target's safety value.

[0116] S504: The safety values ​​corresponding to each target in the radar measurement map and video frame are weighted and summed to obtain the early warning safety value for each target.

[0117] Specifically, the weight of each target in the radar measurement map and video frame is determined, and the corresponding safety values ​​of each target in the radar measurement map and video frame are weighted and summed to reasonably allocate the corresponding safety values ​​of the targets in the radar measurement map and video frame, so as to obtain the early warning safety value of each target.

[0118] In one application, the weights of both the radar measurement map and the video frame are values ​​between 0 and 1, and their sum is 1. The radar measurement map and the video frame have overlapping areas. Within the overlapping area, the weight of the video frame is greater than that of the radar measurement map. Within the non-overlapping area, when the radar does not capture the non-overlapping area, the weight of the video frame within the non-overlapping area is 1. Similarly, within the non-overlapping area, when the camera does not capture the non-overlapping area, the weight of the radar measurement map within the non-overlapping area is 1. Thus, within the overlapping area, the video frame is used as the reference, and within the non-overlapping area, the area that can be captured is used as the reference. The corresponding safety values ​​of each target in the radar measurement map and the video frame are weighted and summed to obtain the warning safety value for each target.

[0119] In another application, the weights of both the radar measurement map and the video frame are values ​​between 0 and 1, and their sum is 1. The area to be monitored is located within the overlapping area of ​​the radar measurement map and the video frame. The initial weight of the video frame is greater than the initial weight of the radar measurement map. The visibility rating within the video frame is determined. Based on the visibility rating and the initial weight of the video frame, the current weight of the video frame is determined, and then the current weight of the radar measurement map is determined. Thus, when the visibility in the video frame is low, the weight of the video frame is reduced to improve the rationality of the safety value. The safety values ​​corresponding to each target in the radar measurement map and the video frame are weighted and summed to obtain the warning safety value corresponding to each target.

[0120] S505: Provide early warnings on the behavior of targets based on the early warning safety values ​​corresponding to each target.

[0121] Specifically, warnings are issued for the behavior of targets based on the warning safety values ​​corresponding to each target, and it is determined whether warning signals are generated for at least some targets.

[0122] In one application method, the warning safety value corresponds to a safety value threshold. Warnings are issued for targets whose safety values ​​are lower than the safety value threshold in order to improve the efficiency of target warnings.

[0123] In another application, a safety threshold is determined for each target based on its distance from the radar and camera. The radar has higher recognition accuracy for targets closer to the radar when measuring targets, and the camera has higher recognition accuracy for targets closer to the radar and camera when collecting video. This allows for setting a larger safety threshold for targets farther from the radar and camera, reducing the impact of recognition accuracy on the warning results, and issuing warnings for targets with safety values ​​below the safety threshold.

[0124] The above scheme obtains radar measurement maps acquired by radar and video frames captured by cameras. A preset warning area is determined in both the radar measurement map and the video frames. Furthermore, there is a mapping relationship between the radar measurement map and the video frames. All targets in the radar measurement map are clustered, obtaining at least one target cluster area. Based on the mapping relationship, this target cluster area is mapped from the radar measurement map to its corresponding position in the video frame. Based on the target's position relative to the preset warning area and the distance of the target relative to the target cluster area, the corresponding safety value of the target in the radar measurement map and the video frame is determined. However, in specific application environments, targets may react based on their own judgment and instincts. Target clusters are formed, and once a cluster is formed, targets closer to the cluster are relatively safer than those farther away. Therefore, by combining the target's position relative to the preset warning area and its distance from the cluster, a target safety value is determined in both radar measurement maps and video frames, improving the accuracy of the safety value. Then, the safety values ​​corresponding to each target in the radar measurement maps and video frames are weighted and summed to reasonably allocate the target's corresponding safety values ​​in both maps and frames, obtaining a warning safety value for each target. Based on the warning safety values ​​corresponding to each target, warnings are issued regarding target behavior, improving the accuracy of behavioral warnings.

[0125] Please see Figure 6 , Figure 6 This is a flowchart illustrating another embodiment of the behavior warning method based on radar-visual fusion in this application. The method includes:

[0126] S601: Obtain radar measurement maps acquired by radar and video frames acquired by camera, and determine the preset warning areas in the radar measurement maps and video frames, wherein there is a mapping relationship between the radar measurement maps and video frames.

[0127] Specifically, please refer to Figure 7 , Figure 7This is a schematic diagram illustrating an application scenario of one embodiment of the behavior warning method based on radar-visual fusion according to this application. The radar and camera are positioned on one side of a preset water body, and a preset warning area is pre-configured in the camera so that the video frames captured by the camera include the preset warning area. The preset warning area is related to the water level of the preset water body and the position of the camera relative to the preset water body. The preset water body includes, but is not limited to, oceans, rivers, and lakes. When the preset water body is ocean, the radar and camera are positioned on a beach or shore; when the preset water body is a river or lake, the radar and camera are positioned on a dam or embankment to facilitate the placement of the radar and camera and to monitor targets near the preset water body.

[0128] Furthermore, a preset warning zone is pre-configured in the camera. This preset warning zone is set based on the water level of a preset water body and the camera's position relative to the preset water body. Specifically, it can be set based on historical averages or historical extreme values, serving as a reference for the target's location. Figure 7 As shown in the upper half, the wavy line represents the preset water body, the dashed rectangle represents the video frame extracted from the video captured by the camera, and the black solid line represents the preset warning area. Thus, when the camera captures video including the preset water body, the video frame extracted from the video includes the preset warning area as shown in the figure.

[0129] In one application scenario, the preset water body is the ocean, and the radar and cameras are set on the beach. The preset warning area can be set based on the lowest water level after the tide recedes to reduce the probability of frequent false alarms, or it can be set based on the highest water level after the tide rises to improve the timeliness of behavioral warnings and ensure the safety of the target.

[0130] In another application scenario, the water body is preset to be a river, and radar and cameras are set on the embankment. The preset warning area can be set based on a fixed distance from the historical average water level, so as to improve the timeliness of behavioral warnings and ensure the safety of the target.

[0131] In one application method, obtaining a radar measurement map acquired by radar and a video frame acquired by a camera, and determining a preset warning area in the radar measurement map and the video frame, includes: obtaining a radar measurement map acquired by radar and a video frame acquired by a camera, determining the location of the preset warning area in the video frame; and mapping the preset warning area in the video frame to the radar measurement map based on the mapping relationship, thereby determining the location of the preset warning area in the radar measurement map.

[0132] Specifically, radar measurement maps acquired by radar and video frames acquired by camera are obtained. The radar measurement maps and video frames are updated based on a preset period, and the timestamps of the radar measurement maps and video frames correspond within the same period. The preset period can be any value within 5-30 seconds.

[0133] Furthermore, the location of the preset warning area in the video frame is determined. Based on the mapping relationship between the video frame and the radar measurement map, the preset warning area in the video frame is mapped onto the radar measurement map, thus determining the location of the preset warning area in the radar measurement map. Figure 7 As shown in the image, the fan-shaped area represents the area corresponding to the radar measurement map. Based on the mapping relationship, the preset warning area in the video frame is mapped onto the radar measurement map, resulting in the following: Figure 7 The area shown in the lower half ensures that the preset warning area is included in both the radar measurement map and the video frame, and that the preset warning areas in the radar measurement map and the video frame correspond to each other, so as to determine the safe value of the target in the radar measurement map and the video frame based on the same standard.

[0134] In one application scenario, the mapping relationship is determined based on at least four calibration point pairs in the overlapping area of ​​the radar measurement map and the video frame, wherein the calibration point pair includes the radar coordinates of the same point in the radar measurement map and the pixel coordinates in the video frame.

[0135] Specifically, at least four identical targets are selected in the overlapping area of ​​the radar measurement map and the video frame. The position of the same target in the radar measurement map and the position in the video frame are recorded as the same calibration point pair. The calibration point pair includes the radar coordinates of the same position in the radar measurement map and the pixel coordinates in the video frame. Then, based on at least four calibration vertices, the transformation relationship between the corresponding coordinate systems of the radar measurement map and the video frame can be obtained, realizing the mutual conversion between radar coordinates and pixel coordinates.

[0136] Furthermore, the calibration points need to form convex polygons as much as possible, enclosing a large area. Therefore, the calibration points need to be distributed widely to avoid concentration. Assume the pixel coordinates are (u i v i ) and radar coordinates are (x i y i There are four calibration point pairs. The process of determining the mapping relationship is expressed by the following formula:

[0137] First, construct the following homogeneous equation:

[0138]

[0139] Furthermore, by further transforming the aligned equation, we obtain:

[0140] x i =a 11 ·u i +α 12 ·v i +α 13 (10)

[0141] Equation (10) can be considered as a constraint, and can be mathematically equivalently treated as follows:

[0142]

[0143]

[0144] Analyzing from the right side of formulas (11) and (12), the solution focuses on the ratio value, which can be set as follows:

[0145]

[0146]

[0147] Formulas (13) and (14) are the formulas for calculating pixel points and radar coordinates. The coefficient matrix can then be constructed as follows:

[0148]

[0149] Where W = [b 11 b 12 b 13 b 21 b 22 b 23 b 31 b 32 ] T Based on formula (15), there are 8 unknowns that need to be solved. Theoretically, only four calibration points are needed to solve the problem. Using four calibration points, the following measurement equation can be constructed:

[0150]

[0151] Formula (16) can be simplified as follows:

[0152] AW=b (17)

[0153] Therefore, we can obtain:

[0154] W = (A T A) -1 A T b (18)

[0155] Where W is the result of the coefficient solution, and when substituted into formulas (13) and (14), it represents the mapping relationship between image pixels and radar coordinates.

[0156] S602: Based on the distance of each target relative to the radar and the distance between each target, cluster all targets to obtain at least one target cluster.

[0157] Specifically, radar has higher accuracy in identifying targets that are closer to it when measuring targets. Therefore, the closer the targets are to the radar, the more accurate the distance between the targets is.

[0158] Furthermore, based on the distance of each target relative to the radar and the distance between each target, all targets are clustered, thereby integrating the radar's identification accuracy into the target clustering process, improving the accuracy of target clustering, and obtaining at least one target cluster, wherein each target cluster includes multiple targets.

[0159] In one application method, based on the distance of each target relative to the radar and the distance between each target, all targets are clustered to obtain at least one target cluster. This includes: determining the clustering distance threshold corresponding to each target based on the distance of each target relative to the radar; wherein the distance of the target relative to the radar corresponds to multiple distance ranges, and each distance range is matched with its own corresponding clustering distance threshold; and clustering all targets based on the relationship between the distance between each target and the clustering distance threshold corresponding to each target to obtain at least one target cluster.

[0160] Specifically, the distance between the target and the radar is set to multiple distance ranges, and each distance range has its own corresponding clustering distance threshold. After obtaining the distance between the target and the radar, the distance range in which the distance between the target and the radar is located is determined, thereby obtaining the clustering distance threshold that matches the distance range.

[0161] Furthermore, for each target, it is determined whether the distance between each target and other targets is less than the clustering distance threshold corresponding to the target. Targets whose distance to each other is less than the clustering distance threshold are clustered into the same target cluster. The specific process described above can be found in formulas (1), (2) and (3), which will not be elaborated upon in this application.

[0162] S603: Based on the area surrounded by the location of the targets in each target cluster, determine the target cluster corresponding to each target cluster, and set a cluster security value for each target cluster, wherein the cluster security value is positively correlated with the number of targets in the target cluster.

[0163] Specifically, the location of targets in each target cluster is determined, the area enclosed by targets in the same target cluster is taken as the target aggregation area corresponding to that target cluster, and an aggregation safety value is set for each target aggregation area.

[0164] Understandably, the areas where targets cluster are usually chosen spontaneously by the targets themselves. When there are risk factors within the target cluster area, most targets will instinctively leave the risky area. Therefore, the more targets included in a target cluster area, the safer the corresponding target cluster area tends to be. Moreover, when there are more targets, they can also help each other. Thus, the cluster safety value corresponding to each target cluster area is positively correlated with the number of targets in the target cluster area, which can improve the rationality of the cluster safety value corresponding to the target cluster area.

[0165] S604: Based on the mapping relationship, the position of the target cluster area in the radar measurement map is mapped to the video frame to determine the position of the target cluster area in the video frame.

[0166] Specifically, based on the mapping relationship, the position of the target cluster area in the radar measurement map is mapped to the video frame to determine the position of the target cluster area in the video frame, thereby improving the efficiency of determining the target cluster area in the video frame and eliminating the influence of weather and environmental factors on determining the target cluster area in the video frame.

[0167] S605: The preset safety value corresponding to the position of the target relative to the preset warning area is taken as the first safety value of the target.

[0168] Specifically, each of the following areas has a preset safety value: the preset warning zone, the area closer to the radar and camera relative to the preset warning zone, and the area farther away from the radar and camera relative to the preset warning zone. Therefore, after obtaining the target's position, the target's position relative to the preset warning zone is determined, thereby obtaining the preset safety value corresponding to the target's position, which is used as the first safety value.

[0169] S606: Based on the cluster safety value corresponding to the target cluster area closest to the target, and the distance of the target relative to the nearest target cluster area, determine the second safety value corresponding to the target, wherein the second safety value is positively correlated with the cluster safety value and negatively correlated with the distance of the target relative to the nearest target cluster area.

[0170] Specifically, the target cluster area closest to the target is determined. Based on the cluster safety value corresponding to the target cluster area and the distance between the target and the nearest target cluster area, a second safety value corresponding to the target is determined. The above process can be referred to formulas (4), (5) and (6), which will not be repeated here.

[0171] S607: The first and second security values ​​corresponding to the target in the radar measurement map are weighted and summed to obtain the security value corresponding to the target in the radar measurement map. The first and second security values ​​corresponding to the target in the video frame are weighted and summed to obtain the security value corresponding to the target in the video frame.

[0172] Specifically, when a target is acquired by both radar and camera, the target has a first security value and a second security value in both the radar measurement map and the video frame. When a target is acquired by only radar or camera, the target has only one set of first security values ​​and second security values.

[0173] Further, the weights corresponding to the first security value and the second security value are determined, and the first security value and the second security value corresponding to the target in the radar measurement map are weighted and summed to obtain the security value corresponding to the target in the radar measurement map; and / or, the first security value and the second security value corresponding to the target in the video frame are weighted and summed to obtain the security value corresponding to the target in the video frame.

[0174] Understandably, when a target is captured by both radar and camera, the first and second safety values ​​corresponding to the target in the radar measurement map are weighted and summed to obtain the target's corresponding safety value in the radar measurement map. Similarly, the first and second safety values ​​corresponding to the target in the video frame are weighted and summed to obtain the target's corresponding safety value in the video frame. When a target is captured only by radar, the first and second safety values ​​corresponding to the target in the radar measurement map are weighted and summed to obtain the target's corresponding safety value in the radar measurement map. When a target is captured only by camera, the first and second safety values ​​corresponding to the target in the video frame are weighted and summed to obtain the target's corresponding safety value in the video frame.

[0175] S608: The safety values ​​corresponding to each target in the radar measurement map and video frame are weighted and summed to obtain the early warning safety value for each target.

[0176] Specifically, the weight of each target in the radar measurement map and video frame is determined, and the corresponding safety values ​​of each target in the radar measurement map and video frame are weighted and summed to obtain the warning safety value for each target.

[0177] In one application method, the light intensity rating and visibility rating in the video frame are obtained. Based on the ratio of the sum of the light intensity rating and visibility rating to a preset coefficient, the first weight corresponding to the video frame is determined. Based on the first weight, the second weight corresponding to the radar measurement map is determined. The first weight and the second weight are both between 0 and 1, and the sum of the first weight and the second weight is 1. Based on the first weight and the second weight, the safety values ​​corresponding to each target in the radar measurement map and the video frame are weighted and summed to obtain the warning safety value corresponding to each target.

[0178] Specifically, the current light intensity rating and visibility rating in the video frame are obtained, the sum of the light intensity rating and visibility rating is determined, the sum of the light intensity rating and visibility rating is divided by a preset coefficient to obtain the first weight corresponding to the video frame, and after obtaining the first weight, the second weight is determined based on the sum of the first weight and the second weight being 1, wherein both the first weight and the second weight are between 0 and 1.

[0179] Furthermore, based on the mapping relationship, the same target in the radar measurement map and video frame is determined. Based on the first weight and the second weight, the corresponding security values ​​of the same target in the radar measurement map and video frame are weighted and summed to obtain the warning security value corresponding to each target.

[0180] In one application scenario, the preset coefficient value is 10. Light intensity and visibility are divided into five levels, each corresponding to an integer from 1 to 5. Higher ratings have larger numerical values, denoted as light intensity rating C. i Visibility rating C v Each video frame has an initial weight β. v It received a light intensity rating of C. i Visibility rating C v Afterwards, based on C i and C v The ratio of the sum of the values ​​to a preset coefficient determines the first weight of the target in the video frame. Based on the sum of weights being 1, the target's second weight in the radar measurement map is... Wherein, the initial weight β v As a customizable weight scaling factor, it can adjust the weight of the target in the video frame. When the illumination intensity rating is C... i Visibility rating C v When all values ​​are at their maximum values, the first weight of the target in the video frame is β. v .

[0181] It should be noted that the light intensity rating and visibility rating in the video frames are updated at preset intervals.

[0182] Furthermore, after obtaining radar measurement maps acquired by radar and video frames acquired by cameras, and determining the preset warning areas in the radar measurement maps and video frames, the process further includes: in response to updates to the light intensity rating and visibility rating, scoring multiple sub-regions within the preset warning areas based on the overlapping and non-overlapping areas of the radar measurement maps and video frames, as well as the updated light intensity rating and visibility rating, to obtain a safety score corresponding to each sub-region; wherein, the safety score of the overlapping area is higher than that of the non-overlapping area, and the safety score is positively correlated with the light intensity rating and visibility rating.

[0183] Specifically, due to changes in weather conditions, the light intensity rating and visibility rating in the video frames are updated periodically with a preset duration to adapt to changes in weather conditions and avoid the computational cost of real-time updates in each video frame.

[0184] Furthermore, after the light intensity rating and visibility rating are updated, the overlapping and non-overlapping areas of the radar measurement map and video frame are determined to adapt to application scenarios where the video frame angle may change. Based on the overlapping and non-overlapping areas of the radar measurement map and video frame, as well as the updated light intensity rating and visibility rating, multiple sub-regions in the preset warning area are scored to obtain the safety score corresponding to each sub-region.

[0185] It is understandable that targets in overlapping areas can be captured by radar and cameras. Therefore, the security score of overlapping areas is higher than that of non-overlapping areas. When the light intensity rating and visibility rating in the video frame are higher, the target is more obvious in the video frame, and thus the security score is also higher. After obtaining the security scores of each sub-area, security personnel can be prompted to pay special attention to the sub-areas with lower security scores in order to allocate security forces reasonably.

[0186] S609: Provide early warnings for the behavior of targets based on the early warning safety values ​​corresponding to each target.

[0187] Specifically, warnings are issued for the behavior of targets based on the warning safety values ​​corresponding to each target, and it is determined whether warning signals are generated for at least some targets.

[0188] In one application scenario, a warning line is determined based on the side of the preset warning area furthest from the radar and camera. For each target, a safety threshold is determined based on the target's location, distance from the target's cluster area, and the angle between the target's movement direction and the normal to the warning line. Based on the relationship between the target's warning safety value and the safety threshold, a warning is issued for the target's behavior. The side of the warning line whose normal faces the radar and camera, the safety threshold is negatively correlated with the target's distance from the radar and camera, the target's distance from the cluster area, and the angle.

[0189] Specifically, please refer to [the relevant document] again. Figure 7 On the side of the pre-defined warning zone furthest from the radar and cameras, the corresponding warning line is determined. Figure 7Within the pre-set warning area, the uppermost line determines the safety threshold for each target based on its position relative to the pre-set warning area, its distance from the target cluster area, and the angle between the target's direction of movement and the normal to the warning line. The safety threshold is negatively correlated with the target's distance from the radar and camera in the direction toward the pre-set warning area. Therefore, when a target is within the pre-set warning area, or when a target has left the pre-set warning area in the direction toward the pre-set warning area, the greater the distance between the target and the radar and camera, the lower the target's safety threshold.

[0190] Furthermore, the greater the distance between the target and the target cluster area, the lower the probability that the target can be assisted by other targets within the group. Therefore, the safety threshold is negatively correlated with the distance between the target and the target cluster area. When the target moves away from the radar and camera relative to the warning line, the target is moving towards a relatively dangerous area. Since the normal of the warning line faces the radar and camera, the angle between the target's movement direction and the normal of the warning line is the largest at this time. Therefore, the safety threshold is negatively correlated with the angle between the target's movement direction and the normal of the warning line.

[0191] Understandably, based on the target's position relative to the preset warning area, the target's distance relative to the target gathering area, and the angle between the target's movement direction and the normal to the warning line, the corresponding safety threshold for the target is comprehensively determined. This allows for setting different safety thresholds for different target states. Based on the relationship between the target's corresponding warning safety value and the safety threshold, warnings are issued for the target's behavior, enabling more accurate and precise behavioral warnings for each target.

[0192] In this embodiment, the radar and camera are positioned on one side of a preset water body. The preset warning area is related to the water level of the preset water body and the position of the camera relative to the preset water body. A radar measurement map acquired by the radar and video frames acquired by the camera are obtained. The preset warning area is determined in the radar measurement map and video frames. Furthermore, there is a mapping relationship between the radar measurement map and the video frames. Based on the distance of each target relative to the radar and the distance between each target, all targets are clustered. This integrates the radar's identification accuracy into the target clustering process, improving the accuracy of target clustering and obtaining at least one target cluster. This determines the target aggregation area, and based on the mapping relationship, the target aggregation area is mapped from the radar measurement map to the corresponding position in the video frame. The safety value corresponding to each target in the radar measurement map and video frame is determined. The illumination intensity rating and visibility rating in the frame determine the first weight of the target in the video frame and the second weight of the target in the radar measurement map. Then, the safety values ​​corresponding to each target in the radar measurement map and video frame are weighted and summed to reasonably allocate the safety values ​​corresponding to the target in the radar measurement map and video frame, and obtain the warning safety value corresponding to each target. Based on the position of the target relative to the preset warning area, the distance of the target relative to the target gathering area, and the angle between the target's movement direction and the normal of the warning line, the safety value threshold corresponding to the target is comprehensively determined. Thus, different safety value thresholds are set for different target states. Based on the relationship between the warning safety value corresponding to the target and the safety value threshold, warnings are issued for the target's behavior, so as to provide more accurate and precise behavioral warnings for each target.

[0193] It should be noted that the side of the preset warning area away from the radar and camera corresponds to the preset warning line of the radar-based behavior warning method, thus integrating the content of the radar-based behavior warning method into the radar-visual fusion-based behavior warning method. In addition, when the area to be monitored is equipped with a camera in addition to the radar, the radar-visual fusion-based behavior warning method can also be integrated into the radar-based behavior warning method. For ease of explanation, this application only illustrates an application scenario with one radar and one camera in the accompanying drawings. However, in actual application scenarios, the number of radars and cameras corresponds to the area of ​​the area to be monitored. Therefore, the number of radars and cameras is at least one.

[0194] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 80 includes a memory 801 and a processor 802 coupled to each other. The memory 801 stores program data (not shown in the figure), and the processor 802 calls the program data to implement the method in any of the above embodiments. For the description of the relevant content, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0195] Please see Figure 9 , Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 90 stores program data 900. When the program data 900 is executed by a processor, it implements the method in any of the above embodiments. For a detailed description of the relevant content, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0196] It should be noted that the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0198] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0199] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A radar-based behavior warning method, characterized in that, The method includes: A radar measurement map acquired by radar is obtained, and the radar measurement map is rasterized to obtain multiple sub-map blocks corresponding to the radar measurement map; wherein, the radar measurement map is provided with a preset warning line; All targets in the radar measurement map are clustered to obtain at least one target clustering region; wherein, the target clustering region corresponds to at least one of the sub-map tiles; Based on the preset warning line and the target aggregation area, a critical sub-map block is determined among all the sub-map blocks. The critical sub-map block and the sub-map blocks on both sides are divided to obtain at least two warning areas bounded by the critical sub-map block. Each warning area corresponds to a preset safety value. The critical sub-map block is determined based on the positional relationship between the preset warning line and all the target aggregation areas. The critical sub-map block and the sub-map blocks on both sides are divided into at least two warning areas bounded by the critical sub-map block. For each target, a warning is issued regarding the target's behavior based on the preset safety value corresponding to the warning area where the target is located, the distance of the target relative to the target cluster area, and the movement direction of the target relative to the critical sub-plot. Specifically, the preset safety value corresponding to the warning area where the target is located serves as the base value; a distance-related adjustment value is determined based on the distance of the target relative to the nearest target cluster area; a direction-related adjustment value is determined based on the movement direction of the target relative to the critical sub-plot; and the base value, the distance-related adjustment value, and the direction-related adjustment value are used to determine the final safety value corresponding to the target. The warning is then issued based on the final safety value.

2. The radar-based behavior warning method according to claim 1, characterized in that, The radar is positioned on one side of a preset body of water, and the preset warning line is related to the water level of the preset body of water.

3. The radar-based behavior warning method according to claim 2, characterized in that, Based on the preset warning line and the target aggregation area, a critical sub-map block is determined among all the sub-map blocks. The critical sub-map block and the sub-map blocks on both sides are divided to obtain at least two warning areas bounded by the critical sub-map block, including: Based on the positional relationship between the preset warning line and all the target cluster areas, the critical sub-block is determined in the sub-block corresponding to the radar measurement map; The sub-map blocks on the side furthest from the radar relative to the critical sub-map block, as well as the critical sub-map block itself, are divided into a first warning area; Based on the distance between the sub-map block and the critical sub-map block, the sub-map block that is closer to the radar side relative to the critical sub-map block is divided to obtain at least one second warning area.

4. The radar-based behavior warning method according to claim 3, characterized in that, After dividing the sub-map tiles closer to the radar side relative to the critical sub-map tile based on the distance between the sub-map tiles and the critical sub-map tile to obtain at least one second warning area, the method further includes: A preset safety value is set for the first warning area and the second warning area; wherein, the preset safety value of the first warning area is less than the preset safety value of the second warning area, and when the number of second warning areas exceeds one, the preset safety value of the second warning area is positively correlated with the distance of the second warning area relative to the critical sub-block.

5. The radar-based behavior warning method according to claim 2, characterized in that, The step of clustering all targets in the radar measurement map to obtain at least one target cluster region includes: Based on the distance of each target relative to the radar and the distance between each target, all targets are clustered to obtain at least one target cluster; Based on the sub-map tile where the target is located in each target cluster, the target cluster corresponding to each target cluster is determined, and a cluster security value is set for each target cluster; wherein, the cluster security value is positively correlated with the number of targets in the target cluster.

6. The radar-based behavior warning method according to claim 5, characterized in that, The method involves clustering all the targets based on their distances relative to the radar and the distances between them, to obtain at least one target cluster, including: Based on the distance of each target relative to the radar, a clustering distance threshold is determined for each target; wherein the distance of each target relative to the radar corresponds to multiple distance ranges, and each distance range is matched with its own corresponding clustering distance threshold; Based on the relationship between the distances between the targets and the clustering distance thresholds corresponding to each target, all targets are clustered to obtain at least one target cluster.

7. The radar-based behavior warning method according to claim 5, characterized in that, The method of issuing a warning about the target's behavior based on the preset safety value corresponding to the warning area where the target is located, the distance of the target relative to the target cluster area, and the movement direction of the target relative to the critical sub-plot includes: The preset safety value corresponding to the warning area where the target is located is taken as the first safety value corresponding to the target; Based on the cluster safety value corresponding to the target cluster area closest to the target, and the distance of the target relative to the nearest target cluster area, a second safety value corresponding to the target is determined; wherein, the second safety value is positively correlated with the cluster safety value and negatively correlated with the distance of the target relative to the nearest target cluster area; The preset warning line is corrected based on the critical sub-plot to obtain the target warning line. The third safety value corresponding to the target is determined based on the angle between the target's movement direction and the normal of the target warning line. The normal of the target warning line is directed toward the preset water body, and the third safety value is negatively correlated with the angle. The first security value, the second security value, and the third security value are weighted and summed to obtain the target security value corresponding to the target; wherein the weights corresponding to the first security value and the second security value are positive numbers, and the weight corresponding to the third security value is a negative number. Based on the relationship between the target's safety value and the safety value threshold, an early warning is issued regarding the target's behavior.

8. The radar-based behavior warning method according to claim 1, characterized in that, The process involves obtaining a radar measurement map acquired by radar, rasterizing the radar measurement map, and obtaining multiple sub-map tiles corresponding to the radar measurement map, including: The radar measurement data is obtained from multiple frames of radar data collected by the radar within a preset period, and the radar measurement map is generated based on the multiple frames of radar measurement data within the preset period. Based on the measurement range and resolution of the radar measurement map, the radar measurement map is rasterized to obtain multiple sub-map blocks corresponding to the radar measurement map.

9. An electronic device, characterized in that, include: A memory and a processor are coupled to each other, wherein the memory stores program data, and the processor invokes the program data to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium storing program data thereon, characterized in that, When the program data is executed by the processor, it implements the method as described in any one of claims 1-8.

Images