Detection method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a detection method and device, electronic equipment and a computer readable medium, and belongs to the technical field of intelligent traffic, and the method comprises the steps: obtaining a first video in a historical time period of a road intersection; screening out a first individual meeting a preset condition from a plurality of sidewalk passing records in the first video to perform target tracking, and obtaining a movement track of each first individual; determining a target source point and a target vanishing point based on the motion track of each first individual; based on the obtained second video of the road intersection in the current time period, determining a real source point and a real vanishing point of each second individual passing the sidewalk in the second video; and performing abnormal behavior detection on the second video based on the distance between the target source point and the real source point of each second individual and the distance between the target vanishing point and the real vanishing point of each second individual. The target source point and the target vanishing point used for detecting the abnormal behavior can be automatically determined, and the intelligent level is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and more specifically, to a detection method, device, electronic device, and computer-readable medium. Background Art

[0002] The existing intelligent video analysis system needs to set reference values ​​based on manual experience, and judgments need to be made based on the manually set reference values ​​during the detection phase. The labor cost is high, and the accuracy and intelligence level of the video analysis module are low. Summary of the Invention

[0003] This application proposes a detection method, device, electronic device and computer-readable medium to improve the above-mentioned defects.

[0004] In a first aspect, the present application provides a detection method, which is applied to a processing module of a monitoring device, and the method includes: obtaining a first video within a historical time period of a road intersection; screening out a first individual that meets preset conditions from multiple records of passing through a sidewalk in the first video for target tracking, and obtaining a motion trajectory of each first individual; determining a target source point and a target vanishing point based on the motion trajectory of each first individual; determining a real source point and a real vanishing point of each second individual crossing the sidewalk in the second video based on a second video of the road intersection obtained within a current time period; and performing abnormal behavior detection on the second video based on the distance between the target source point and the real source point of each second individual, and the distance between the target vanishing point and the real vanishing point of each second individual.

[0005] Optionally, for a possible implementation, before selecting the first individual that meets the preset conditions from multiple records of passing the sidewalk in the first video for target tracking and obtaining the motion trajectory of each first individual, it also includes: determining the first number of frames in which each object passing the sidewalk in the first video is continuously detected and the second number of frames in which it is not detected; and taking the object among the multiple objects whose first frame number is greater than or equal to N1 and whose second frame number is less than N2 as the first individual that meets the preset conditions, where N1 is greater than N2.

[0006] Optionally, for a possible implementation, determining the target source point and the target vanishing point based on the motion trajectory of each first individual includes: traversing each of the motion trajectories to obtain a source point set and a vanishing point set that pass through the sidewalk; determining the number of clusters of the source point set to be M1 based on a first evaluation index and a second evaluation index, wherein the first evaluation index is used to characterize the similarity between different clusters, and the second evaluation index is used to characterize the intra-cluster compactness and inter-cluster separation; determining the number of clusters of the vanishing point set to be M2 based on the first evaluation index and the second evaluation index; dividing the source point set into M1 source point clusters based on a Kmeans clustering algorithm, with the center of the source point cluster being the target source point; dividing the vanishing point set into M2 vanishing point clusters based on the Kmeans clustering algorithm, with the center of the vanishing point cluster being the target vanishing point.

[0007] Optionally, for a possible implementation, the clustering number of the source point set is determined to be M1 based on the first evaluation index and the second evaluation index, including: mapping the source point set to the source point two-dimensional space plane through a two-dimensional Gaussian function; extracting n discrete points from the source point two-dimensional space plane as the initial cluster center of the source point set according to a preset rule; based on the initial cluster center of the source point set, the first evaluation index, And the second evaluation index Determine the first cluster number of the source point set as M1, where a represents the average distance between the data point v and other data points in the same cluster, and b represents the average distance between the data point v and all data points in the nearest cluster. represents the average distance from the data points in cluster i to the cluster center, represents the average distance from the data points in cluster j to the cluster center, c i with c j They represent cluster i and cluster j respectively, and K represents the number of clusters.

[0008] Optionally, for a possible implementation, the determining the number of clusters of the vanishing point set as M2 based on the first evaluation index and the second evaluation index includes: mapping the vanishing point set to a two-dimensional space plane of vanishing points through a two-dimensional Gaussian function; extracting n discrete points from the two-dimensional space plane of vanishing points according to a preset rule as the initial cluster centers of the vanishing points; and determining the number of clusters of the vanishing point set as M2 based on the initial cluster centers of the vanishing point set and the first evaluation index. And the second evaluation index Determine the first cluster number of the vanishing point set as M1, where a represents the average distance between a data point v and other data points in the same cluster, b represents the average distance between a data point v and all data points in the nearest cluster, represents the average distance from the data points in cluster i to the cluster center, represents the average distance from the data points in cluster j to the cluster center, ci with c j They represent cluster i and cluster j respectively, and K represents the number of clusters.

[0009] Optionally, for a possible implementation, there is one target source point and one target vanishing point, and abnormal behavior detection is performed on the second video based on the distance between the target source point and the real source point of each second individual, as well as the distance between the target vanishing point and the real vanishing point of each second individual, including: if the distance between the real source point and the target source point is greater than a first threshold, or the distance between the real vanishing point and the target vanishing point is greater than a second threshold, determining that abnormal behavior exists in the second video.

[0010] Optionally, for a possible implementation, there are at least two target source points and at least two target vanishing points, and abnormal behavior detection is performed on the second video based on the distance between the target source point and the real source point of each second individual, as well as the distance between the target vanishing point and the real vanishing point of each second individual, including: if the distance between the real source point and the nearest target source point is greater than a first threshold, or the distance between the real vanishing point and the nearest target vanishing point is greater than a second threshold, then it is determined that abnormal behavior exists in the second video.

[0011] In a second aspect, the present application also provides a detection device, which is applied to a processing module of a monitoring device, and the device includes: an acquisition unit, which is used to acquire a first video within a historical time period of a road intersection; an extraction unit, which is used to screen out a first individual that meets preset conditions from multiple records of passing through a sidewalk in the first video for target tracking, and obtain the motion trajectory of each first individual; a determination unit, which is used to determine a target source point and a target vanishing point based on the motion trajectory of each first individual; an acquisition unit, which is used to determine a real source point and a real vanishing point of each second individual crossing the sidewalk in the second video based on a second video of the road intersection acquired within the current time period; and a detection unit, which is used to perform abnormal behavior detection on the second video based on the distance between the target source point and the real source point of each second individual, and the distance between the target vanishing point and the real vanishing point of each second individual.

[0012] In a third aspect, the present application also provides an electronic device comprising: one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory, the one or more applications are configured to be executed by the one or more processors, and the one or more applications are configured to execute the above method.

[0013] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the above method.

[0014] In a fifth aspect, the present application also provides a computer program product, comprising a computer program / instruction, which implements the above method when executed by a processor.

[0015] An embodiment of the present application provides a detection method, which includes: first, obtaining a first video within a historical time period of a road intersection; second, screening out a first individual that meets preset conditions from multiple records of crossing a sidewalk in the first video for target tracking, and obtaining a motion trajectory of each first individual; determining a target source point and a target vanishing point based on the motion trajectory of each first individual; then, based on a second video of the road intersection obtained within a current time period, determining a true source point and a true vanishing point of each second individual crossing the sidewalk in the second video; finally, performing abnormal behavior detection on the second video based on the distance between the target source point and the true source point of each second individual, and the distance between the target vanishing point and the true vanishing point of each second individual.

[0016] On the one hand, compared to methods that capture video in real time and perform anomaly detection on the video based on manually specified parameters, the present application extracts the target source point and target vanishing point from the first video data of a historical time period, and can automatically determine the target source point and target vanishing point for detecting abnormal behavior, thereby improving the level of intelligence. On the other hand, the present application detects abnormal behavior based on the distance between the real vanishing point and the target vanishing point, as well as the distance between the real source point and the target source point. There is no need to wait until the pedestrian has crossed the sidewalk before detecting the movement trajectory, and abnormal behavior can be detected in a timely manner. When abnormal behavior is detected, the present application can quickly determine the abnormal location through the position information of the target source point and the target vanishing point, and thus timely control.

[0017] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flow chart of the detection method provided in an embodiment of the present application is shown;

[0020] Figure 2 A flow chart of a detection method provided by another embodiment of the present application is shown;

[0021] Figure 3 A flow chart of a detection method provided in another embodiment of the present application is shown;

[0022] Figure 4 A schematic diagram of a road intersection provided by an embodiment of the present application is shown;

[0023] Figure 5 Shown Figure 4 Schematic diagram of the motion trajectory in ;

[0024] Figure 6 Shown Figure 4 Schematic diagram of the target source point and target vanishing point in ;

[0025] Figure 7 The following is a structural diagram of a detection method and apparatus provided in an embodiment of the present application;

[0026] Figure 8 A structural block diagram of an electronic device provided in an embodiment of the present application is shown;

[0027] Figure 9 A structural block diagram of a computer-readable storage medium provided in an embodiment of the present application is shown;

[0028] Figure 10 The structural block diagram of the computer program product provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0031] The existing intelligent video analysis system needs to set reference values ​​based on manual experience, and judgments need to be made based on the manually set reference values ​​during the detection phase. The labor cost is high, and the accuracy and intelligence level of the video analysis module are low.

[0032] Therefore, in the embodiments of the present application, a detection method, device, electronic device and computer-readable medium are provided to solve or partially solve the above-mentioned problems.

[0033] See also Figure 1 , which shows a flow chart of a detection provided by an embodiment of the present application, and the processing module applied to the monitoring device specifically includes steps S101 to S105.

[0034] Step S101: Acquire the first video within a historical time period of a road intersection.

[0035] It should be noted that the first video can be acquired from a camera installed at a road intersection. The camera's installation position and shooting angle must be sufficient to fully capture the sidewalk and stop line at the intersection. The first video is captured from a fixed camera at the intersection over a historical period. The target source point and target vanishing point are extracted from the first video. A second video of the same intersection is then acquired in real time from the same camera. Abnormal behavior detection is performed on the second video based on the target source point and target vanishing point. For details, please refer to the subsequent embodiments.

[0036] The road can be a straight road, a T-shaped road or a cross road. It can be known that the first video stores a video of a target group crossing the sidewalk, and the target group includes pedestrians crossing the sidewalk, people crossing the sidewalk by bicycles or electric bicycles, and people crossing the sidewalk by motorcycles or electric motorcycles.

[0037] The historical time period is the time period before the current time period. The length of the historical time period can be one day, one month, or one quarter, which can be determined according to actual conditions.

[0038] Step S102: Filtering out first individuals that meet preset conditions from multiple records of passing through the sidewalk in the first video for target tracking, and obtaining a motion trajectory of each first individual.

[0039] It should be noted that the motion trajectory of each first individual can be obtained based on target tracking algorithms, spatiotemporal context information or deep learning methods.

[0040] An exemplary target tracking algorithm mainly includes target detection and target tracking. Target detection uses a detection algorithm to extract the target object in each frame of the image. Target tracking then associates the detected target objects in consecutive frames to form a trajectory. Furthermore, the trajectory can be analyzed and optimized.

[0041] In an exemplary embodiment, spatiotemporal context information can be combined with the target's position, speed, direction and other spatiotemporal features to determine the motion trajectory.

[0042] It should be noted that the trajectory represents the trajectory of the first person crossing the sidewalk. Each trajectory is a smooth curve formed by connecting multiple consecutive points. The average of w consecutive points at the beginning of the trajectory is used as the source point of the trajectory, and the average of w consecutive points at the end of the trajectory is used as the vanishing point of the trajectory. For example, w is 3.

[0043] It should be noted that when many people pass by the sidewalk, some people in the first video may be obscured. When pedestrians are obscured for a long time, it is difficult to accurately track the pedestrians. Therefore, it is necessary to remove some target groups that do not meet the preset conditions. The first individual can be determined based on the number of frames in which the object crossing the sidewalk is detected and the number of frames in which it is not detected. Specifically, the first number of frames in which each object crossing the sidewalk is continuously detected and the second number of frames in which it is not detected in the first video are determined; among multiple objects, the object with the first number of frames greater than or equal to N1 and the second number of frames less than N2 is regarded as the first individual that meets the preset conditions. For details, please refer to the subsequent embodiments.

[0044] Step S103: determining a target source point and a target vanishing point based on the motion trajectory of each of the first individuals.

[0045] It should be noted that the first person entering and leaving the surveillance field of view are defined as events A and B, respectively. Based on the principles of probability statistics, the locations with the highest frequency of events A and B are defined as the "source point" and "vanishing point." Under a specific shooting angle, the source point and vanishing point of the first person crossing the street are physically relatively fixed. Normally, the starting and ending points of pedestrian crossings are often located at either end of the intersection. If the "source point" and "vanishing point" of a pedestrian's trajectory at a given moment show significant fluctuations, it can be considered that the target is exhibiting abnormal behavior.

[0046] It should be noted that, while a source point and vanishing point can be determined based on each motion trajectory, multiple source points and vanishing points can be determined based on the motion trajectory of each first object. The multiple source points are considered a source point set, and the multiple vanishing points are considered a vanishing point set. Based on the source point set and vanishing point set, a target source point and a target vanishing point can be determined. The target source point represents the point where a pedestrian is likely to pass through and enter the sidewalk, and the target vanishing point represents the point where a pedestrian is likely to pass through after exiting the sidewalk. It should be noted that the points here can be 2D pixels or 2D image regions in a video image, and correspond to the actual coordinates of a location on an actual road.

[0047] Specifically, a set of source points and a set of vanishing points that cross the sidewalk can be obtained by traversing each motion trajectory; the number of clusters of the source point set is determined to be M1 based on a first evaluation metric and a second evaluation metric, where the first evaluation metric is used to characterize the similarity between different clusters, and the second evaluation metric is used to characterize the intra-cluster closeness and inter-cluster separation; the number of clusters of the vanishing point set is determined to be M2 based on the first evaluation metric and the second evaluation metric; the source point set is divided into M1 source point clusters using a Kmeans clustering algorithm, with the target source point as the center of each source point cluster; and the vanishing point set is divided into M2 vanishing point clusters using the Kmeans clustering algorithm, with the target vanishing point as the center of each vanishing point cluster. For details, please refer to the subsequent embodiments.

[0048] Step S104: Based on the second video of the road intersection obtained in the current time period, determine the real source point and the real vanishing point of each second object crossing the sidewalk in the second video.

[0049] It should be noted that the shooting screen sizes of the first video and the second video are the same, and the first video and the second video are obtained based on the same model of camera with the same installation position and the same shooting angle.

[0050] It can be known that the second video also includes videos of target groups crossing the sidewalk, and the target groups include pedestrians crossing the sidewalk, people crossing the sidewalk on bicycles or electric bicycles, and people crossing the sidewalk on motorcycles or electric motorcycles.

[0051] Specifically, based on the second video, target tracking can be performed on people crossing the sidewalk to obtain the movement intention of each person, and then the real source point and real vanishing point of the second individual can be determined based on the movement intention.

[0052] In an exemplary embodiment, the road intersection is a crossroads with four zebra crossings. Target tracking can be performed on a person crossing the crosswalk, and the pedestrian's movement intention is determined to cross the first crosswalk from end A to end B. Thus, the pedestrian's true source point can be determined to be at end A, and the pedestrian's true vanishing point can be determined to be at end B.

[0053] In an exemplary embodiment, the road intersection is a T-junction with three zebra crossings. Target tracking can be performed on a person crossing the crosswalk, and the pedestrian's movement intention is determined to cross the second crosswalk from end B to end A. Thus, the pedestrian's true source point can be determined to be at end B, and the pedestrian's true vanishing point can be determined to be at end A.

[0054] In an optional embodiment, the position of a pedestrian in at least two frames of images can be detected based on the second video, and the pedestrian's movement intention can be determined based on the movement pattern of the pedestrian in the image. In this way, the movement intention of each person can be obtained, and then the real source point and real vanishing point of the second individual can be determined based on the movement intention.

[0055] For example, the average position of the pedestrian in the first 3 seconds before entering the picture can be determined as the true source point, and the average position of the pedestrian in the last 3 seconds after leaving the picture can be determined as the true vanishing point.

[0056] In another exemplary embodiment, the average position of the pedestrian within 3 seconds after entering the preset area of ​​the zebra crossing is used as the true source point, and the average position of the pedestrian within 3 seconds before leaving the preset area of ​​the zebra crossing is used as the true vanishing point.

[0057] Step S105: performing abnormal behavior detection on the second video based on the distance between the target source point and the true source point of each second individual, and the distance between the target vanishing point and the true vanishing point of each second individual.

[0058] It should be noted that the closer the distance between the true source point and the target source point, the lower the probability of the pedestrian corresponding to that source point exhibiting abnormal behavior. The farther the distance between the true source point and the target source point, the higher the probability of the pedestrian corresponding to that source point exhibiting abnormal behavior. Similarly, the closer the distance between the true vanishing point and the target vanishing point, the lower the probability of the pedestrian corresponding to that vanishing point exhibiting abnormal behavior. The farther the distance between the true vanishing point and the target vanishing point, the higher the probability of the pedestrian corresponding to that vanishing point exhibiting abnormal behavior.

[0059] To this end, abnormal behavior detection can be performed on the second video based on the distance between the target source point and the true source point of each second individual, and the distance between the target vanishing point and the true vanishing point of each second individual. For details, please refer to the subsequent embodiments.

[0060] In an exemplary embodiment, a first video of a road intersection within a historical time period is obtained, and based on the first video, the motion trajectory of each first individual is extracted. Based on the motion trajectory of each first individual, a target source point and a target vanishing point are determined as reference objects for anomaly detection. Based on a second video of the road intersection within a current time period obtained in real time, the true source point and true vanishing point of each second individual crossing the sidewalk are determined based on the second video, and abnormal behavior detection is then performed on the second video based on the distance between the true source point and the target source point, and the distance between the true vanishing point and the target vanishing point.

[0061] An embodiment of the present application provides a detection method, which includes: first, obtaining a first video within a historical time period of a road intersection; second, screening out a first individual that meets preset conditions from multiple records of crossing a sidewalk in the first video for target tracking, and obtaining a motion trajectory of each first individual; determining a target source point and a target vanishing point based on the motion trajectory of each first individual; then, based on a second video of the road intersection obtained within a current time period, determining a true source point and a true vanishing point of each second individual crossing the sidewalk in the second video; finally, performing abnormal behavior detection on the second video based on the distance between the target source point and the true source point of each second individual, and the distance between the target vanishing point and the true vanishing point of each second individual.

[0062] On the one hand, compared to methods that capture video in real time and perform anomaly detection on the video based on manually specified parameters, the present application extracts the target source point and target vanishing point from the first video data of a historical time period, and can automatically determine the target source point and target vanishing point for detecting abnormal behavior, thereby improving the level of intelligence. On the other hand, the present application detects abnormal behavior based on the distance between the real vanishing point and the target vanishing point, as well as the distance between the real source point and the target source point. There is no need to wait until the pedestrian has crossed the sidewalk before detecting the movement trajectory, and abnormal behavior can be detected in a timely manner. When abnormal behavior is detected, the present application can quickly determine the abnormal location through the position information of the target source point and the target vanishing point, and thus timely control.

[0063] See also Figure 2 , which shows a flow chart of a detection provided by an embodiment of the present application, and the processing module applied to the monitoring device specifically includes steps S201 to S207.

[0064] Step S201: Acquire the first video within a historical time period of a road intersection.

[0065] Among them, step S201 has been described in detail in the above embodiment and will not be repeated here.

[0066] Step S202: Determine a first number of frames in which each object crossing the sidewalk is continuously detected and a second number of frames in which each object is not detected in the first video.

[0067] It should be noted that the target group in the first video includes pedestrians, cyclists, and motorcyclists, and each person has different heights and weights. Some people in the first video are obscured. When pedestrians are obscured for a long time, it is difficult to accurately track them. Therefore, it is not necessary to extract the trajectory of every person crossing the sidewalk in the first video. Instead, we extract the trajectory of only those who meet the preset conditions and use them as the first subject.

[0068] Based on prior knowledge, if the number of consecutive frames in which each object is detected is greater than a certain value and the number of frames in which it is not detected is less than a certain value, the object is considered to meet the preset conditions. Based on the first video, the number of consecutive frames in which each object crossing the sidewalk is successfully detected is defined as the first number of frames, and the number of frames in which it is not detected is defined as the second number of frames.

[0069] Step S203: Among the multiple objects, the object whose first frame number is greater than or equal to N1 and whose second frame number is less than N2 is taken as the first object that meets the preset condition, and N1 is greater than N2.

[0070] If the first frame number of the object is greater than or equal to N1 and the second frame number is less than N2, it means that the object meets the preset conditions and the object is taken as the first body, thereby determining the first body in the first video. The values ​​of N1 and N2 can be obtained based on prior knowledge.

[0071] Step S204: performing target tracking on a plurality of first individuals crossing the sidewalk that meet preset conditions in the first video to obtain a motion trajectory of each of the first individuals.

[0072] Step S205: determining a target source point and a target vanishing point based on the motion trajectory of each of the first individuals.

[0073] Step S206: Based on the second video of the road intersection obtained in the current time period, determine the real source point and the real vanishing point of each second object crossing the sidewalk in the second video.

[0074] Step S207: performing abnormal behavior detection on the second video based on the distance between the target source point and the true source point of each second individual, and the distance between the target vanishing point and the true vanishing point of each second individual.

[0075] Among them, step S204 and step S207 have been described in detail in the above embodiments and will not be repeated here.

[0076] The present application screens the objects crossing the sidewalk in the first video and takes the objects that meet the preset conditions as the first objects, which can improve the effectiveness of the motion trajectory and further improve the effectiveness of abnormal behavior detection.

[0077] See also Figure 3 , which shows a flow chart of a detection provided by an embodiment of the present application, and the processing module applied to the monitoring device specifically includes steps S301 to S305.

[0078] Step S301: Acquire the first video within a historical time period of a road intersection.

[0079] Step S302: Filtering out the first individuals that meet preset conditions from the multiple records of passing through the sidewalk in the first video for target tracking, and obtaining the motion trajectory of each of the first individuals.

[0080] Among them, step S301 and step S302 have been described in detail in the above embodiments and will not be repeated here.

[0081] Step S303: traverse each of the motion trajectories to obtain a set of source points and a set of vanishing points that pass through the sidewalk.

[0082] A source point and a vanishing point can be determined based on each motion trajectory, and multiple source points and multiple vanishing points can be determined based on the motion trajectory of each first body. The multiple source points are regarded as a source point set, and the multiple vanishing points are regarded as a vanishing point set.

[0083] Step S304: Determine the number of clusters of the source point set as M1 based on a first evaluation index and a second evaluation index, wherein the first evaluation index is used to characterize the similarity between different clusters, and the second evaluation index is used to characterize the intra-cluster compactness and inter-cluster separation. Specifically, the process includes steps S3041 to S3043.

[0084] Step S3041: Map the source point set onto the source point two-dimensional space plane through a two-dimensional Gaussian function.

[0085] Compared with processing directly based on the source point set, mapping the source points to the probability space through a two-dimensional Gaussian function can convert complex geometric relationships or spatial distributions into probability distributions, thereby enabling more efficient searches in the probability space.

[0086] Step S3042: extracting n discrete points from the two-dimensional spatial plane of the source points according to a preset rule as initial cluster centers of the source point set.

[0087] It should be noted that n is the initialization value, and the default rule is to select the initialization centers based on probability without replacement. To avoid biased initialization centers based solely on probability sampling, the selection process must ensure that if a newly selected discrete point is to be used as a new cluster initialization center, its distance from any existing cluster initialization center must be greater than the minimum distance F. To this end, n discrete points are selected based on probability without replacement as the initial cluster centers of the source point set. F is a value pre-set based on experience.

[0088] Step S3043: Based on the initial cluster center of the source point set and the first evaluation index And the second evaluation index Determine the first cluster number of the source point set as M1, where a represents the average distance between the data point v and other data points in the same cluster, and b represents the average distance between the data point v and all data points in the nearest cluster. represents the average distance from the data points in cluster i to the cluster center, represents the average distance from the data points in cluster j to the cluster center, c i with c j They represent cluster i and cluster j respectively, and K represents the number of clusters.

[0089] The first evaluation index is the silhouette coefficient S, and the second evaluation index is the DB index. The silhouette coefficient S and DB index are introduced as two clustering result evaluation indicators, and the KMeans algorithm is executed to assist in inferring the number of the most appropriate source or vanishing point cluster centers. The silhouette coefficient S is considered to contain a set of N data points. The silhouette coefficient St of each sample v after clustering is defined, where v = 1, 2, 3, ..., N, satisfies in, |C| represents the number of elements in set C, d(v,x) represents the distance between sample v and sample x, and traverses the set C. m For any sample point v, calculate its difference with class C m The average distance of all other sample points in the cluster is a, which measures the degree to which the sample belongs to the cluster. m All samples of , calculate the distance between each sample v and the nearest cluster center C k The average distance between all samples in the corresponding cluster is b, which measures the degree to which the sample belongs to other clusters. For a set of N data points, the S coefficient is defined for the clustering result of the entire sample data. From the perspective of a single sample data, it comprehensively considers the affinity between the sample point and the cluster center, as well as the difference from other types of samples.

[0090] It should be noted that the coefficient S ranges from [-1, 1]. In the clustering results, the closer the distance between samples of the same category is, the farther the distance between samples of different categories is, and the larger S is and closer to 1, the better the clustering effect is.

[0091] in, n i 、n j Respectively represent the number of samples contained in the i-th cluster and the j-th cluster, c i 、c j Respectively represent the class centers of the i-th and j-th clusters, x p Represents cluster sample points.

[0092] It represents the mean distance between all samples of cluster i and the center of the cluster. The DB index represents the mean distance between all samples in cluster j and the cluster center. The DB index measures the similarity between clusters. A smaller DB index indicates lower inter-cluster similarity and better clustering. The maximum S coefficient and minimum DB index determine the optimal number of clusters, thereby obtaining the best estimate of the target source points for pedestrians and cyclists crossing the street. The resulting number of clusters for the source point set is M1.

[0093] Step S305: Determine the number of clusters of the vanishing point set as M2 based on the first evaluation index and the second evaluation index.

[0094] Specifically, it includes steps S3051 to S3053.

[0095] Step S3051: Mapping the vanishing point set to the vanishing point two-dimensional space plane through a two-dimensional Gaussian function.

[0096] Compared with directly processing based on the vanishing point set, mapping the vanishing points to the probability space through a two-dimensional Gaussian function can convert complex geometric relationships or spatial distributions into probability distributions, thereby performing more efficient searches in the probability space.

[0097] Step S3052: extracting n discrete points from the two-dimensional space of the vanishing points as the initial cluster centers of the vanishing points according to a preset rule.

[0098] N discrete points are extracted from the vanishing point two-dimensional space plane according to probability and without replacement as the initial cluster centers of the source point set.

[0099] Step S3053: Based on the initial cluster center of the vanishing point and the first evaluation index And the second evaluation index Determine the second cluster number of the vanishing point set as M2, where a represents the average distance between the data point v and other data points in the same cluster, and b represents the average distance between the data point v and all data points in the nearest cluster. represents the average distance from the data points in cluster i to the cluster center, represents the average distance from the data points in cluster j to the cluster center, C i with C j They represent cluster i and cluster j respectively, and K represents the number of clusters.

[0100] For detailed principles, please refer to the previous example. The DB index calculates the similarity between different clusters. The smaller the DB index, the lower the inter-cluster similarity and the better the clustering effect. The optimal number of clusters can be determined by the maximum S coefficient and the minimum DB index, thereby obtaining the best estimate of the target vanishing point for pedestrians and cyclists crossing the street. The number of clusters for the vanishing point set is M2.

[0101] Step S306: Divide the source point set into M1 source point clusters based on the Kmeans clustering algorithm, with the center of the source point cluster being the target source point.

[0102] That is, M1 target source points are obtained.

[0103] Step S307: Divide the vanishing point set into M2 vanishing point clusters based on the Kmeans clustering algorithm, with the center of each vanishing point cluster being the target vanishing point.

[0104] That is, M2 target vanishing points are obtained.

[0105] For an example, see Figure 4 , which shows a frame of image information of the road intersection in the first video, see Figure 5 , which actually shows the trajectory of each first body, see Figure 6 , which shows the coordinates of the target source point and target vanishing point in the image obtained based on the method of this application, as shown by the location of the white square. This method of this application automatically and reasonably estimates and extracts the starting point and vanishing point of the crossing trajectory of pedestrians, bicycle groups, and electric bicycle groups in the monitoring field of view without human guidance.

[0106] This application collects a large number of valid street crossing trajectories and proposes a K-means algorithm with probability sampling initialization under the constraints of the S coefficient and DB index to determine the target source and target vanishing points. Based on computer vision technology, it automatically analyzes and summarizes the semantic information of the spatiotemporal dimensions of surveillance video, adaptively assesses security events (such as abnormal street crossing behavior) in the video field of view, and generates warnings. This improves the intelligence of video surveillance, contributes to efficient urban management, and reduces the workload of monitoring personnel.

[0107] Step S308: Based on the second video of the road intersection obtained in the current time period, determine the real source point and the real vanishing point of each second object crossing the sidewalk in the second video.

[0108] Among them, step S308 has been described in detail in the above embodiment and will not be repeated here.

[0109] Step S309: performing abnormal behavior detection on the second video based on the distance between the target source point and the real source point of each second individual, and the distance between the target vanishing point and the real vanishing point of each second individual.

[0110] In an optional embodiment, there is only one sidewalk at the road intersection, and there is one target source point and one target vanishing point. If the distance between the true source point and the target source point is greater than a first threshold, or the distance between the true vanishing point and the target vanishing point is greater than a second threshold, it is determined that abnormal behavior exists in the second video.

[0111] In an optional embodiment, at least two target source points and at least two target vanishing points are obtained. If the distance between the true source point and the nearest target source point is greater than a first threshold, or the distance between the true vanishing point and the nearest target vanishing point is greater than a second threshold, it is determined that abnormal behavior exists in the second video.

[0112] Furthermore, if abnormal behavior is detected, an alarm message is issued to facilitate staff to take timely action.

[0113] In an optional embodiment, each of the motion trajectories includes a time feature, an age feature, a road feature, and a motion feature. Determining a target source point and a target vanishing point based on the motion trajectory of each of the first individuals includes: determining a target scene based on the time feature, the age feature, and the road feature of each of the first individuals' motion trajectories; and determining a target source point and a target vanishing point corresponding to the target scene based on the motion features of each of the first individuals' motion trajectories. The target scene and the corresponding target source point and target vanishing point are stored in a database.

[0114] When a second video of the current time period is captured, the actual target scene corresponding to the second video is determined based on the time characteristics, age characteristics, road characteristics, or motion characteristics of each second individual crossing the sidewalk in the second video. The target source points and target vanishing points corresponding to the target scene identical to the actual target scene are extracted from the database as the target source points and target vanishing points for this detection.

[0115] For example, a first video is collected from 10:00 to 11:00 in the morning, and the time characteristics, age characteristics, road characteristics and motion characteristics of the people crossing the sidewalk in the first video are extracted. Based on the above information, the target scene corresponding to the first video is determined to be "elderly people buying groceries crossing the street in a concentrated manner". Based on the first video, the target source point and target vanishing point corresponding to "elderly people buying groceries crossing the street in a concentrated manner" can be determined. If the actual target scene corresponding to the second video collected in real time is "elderly people buying groceries crossing the street in a concentrated manner", the target source point and target vanishing point corresponding to "elderly people buying groceries crossing the street in a concentrated manner" will be used as the target source point and target vanishing point of this detection.

[0116] In an optional embodiment, each of the motion trajectories includes time features, age features, road features, and motion features. Based on the motion trajectory of each first individual, the changing pattern of the thermal distribution of the crowd crossing the street within a day can be described, thereby further improving the accuracy of abnormal behavior detection.

[0117] See also Figure 7 , which shows a structural block diagram of a detection method device 600 provided in an embodiment of the present application. The test device 600 includes: an acquisition unit 610, an extraction unit 620, a determination unit 630, a collection unit 640 and a detection unit 650.

[0118] The acquisition unit 610 is configured to acquire a first video within a historical time period of a road intersection.

[0119] The extraction unit 620 is configured to select a first individual that meets a preset condition from multiple records of passing through the sidewalk in the first video, perform target tracking, and obtain a motion trajectory of each first individual.

[0120] Furthermore, the extraction unit 620 can also be used to determine the first number of frames in which each object crossing the sidewalk in the first video is continuously detected and the second number of frames in which it is not detected; among multiple objects, the object whose first number of frames is greater than or equal to N1 and whose second number of frames is less than N2 is regarded as the first object that meets the preset conditions, and N1 is greater than N2.

[0121] The determining unit 630 is configured to determine a target source point and a target vanishing point based on the motion trajectory of each of the first individuals.

[0122] Furthermore, the determination unit 630 can also be used to traverse each of the motion trajectories to obtain a set of source points and a set of vanishing points that pass through the sidewalk; based on the first evaluation index and the second evaluation index, the number of clusters of the source point set is determined to be M1, the first evaluation index is used to characterize the similarity between different clusters, and the second evaluation index is used to characterize the intra-cluster compactness and inter-cluster separation; based on the first evaluation index and the second evaluation index, the number of clusters of the vanishing point set is determined to be M2; based on the Kmeans clustering algorithm, the source point set is divided into M1 source point clusters, and the center of the source point cluster is the target source point; based on the Kmeans clustering algorithm, the vanishing point set is divided into M2 vanishing point clusters, and the center of the vanishing point cluster is the target vanishing point.

[0123] Furthermore, the determination unit 630 can also be used to map the source point set to the source point two-dimensional space plane through a two-dimensional Gaussian function; extract n discrete points from the source point two-dimensional space plane as the initial cluster center of the source point set according to a preset rule; based on the initial cluster center of the source point set, the first evaluation index And the second evaluation index Determine the first cluster number of the source point set as M1, where a represents the average distance between the data point v and other data points in the same cluster, and b represents the average distance between the data point v and all data points in the nearest cluster. represents the average distance from the data points in cluster i to the cluster center, represents the average distance from the data points in cluster j to the cluster center, c i with c j They represent cluster i and cluster j respectively, and K represents the number of clusters.

[0124] Furthermore, the determination unit 630 may also be configured to map the vanishing point set to a vanishing point two-dimensional space plane through a two-dimensional Gaussian function; extract n discrete points from the vanishing point two-dimensional space plane as the initial cluster centers of the vanishing points according to a preset rule; and perform a clustering operation based on the initial cluster centers of the vanishing point set and the first evaluation index. And the second evaluation index Determine the first cluster number of the vanishing point set as M1, where a represents the average distance between a data point v and other data points in the same cluster, and b represents the average distance between a data point v and all data points in the nearest cluster. represents the average distance from the data points in cluster i to the cluster center, represents the average distance from the data points in cluster j to the cluster center, c i with c j They represent cluster i and cluster j respectively, and K represents the number of clusters.

[0125] The acquisition unit 640 is configured to determine, based on the second video of the road intersection acquired within the current time period, a true source point and a true vanishing point of each second object crossing the sidewalk in the second video.

[0126] The detection unit 650 is configured to perform abnormal behavior detection on the second video based on the distance between the target source point and the true source point of each second individual, and the distance between the target vanishing point and the true vanishing point of each second individual.

[0127] Furthermore, the detection unit 650 can also be used to determine that abnormal behavior exists in the second video if the distance between the real source point and the target source point is greater than a first threshold, or the distance between the real vanishing point and the target vanishing point is greater than a second threshold.

[0128] Furthermore, the detection unit 650 can also be used to determine that abnormal behavior exists in the second video if the distance between the real source point and the nearest target source point is greater than a first threshold, or the distance between the real vanishing point and the nearest target vanishing point is greater than a second threshold.

[0129] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0130] In several embodiments provided in this application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0131] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0132] Please refer to Figure 8 , which shows a structural block diagram of an electronic device 700 provided in an embodiment of the present application. The electronic device 700 may be a vehicle-mounted system, which may be installed in a vehicle. The electronic device 700 in the present application may include one or more of the following components: a processor 711, a memory 712, and one or more application programs, wherein the processor 711 is electrically connected to the memory 712, and the one or more application programs are configured to execute the methods described in the various embodiments of the aforementioned testing method.

[0133] The processor 711 may include one or more processing cores. The processor 711 utilizes various interfaces and circuits to connect various components within the electronic device 700. It executes instructions, programs, code sets, or instruction sets stored in the memory 712, and accesses data stored in the memory 712 to perform various functions and process data within the electronic device 700. Optionally, the processor 711 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 711 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and computer programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 711 but may be implemented separately via a communications chip. Specifically, the methods described in the aforementioned embodiments may be executed by one or more processors 711.

[0134] In some embodiments, the memory 712 may include a random access memory (RAM) or a read-only memory (ROM). The memory 712 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 712 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, and the like. The data storage area may also store data created by the electronic device 700 during use.

[0135] Please refer to Figure 9 , which shows a block diagram of a computer-readable medium provided in an embodiment of the present application. The computer-readable medium 800 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0136] The computer-readable medium 800 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable medium 800 includes a non-transitory computer-readable storage medium. The computer-readable medium 800 has storage space for program code 810 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in a suitable form.

[0137] Please refer to Figure 10 , which shows a structural block diagram 900 of a computer program product provided by an embodiment of the present application. The computer program product 900 includes a computer program / instruction 910, which implements the steps of the above method when executed by a processor.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A detection method, characterized in that: The processing module applied to the monitoring device comprises: Get the first video within the historical time period of the road intersection; Selecting a first individual that meets a preset condition from multiple records of passing through the sidewalk in the first video for target tracking, and obtaining a motion trajectory of each first individual; Determining a target source point and a target vanishing point based on the motion trajectory of each of the first individuals; determining, based on a second video of the road intersection obtained during a current time period, a true source point and a true vanishing point of each second object crossing the sidewalk in the second video; Abnormal behavior detection is performed on the second video based on the distance between the target source point and the true source point of each second individual, and the distance between the target vanishing point and the true vanishing point of each second individual.

2. The method according to claim 1, characterized in that Before selecting a first individual that meets a preset condition from a plurality of records of passing through the sidewalk in the first video for target tracking and obtaining a motion trajectory of each first individual, the method further includes: Determining, in the first video, a first number of consecutive frames in which each object crossing the sidewalk is detected and a second number of consecutive frames in which each object crossing the sidewalk is not detected; Among the multiple objects, the object whose first frame number is greater than or equal to N1 and whose second frame number is less than N2 is taken as the first object that meets the preset condition, and N1 is greater than N2.

3. The method according to claim 1, characterized in that The determining of a target source point and a target vanishing point based on the motion trajectory of each of the first individuals includes: Traversing each of the motion trajectories to obtain a set of source points and a set of vanishing points that pass through the sidewalk; Determining the number of clusters of the source point set as M1 based on a first evaluation index and a second evaluation index, wherein the first evaluation index is used to characterize the similarity between different clusters, and the second evaluation index is used to characterize the intra-cluster compactness and inter-cluster separation; Determining the number of clusters of the vanishing point set as M2 based on the first evaluation index and the second evaluation index; Based on the Kmeans clustering algorithm, the source point set is divided into M1 source point clusters, and the center of the source point cluster is the target source point; The vanishing point set is divided into M2 vanishing point clusters based on the Kmeans clustering algorithm, and the center of the vanishing point cluster is the target vanishing point.

4. The method according to claim 3, characterized in that The determining the number of clusters of the source point set as M1 based on the first evaluation index and the second evaluation index includes: Mapping the source point set onto the two-dimensional spatial plane of the source point through a two-dimensional Gaussian function; Extracting n discrete points from the two-dimensional spatial plane of the source points as the initial cluster centers of the source point set according to a preset rule; Based on the initial cluster center of the source point set and the first evaluation index And the second evaluation index Determine the first cluster number of the source point set as M1, where a represents the average distance between the data point v and other data points in the same cluster, and b represents the average distance between the data point v and all data points in the nearest cluster. represents the average distance from the data points in cluster i to the cluster center, represents the average distance from the data points in cluster j to the cluster center, c i with c j They represent cluster i and cluster j respectively, and K represents the number of clusters.

5. The method according to claim 3, characterized in that The determining, based on the first evaluation index and the second evaluation index, that the number of clusters of the vanishing point set is M2 includes: Mapping the vanishing point set to a two-dimensional space plane of vanishing points through a two-dimensional Gaussian function; Extracting n discrete points from the two-dimensional space plane of the vanishing points according to a preset rule as the initial cluster centers of the vanishing points; Based on the initial cluster center of the vanishing point set, the first evaluation index And the second evaluation index Determine the first cluster number of the vanishing point set as M1, where a represents the average distance between a data point v and other data points in the same cluster, and b represents the average distance between a data point v and all data points in the nearest cluster. represents the average distance from the data points in cluster i to the cluster center, represents the average distance from the data points in cluster j to the cluster center, c i with c j They represent cluster i and cluster j respectively, and K represents the number of clusters.

6. The method according to claim 1, wherein There is one target source point and one target vanishing point, and performing abnormal behavior detection on the second video based on a distance between the target source point and a true source point of each second individual, and a distance between the target vanishing point and a true vanishing point of each second individual, includes: If the distance between the real source point and the target source point is greater than a first threshold, or the distance between the real vanishing point and the target vanishing point is greater than a second threshold, it is determined that abnormal behavior exists in the second video.

7. The method according to claim 1, characterized in that There are at least two target source points and at least two target vanishing points, and performing abnormal behavior detection on the second video based on the distances between the target source points and the true source points of each second individual, and the distances between the target vanishing points and the true vanishing points of each second individual, includes: If the distance between the real source point and the nearest target source point is greater than a first threshold, or the distance between the real vanishing point and the nearest target vanishing point is greater than a second threshold, it is determined that abnormal behavior exists in the second video.

8. A detection device, characterized in that: A processing module applied to a monitoring device, the device comprising: an acquisition unit, configured to acquire a first video within a historical time period of a road intersection; an extraction unit, configured to select a first individual meeting a preset condition from a plurality of records of passing through the sidewalk in the first video, perform target tracking, and obtain a motion trajectory of each first individual; a determining unit, configured to determine a target source point and a target vanishing point based on the motion trajectory of each of the first individuals; an acquisition unit, configured to determine, based on a second video of the road intersection acquired during a current time period, a true source point and a true vanishing point of each second object crossing the sidewalk in the second video; A detection unit is configured to perform abnormal behavior detection on the second video based on a distance between the target source point and a true source point of each second individual, and a distance between the target vanishing point and a true vanishing point of each second individual.

9. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory, the one or more applications are configured to be executed by the one or more processors, and the one or more applications are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images