Target position anomaly detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a target position anomaly detection method and device, electronic equipment and a storage medium, and belongs to the technical field of video surveillance, and the method comprises the steps: periodically obtaining a video stream corresponding to a target monitoring area, and carrying out the target detection of a video frame in the video stream; if it is detected that the video frame comprises the to-be-detected target and the reference target, determining motion tracks of the to-be-detected target and the reference target in the obtained video stream; determining the relative position of the to-be-measured target and the reference target according to the motion trails of the to-be-measured target and the reference target; and performing position anomaly detection on the to-be-detected target based on the relative position and a preset anomaly detection rule corresponding to the to-be-detected target. Thus, anomaly detection is carried out through the relative positions of the multiple targets, the targets with abnormal positions can be predicted more timely and accurately, potential safety hazards are effectively prevented, and compared with an anomaly detection method of a single fixed target type in the prior art, the comprehensiveness and accuracy of a monitoring system can be improved.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and in particular to a method, device, electronic device and storage medium for detecting anomalies in a target position. Background Art

[0002] With the rapid development of artificial intelligence technology and monitoring equipment, intelligent video surveillance systems have become one of the core technologies in modern security, traffic management, industrial automation and other fields.

[0003] However, most existing surveillance video alarm methods still focus on warnings of a single fixed target type, such as animals entering or leaving a warning area, monitoring human activities in a specific area, etc. This single-target alarm mechanism cannot handle the situation where multiple types of targets exist at the same time, and it is difficult to cope with complex real-world scenarios. It also fails to fully consider the interactive relationship between different targets, resulting in insufficient understanding of complex events, and alarm rules customized for specific applications have limited adaptability when facing new scenarios or changes.

[0004] Therefore, the anomaly detection method of a single fixed target type in the prior art lacks analysis of the interactive relationship between multiple targets, resulting in delayed identification and response to potential security threats, affecting the real-time and effectiveness of the system. Summary of the invention

[0005] The embodiments of the present application provide a target position anomaly detection method, device, electronic device and storage medium to solve the problem that the single fixed target type anomaly detection method in the prior art lacks analysis of the interactive relationship between multiple targets, resulting in delayed identification and response to potential security threats, affecting the real-time and effectiveness of the system.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting anomaly of a target position, the method comprising: Periodically acquiring a video stream corresponding to a target monitoring area, and performing target detection on video frames in the video stream; If it is detected that the video frame includes the target to be measured and the reference target, then determining the motion trajectories of the target to be measured and the reference target in the acquired video stream; Determining the relative position of the target to be measured and the reference target according to the motion trajectories of the target to be measured and the reference target; Based on the relative position and a preset anomaly detection rule corresponding to the target to be detected, position anomaly detection is performed on the target to be detected.

[0007] In some embodiments, if it is detected that the video frame includes a target to be detected and a reference target, determining the motion trajectories of the target to be detected and the reference target in the acquired video stream includes: If it is detected that the video frame includes a target to be detected and a reference target, then based on the position information and category information in the detection result, the detection information of the target to be detected and the detection information of the reference target are determined, wherein the position information represents the coordinate information of the target to be detected and the reference target, and the category information represents the target category of the target to be detected and the reference target; Inputting the detection information of the target to be detected and the detection information of the reference target into a pre-trained target tracking model to obtain tracking results of the target to be detected and the reference target; Based on the tracking results of the target to be measured and the reference target, the motion trajectories of the target to be measured and the reference target are determined.

[0008] In some embodiments, determining the relative position of the target to be measured and the reference target according to the motion trajectories of the target to be measured and the reference target includes: Determine, according to the motion trajectories of the target to be measured and the reference target, a first coordinate of the target to be measured and a second coordinate of the reference target in each target video frame, wherein the target video frame is a video frame in the video stream including the target to be measured and the reference target; For any target video frame, the relative position of the target to be measured and the reference target is determined based on the first coordinates of the target to be measured and the second coordinates of the reference target.

[0009] In some embodiments, determining the relative position of the target to be measured and the reference target based on the first coordinates of the target to be measured and the second coordinates of the reference target includes: Determining a distance between the target to be measured and the reference target based on the first coordinates of the target to be measured and the second coordinates of the reference target; Based on the distance and a preset reference direction, determining an angle between the target to be measured and the reference target; Based on the distance and the angle, a relative position of the target to be measured and the reference target is determined.

[0010] In some embodiments, the performing position anomaly detection on the target to be detected based on the relative position and a preset anomaly detection rule corresponding to the target to be detected includes: Determining a first comparison result based on the distance in the relative position and the preset distance value in the anomaly detection rule, and Determining a second comparison result based on the angle in the relative position and the angle preset value in the anomaly detection rule; Based on the first comparison result and the second comparison result, if it is determined that an abnormal condition is met, it is determined that the position of the target to be measured is abnormal.

[0011] In some embodiments, the abnormal condition includes at least one of the following: The first comparison result is greater than the preset distance value, and the second comparison result is greater than the preset angle value; The first comparison result is greater than the preset distance value, or the second comparison result is greater than the preset angle value; The first comparison result is not greater than the preset distance value, and the second comparison result is not greater than the preset angle value; The first comparison result is not greater than the preset distance value, or the second comparison result is not greater than the preset angle value.

[0012] In a second aspect, an embodiment of the present application provides a target position abnormality detection device, the device comprising: An acquisition module is used to periodically acquire a video stream corresponding to a target monitoring area and perform target detection on video frames in the video stream; A first determination module is used to determine the motion trajectories of the target to be detected and the reference target in the acquired video stream if it is detected that the video frame includes the target to be detected and the reference target; A second determination module is used to determine the relative position of the target to be measured and the reference target according to the motion trajectories of the target to be measured and the reference target; The detection module is used to perform position anomaly detection on the target to be detected based on the relative position and a preset anomaly detection rule corresponding to the target to be detected.

[0013] In some embodiments, the first determining module is specifically configured to: If it is detected that the video frame includes a target to be detected and a reference target, then based on the position information and category information in the detection result, the detection information of the target to be detected and the detection information of the reference target are determined, wherein the position information represents the coordinate information of the target to be detected and the reference target, and the category information represents the target category of the target to be detected and the reference target; Inputting the detection information of the target to be detected and the detection information of the reference target into a pre-trained target tracking model to obtain tracking results of the target to be detected and the reference target; Based on the tracking results of the target to be measured and the reference target, the motion trajectories of the target to be measured and the reference target are determined.

[0014] In some embodiments, the second determining module is specifically configured to: Determine, according to the motion trajectories of the target to be measured and the reference target, a first coordinate of the target to be measured and a second coordinate of the reference target in each target video frame, wherein the target video frame is a video frame in the video stream including the target to be measured and the reference target; For any target video frame, the relative position of the target to be measured and the reference target is determined based on the first coordinates of the target to be measured and the second coordinates of the reference target.

[0015] In some embodiments, the second determining module is specifically configured to: Determining a distance between the target to be measured and the reference target based on the first coordinates of the target to be measured and the second coordinates of the reference target; Based on the distance and a preset reference direction, determining an angle between the target to be measured and the reference target; Based on the distance and the angle, a relative position of the target to be measured and the reference target is determined.

[0016] In some embodiments, the detection module is specifically used to: Determining a first comparison result based on the distance in the relative position and the preset distance value in the anomaly detection rule, and Determining a second comparison result based on the angle in the relative position and the angle preset value in the anomaly detection rule; Based on the first comparison result and the second comparison result, if it is determined that an abnormal condition is met, it is determined that the position of the target to be measured is abnormal.

[0017] In some embodiments, the abnormal condition includes at least one of the following: The first comparison result is greater than the preset distance value, and the second comparison result is greater than the preset angle value; The first comparison result is greater than the preset distance value, or the second comparison result is greater than the preset angle value; The first comparison result is not greater than the preset distance value, and the second comparison result is not greater than the preset angle value; The first comparison result is not greater than the preset distance value, or the second comparison result is not greater than the preset angle value.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein: The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned target position abnormality detection method.

[0019] In a fourth aspect, an embodiment of the present application provides a storage medium. When a computer program in the storage medium is executed by a processor of an electronic device, the electronic device can execute the above-mentioned target position anomaly detection method.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when called and executed by an electronic device, enables the electronic device to execute the above-mentioned target position anomaly detection method.

[0021] In the embodiment of the present application, the video stream corresponding to the target monitoring area is periodically acquired, and target detection is performed on the video frames in the video stream; if it is detected that the video frame includes the target to be measured and the reference target, the motion trajectory of the target to be measured and the reference target is determined in the acquired video stream; the relative position of the target to be measured and the reference target is determined according to the motion trajectory of the target to be measured and the reference target; based on the relative position and the pre-set anomaly detection rules corresponding to the target to be measured, the position anomaly detection is performed on the target to be measured. In this way, by performing anomaly detection on the relative positions between multiple targets, targets with abnormal positions can be predicted more timely and accurately, and potential safety hazards can be effectively prevented. Compared with the anomaly detection method of a single fixed target type in the prior art, the comprehensiveness and accuracy of the monitoring system can be improved, and more refined and intelligent safety management and risk warning can be achieved.

[0022] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a target position abnormality detection method provided in an embodiment of the present application; Figure 2 A flowchart of another target position abnormality detection method provided in an embodiment of the present application; Figure 3 A schematic diagram of the distance position relationship between a target to be measured and a reference target provided in an embodiment of the present application; Figure 4A schematic diagram of the angular position relationship between a target to be measured and a reference target provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a target position abnormality detection device provided in an embodiment of the present application; Figure 6 A schematic diagram of the hardware structure of an electronic device for implementing a method for detecting anomalies in target positions provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0025] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

[0026] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0027] In the technical solution of this application, the acquisition, transmission, storage, and use of data are in compliance with the requirements of relevant national laws and regulations.

[0028] Before introducing the target position anomaly detection method provided in the embodiment of the present application, in order to facilitate understanding, the technical background of the embodiment of the present application is first introduced in detail below.

[0029] With the rapid development of artificial intelligence technology and monitoring equipment, intelligent video surveillance systems have become one of the core technologies in modern security, traffic management, industrial automation and other fields.

[0030] However, most existing surveillance video alarm methods still focus on warnings of a single fixed target type, such as animals entering or leaving a warning area, monitoring human activities in a specific area, etc. This single-target alarm mechanism cannot handle the situation where multiple types of targets exist at the same time, and it is difficult to cope with complex real-world scenarios. It also fails to fully consider the interactive relationship between different targets, resulting in insufficient understanding of complex events, and alarm rules customized for specific applications have limited adaptability when facing new scenarios or changes.

[0031] In view of this, in order to solve the problem that the anomaly detection method of a single fixed target type in the prior art lacks the analysis of the interactive relationship between multiple targets, resulting in delayed identification and response to potential security threats, affecting the real-time and effectiveness of the system, the embodiment of the present application provides a target position anomaly detection method, device, electronic device and storage medium. The following describes some preferred embodiments of the present application in conjunction with the drawings of the specification.

[0032] It should be noted that a target position anomaly detection method provided in an embodiment of the present application is suitable for scenarios where position anomaly detection is performed on a target to be measured within a target monitoring area. For example, in the application scenario of intelligent traffic management, it can be used to monitor the movement of vehicles and pedestrians. Taking the target to be measured as a pedestrian and the reference target as a vehicle as an example, position anomaly detection can be performed on pedestrians based on the relative position between the pedestrian and the vehicle and the pre-set anomaly detection rules corresponding to the pedestrian. If the target to be measured is a vehicle and the reference target is a pedestrian, position anomaly detection can be performed on the vehicle based on the relative position between the pedestrian and the vehicle and the pre-set anomaly detection rules corresponding to the vehicle, so as to provide early warning of potential traffic accidents. In application scenarios such as factory workshops, it can be used to monitor operating personnel and dangerous objects. The positional relationship between the objects can be determined by the position detection method. For example, taking the target to be detected as the operator and the reference target as the dangerous object, the position abnormality detection of the operator can be performed based on the relative position between the operator and the dangerous object and the pre-set abnormality detection rules corresponding to the operator. For example, when the operator approaches the dangerous object at a certain distance, the position abnormality is determined and an alarm is issued. It can also be used to monitor the positional relationship between the operator and the safety rope. For example, taking the target to be detected as the operator and the reference target as the safety rope, the position abnormality detection of the operator can be performed based on the relative position between the operator and the safety rope and the pre-set abnormality detection rules corresponding to the operator. For example, when the angle between the operator and the safety rope deviates from the preset angle, the position abnormality is determined and an alarm is issued.

[0033] It is worth noting that the application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0034] After introducing the application scenarios of the embodiments of the present application, the target position anomaly detection method proposed in the present application is described below with specific embodiments.

[0035] See also Figure 1 , Figure 1 A flow chart of a method for detecting anomaly of a target position provided in an embodiment of the present application, the method comprising the following steps.

[0036] In step 101, a video stream corresponding to a target monitoring area is periodically acquired, and target detection is performed on video frames in the video stream.

[0037] During specific implementation, when it is determined that the preset collection period has been reached, the video stream corresponding to the collected target monitoring area can be obtained, where the preset collection period, such as 5s, 10s, 30s, or 1 minute, can be specifically set by technical personnel, and this application is not limited here. The target monitoring area refers to the spatial range that needs to be paid attention to and analyzed, which can be a fixed physical area (for example, station entrances / exits, traffic intersections, shopping mall entrances / exits, warehouses, office areas, etc.) or a dynamically adjusted virtual boundary (for example, when tracking a person or object, the monitoring area will be dynamically adjusted as the target moves). The selection and setting of the target monitoring area depends on the specific monitoring needs and application scenarios. The video stream can come from a variety of sources, such as cameras, webcams, local video files, or real-time streaming services.

[0038] In specific implementation, after obtaining the video stream corresponding to the target monitoring area, the continuous video frames corresponding to the video stream can be determined based on the video stream; in this way, it can be ensured that the subsequent target detection and tracking process can proceed smoothly. After that, the pre-trained target detection model is used to perform target detection on each video frame in the continuous video frames. The target image sequence can also be determined based on the continuous video frames, and the target image sequence is detected using the pre-trained target detection model. The target detection model includes but is not limited to the YOLO (You Only Look Once) series or the single-stage target detection model (Single Shot MultiBox Detector, SSD), etc.

[0039] In the specific implementation, when selecting a multi-target detection model, multiple factors can be considered according to the detection requirements. The factors to be considered include but are not limited to the model's accuracy, speed, resource consumption, and specific requirements of the application scenario (such as detection of vehicles and pedestrians). The specific selection can be made by technical personnel, and this application does not limit this.

[0040] In step 102, if it is detected that the video frame includes the target to be detected and the reference target, the motion trajectories of the target to be detected and the reference target are determined in the acquired video stream.

[0041] In a specific implementation, if it is detected that the video frame includes a target to be measured and a reference target, the detection information of the target to be measured and the detection information of the reference target can be determined based on the position information and category information in the detection result, wherein the position information represents the coordinate information of the target to be measured and the reference target, and the category information represents the target category of the target to be measured and the reference target. The detection information of the target to be measured and the detection information of the reference target are input into a pre-trained target tracking model to obtain the tracking results of the target to be measured and the reference target; based on the tracking results of the target to be measured and the reference target, the motion trajectory of the target to be measured and the reference target is determined.

[0042] In a specific implementation, for example, there is a video stream with 20 video frames in total. If the target to be measured and the reference target are detected in the second video frame, the detection information of the target to be measured and the detection information of the reference target can be determined based on the position information and category information in the detection result. Among them, the position information in the detection result of the target detection model, such as the bounding box coordinates of each target or the center point coordinates of the bounding box, and the category information, such as the category label (such as "person", "car", etc.), can be used to separate the target to be measured (such as pedestrians) and the reference target (such as vehicles) according to the position information and the category information, and determine the detection information of the target to be measured and the reference target based on the position information and the category information.

[0043] In specific implementation, after determining the detection information of the target to be detected and the detection information of the reference target, the pre-trained target tracking model can be initialized to assign a unique ID to the target to be detected and the reference target detected for the first time, and record their initial position information. Among them, the target tracking model includes but is not limited to a multi-target tracking algorithm (Simple Online and Realtime Tracking, SORT), a deep learning-based visual target tracking algorithm (Deep learning based SORT, DeepSORT), etc.

[0044] In specific implementation, the detection information of the target to be measured and the detection information of the reference target are input into a pre-trained target tracking model, and the target detection model can track the target to be measured and the reference target in the subsequent video frames (frames 3-20) to obtain the tracking results of the target to be measured and the reference target, wherein the tracking results include the latest position information, ID, etc. of the target to be measured and the reference target. Finally, for the target to be measured and the reference target in each frame of the video, its position information is recorded in each frame, and a complete motion trajectory is formed based on the recorded position information.

[0045] In step 103, the relative positions of the target to be measured and the reference target are determined according to the motion trajectories of the target to be measured and the reference target.

[0046] In a specific implementation, the first coordinates of the target to be measured and the second coordinates of the reference target in each target video frame are determined according to the motion trajectories of the target to be measured and the reference target, wherein the target video frame is a video frame in the video stream that includes the target to be measured and the reference target; for any target video frame, the relative positions of the target to be measured and the reference target are determined based on the first coordinates of the target to be measured and the second coordinates of the reference target.

[0047] The first coordinate and the second coordinate can be the center point coordinates of the target to be measured and the reference target in the current target video frame, or can be the bounding box coordinates of the target to be measured and the reference target in the current target video frame, wherein the bounding box coordinates can be the upper left corner of the current target video frame as the coordinate origin, and the upper left corner of the bounding box x Coordinates and y Coordinates and lower right corner x Coordinates and y Coordinate composition.

[0048] In specific implementation, the distance between the target to be measured and the reference target can be determined based on the first coordinates of the target to be measured and the second coordinates of the reference target; the angle between the target to be measured and the reference target can be determined based on the distance and the preset reference direction; and the relative position of the target to be measured and the reference target can be determined based on the distance and the angle.

[0049] For example, taking the first coordinate and the second coordinate as the center point coordinates of the target to be measured and the reference target in the current target video frame, and the coordinate value of the second coordinate is greater than the coordinate value of the first coordinate, the distance between the target to be measured and the reference target can be determined according to the Euclidean distance formula, for example, y Axis coordinate value and first coordinate y The difference between the axis coordinate values ​​is determined as the difference between the target to be measured and the reference target. y The first distance between the axes, the second coordinate x Axis coordinate value and first coordinate x The difference between the axis coordinate values ​​is determined as the difference between the target to be measured and the reference target. x Then, a square root is taken according to the sum of the squares of the first distance and the second distance, and the obtained square root is determined as the distance between the target to be measured and the reference target.

[0050] In a specific implementation, after obtaining the distance between the target to be measured and the reference target, the angle between the target to be measured and the reference target can be determined based on the distance and the preset reference direction, wherein the preset reference direction can be x Axis direction, can also be y Axis direction, with the preset reference direction as x Taking the axis direction as an example, the angle between the target to be measured and the reference target can be determined according to the inverse sine function or the inverse cosine function.

[0051] In a specific implementation, after determining the first distance between the target to be measured and the reference target and determining the second distance between the target to be measured and the reference target, the angle between the target to be measured and the reference target may be directly determined according to the inverse tangent function.

[0052] In step 104, based on the relative position and a preset anomaly detection rule corresponding to the target to be detected, position anomaly detection is performed on the target to be detected.

[0053] In specific implementation, the first comparison result can be determined based on the distance in the relative position and the preset distance value in the anomaly detection rule, and the second comparison result can be determined based on the angle in the relative position and the preset angle value in the anomaly detection rule; based on the first comparison result and the second comparison result, if it is determined that the abnormal condition is met, it is determined that the position of the target to be measured is abnormal.

[0054] The abnormal condition includes at least one of the following: The first comparison result is greater than the preset distance value, and the second comparison result is greater than the preset angle value; that is, if the distance between the target to be measured and the reference target exceeds the preset value, and the angle between them also exceeds the preset value, the position of the target to be measured is considered abnormal. For example, to monitor whether a person has entered a restricted area, if the distance between the operator and the boundary line exceeds the preset distance value, and the angle of the moving direction is greater than the preset angle value, the position of the operator is determined to be abnormal.

[0055] The first comparison result is greater than the preset distance value, or the second comparison result is greater than the preset angle value; that is, as long as one of the conditions (distance or angle) exceeds the preset value, the position of the target to be measured is considered abnormal. For example, in scenarios where a safety rope is required, such as high-altitude operations, rock climbing, and rescue, ensure that the operator always maintains a safe angle range with the safety rope. When it is detected that the angle between the operator and the safety rope is greater than the preset angle value, the operator's position is considered abnormal.

[0056] The first comparison result is not greater than the preset distance value, and the second comparison result is not greater than the preset angle value; that is, if the distance between the target to be measured and the reference target is not greater than the preset value, and the angle between them is not greater than the preset value, the position of the target to be measured is considered abnormal. For example, at a crosswalk or intersection, the relative position between pedestrians and vehicles is determined by angle and distance. When the distance between the pedestrian and the vehicle is less than the preset distance value, and the angle is less than the preset angle value, the position of the pedestrian or vehicle is considered abnormal.

[0057] The first comparison result is not greater than the preset distance value, or the second comparison result is not greater than the preset angle value, that is, as long as one of the conditions (distance or angle) is not greater than the preset value, the position of the target to be measured is considered abnormal. For example, on an automated production line, if the distance between the robot and the operating device is not greater than the preset distance value, or the angle between the robot and the operating device is not greater than the preset angle value, the robot position is considered abnormal.

[0058] During specific implementation, after determining that the position of the target to be measured is abnormal, an alarm prompt may also be issued.

[0059] It should be noted that if it is detected that the video frame includes multiple targets to be detected and multiple reference targets, the relative position of each target to be detected and each reference target is calculated respectively, and based on the relative position, it is determined whether the target to be detected is a position abnormality detection.

[0060] In an embodiment of the present application, a video stream corresponding to a target monitoring area is periodically acquired, and target detection is performed on video frames in the video stream; if it is detected that the video frame includes a target to be measured and a reference target, the motion trajectories of the target to be measured and the reference target are determined in the acquired video stream; based on the motion trajectories of the target to be measured and the reference target, the relative position of the target to be measured and the reference target is determined; based on the relative position and a pre-set anomaly detection rule corresponding to the target to be measured, position anomaly detection is performed on the target to be measured.

[0061] In this way, anomaly detection is performed through the relative positions of multiple targets, and targets with abnormal positions can be predicted more timely and accurately, effectively preventing potential safety hazards. Compared with the anomaly detection method of a single fixed target type in the prior art, the comprehensiveness and accuracy of the monitoring system can be improved, and more refined and intelligent security management and risk warning can be achieved. Users can also decide by themselves what relative position relationship to use for anomaly alarms by flexibly combining multiple targets, which solves the problems of poor portability, poor alarm effect and single function of existing solutions.

[0062] The following is an example of an application scenario in intelligent traffic management, in which the movement of vehicles and pedestrians is monitored to prevent traffic accidents, wherein the target to be measured is a vehicle, the reference target is a pedestrian, and the pre-set abnormality detection rule corresponding to the target to be measured is that when the distance between the pedestrian and the vehicle is less than a preset distance value, and the angle is less than a preset angle value, the position of the vehicle is considered abnormal. A target position anomaly detection method provided in an embodiment of the present application is introduced in detail.

[0063] See also Figure 2 , Figure 2 A flowchart of another target position anomaly detection method provided in an embodiment of the present application, the method comprising the following steps.

[0064] In step 201, a video stream corresponding to a target monitoring area is periodically acquired, and target detection is performed on video frames in the video stream.

[0065] In step 202, if it is detected that the video frame includes a vehicle and a pedestrian, the detection information of the vehicle and the detection information of the pedestrian are determined based on the position information and the category information in the detection result.

[0066] Among them, the position information represents the coordinate information of the vehicle and the pedestrian, and the category information represents the target category of the vehicle and the pedestrian.

[0067] In step 203, the detection information of the vehicle and the detection information of the pedestrian are input into a pre-trained target tracking model to obtain the tracking results of the vehicle and the pedestrian.

[0068] In step 204, based on the tracking results of the vehicle and the pedestrian, the movement trajectories of the vehicle and the pedestrian are determined.

[0069] In step 205, the relative positions of the vehicle and the pedestrian are determined according to the movement trajectories of the vehicle and the pedestrian.

[0070] In specific implementation, the first coordinates of the vehicle and the second coordinates of the pedestrian in each target video frame are determined according to the movement trajectories of the vehicle and the pedestrian, wherein the target video frame is a video frame in the video stream that includes the vehicle and the pedestrian; for any target video frame, the relative positions of the vehicle and the pedestrian are determined based on the first coordinates of the vehicle and the second coordinates of the pedestrian.

[0071] In a specific implementation, the relative position of the vehicle and the pedestrian includes distance and angle, where Figure 3 As shown, Figure 3 A schematic diagram of the distance position relationship between a vehicle and a pedestrian provided in an embodiment of the present application.

[0072] Among them, the distance between the vehicle and the pedestrian can be calculated according to the first coordinate of the vehicle and the second coordinate of the pedestrian using the Euclidean distance formula. The specific implementation process can refer to step 103, and this application will not repeat it here.

[0073] like Figure 4 As shown, Figure 4 A schematic diagram of the angular position relationship between a vehicle and a pedestrian provided in an embodiment of the present application.

[0074] Among them, after obtaining the distance between the vehicle and the pedestrian, the angle between the vehicle and the pedestrian can be determined based on the distance and the preset reference direction. The specific implementation process can refer to step 103, and this application will not repeat it here.

[0075] In step 206, based on the relative position and a preset abnormality detection rule for the vehicle, position abnormality detection is performed on the vehicle.

[0076] Among them, if the distance between the pedestrian and the vehicle is less than the preset distance value, and the angle is less than the preset angle value, the vehicle's position is considered to be abnormal. After determining that the vehicle's position is abnormal, an alarm can also be issued.

[0077] In this way, the interactive correlation relationship of general multiple targets is used to issue abnormal alarms in video surveillance, which effectively avoids the problems of only considering a single category and dealing with a single problem, few application scenarios, and poor alarm effects. It further broadens the practicality and applicability of the field of intelligent video surveillance, enriches the usage scenarios, and allows users to decide for themselves what relative position relationship to use for flexible combination of multiple targets for abnormal alarms, solving the problems of poor portability, poor alarm effects, and single functions of existing solutions.

[0078] Based on the same technical concept, an embodiment of the present application also provides a target position abnormality detection device. The principle of solving the problem by the target position abnormality detection device is similar to that of the above-mentioned target position abnormality detection method. Therefore, the implementation of the target position abnormality detection device can refer to the implementation of the target position abnormality detection method, and the repeated parts will not be repeated.

[0079] Figure 5 A structural schematic diagram of a target position anomaly detection device provided in an embodiment of the present application includes an acquisition module 501, a first determination module 502, a second determination module 503, and a detection module 504.

[0080] The acquisition module 501 is used to periodically acquire the video stream corresponding to the target monitoring area and perform target detection on the video frames in the video stream; A first determination module 502 is configured to determine the motion trajectories of the target to be detected and the reference target in the acquired video stream if it is detected that the video frame includes the target to be detected and the reference target; A second determination module 503 is used to determine the relative position of the target to be measured and the reference target according to the motion trajectories of the target to be measured and the reference target; The detection module 504 is used to perform position anomaly detection on the target to be detected based on the relative position and a preset anomaly detection rule corresponding to the target to be detected.

[0081] In some embodiments, the first determining module 502 is specifically configured to: If it is detected that the video frame includes a target to be detected and a reference target, then based on the position information and category information in the detection result, the detection information of the target to be detected and the detection information of the reference target are determined, wherein the position information represents the coordinate information of the target to be detected and the reference target, and the category information represents the target category of the target to be detected and the reference target; Inputting the detection information of the target to be detected and the detection information of the reference target into a pre-trained target tracking model to obtain tracking results of the target to be detected and the reference target; Based on the tracking results of the target to be measured and the reference target, the motion trajectories of the target to be measured and the reference target are determined.

[0082] In some embodiments, the second determining module 503 is specifically configured to: Determine, according to the motion trajectories of the target to be measured and the reference target, a first coordinate of the target to be measured and a second coordinate of the reference target in each target video frame, wherein the target video frame is a video frame in the video stream including the target to be measured and the reference target; For any target video frame, the relative position of the target to be measured and the reference target is determined based on the first coordinates of the target to be measured and the second coordinates of the reference target.

[0083] In some embodiments, the second determining module 503 is specifically configured to: Determining a distance between the target to be measured and the reference target based on the first coordinates of the target to be measured and the second coordinates of the reference target; Based on the distance and a preset reference direction, determining an angle between the target to be measured and the reference target; Based on the distance and the angle, a relative position of the target to be measured and the reference target is determined.

[0084] In some embodiments, the detection module 504 is specifically configured to: Determining a first comparison result based on the distance in the relative position and the preset distance value in the anomaly detection rule, and Determining a second comparison result based on the angle in the relative position and the angle preset value in the anomaly detection rule; Based on the first comparison result and the second comparison result, if it is determined that an abnormal condition is met, it is determined that the position of the target to be measured is abnormal.

[0085] In some embodiments, the abnormal condition includes at least one of the following: The first comparison result is greater than the preset distance value, and the second comparison result is greater than the preset angle value; The first comparison result is greater than the preset distance value, or the second comparison result is greater than the preset angle value; The first comparison result is not greater than the preset distance value, and the second comparison result is not greater than the preset angle value; The first comparison result is not greater than the preset distance value, or the second comparison result is not greater than the preset angle value.

[0086] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional modules in the embodiments of the present application may be integrated in a processor, or may exist physically separately, or two or more modules may be integrated in one module. The coupling between the modules may be achieved through some interfaces, which are usually electrical communication interfaces, but it is not excluded that they may be mechanical interfaces or other forms of interfaces. Therefore, the modules described as separate components may or may not be physically separated, and may be located in one place or distributed to different locations of the same or different devices. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0087] Based on the same technical concept, the embodiment of the present application provides an electronic device, which can realize the function of detecting abnormal target position discussed above, referring to Figure 6 The electronic device 130 implemented according to this embodiment of the present application is described. Figure 6 The electronic device 130 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0088] like Figure 6 As shown, the electronic device 130 is in the form of a general electronic device. The components of the electronic device 130 may include but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0089] Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0090] The memory 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory 1322 , and may further include a read-only memory (ROM) 1323 .

[0091] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0092] The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 130, and / or communicate with any device that enables the electronic device 130 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 135. In addition, the electronic device 130 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 136. As shown, the network adapter 136 communicates with other modules for the electronic device 130 through a bus 133. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0093] Based on the same inventive concept, the embodiment of the present application provides a computer-readable storage medium, and the computer program product includes: computer program code, when the computer program code is run on a computer, the computer executes any of the target position anomaly detection methods discussed above. Since the principle of solving the problem by the above-mentioned computer-readable storage medium is similar to that of the target position anomaly detection method, the implementation of the above-mentioned computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be repeated.

[0094] Based on the same inventive concept, the embodiment of the present application also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, the computer executes any of the target position abnormality detection methods discussed above. Since the principle of solving the problem by the above computer program product is similar to that of the target position abnormality detection method, the implementation of the above computer program product can refer to the implementation of the method, and the repeated parts will not be repeated.

[0095] Furthermore, the computer program product may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0096] The program product for target position abnormality detection in the embodiment of the present application can adopt a CD-ROM and include program code, and can be run on a computing device. However, the program product of the present application is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, apparatus or device.

[0097] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0098] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0099] The technical solution provided by the present application is introduced in detail above. The principles and implementation methods of the present application are explained by using specific examples in the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0100] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0101] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0104] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting anomaly of a target position, characterized in that: The method comprises: Periodically acquiring a video stream corresponding to a target monitoring area, and performing target detection on video frames in the video stream; If it is detected that the video frame includes the target to be measured and the reference target, then determining the motion trajectories of the target to be measured and the reference target in the acquired video stream; Determining the relative position of the target to be measured and the reference target according to the motion trajectories of the target to be measured and the reference target; Based on the relative position and a preset anomaly detection rule corresponding to the target to be detected, position anomaly detection is performed on the target to be detected.

2. The method according to claim 1, characterized in that If it is detected that the video frame includes the target to be detected and the reference target, then determining the motion trajectories of the target to be detected and the reference target in the acquired video stream includes: If it is detected that the video frame includes a target to be detected and a reference target, then based on the position information and category information in the detection result, the detection information of the target to be detected and the detection information of the reference target are determined, wherein the position information represents the coordinate information of the target to be detected and the reference target, and the category information represents the target category of the target to be detected and the reference target; Inputting the detection information of the target to be detected and the detection information of the reference target into a pre-trained target tracking model to obtain tracking results of the target to be detected and the reference target; Based on the tracking results of the target to be measured and the reference target, the motion trajectories of the target to be measured and the reference target are determined.

3. The method according to claim 1, characterized in that The determining the relative position of the target to be measured and the reference target according to the motion trajectories of the target to be measured and the reference target comprises: Determine, according to the motion trajectories of the target to be measured and the reference target, a first coordinate of the target to be measured and a second coordinate of the reference target in each target video frame, wherein the target video frame is a video frame in the video stream including the target to be measured and the reference target; For any target video frame, the relative position of the target to be measured and the reference target is determined based on the first coordinates of the target to be measured and the second coordinates of the reference target.

4. The method according to claim 3, characterized in that The determining the relative position of the target to be measured and the reference target based on the first coordinate of the target to be measured and the second coordinate of the reference target includes: Determining a distance between the target to be measured and the reference target based on the first coordinates of the target to be measured and the second coordinates of the reference target; Based on the distance and a preset reference direction, determining an angle between the target to be measured and the reference target; Based on the distance and the angle, a relative position of the target to be measured and the reference target is determined.

5. The method according to claim 1, characterized in that The performing position anomaly detection on the target to be detected based on the relative position and a preset anomaly detection rule corresponding to the target to be detected includes: Determining a first comparison result based on the distance in the relative position and the preset distance value in the anomaly detection rule, and Determining a second comparison result based on the angle in the relative position and the angle preset value in the anomaly detection rule; Based on the first comparison result and the second comparison result, if it is determined that an abnormal condition is met, it is determined that the position of the target to be measured is abnormal.

6. The method according to claim 5, characterized in that The abnormal condition includes at least one of the following: The first comparison result is greater than the preset distance value, and the second comparison result is greater than the preset angle value; The first comparison result is greater than the preset distance value, or the second comparison result is greater than the preset angle value; The first comparison result is not greater than the preset distance value, and the second comparison result is not greater than the preset angle value; The first comparison result is not greater than the preset distance value, or the second comparison result is not greater than the preset angle value.

7. A target position abnormality detection device, characterized in that: The device comprises: An acquisition module is used to periodically acquire a video stream corresponding to a target monitoring area and perform target detection on video frames in the video stream; A first determination module is used to determine the motion trajectories of the target to be detected and the reference target in the acquired video stream if it is detected that the video frame includes the target to be detected and the reference target; A second determination module is used to determine the relative position of the target to be measured and the reference target according to the motion trajectories of the target to be measured and the reference target; The detection module is used to perform position anomaly detection on the target to be detected based on the relative position and a preset anomaly detection rule corresponding to the target to be detected.

8. The device according to claim 7, characterized in that The first determining module is specifically configured to: If it is detected that the video frame includes a target to be detected and a reference target, then based on the position information and category information in the detection result, the detection information of the target to be detected and the detection information of the reference target are determined, wherein the position information represents the coordinate information of the target to be detected and the reference target, and the category information represents the target category of the target to be detected and the reference target; Inputting the detection information of the target to be detected and the detection information of the reference target into a pre-trained target tracking model to obtain tracking results of the target to be detected and the reference target; Based on the tracking results of the target to be measured and the reference target, the motion trajectories of the target to be measured and the reference target are determined.

9. An electronic device, characterized in that: include: at least one processor, and a memory communicatively connected to the at least one processor, wherein: The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 6.

10. A storage medium, characterized in that: When the computer program in the storage medium is executed by a processor of an electronic device, the electronic device can execute the method according to any one of claims 1 to 6.

Images