Global tracking monitoring method and system based on multi-path data collaboration

Abstract

The invention provides a global tracking monitoring method and system based on multi-path data collaboration, and the method comprises the steps: constructing a global event feature pool for storing an object event of a tracking object in a monitoring region through the configuration of tracking object information, receiving monitoring image data transmitted by a monitoring device, and transmitting the monitoring image data to a server; and extracting event features of an object event of the tracking object from the monitoring picture data, writing the event features into the global event feature pool, periodically executing event correlation analysis on the event features in the global event feature pool, and implementing global risk prediction according to a correlation analysis result. The risk prediction capability of the security and protection monitoring system can be effectively improved, sufficient coping decision time is provided before a risk event occurs, and the purpose of nipping in the bud is truly achieved.

Description

Technical Field

[0001] This invention relates to the field of security monitoring technology, and in particular to a global tracking and monitoring method and system based on multi-channel data collaboration. Background Technology

[0002] With the development and advancement of related technologies such as the Internet of Things (IoT) and artificial intelligence (AI), security monitoring systems are also moving towards comprehensive intelligence, with increasingly powerful risk identification and early warning capabilities. Anomaly identification is the foundation for risk identification and early warning in security monitoring systems; therefore, the accuracy of anomaly identification and analysis is a crucial indicator of the system's risk warning capability. Thanks to the development of AI technology, especially the proliferation of various specialized and general-purpose image analysis models, the accuracy and efficiency of anomaly identification using surveillance images have significantly improved. This has made it possible to implement global tracking of multiple objects over a large geographical area and perform real-time anomaly analysis based on the tracked images. Existing security monitoring systems typically focus only on anomalies and analyze them as isolated events, often overlooking potential risks. This leads to untimely risk warnings, insufficient decision-making time, and the inability to prevent risks from escalating beyond post-incident remediation. Summary of the Invention

[0003] Based on the above-mentioned problems, this invention proposes a global tracking and monitoring method and system based on multi-channel data collaboration, which can effectively improve the risk prediction capability of security monitoring systems, provide sufficient time for response and decision-making before risk events occur, and truly achieve prevention before problems occur.

[0004] In view of this, the first aspect of the present invention proposes a global tracking and monitoring method based on multi-channel data collaboration, comprising: Configure tracking object information, which includes the type information of the tracking object; Construct a global event feature pool for storing object events of the tracked objects within the monitoring area; Receive monitoring screen data transmitted by the monitoring device; Extract the event features of the object events of the tracked object from the monitoring screen data and write them into the global event feature pool; Periodically perform event correlation analysis on the event features in the global event feature pool; Implement global risk prediction based on the results of correlation analysis.

[0005] Optionally, the step of extracting the event features of the tracked object's events from the monitoring screen data and writing them into the global event feature pool specifically includes: The system receives event information sent by the monitoring device. The event information includes the tracked object associated with the object event, the event type of the object event, and the time, location, and duration of the object event. The feature information of the object event is extracted from the monitoring screen data. The feature information includes several feature keywords used to describe the dynamic attributes of the object event. Generate an event feature instance of the object event; Write the event information and feature keywords of the object event into the corresponding attribute variables of the event feature instance; The event feature instance containing the event information and feature information of the object event is written into the global event feature pool.

[0006] Optionally, the step of extracting the feature information of the object event from the monitoring screen data specifically includes: The occurrence time of the object event is read from the event information of the object event, where the occurrence time of the object event is the time when the monitoring device detects the object event; Identify at least one event image from monitoring footage data after the time of occurrence; Extract a partial image from the event image that contains the event-related entities of the object event, wherein the event-related entities are the necessary entity objects that participate in the object event; The local image is input into a pre-trained event feature extraction model to extract feature keywords used to describe the object event.

[0007] Optionally, the step of determining several event images from monitoring screen data after the occurrence time specifically includes: Obtain the pre-configured minimum number of images and image sampling duration; Multiple candidate event images are sampled from the monitoring screen data after the occurrence time, with the image sampling duration as the time interval; When a candidate event image is obtained by sampling, it is determined whether the candidate event image meets the event feature extraction conditions; When the candidate event image meets the event feature extraction conditions, the candidate event image is determined as an event image; The process involves repeatedly performing the following steps: when a candidate event image is obtained through sampling, determining whether the candidate event image meets the event feature extraction conditions; and when the candidate event image meets the event feature extraction conditions, determining the candidate event image as an event image, until the number of event images is greater than or equal to the minimum number of images.

[0008] Optionally, the step of determining whether the candidate event image meets the event feature extraction conditions specifically includes: Load the list of event-associated entities of the object event, and the identification features of each event-associated entity in the list of event-associated entities, wherein the identification features are the appearance features of the event-associated entity; Based on the identification features, determine whether the candidate event image contains all event-related entities in the event-related entity list; When the candidate event image contains all event-related entities in the event-related entity list, the candidate event image is determined as the event image of the object event.

[0009] Optionally, the step of periodically performing event correlation analysis on the event features in the global event feature pool specifically includes: Obtain the pre-configured global event analysis cycle and spatiotemporal correlation strength threshold; In each global event analysis cycle, the event features in the global event feature pool are traversed to calculate the spatiotemporal correlation strength between each pair of event features; Two event features with a spatiotemporal correlation strength greater than the spatiotemporal correlation strength threshold are input into a pre-trained event correlation matching model for event matching; Output the matching identifier for the corresponding event characteristics and the parent event ID.

[0010] Optionally, the step of representing the two event features for which the event correlation strength needs to be calculated as the first event feature and the second event feature, and traversing the event features in the global event feature pool to calculate the spatiotemporal correlation strength between each pair of event features specifically includes: Calculate the first distance between the first event feature and the second event feature on a spatial scale. And the second distance between the two on the time scale. ; Based on the first distance and the second distance Calculate the spatiotemporal correlation strength between the first event feature and the second event feature: .

[0011] Optionally, calculate the first distance between the first event feature and the second event feature on the time scale. And the second distance between the two on a spatial scale. The specific steps include: Obtain the spatial dimensions of the monitored area on a two-dimensional plane projection. ; Determine the first tracking object and the first object event associated with the first event feature, and the second tracking object and the second object event associated with the second event feature; Generate a first movement trajectory of the first tracked object on a two-dimensional plane during the occurrence of the first object event, and a second movement trajectory of the second tracked object on a two-dimensional plane during the occurrence of the second object event; Calculate the minimum spatial distance between the first movement trajectory and the second movement trajectory. ; Calculate the first distance between the first event feature and the second event feature in terms of spatial scale: .

[0012] Determine the minimum spatial distance corresponding to the first movement trajectory. The first coordinate point, and the second coordinate point on the second movement trajectory corresponding to the minimum spatial distance; Determine the first moment when the first tracked object passes through the first coordinate point. and the second moment when the second tracked object passes through the second coordinate point. ; Get the duration of the event of the first object. and the duration of the second object event. ; Calculate the first distance between the first event feature and the second event feature in terms of spatial scale: .

[0013] Optionally, the steps for implementing global risk prediction based on the correlation analysis results specifically include: Obtain the pre-configured prediction duration of the action trajectory; Obtain the constraint data corresponding to the parent event ID, wherein the constraint data includes one or more of time constraint data, position constraint data, and velocity constraint data; Predict the movement trajectory of the first or second tracked object within a future period of time with a duration equal to the predicted movement trajectory duration; Determine whether the predicted movement trajectory of the first or second tracked object conforms to the constraint conditions of the constraint data.

[0014] A second aspect of the present invention proposes a global tracking and monitoring system based on multi-channel data collaboration, comprising: Several monitoring devices are used to acquire monitoring images within the monitored area; A monitoring server, which is communicatively connected to the monitoring device, is used to perform global tracking and monitoring of the tracked objects in the monitoring area based on the monitoring screen. The monitoring device includes an image sensing unit for generating monitoring screen data, an image processing unit for processing the monitoring screen data, and a tracking detection unit for tracking and detecting a target object using the image content processed by the image processing unit. The tracking detection unit is configured as follows: Detect the tracked object in the monitoring screen; When the tracked object is detected on the monitoring screen, the position and status of the tracked object are tracked and monitored; Based on changes in the location and / or state of the tracked object, perform object event recognition on the monitoring screen containing the tracked object; Determine whether the object events of the tracked object meet preset conditions; When the object event of the tracked object meets the preset conditions, the event information of the object event is sent to the monitoring server. The event information includes the tracked object associated with the object event, the event type of the object event, and the time, location, and duration of the object event. The monitoring screen data is transmitted to the monitoring server, so that the monitoring server can extract the feature information of the object event from the monitoring screen data; The monitoring server is configured to implement the global tracking and monitoring method based on multi-channel data collaboration as described in any of the first aspects of the present invention.

[0015] This invention proposes a global tracking and monitoring method and system based on multi-channel data collaboration. By configuring tracking object information, a global event feature pool is constructed to store object events of the tracked objects within the monitoring area. Monitoring screen data transmitted by the monitoring device is received, and event features of the object events of the tracked objects are extracted from the monitoring screen data and written into the global event feature pool. Event correlation analysis is periodically performed on the event features in the global event feature pool, and global risk prediction is implemented based on the correlation analysis results. This can effectively improve the risk prediction capability of the security monitoring system, provide sufficient time for response and decision-making before risk events occur, and truly achieve prevention before problems arise. Attached Figure Description

[0016] Figure 1 This is a flowchart of a global tracking and monitoring method based on multi-channel data collaboration provided in one embodiment of the present invention; Figure 2 This is a schematic diagram of a global tracking and monitoring system based on multi-channel data collaboration provided in one embodiment of the present invention. Detailed Implementation

[0017] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0019] In the description of this invention, the term "multiple" refers to two or more. Unless otherwise explicitly defined, the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. The terms "connect," "install," "fix," etc., should be interpreted broadly. For example, "connect" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "multiple" means two or more.

[0020] In the description of this specification, the terms "one embodiment," "some implementations," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0021] The following description, with reference to the accompanying drawings, illustrates a global tracking and monitoring method and system based on multi-channel data collaboration, according to some embodiments of the present invention.

[0022] like Figure 1 As shown, the first aspect of the present invention proposes a global tracking and monitoring method based on multi-channel data collaboration, comprising: Configure tracking object information, which includes the type information of the tracking object; Construct a global event feature pool for storing object events of the tracked objects within the monitoring area; Receive monitoring screen data transmitted by the monitoring device; Extract the event features of the object events of the tracked object from the monitoring screen data and write them into the global event feature pool; Periodically perform event correlation analysis on the event features in the global event feature pool; Implement global risk prediction based on the results of correlation analysis.

[0023] In the technical solution of this invention, the tracked object can be any moving object within the monitoring area, including objects with active movement capabilities such as pedestrians or vehicles, as well as passively moving objects carried or transported by objects with active movement capabilities, such as suitcases or goods. In specific implementations, the types of tracked objects to be monitored can be configured according to the different functional attributes of the monitoring area. For example, in a residential area, only pedestrians can be configured as tracked objects, while in an industrial park, multiple types of moving objects such as pedestrians, forklifts, and goods need to be configured as tracked objects.

[0024] In the above implementation method, the step of configuring tracking object information is to configure the type information of the tracking object, such as tracking pedestrians, vehicles or goods in the monitoring area. That is, when the tracking object information includes pedestrians, all pedestrians in the monitoring area are tracked and monitored.

[0025] In some embodiments of the present invention, the tracking object information may further include image feature information of a specific tracking object, such as facial feature information of a specific pedestrian, or appearance features and license plate number information of a specific vehicle, thereby enabling the tracking and monitoring of the specific tracking object within the monitoring area.

[0026] The object events referred to in this invention include any change events in the position or state of the tracked object, as well as interactive events in which the tracked object interacts with any person or object within the monitoring area, such as pedestrian walking events, running events, conversation events, and door opening events, or object handling events, disassembly and assembly events, and specific usage events.

[0027] The event characteristics include event information and feature information. The event information includes the tracked object associated with the object event, the event type, and information such as the time, location, and duration of the object event. The feature information includes several feature keywords used to describe the dynamic attributes of the object event. The feature keywords in the feature information are related to the event type of the object event. For example, when the object event is a position change event of the tracked object, it can be feature keywords describing the movement direction, movement speed, and other movement states of the tracked object. When the object event is a state change event of the tracked object, it can be feature keywords describing changes in the appearance, posture, or action of the tracked object. When the object event is an interaction event of the tracked object, the feature information can be feature keywords describing the interaction object, interaction type, interaction state, and other interaction event attributes.

[0028] The global event feature pool is used to temporarily store the event features of object events that meet preset conditions in multi-channel monitoring screen data within a certain period of time, and is used to perform event correlation analysis and global risk prediction.

[0029] Optionally, the step of extracting the event features of the tracked object's events from the monitoring screen data and writing them into the global event feature pool specifically includes: The system receives event information sent by the monitoring device. The event information includes the tracked object associated with the object event, the event type of the object event, and the time, location, and duration of the object event. The feature information of the object event is extracted from the monitoring screen data. The feature information includes several feature keywords used to describe the dynamic attributes of the object event. Generate an event feature instance of the object event; Write the event information and feature keywords of the object event into the corresponding attribute variables of the event feature instance; The event feature instance containing the event information and feature information of the object event is written into the global event feature pool.

[0030] In the technical solution of the above embodiments, before the step of extracting the event features of the tracked object's events from the monitoring screen data and writing them into the global event feature pool, the method further includes constructing an event feature model for storing event feature information. The event feature model is a data storage model for event features in the cache space, which may include a model name, an identifier attribute variable for storing the event ID of the object event, and event feature attribute variables for storing various event information and feature information.

[0031] In the step of generating event feature instances of the object event, the event feature model is used to instantiate the event features of each object event, so as to allocate cache space in the cache space for storing the event feature data of the object event, and at the same time assign an event ID to each event feature instance.

[0032] Optionally, the step of extracting the feature information of the object event from the monitoring screen data specifically includes: The occurrence time of the object event is read from the event information of the object event, where the occurrence time of the object event is the time when the monitoring device detects the object event; Identify at least one event image from monitoring footage data after the time of occurrence; Extract a partial image from the event image that contains the event-related entities of the object event, wherein the event-related entities are the necessary entity objects that participate in the object event; The local image is input into a pre-trained event feature extraction model to extract feature keywords used to describe the object event.

[0033] Besides the object being tracked, the associated entities for different types of object events often differ. For example, in a pedestrian walking event, the necessary participants, i.e., the entity elements, include the pedestrian themselves. However, for a car driving event, the necessary participants include the car itself and the driver, while the passengers in the car are not necessary participants.

[0034] The event feature extraction model is a pre-trained artificial intelligence generation model used to extract feature keywords from event images. It takes an event image as input data and uses a multi-layer convolutional neural network to generate several descriptive words for the event image as the feature keywords. In some implementations, multiple event images of the target event can be extracted from the monitoring screen data. Each event image is input into the model to obtain a keyword sequence corresponding to each event image. The multiple output keyword sequences are then deduplicated and merged to obtain the feature information of the target event.

[0035] Optionally, the step of determining several event images from monitoring screen data after the occurrence time specifically includes: Obtain the pre-configured minimum number of images and image sampling duration; Multiple candidate event images are sampled from the monitoring screen data after the occurrence time, with the image sampling duration as the time interval; When a candidate event image is obtained by sampling, it is determined whether the candidate event image meets the event feature extraction conditions; When the candidate event image meets the event feature extraction conditions, the candidate event image is determined as an event image; The process involves repeatedly performing the following steps: when a candidate event image is obtained through sampling, determining whether the candidate event image meets the event feature extraction conditions; and when the candidate event image meets the event feature extraction conditions, determining the candidate event image as an event image, until the number of event images is greater than or equal to the minimum number of images.

[0036] In the technical solution of this invention, the term "entity object" refers to a physical entity that can be captured by the monitoring device and is visible in the monitoring screen. Since the participating entities in different object events will vary in size, position, and posture, in some implementations, a separate minimum number of images can be configured for different event types. This allows the global tracking and monitoring system to use multiple event images to collect descriptive data of each participating entity in different orientations or postures. Of course, to improve processing efficiency, a uniform minimum number of images can also be used for all types of event objects.

[0037] Furthermore, before the step of extracting the event features of the tracked object's events from the monitoring screen data and writing them into the global event feature pool, the method also includes configuring the event image extraction duration. In the process of iteratively executing the steps of determining whether a candidate event image meets the event feature extraction conditions and determining a candidate event image as an event image when the candidate event image meets the event feature extraction conditions, and continuing until the number of event images is greater than or equal to the minimum number of images, if the number of event images collected within the event image extraction time is less than the minimum number of images, the corresponding object event is discarded.

[0038] Optionally, the step of determining whether the candidate event image meets the event feature extraction conditions specifically includes: Load the list of event-associated entities of the object event, and the identification features of each event-associated entity in the list of event-associated entities, wherein the identification features are the appearance features of the event-associated entity; Based on the identification features, determine whether the candidate event image contains all event-related entities in the event-related entity list; When the candidate event image contains all event-related entities in the event-related entity list, the candidate event image is determined as the event image of the object event.

[0039] The event-related entity list contains information on all necessary participants in the object event, i.e., event-related entities. In the step of determining whether a candidate event image contains all event-related entities in the event-related entity list based on the identification features, the candidate event image is considered to contain the corresponding event-related entity if and only if there is an event-related entity in the candidate event image that contains a portion of the identification features. For example, using a pedestrian's facial features as the identification feature, when a candidate event image contains a pedestrian's facial image, it is considered that the candidate event image contains the corresponding pedestrian, without needing to capture a complete image of the pedestrian.

[0040] Furthermore, after the step of determining whether the candidate event image contains all event-related entities in the event-related entity list based on the recognition features, the method further includes: When the candidate event image does not contain all the event-related entities in the event-related entity list, that is, when the number of event-related entities contained in the candidate event image is less than the number of event-related entities in the event-related entity list, the candidate event image is discarded.

[0041] Optionally, the step of periodically performing event correlation analysis on the event features in the global event feature pool specifically includes: Obtain the pre-configured global event analysis cycle and spatiotemporal correlation strength threshold; In each global event analysis cycle, the event features in the global event feature pool are traversed to calculate the spatiotemporal correlation strength between each pair of event features; Two event features with a spatiotemporal correlation strength greater than the spatiotemporal correlation strength threshold are input into a pre-trained event correlation matching model for event matching; Output the matching identifier for the corresponding event characteristics and the parent event ID.

[0042] The event correlation matching model is a pre-trained artificial intelligence classification model used to match the features of two events to identify whether they have a common parent event. It takes two quantized event features, which consist of quantized event information and quantized feature information, as input data, and uses a trained classification mechanism, such as a multi-layer convolutional neural network, to classify the input data to determine whether they have a common parent event.

[0043] More specifically, the output data of the event correlation matching model is a data sequence consisting of a matching identifier bit and a parent event ID. For example, the matching identifier bit can be 0 or 1. When the matching identifier bit is 0, it indicates that the two input event features do not have a common parent event, and the parent ID data bit in the output data can be represented by a data sequence of all 0s. Conversely, when the matching identifier bit is 1, it indicates that the two input event features have a common parent event, and the parent ID data bit in the output data is the ID encoding of their common parent event.

[0044] The parent event is the common upper-level event of two object events associated with two event characteristics; that is, both object events can be understood as child events belonging to the same parent event. For example, in a visitor's arrival event, from the moment the visitor enters the park by car, a series of child events occur, such as driving, registering, walking, opening doors, and conversing. The arrival event is the common parent event of these child events. The parent event ID is a pre-configured, unique identifier for each type of parent event stored in the database, used to uniquely represent a parent event.

[0045] Optionally, the step of representing the two event features for which the event correlation strength needs to be calculated as the first event feature and the second event feature, and traversing the event features in the global event feature pool to calculate the spatiotemporal correlation strength between each pair of event features specifically includes: Calculate the first distance between the first event feature and the second event feature on a spatial scale. And the second distance between the two on the time scale. ; Based on the first distance and the second distance Calculate the spatiotemporal correlation strength between the first event feature and the second event feature: .

[0046] In some embodiments of the present invention, when the first distance... At that time, let the spatiotemporal correlation strength of the first event feature and the second event feature be... .

[0047] Optionally, calculate the first distance between the first event feature and the second event feature on the time scale. And the second distance between the two on a spatial scale. The specific steps include: Obtain the spatial dimensions of the monitored area on a two-dimensional plane projection. ; Determine the first tracking object and the first object event associated with the first event feature, and the second tracking object and the second object event associated with the second event feature; Generate a first movement trajectory of the first tracked object on a two-dimensional plane during the occurrence of the first object event, and a second movement trajectory of the second tracked object on a two-dimensional plane during the occurrence of the second object event; Calculate the minimum spatial distance between the first movement trajectory and the second movement trajectory. ; Calculate the first distance between the first event feature and the second event feature in terms of spatial scale: .

[0048] Determine the minimum spatial distance corresponding to the first movement trajectory. The first coordinate point, and the second coordinate point on the second movement trajectory corresponding to the minimum spatial distance; Determine the first moment when the first tracked object passes through the first coordinate point. and the second moment when the second tracked object passes through the second coordinate point. ; Get the duration of the event of the first object. and the duration of the second object event. ; Calculate the first distance between the first event feature and the second event feature in terms of spatial scale: .

[0049] The spatial scale of the monitored area on a two-dimensional plane projection It can be any dimension used to represent the size of the projected area of ​​the monitored area on a two-dimensional plane parallel to the ground. For example, it can be the perimeter or maximum width of an industrial park on a two-dimensional map.

[0050] The first coordinate point and the second coordinate point are at the minimum spatial distance. Correspondingly, this refers to the distance between the first coordinate point and the second coordinate point being... On both the first and second movement trajectories, except for the first and second coordinate points, the length of the line connecting any other point on the first movement trajectory to any other point on the second movement trajectory is greater than or equal to the minimum spatial distance. The minimum spatial distance between the first movement trajectory and the second movement trajectory is calculated. In the steps, when there is an intersection between the first movement trajectory and the second movement trajectory, then .

[0051] When there are multiple first coordinate points or second coordinate points, the earliest time when the tracked object passes through these coordinate points is taken as the first time or the second time.

[0052] Optionally, the steps for implementing global risk prediction based on the correlation analysis results specifically include: Obtain the pre-configured prediction duration of the action trajectory; Obtain the constraint data corresponding to the parent event ID, wherein the constraint data includes one or more of time constraint data, position constraint data, and velocity constraint data; Predict the movement trajectory of the first or second tracked object within a future period of time with a duration equal to the predicted movement trajectory duration; Determine whether the predicted movement trajectory of the first or second tracked object conforms to the constraint conditions of the constraint data.

[0053] The constraint data refers to the pre-configured safety range evaluation data for the parent event. Similarly, taking a visitor's visit as an example, certain restricted areas can be configured as location constraint data within the park, or the visitor's dwell time within the park can be configured as time constraint data, etc.

[0054] In some implementations, the step of predicting the movement trajectory of the first tracked object or the second tracked object over a future period of time with a duration equal to the predicted movement trajectory duration includes: Determine the third occurrence time of the first object event. and the fourth occurrence time of the second object event The "occurrence time" refers to the earliest moment when the monitoring device or the monitoring server identifies the corresponding object event. When the third occurrence time Later than the fourth occurrence time mentioned above When the time is right, the movement trajectory of the first tracked object is predicted within a future period of time with a time length equal to the movement trajectory prediction time, and the step of determining whether the predicted movement trajectory of the first tracked object meets the constraint conditions of the constraint data is executed. Conversely, when the third occurrence time... Earlier than the fourth occurrence time mentioned above When the time is right, the movement trajectory of the second tracked object is predicted for a future period of time with a time length equal to the predicted movement trajectory duration, and the step of determining whether the predicted movement trajectory of the second tracked object meets the constraint conditions of the constraint data is executed.

[0055] Furthermore, after determining whether the predicted trajectory of the first or second tracked object conforms to the constraints of the constraint data, the method further includes: When the predicted movement trajectory of the first or second tracked object does not meet the constraint conditions of any of the constraint data, it is determined that the first or second tracked object has a high probability of experiencing a risk event in the future.

[0056] like Figure 2 As shown, a second aspect of the present invention proposes a global tracking and monitoring system based on multi-channel data collaboration, comprising: Several monitoring devices are used to acquire monitoring images within the monitored area; A monitoring server, which is communicatively connected to the monitoring device, is used to perform global tracking and monitoring of the tracked objects in the monitoring area based on the monitoring screen. The monitoring device includes an image sensing unit for generating monitoring screen data, an image processing unit for processing the monitoring screen data, and a tracking detection unit for tracking and detecting a target object using the image content processed by the image processing unit. The tracking detection unit is configured as follows: Detect the tracked object in the monitoring screen; When the tracked object is detected on the monitoring screen, the position and status of the tracked object are tracked and monitored; Based on changes in the location and / or state of the tracked object, perform object event recognition on the monitoring screen containing the tracked object; Determine whether the object events of the tracked object meet preset conditions; When the object event of the tracked object meets the preset conditions, the event information of the object event is sent to the monitoring server. The event information includes the tracked object associated with the object event, the event type of the object event, and the time, location, and duration of the object event. The monitoring screen data is transmitted to the monitoring server, so that the monitoring server can extract the feature information of the object event from the monitoring screen data; The monitoring server is configured to implement the global tracking and monitoring method based on multi-channel data collaboration as described in any of the first aspects of the present invention.

[0057] Furthermore, determining whether the object events of the tracked object meet the preset conditions specifically includes: Obtain pre-configured abnormal behavior parameters of the tracked object, the abnormal behavior parameters including one or more of the tracked object's position, movement speed, behavior, and posture when the object event occurs; Extract the position, speed of movement, behavior and / or posture of the tracked object when the object event occurs from the monitoring screen; When the position, speed, behavior, and / or posture of the tracked object at the time of the object event match any of the abnormal behavior parameters, it is determined that the object event of the tracked object meets the preset conditions.

[0058] Furthermore, the global tracking and monitoring system also includes a storage device. In the technical solution of the above embodiment, the monitoring device is directly connected to the storage device and stores the collected monitoring screen data in the storage device.

[0059] In other embodiments, the storage device is connected to the monitoring server, the monitoring device transmits all monitoring screen data to the monitoring server, the monitoring server stores the monitoring screen data in the storage device, and the monitoring server performs tracking object identification and event identification, and then implements the global tracking and monitoring method based on multi-channel data collaboration as described in any of the first aspects of the present invention based on the event identification results.

[0060] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] As described above, these embodiments of the present invention do not exhaustively cover all details, nor do they limit the invention to the specific embodiments described. Clearly, many modifications and variations can be made based on the above description. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to effectively utilize the invention and its modifications. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A global tracking and monitoring method based on multi-channel data collaboration, characterized in that, include: Configure tracking object information, which includes the type information of the tracking object; Construct a global event feature pool for storing object events of the tracked objects within the monitoring area; Receive monitoring screen data transmitted by the monitoring device; Extract the event features of the object events of the tracked object from the monitoring screen data and write them into the global event feature pool; Periodically perform event correlation analysis on the event features in the global event feature pool; Implement global risk prediction based on the results of correlation analysis.

2. The global tracking and monitoring method based on multi-channel data collaboration according to claim 1, characterized in that, The steps of extracting event features of the tracked object's events from the monitoring screen data and writing them into the global event feature pool specifically include: The system receives event information sent by the monitoring device. The event information includes the tracked object associated with the object event, the event type of the object event, and the time, location, and duration of the object event. The feature information of the object event is extracted from the monitoring screen data. The feature information includes several feature keywords used to describe the dynamic attributes of the object event. Generate an event feature instance of the object event; Write the event information and feature keywords of the object event into the corresponding attribute variables of the event feature instance; The event feature instance containing the event information and feature information of the object event is written into the global event feature pool.

3. The global tracking and monitoring method based on multi-channel data collaboration according to claim 2, characterized in that, The steps for extracting feature information of the object event from the monitoring screen data specifically include: The occurrence time of the object event is read from the event information of the object event, where the occurrence time of the object event is the time when the monitoring device detects the object event; Identify at least one event image from monitoring footage data after the time of occurrence; Extract a partial image from the event image that contains the event-related entities of the object event, wherein the event-related entities are the necessary entity objects that participate in the object event; The local image is input into a pre-trained event feature extraction model to extract feature keywords used to describe the object event.

4. The global tracking and monitoring method based on multi-channel data collaboration according to claim 3, characterized in that, The steps for identifying several event images from monitoring footage data after the occurrence time specifically include: Obtain the pre-configured minimum number of images and image sampling duration; Multiple candidate event images are sampled from the monitoring screen data after the occurrence time, with the image sampling duration as the time interval; When a candidate event image is obtained by sampling, it is determined whether the candidate event image meets the event feature extraction conditions; When the candidate event image meets the event feature extraction conditions, the candidate event image is determined as an event image; The process involves repeatedly performing the following steps: when a candidate event image is obtained through sampling, determining whether the candidate event image meets the event feature extraction conditions; and when the candidate event image meets the event feature extraction conditions, determining the candidate event image as an event image, until the number of event images is greater than or equal to the minimum number of images.

5. The global tracking and monitoring method based on multi-channel data collaboration according to claim 4, characterized in that, The steps for determining whether the candidate event image meets the event feature extraction conditions specifically include: Load the list of event-associated entities of the object event, and the identification features of each event-associated entity in the list of event-associated entities, wherein the identification features are the appearance features of the event-associated entity; Based on the identification features, determine whether the candidate event image contains all event-related entities in the event-related entity list; When the candidate event image contains all event-related entities in the event-related entity list, the candidate event image is determined as the event image of the object event.

6. The global tracking and monitoring method based on multi-channel data collaboration according to claim 1, characterized in that, The specific steps of periodically performing event correlation analysis on the event features in the global event feature pool include: Obtain the pre-configured global event analysis cycle and spatiotemporal correlation strength threshold; In each global event analysis cycle, the event features in the global event feature pool are traversed to calculate the spatiotemporal correlation strength between each pair of event features; Two event features with a spatiotemporal correlation strength greater than the spatiotemporal correlation strength threshold are input into a pre-trained event correlation matching model for event matching; Output the matching identifier for the corresponding event characteristics and the parent event ID.

7. The global tracking and monitoring method based on multi-channel data collaboration according to claim 6, characterized in that, The steps of representing the two event features for which the event correlation strength needs to be calculated as the first event feature and the second event feature, and traversing the event features in the global event feature pool to calculate the spatiotemporal correlation strength between each pair of event features specifically include: Calculate the first distance between the first event feature and the second event feature on a spatial scale. And the second distance between the two on the time scale. ; Based on the first distance and the second distance Calculate the spatiotemporal correlation strength between the first event feature and the second event feature: 。 8. The global tracking and monitoring method based on multi-channel data collaboration according to claim 7, characterized in that, Calculate the first distance between the first event feature and the second event feature on the time scale. And the second distance between the two on a spatial scale. The specific steps include: Obtain the spatial dimensions of the monitored area on a two-dimensional plane projection. ; Determine the first tracking object and the first object event associated with the first event feature, and the second tracking object and the second object event associated with the second event feature; Generate a first movement trajectory of the first tracked object on a two-dimensional plane during the occurrence of the first object event, and a second movement trajectory of the second tracked object on a two-dimensional plane during the occurrence of the second object event; Calculate the minimum spatial distance between the first movement trajectory and the second movement trajectory. ; Calculate the first distance between the first event feature and the second event feature in terms of spatial scale: ； Determine the minimum spatial distance corresponding to the first movement trajectory. The first coordinate point, and the second coordinate point on the second movement trajectory corresponding to the minimum spatial distance; Determine the first moment when the first tracked object passes through the first coordinate point. and the second moment when the second tracked object passes through the second coordinate point. ; Get the duration of the event of the first object. and the duration of the second object event. ; Calculate the first distance between the first event feature and the second event feature in terms of spatial scale: 。 9. The global tracking and monitoring method based on multi-channel data collaboration according to claim 8, characterized in that, The specific steps for implementing global risk prediction based on the correlation analysis results include: Obtain the pre-configured prediction duration of the action trajectory; Obtain the constraint data corresponding to the parent event ID, wherein the constraint data includes one or more of time constraint data, position constraint data, and velocity constraint data; Predict the movement trajectory of the first or second tracked object within a future period of time with a duration equal to the predicted movement trajectory duration; Determine whether the predicted movement trajectory of the first or second tracked object conforms to the constraint conditions of the constraint data.

10. A global tracking and monitoring system based on multi-channel data collaboration, characterized in that, include: Several monitoring devices are used to acquire monitoring images within the monitored area; A monitoring server, which is communicatively connected to the monitoring device, is used to perform global tracking and monitoring of the tracked objects in the monitoring area based on the monitoring screen. The monitoring device includes an image sensing unit for generating monitoring screen data, an image processing unit for processing the monitoring screen data, and a tracking detection unit for tracking and detecting a target object using the image content processed by the image processing unit. The tracking detection unit is configured as follows: Detect the tracked object in the monitoring screen; When the tracked object is detected on the monitoring screen, the position and status of the tracked object are tracked and monitored; Based on changes in the location and / or state of the tracked object, perform object event recognition on the monitoring screen containing the tracked object; Determine whether the object events of the tracked object meet preset conditions; When the object event of the tracked object meets the preset conditions, the event information of the object event is sent to the monitoring server. The event information includes the tracked object associated with the object event, the event type of the object event, and the time, location, and duration of the object event. The monitoring screen data is transmitted to the monitoring server, so that the monitoring server can extract the feature information of the object event from the monitoring screen data; The monitoring server is configured to implement the global tracking and monitoring method based on multi-channel data collaboration as described in any one of claims 1-9.

Images