Event identification method and device based on multiple video streams, equipment and storage medium

Abstract

According to the event identification method and device based on the multiple video streams, the equipment and the storage medium provided by the invention, the video stream data of the multiple acquisition equipment are acquired, and the video frame information is extracted from the video stream data; inputting the video frame information into an MLP event classification model to obtain an event classification tag corresponding to the video frame information; the event classification labels are stored in a label pool, the label pool is used for storing the event classification labels corresponding to the video frame information of the multiple collection devices, and a unique identifier is distributed to each label; and within a preset time range, when the event classification tag in the tag pool is greater than a preset threshold value, generating a task instruction. According to the method, the video frame information is classified by adopting the MLP event classification model, the event classification label can be quickly and efficiently acquired, and the event identification accuracy is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an event recognition method, device, equipment and storage medium based on multiple video streams. Background Art

[0002] With the rapid development of intelligent monitoring technology, the collaborative work of multiple acquisition devices has become an important direction for event recognition systems. Traditional event recognition methods usually rely on single video stream data for analysis, which makes it difficult to effectively integrate multi-source data in complex scenarios, resulting in insufficient accuracy and timeliness of event detection. In addition, for the time synchronization problem of multiple video streams, traditional methods mostly use simple time alignment strategies, which cannot provide accurate synchronization results in the case of data loss or large deviations.

[0003] In summary, the problems existing in the prior art need to be solved urgently. Summary of the invention

[0004] The present invention provides an event recognition method, device, equipment and storage medium based on multiple video streams, which are used to solve the defects in the prior art and improve the timeliness and intelligence of event processing.

[0005] The present invention provides an event recognition method based on multiple video streams, comprising: Acquire video stream data of multiple acquisition devices, and extract video frame information from the video stream data; Inputting the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information; The event classification labels are stored in a label pool, wherein the label pool is used to store event classification labels corresponding to video frame information of multiple acquisition devices, and a unique identifier is assigned to each label; Within a preset time range, when the event classification tags in the tag pool are greater than a preset threshold, a task instruction is generated.

[0006] According to an event recognition method based on multiple video streams provided by the present invention, after the step of inputting the video frame information into the MLP event classification model to obtain the event classification label corresponding to the video frame information, the method further includes: The video stream data of each acquisition device is synchronized according to the timestamp.

[0007] According to an event recognition method based on multiple video streams provided by the present invention, the step of synchronizing the video stream data of each acquisition device according to the timestamp specifically includes: Interpolating and completing the video stream data; The first video stream data after interpolation and completion is used as a reference video stream, and the timestamp in the first video stream data is used as a reference timestamp; In the second video stream, a target timestamp is determined, and a difference between the target timestamp and a reference timestamp is within a preset range.

[0008] According to an event recognition method based on multiple video streams provided by the present invention, the step of storing the event classification label in a label pool specifically includes: Storing the event classification tags in the tag pool in chronological order; The event classification tags in the tag pool are updated regularly to remove event classification tags that exceed the preset event range.

[0009] According to an event recognition method based on multiple video streams provided by the present invention, within a preset time range, when the event classification label in the label pool is greater than a preset threshold, the step of generating a task instruction specifically includes: Count the event classification tags in the tag pool to determine the number of event classification tags within the preset time range When the event classification tags in the tag pool are greater than a preset threshold, a corresponding task instruction is generated according to a preset rule.

[0010] According to an event recognition method based on multiple video streams provided by the present invention, when the event classification label in the label pool is greater than a preset threshold, the step of generating a corresponding task instruction according to a preset rule specifically includes: When the event classification tag in the tag pool is greater than a preset threshold, the target device ID is determined according to the unique identifier of the event classification tag; According to the target device ID and the type of event classification label, a corresponding task instruction is generated.

[0011] According to an event recognition method based on multiple video streams provided by the present invention, the task instructions include: an alarm start instruction and a door and window closing instruction; The alarm start instruction is used to generate alarm information according to the event classification label and send the alarm information to the target monitoring terminal; The door and window closing instruction is used to control the target device to close.

[0012] The present invention also provides an event recognition device based on multiple video streams, comprising: A data acquisition module, used to acquire video stream data of multiple acquisition devices and extract video frame information from the video stream data; An event classification module, used to input the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information; A label storage module, used to store the event classification label in a label pool, wherein the label pool is used to store event classification labels corresponding to video frame information of multiple acquisition devices, and assign a unique identifier to each label; The task instruction module is used to generate a task instruction when the event classification tag in the tag pool is greater than a preset threshold within a preset time range.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the event recognition method based on multiple video streams as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the event recognition method based on multiple video streams as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned event recognition methods based on multiple video streams.

[0016] The event recognition method, device, equipment and storage medium based on multiple video streams provided by the present invention obtain video stream data of multiple acquisition devices, and extract video frame information from the video stream data; input the video frame information into the MLP event classification model to obtain the event classification label corresponding to the video frame information; store the event classification label in a label pool, wherein the label pool is used to store event classification labels corresponding to the video frame information of multiple acquisition devices, and assign a unique identifier to each label; within a preset time range, when the event classification label in the label pool is greater than a preset threshold, a task instruction is generated. The present invention classifies video frame information by adopting an MLP (multi-layer perceptron) event classification model, can quickly and efficiently obtain event classification labels, significantly improve the accuracy of event recognition, and within a preset time range, by counting the number of event classification labels and generating task instructions based on preset rules, can realize automated task response, and improve the timeliness and intelligence of event processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of an event recognition method based on multiple video streams provided by the present invention; Figure 2 It is a structural schematic diagram of an event recognition device based on multiple video streams provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] In order to solve the problems in the prior art, the present invention proposes an event recognition method based on multiple video streams to improve the timeliness and intelligence of event processing. The event recognition method based on multiple video streams is described below. Figure 1 As shown, including but not limited to the following steps: Step 110: Acquire video stream data from multiple acquisition devices, and extract video frame information from the video stream data.

[0021] In this embodiment, it is first necessary to obtain data from multiple video acquisition devices, which may be surveillance cameras, sensors, or other video acquisition devices. The video stream collected by each device contains continuous frame information (image frames), which contain visual information when an event occurs. The process of obtaining video stream data can be transmitted through a network or local data transmission method, and the video stream is processed in chronological order.

[0022] The process of extracting video frame information includes intercepting image frames at fixed time intervals from the video stream and preprocessing them. The preprocessing steps can include operations such as denoising, resolution adjustment, and color space conversion so that the subsequent event recognition and classification model can better process and analyze them. Video frame information can be stored in the form of image matrices, each of which represents a video frame.

[0023] Step 120: Input the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information.

[0024] Next, the video frame information extracted from multiple video streams is input into a multi-layer perceptron (MLP) event classification model. MLP is a feed-forward neural network consisting of an input layer, a hidden layer, and an output layer. In this embodiment, the video frame information is used as input data, and after training the MLP model, the corresponding event classification label can be generated. The output of the model is a multi-category label, indicating the event type corresponding to each video frame, such as "intrusion", "fire", "fall", etc.

[0025] During the training process, the MLP model uses a large amount of labeled video data for supervised learning, and optimizes the model parameters through the back propagation algorithm, so that the model can efficiently and accurately classify events in actual scenarios. After training, the model can analyze any input video frame in real time and generate event classification labels.

[0026] Step 130: Store the event classification label in a label pool, wherein the label pool is used to store event classification labels corresponding to video frame information of multiple acquisition devices, and assign a unique identifier to each label.

[0027] After the event classification tags are generated, they will be stored in the tag pool. The tag pool is a data structure used to store all generated event classification tags. To ensure the uniqueness of each tag, each tag will be assigned a unique identifier (ID) to distinguish them in subsequent processing.

[0028] The tag pool can be stored in a time series-based manner, storing all tags in the order in which the events occurred. Each time a new tag is generated, it is stored in the tag pool and the current timestamp or other relevant metadata is attached. These tags are matched with their corresponding video stream data, so that the specific time of the event and the source of the video stream can be traced back.

[0029] In addition, the tag pool can be updated regularly as needed, tags in the tag pool can be cleaned up or expired, and tags that exceed the preset event range can be removed, thereby maintaining the efficiency and simplicity of the tag pool.

[0030] It is understandable that in this embodiment, the system can directly obtain AI event data from the video stream or related sensor devices, without necessarily performing complex analysis on each frame of video data. AI events may be generated by a pre-trained artificial intelligence model, which can directly identify a specific type of event (such as intrusion, fire, etc.) from the video stream based on historical data or event patterns.

[0031] In addition, AI events can also be collected and generated by other smart devices such as sensors (temperature sensors, motion sensors, smoke detectors, etc.). These events are collected and transmitted through conventional technical means (such as data interfaces or APIs), without the need to analyze video streams frame by frame, simplifying the processing process.

[0032] Step 140: Within a preset time range, when the event classification tags in the tag pool are greater than a preset threshold, a task instruction is generated.

[0033] In the tag pool, the stored event classification tags are counted and analyzed according to the preset time range. When the number of event classification tags in the tag pool exceeds the preset threshold, the system will automatically generate a task instruction. The task instruction is a control instruction generated for a specific event type, which can trigger specific response measures.

[0034] For example, in some security monitoring systems, when the number of tags for "intrusion" events in the tag pool reaches a certain threshold within a predetermined time, the system will generate an alarm start instruction based on the event, start the alarm device and notify the monitoring personnel. Or, when the number of tags for "fire" events in the tag pool exceeds the threshold, the system can generate a fire system start instruction to automatically start the fire extinguishing system.

[0035] The specific rules for generating task instructions can be flexibly configured according to the application scenario. For example, task instructions can include alarm instructions, equipment control instructions (such as closing doors and windows), automatic inspection instructions, etc. The system automatically generates and executes corresponding task instructions based on the event type, the time when the event occurred, and the number of tags in the tag pool, greatly improving the efficiency and intelligence level of event response.

[0036] As a further optional embodiment, after the step of inputting the video frame information into the MLP event classification model to obtain the event classification label corresponding to the video frame information, the method further includes: The video stream data of each acquisition device is synchronized according to the timestamp.

[0037] In the video streams of multiple acquisition devices, although each device collects data according to its own acquisition frequency and clock, due to factors such as network delay and device performance differences, the data timestamps of each video stream may have certain deviations. Therefore, the video stream data of multiple acquisition devices needs to be time synchronized in the step to ensure that events occurring at the same time can be accurately recorded and captured by all related devices.

[0038] As a further optional embodiment, the step of synchronizing the video stream data of each acquisition device according to the timestamp specifically includes: Interpolating and completing the video stream data; The first video stream data after interpolation and completion is used as a reference video stream, and the timestamp in the first video stream data is used as a reference timestamp; In the second video stream, a target timestamp is determined, and a difference between the target timestamp and a reference timestamp is within a preset range.

[0039] In this embodiment, first, interpolation and completion are performed on the video stream data of each acquisition device. Since the acquisition frequencies and timestamps of different acquisition devices may be inconsistent, interpolation and completion are used to fill in missing timestamps or smooth data at irregular time intervals. Common interpolation methods include linear interpolation, spline interpolation, or Lagrange interpolation. Through this interpolation and completion, a relatively uniform time series can be generated so that the data of each video stream can be presented in the same time interval.

[0040] During the synchronization of multiple acquisition devices, select one video stream as the reference video stream. Usually, the video stream with the most frames and the most stable timestamp is selected as the reference video stream. The timestamp in the reference video stream is used as a reference to ensure that the timestamps of other video streams can be aligned with it. The reference timestamp will be used as a reference point to calibrate the timestamps of other video streams.

[0041] During synchronization, for each frame of data in the second video stream, the target timestamp of the frame data needs to be determined. The difference between the target timestamp and the timestamp of the reference video stream needs to be within a preset time window, such as ±10ms. If the difference is within the preset range, the frame data is considered to be aligned in time and can be processed later. If the difference between the target timestamp and the reference timestamp exceeds the preset range, further interpolation or calibration operations may be required to accurately align the timestamps between the two video streams.

[0042] Specifically, it is assumed that the timestamps of the first video stream and the second video stream are {T1_1, T1_2, ..., T1_n} and {T2_1, T2_2, ..., T2_m}, respectively, where n and m are the frame numbers of the two video streams. Since the timestamps are not completely aligned, after interpolation and completion, the completed timestamp sequence can be expressed as {T1'_1, T1'_2, ..., T1'_n} and {T2'_1, T2'_2, ..., T2'_m}, so that the timestamps of the two video streams are more closely aligned at the same time point.

[0043] Select the first video stream (assuming that the first video stream after interpolation and completion is T1'_1, T1'_2, ..., T1'_n) as the reference video stream, and use its timestamp sequence T1' as a reference for time synchronization. The timestamp T1'_i of the reference video stream is used as the reference timestamp.

[0044] For each frame in the second video stream, its timestamp is T2'_j, which needs to be compared with the reference timestamp T1'_i. If |T2'_j - T1'_i| ≤ ΔT, where ΔT is a preset time window (e.g., 10ms), then T2'_j is considered to be aligned with T1'_i and subsequent processing can continue. Otherwise, T2'_j may need to be interpolated or corrected to align it with T1'_i.

[0045] As a further optional embodiment, the step of storing the event classification tag in the tag pool specifically includes: Storing the event classification tags in the tag pool in chronological order; The event classification tags in the tag pool are updated regularly to remove event classification tags that exceed the preset event range.

[0046] In this embodiment, after all event classification tags from multiple acquisition devices are generated, they are stored in the tag pool in sequence according to the generation time order. By storing in chronological order, it can be ensured that event tags can be queried and processed based on time. In a specific implementation, timestamps can be used as the sorting basis in the tag pool to ensure that each event classification tag is closely associated with its corresponding time point, so that in the subsequent task instruction generation and event processing, effective screening and judgment can be performed according to the time series.

[0047] For example, whenever a new event classification tag is generated, the tag pool will insert or append the tag to the tag pool according to the generation time and record the timestamp corresponding to the tag. In this way, the event classification tags in the tag pool will always be arranged in chronological order, which is conducive to statistical analysis within the subsequent time range.

[0048] In order to avoid accumulating too many outdated tags in the tag pool and affecting the operating efficiency of the system, it is necessary to regularly clean up the expired data in the tag pool. This step includes checking the time range of the event classification tags in the tag pool and removing those tags that are beyond the preset event range.

[0049] For example, a label pool can be set with a time window (such as the last 10 minutes, 30 minutes, or 1 hour), and any event classification label that exceeds this time window will be considered expired. By regularly updating the label pool and removing expired labels, redundant data in the label pool can be effectively reduced, ensuring that the label pool always retains the latest and relevant event classification labels, ensuring that the data used in the subsequent generation of task instructions is more accurate and timely.

[0050] This update mechanism can be implemented using a timer or based on event triggering. For example, at regular intervals (such as every hour or every work cycle), the system automatically checks the event classification tags in the tag pool and deletes tags that are out of time to keep the tag pool streamlined and efficient.

[0051] Assume that each event classification tag contains the event type, the timestamp of the event, and other related information. When a new event classification tag is generated, it will contain timestamp information, such as "2024-12-24 10:15:00". The newly generated event classification tags will be inserted into the tag pool in the order of timestamps. If the tag pool is empty, the event classification tag will be directly stored; if there are other tags in the tag pool, they will be inserted into the correct position to ensure that the event classification tags in the tag pool are arranged in chronological order.

[0052] For example, in a tag pool, event classification tags can be stored in the following format: Tag 1: [Event type A, Timestamp: 2024-12-24 10:10:00] Tag 2: [Event type B, Timestamp: 2024-12-24 10:15:00] Tag 3: [Event type A, Timestamp: 2024-12-24 10:20:00] In this way, tags in the tag pool can always be organized and queried according to time series.

[0053] The system will scan the tag pool regularly (such as every 5 minutes or every 30 minutes) to check the timestamp of each event classification tag. If the timestamp of a tag exceeds the preset event range (for example, exceeds the current time window), the tag will be deleted from the tag pool.

[0054] Assume that the current time is "2024-12-24 10:30:00", and the tags in the tag pool include: Tag 1: [Event type A, Timestamp: 2024-12-24 10:10:00] Tag 2: [Event type B, Timestamp: 2024-12-24 10:15:00] Tag 3: [Event type A, Timestamp: 2024-12-24 10:20:00] If the preset event range is 30 minutes, then at the current time "2024-12-24 10:30:00", tag 1 will be removed because its timestamp "2024-12-24 10:10:00" exceeds the preset time window. Tags 2 and 3 will be retained because their timestamps are still within the valid range.

[0055] In addition, the frequency of regular updates can be adjusted in a timely manner according to the size of the tag pool and the system load to ensure that the system resources are effectively utilized.

[0056] As a further optional embodiment, within the preset time range, when the event classification tag in the tag pool is greater than a preset threshold, the step of generating a task instruction specifically includes: Count the event classification tags in the tag pool to determine the number of event classification tags within the preset time range When the event classification tags in the tag pool are greater than a preset threshold, a corresponding task instruction is generated according to a preset rule.

[0057] Within the preset time range, the system will count the number of event classification tags in the tag pool. The counting process traverses all event classification tags in the tag pool and calculates the number of event classification tags that appear in the specified time window.

[0058] For example, the system will determine whether each event classification tag is within a preset time range based on the timestamp of the tag. The time range can be seconds, minutes, hours, etc., depending on the needs of the application scenario. If the timestamp of the event classification tag is within the interval between the current time and the set time range, the tag is considered valid and will be counted.

[0059] Assuming that the preset time range is 10 minutes, if the current time is "2024-12-24 10:30:00", the system checks the number of event classification labels in the past 10 minutes (that is, from "2024-12-24 10:20:00" to "2024-12-24 10:30:00").

[0060] After the statistics are completed, the system will compare the number of event classification tags obtained with the preset threshold. The preset threshold can be flexibly set according to different application scenarios. For example, if there are more than 3 event classification tags within 10 minutes, it is considered an important event and a task instruction is generated.

[0061] If the number of event classification tags in the tag pool is greater than the preset threshold, the process of generating task instructions is triggered. The selection of the preset threshold is usually optimized based on the design requirements of the system or the complexity of the actual scenario. Smaller thresholds are suitable for scenarios that require a higher response frequency, while larger thresholds are suitable for environments where fewer events occur.

[0062] When the number of event classification tags in the tag pool exceeds the preset threshold, the system will generate corresponding task instructions according to the preset rules. The content of the task instruction is determined by the type of event and the content of the classification tag, usually including but not limited to alarm, triggering device actions (such as door and window opening, device opening, video surveillance switching, etc.).

[0063] For example, suppose the preset rule is that when multiple "fire" events occur in the tag pool (tag type is fire), an "alarm start instruction" is generated and notified to the monitoring center. If an "abnormal intrusion" event occurs, a "door and window closing instruction" is generated to lock the scene.

[0064] The generated task instructions will be sent to the relevant equipment or control system to perform specific operations to ensure that the event is responded to in a timely manner. This rule can be flexibly configured according to different event types, equipment types and task priorities.

[0065] As a further optional embodiment, when the event classification tag in the tag pool is greater than a preset threshold, the step of generating a corresponding task instruction according to a preset rule specifically includes: When the event classification tag in the tag pool is greater than a preset threshold, the target device ID is determined according to the unique identifier of the event classification tag; According to the target device ID and the type of event classification label, a corresponding task instruction is generated.

[0066] In this embodiment, when the number of event classification tags in the tag pool is greater than a preset threshold, the system determines the target device that needs to be processed based on the unique identifier of the event classification tag (e.g., tag ID). Each event classification tag is usually associated with a specific device or device group, which can be a camera, sensor, alarm device, door and window control system, etc.

[0067] In actual applications, event classification tags not only contain event type information, but also device information related to the event. For example, a device ID or device category information may be embedded in the tag, which can quickly determine which device has an abnormality or needs a response task.

[0068] For example, when a "fire" event is detected, the system may look up the fire alarm device associated with the tag based on the unique identifier in the tag to determine the target device ID.

[0069] Once the target device ID is determined, the system will generate the corresponding task instructions based on the type of event classification label (such as fire, intrusion, gas leak, etc.). Task instructions will be automatically generated based on the type of device and the characteristics of the event to ensure that the device can take appropriate response measures.

[0070] For example, when the event classification label indicates the "fire" type and the target device ID points to a fire alarm device, the system can generate an "alarm start instruction" to trigger the target device to issue a fire alarm; if the event classification label is the "intrusion" type and the target device ID points to the access control system, the system can generate a "door and window closing instruction" to immediately close the doors and windows and lock the scene.

[0071] The generated task instructions not only depend on the type of event, but also adjust according to the capabilities of the equipment and the urgency of the task. For example, some equipment may automatically execute according to the instructions, while others may require manual confirmation or further operation.

[0072] Specifically, an example of associating event classification tags with devices: Assume that the event classification tags in the tag pool are as follows: Tag 1: [Event Type: Fire, Tag ID: T1, Device ID: D1, Timestamp: 2024-12-24 10:15:00] Tag 2: [Event Type: Intrusion, Tag ID: T2, Device ID: D2, Timestamp: 2024-12-24 10:20:00] Tag 3: [Event Type: Fire, Tag ID: T3, Device ID: D1, Timestamp: 2024-12-24 10:25:00] When the system detects that the number of event classification tags in the tag pool is greater than the preset threshold (such as greater than 2), it first confirms the target device based on the tag ID (T1, T3). Assuming that tags T1 and T3 point to device D1 (for example, a fire alarm device), the target device ID is determined to be D1.

[0073] Based on the event type (fire) and target device ID (D1) indicated in the tag, the system generates a task instruction according to the preset rules. If the event is "fire", the system generates an "alarm start instruction" and sends it to device D1, triggering a fire alarm.

[0074] If the event type indicated by the tag is "intrusion", the task instruction will generate a "door and window closing instruction" based on the device ID (such as D2, pointing to the access control system) and automatically close the doors and windows to ensure safety.

[0075] The generated task instructions will be sent to the corresponding target devices through the network, control system, etc. to perform corresponding operations. For example, the alarm device will send out an alarm signal, the access control device will lock the doors and windows, and may send a warning message to the security monitoring center or relevant personnel.

[0076] As a further optional embodiment, the task instructions include: an alarm activation instruction and a door and window closing instruction; The alarm start instruction is used to generate alarm information according to the event classification label and send the alarm information to the target monitoring terminal; The door and window closing instruction is used to control the target device to close.

[0077] When the event classification label indicates that an emergency event (such as fire, intrusion, etc.) has occurred, the system will generate an alarm message based on the type of event, the time of occurrence, the severity of the event, etc. The alarm message usually includes the type of event, the location of the event, the time of occurrence, the event description, etc. The generated alarm message can be in the form of text, image, or voice.

[0078] The generated alarm information will be sent to the target monitoring terminal, which can be the computer system of the monitoring center, smartphone application, or other devices that can receive alarm information. After the monitoring terminal receives the alarm information, the staff can judge the urgency of the event according to the alarm content and respond.

[0079] The door and window closing command is used to send a closing command to the target device through the automation control system under specific circumstances. The target device can be an access control system, a window control system, or any device that can be closed through automation control. When the event classification tag identifies an event that requires an immediate response (such as an intrusion, fire, etc.), the system will automatically generate and send a door and window closing command to enhance on-site security.

[0080] The event recognition device based on multiple video streams provided by the present invention is described below. Figure 2 As shown, the event recognition device based on multiple video streams described below and the event recognition method based on multiple video streams described above can refer to each other.

[0081] An event recognition device based on multiple video streams, comprising: The data acquisition module 210 is used to acquire video stream data of multiple acquisition devices and extract video frame information from the video stream data; An event classification module 220, configured to input the video frame information into an MLP event classification model to obtain an event classification label corresponding to the video frame information; The tag storage module 230 is used to store the event classification tags in a tag pool, wherein the tag pool is used to store event classification tags corresponding to video frame information of multiple acquisition devices, and assign a unique identifier to each tag; The task instruction module 240 is used to generate a task instruction when the event classification tag in the tag pool is greater than a preset threshold within a preset time range.

[0082] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communications interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the event recognition method based on multiple video streams, and the method includes: Acquire video stream data of multiple acquisition devices, and extract video frame information from the video stream data; Inputting the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information; The event classification labels are stored in a label pool, wherein the label pool is used to store event classification labels corresponding to video frame information of multiple acquisition devices, and a unique identifier is assigned to each label; Within a preset time range, when the event classification tags in the tag pool are greater than a preset threshold, a task instruction is generated.

[0083] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0084] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the event recognition method based on multiple video streams provided by the above methods, the method comprising: Acquire video stream data of multiple acquisition devices, and extract video frame information from the video stream data; Inputting the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information; The event classification labels are stored in a label pool, wherein the label pool is used to store event classification labels corresponding to video frame information of multiple acquisition devices, and a unique identifier is assigned to each label; Within a preset time range, when the event classification tags in the tag pool are greater than a preset threshold, a task instruction is generated.

[0085] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the event recognition method based on multiple video streams provided by the above methods, the method comprising: Acquire video stream data of multiple acquisition devices, and extract video frame information from the video stream data; Inputting the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information; The event classification labels are stored in a label pool, wherein the label pool is used to store event classification labels corresponding to video frame information of multiple acquisition devices, and a unique identifier is assigned to each label; Within a preset time range, when the event classification tags in the tag pool are greater than a preset threshold, a task instruction is generated.

[0086] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An event recognition method based on multiple video streams, characterized in that: include: Acquire video stream data of multiple acquisition devices, and extract video frame information from the video stream data; Inputting the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information; The event classification labels are stored in a label pool, wherein the label pool is used to store event classification labels corresponding to video frame information of multiple acquisition devices, and a unique identifier is assigned to each label; Within a preset time range, when the event classification tags in the tag pool are greater than a preset threshold, a task instruction is generated.

2. The event recognition method based on multiple video streams according to claim 1, characterized in that: After the step of inputting the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information, the method further includes: The video stream data of each acquisition device is synchronized according to the timestamp.

3. The event recognition method based on multiple video streams according to claim 2 is characterized in that: The step of synchronizing the video stream data of each acquisition device according to the timestamp specifically includes: Interpolating and completing the video stream data; The first video stream data after interpolation and completion is used as a reference video stream, and the timestamp in the first video stream data is used as a reference timestamp; In the second video stream, a target timestamp is determined, and a difference between the target timestamp and a reference timestamp is within a preset range.

4. The event recognition method based on multiple video streams according to claim 1, characterized in that: The step of storing the event classification label in the label pool specifically includes: Storing the event classification tags in the tag pool in chronological order; The event classification tags in the tag pool are updated regularly to remove event classification tags that exceed the preset event range.

5. The event recognition method based on multiple video streams according to claim 1, characterized in that: The step of generating a task instruction when the event classification tag in the tag pool is greater than a preset threshold within the preset time range specifically includes: Count the event classification tags in the tag pool to determine the number of event classification tags within a preset time range; When the event classification tags in the tag pool are greater than a preset threshold, a corresponding task instruction is generated according to a preset rule.

6. The event recognition method based on multiple video streams according to claim 5, characterized in that: When the event classification tag in the tag pool is greater than a preset threshold, the step of generating a corresponding task instruction according to a preset rule specifically includes: When the event classification tag in the tag pool is greater than a preset threshold, the target device ID is determined according to the unique identifier of the event classification tag; According to the target device ID and the type of event classification label, a corresponding task instruction is generated.

7. The event recognition method based on multiple video streams according to claim 1, characterized in that: Task instructions include: alarm start instructions and door and window closing instructions; The alarm start instruction is used to generate alarm information according to the event classification label and send the alarm information to the target monitoring terminal; The door and window closing instruction is used to control the target device to close.

8. An event recognition device based on multiple video streams, characterized in that: include: A data acquisition module, used to acquire video stream data of multiple acquisition devices and extract video frame information from the video stream data; An event classification module, used to input the video frame information into the MLP event classification model to obtain an event classification label corresponding to the video frame information; A label storage module, used to store the event classification label in a label pool, wherein the label pool is used to store event classification labels corresponding to video frame information of multiple acquisition devices, and assign a unique identifier to each label; The task instruction module is used to generate a task instruction when the event classification tag in the tag pool is greater than a preset threshold within a preset time range.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the event recognition method based on multiple video streams as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the event recognition method based on multiple video streams as described in any one of claims 1 to 7 is implemented.

Images