Anomaly detection method and apparatus based on multi-dimensional information

Abstract

The application discloses an abnormality detection method and device based on multi-dimensional information, and relates to the field of safety monitoring. A specific embodiment of the method comprises: marking video streams captured by video monitoring devices according to information such as the positions of the video monitoring devices, the monitoring objects thereof and the types of the regions, and searching, clustering and outputting the video streams with corresponding marks according to keywords input by a user. Furthermore, the video streams output by the video monitoring devices are bound to alarm signals output by alarm devices according to the positions of the video monitoring devices and the alarm devices. The embodiment reduces the technical problems of low accuracy of monitoring of abnormal events, poor adaptability and timeliness of emergency measures in the prior art, and can achieve the technical effect of timely and accurate monitoring of abnormal events.

Description

Technical Field

[0001] This invention relates to the field of security monitoring, and in particular to an anomaly detection method and apparatus based on multidimensional information. Background Technology

[0002] Currently, video surveillance systems are widely used in scenarios requiring security monitoring such as fire prevention and theft prevention, including operations in logistics, warehousing, petrochemicals, and mining. Typically, a video surveillance system comprises multiple video surveillance devices located in different positions, with different shooting directions and / or different device parameters. Traditionally, the video feeds from these devices are manually polled, and the content is judged, classified, and anomaly identified.

[0003] In addition to video surveillance systems, other types of alarm devices, such as smoke detectors, temperature alarms, and air sampling alarms, are typically installed in the aforementioned work scenarios for safety monitoring. Similar to video surveillance systems, these alarm devices are located in different locations and usually have different parameters. Traditionally, these alarm devices detect anomalies and issue alarm signals independently of the video surveillance system.

[0004] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art: As the industry develops and demand increases, the number of video surveillance devices deployed simultaneously often reaches the millions. Traditional manual methods of polling millions of video feeds become inefficient, unable to cover the entire monitored area in a timely manner, and difficult to accurately retrieve the required video streams. Video surveillance scenarios are also susceptible to factors such as obstructions, lighting conditions, and poor shooting angles, leading to monitoring failures.

[0005] Meanwhile, because other types of alarm devices operate independently of the video surveillance system, it is difficult to cross-verify between different systems in a timely manner, resulting in a high false alarm rate. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide an anomaly detection method and apparatus based on multidimensional information, which can solve the technical problems of low accuracy and poor adaptability and timeliness of emergency measures in the prior art for monitoring abnormal events such as fires.

[0007] To achieve the above objectives, according to one aspect of the present invention, an anomaly detection method based on multidimensional information is provided, comprising: an information acquisition step, acquiring video streams and information related to the video surveillance devices from each of a plurality of video surveillance devices, and acquiring alarm signals and information related to the alarm devices from each of a plurality of alarm devices; a multidimensional information generation step, associating the acquired information related to the video surveillance devices with the video streams acquired by the corresponding video surveillance devices, and associating the acquired alarm signals with at least one video stream based on the information related to the video surveillance devices and the information related to the alarm devices, thereby forming multidimensional information; a video stream clustering step, retrieving information matching keywords input from an external source from the information related to the video surveillance devices included in the multidimensional information, clustering the video streams associated with the retrieved information in the multidimensional information, and outputting the clustered video streams; a video stream anomaly judgment step, polling the clustered video streams to detect anomalies; a multidimensional information anomaly judgment step, detecting anomalies based on the alarm signals and associated video streams included in the multidimensional information; and a warning information sending step, generating and sending warning information based on the multidimensional information when an anomaly is detected.

[0008] According to one aspect of the present invention, the information related to the video surveillance device includes: identification information of the video surveillance device, location information of the video surveillance device, performance parameter information of the video surveillance device, and service tag information related to the monitored object of the video surveillance device; and the information related to the alarm device includes: identification information of the alarm device, location information of the alarm device, performance parameter information of the alarm device, and service tag information related to the monitored object of the alarm device.

[0009] According to one aspect of the present invention, in the multi-dimensional information generation step, based on the location information of the video surveillance device and the location information of the alarm device, a video surveillance device near the alarm device is identified, and the alarm signal of the alarm device is associated with the video stream captured by the identified video surveillance device.

[0010] According to one aspect of the present invention, the video stream clustering step further includes determining the video quality of the clustered video streams; and, in response to the video quality of the clustered video streams being unqualified, adjusting the corresponding video surveillance equipment to improve the video quality based on the information related to the video surveillance equipment included in the multidimensional information.

[0011] According to one aspect of the present invention, the multi-dimensional information anomaly judgment step further includes setting an anomaly detection threshold based on the video stream, information related to the video surveillance equipment, alarm signals, and information related to the alarm equipment included in the multi-dimensional information.

[0012] According to one aspect of the present invention, the anomaly detection method further includes: an artificial intelligence model training step, which trains an artificial intelligence model based on the video stream and information related to the video surveillance equipment acquired in the information acquisition step, using a standard library of pre-stored standard video frames; and uses the trained artificial intelligence model to perform the video stream clustering step, the video stream anomaly judgment step, the multi-dimensional information anomaly judgment step, and the warning information sending step.

[0013] According to one aspect of the present invention, an anomaly detection device based on multidimensional information is also provided, comprising: an information acquisition module, which acquires video streams and information related to the video surveillance devices from each of a plurality of video surveillance devices, and acquires alarm signals and information related to the alarm devices from each of a plurality of alarm devices; a multidimensional information generation module, which associates the acquired information related to the video surveillance devices with the video streams acquired by the corresponding video surveillance devices, and associates the acquired alarm signals with at least one video stream based on the information related to the video surveillance devices and the information related to the alarm devices, thereby forming multidimensional information; a video stream clustering module, which retrieves information matching keywords input from an external source from the information related to the video surveillance devices included in the multidimensional information, clusters the video streams associated with the retrieved information in the multidimensional information, and outputs the clustered video streams; a video stream anomaly judgment module, which polls the clustered video streams to detect anomalies; a multidimensional information anomaly judgment module, which detects anomalies based on the alarm signals and associated video streams included in the multidimensional information; and a warning information sending module, which generates and sends warning information based on the multidimensional information when an anomaly is detected.

[0014] According to one aspect of the present invention, a server for anomaly detection based on multidimensional information is also provided, characterized in that it includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the anomaly detection method as described above.

[0015] According to one aspect of the present invention, a computer-readable medium is also provided, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the anomaly detection method as described above.

[0016] According to one aspect of the present invention, a computer program product is also provided, comprising a computer program, characterized in that, when the computer program is executed by a processor, it implements the anomaly detection method as described above.

[0017] One embodiment of the above invention has the following advantages or beneficial effects: by marking the video stream captured by the video surveillance equipment according to information such as the location of the video surveillance equipment, its monitored object and area type, and by retrieving, clustering and outputting the video stream with corresponding markings according to the keywords input by the user, it can overcome the technical problems of low accuracy and poor timeliness in monitoring abnormal events such as fires in the prior art, and can achieve the technical effect of timely and accurate monitoring of abnormal events in specific types of business functions.

[0018] Furthermore, by binding the video stream output by the video surveillance equipment with the alarm signal output by the alarm equipment according to their locations, anomalies can be identified collaboratively. This overcomes the technical problems of low accuracy in anomaly identification and poor adaptability and timeliness of emergency measures in existing technologies, and achieves the technical effect of improving the timeliness and accuracy of anomaly event identification, anomaly level confirmation, and handling strategy determination.

[0019] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0020] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a schematic diagram of the main flow of the anomaly detection method based on multidimensional information according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the main flow of the artificial intelligence model training steps in the anomaly detection method based on multidimensional information according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the main process of the video stream clustering step in the anomaly detection method based on multidimensional information according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the main modules of an anomaly detection device based on multidimensional information according to an embodiment of the present invention; Figure 5 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied; Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0021] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] It should be noted that the collection, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security. For example, after collecting video information, we will use technical means to de-identify the data. Furthermore, when displaying video information, we will use methods including de-identification or anonymization to de-sensitize your information to protect your information security.

[0023] Figure 1 This is a schematic diagram of the main flow of an anomaly detection method based on multidimensional information according to an embodiment of the present invention. Figure 1 As shown, the anomaly detection method based on multidimensional information according to an embodiment of the present invention includes: information acquisition step S101, multidimensional information generation step S102, artificial intelligence model training step S103, video stream clustering step S104, video stream anomaly judgment step S105, multidimensional information anomaly judgment step S106, and warning information sending step S107.

[0024] Information acquisition step S101 is a step of acquiring video streams and information related to video surveillance devices from each of multiple video surveillance devices, and acquiring alarm signals and information related to alarm devices from each of multiple alarm devices.

[0025] According to embodiments of the present invention, video surveillance equipment can be various hardware and software systems used to capture, record, and transmit video images to achieve security monitoring, remote observation, and event recording. Video surveillance equipment according to embodiments of the present invention includes, but is not limited to, analog cameras, PTZ (Pan-Tilt-Zoom) cameras, infrared cameras, etc. Video surveillance equipment according to embodiments of the present invention can transmit its own information and the video streams it acquires to a higher-level unit, such as a server for processing this information, via wired or wireless transmission protocols.

[0026] The information acquired during the information collection step related to the video surveillance equipment includes: identification information for recognizing the video surveillance equipment, location information indicating the installation location of the video surveillance equipment, performance parameter information of the video surveillance equipment, and service tagging information related to the monitored objects of the video surveillance equipment. The performance parameter information of the video surveillance equipment may include information indicating the resolution, viewing angle, rotation capability, zoom capability, and night vision capability of the video surveillance equipment. The service tagging information related to the monitored objects of the video surveillance equipment is used to identify the services that are monitored by the video surveillance equipment.

[0027] For example, when this invention is applied to a logistics scenario, the business marking information related to the monitored object of the video surveillance equipment could be "flammable material," indicating that the object being filmed by the video surveillance equipment is a stored flammable item. As another example, the business marking information could be "picking area," indicating that the location of the monitored object of the video surveillance equipment is an area used for sorting goods.

[0028] The application scenarios of this invention are not limited to logistics scenarios, but can also include high-risk operation scenarios such as petroleum, chemical, and mining, as well as scenarios that require real-time safety monitoring, such as public places.

[0029] According to embodiments of the present invention, alarm devices can be various devices and systems used to detect abnormal signs such as fire, explosion, and leakage and to issue alarms so that corresponding measures can be taken in a timely manner. Alarm devices according to embodiments of the present invention may include, but are not limited to, smoke alarm devices, temperature alarm devices, and air sampling alarm devices. Alarm devices according to embodiments of the present invention can transmit their own information and the alarm signals they generate to a higher-level unit, such as a server for processing this information, via wired or wireless transmission protocols.

[0030] The alarm signal output by the alarm device can be an abnormal alarm signal output when the alarm device detects an abnormal event. The alarm device can also transmit electrical signal parameters (such as voltage value, current value) or parameters indicated or derived by the electrical signal parameters (such as temperature value, concentration value) as alarm signals to an upper-level unit, such as a server for processing these detection signals, so that the server can process the detection signals to identify abnormal events.

[0031] The information acquired during the information collection step related to the alarm device includes: identification information for recognizing the alarm device, location information indicating the installation location of the alarm device, performance parameter information of the alarm device, and service tagging information related to the monitored object of the alarm device. The performance parameter information of the alarm device may include information indicating the alarm device type, detection distance, adjustable parameters, etc. The service tagging information related to the monitored object of the alarm device is used to identify the service being monitored by the alarm device, similar to the service tagging information related to the monitored object of video surveillance equipment.

[0032] The multi-dimensional information generation step S102 is a step of binding the information obtained from the alarm device with the information obtained from the video surveillance device to generate multi-dimensional information.

[0033] According to an embodiment of the present invention, information related to the video surveillance device is associated with the video stream acquired by the corresponding video surveillance device, and based on the information related to the video surveillance device and the information related to the alarm device, the acquired alarm signal is associated with at least one video stream, thereby forming multi-dimensional information.

[0034] According to an embodiment of the present invention, the video stream acquired by the video surveillance equipment, the identification information, location information, performance parameter information of the video surveillance equipment, and service tag information related to the monitored object of the video surveillance equipment are bound together. Furthermore, based on the acquired location information indicating the installation location of the video surveillance equipment and the location information indicating the installation location of the alarm equipment, the alarm equipment and nearby video surveillance equipment are identified. Then, the alarm signal generated by the alarm equipment, the identification information, location information, performance parameter information of the alarm equipment, and service tag information related to the monitored object of the alarm equipment are bound together with the video stream captured by at least one nearby video surveillance equipment. The bound information is stored as multi-dimensional information, for example, in the form of a mapping table.

[0035] According to an embodiment of the present invention, alarm devices and nearby video surveillance devices can be identified in the following way: the as-built CAD drawings indicating the monitoring points where video surveillance devices and alarm devices are installed are imported into the system, the relative installation positions of the devices are automatically identified by the legends representing the monitoring points on the drawings, and then manually corrected and calibrated to see if they match the actual situation.

[0036] According to embodiments of the present invention, by forming multi-dimensional information, for example, the identification information, location information, and service tag information of video surveillance equipment can be used as tags for corresponding video streams. Therefore, the corresponding video stream can be retrieved and obtained using the identification information, location information, and service tag information of the video surveillance equipment as an index. For example, upon receiving a warning message from an alarm device, the video stream obtained by a video surveillance device near the alarm device can be automatically retrieved based on the multi-dimensional information.

[0037] The artificial intelligence model training step S103 is a step that uses information obtained from the video surveillance equipment in the information acquisition step to train the artificial intelligence model. The following will refer to... Figure 2 Describe in detail the steps involved in training an artificial intelligence model.

[0038] Figure 2 This is a schematic diagram of the main process of the artificial intelligence model training step S103 in the anomaly detection method based on multidimensional information according to an embodiment of the present invention.

[0039] like Figure 2 As shown, in step S201, the acquired information of the video surveillance device is compared with the information stored in the standard library. According to an embodiment of the present invention, the standard library stores standard video frames according to functional area type and device type. In step S201, based on the identification information, location information, performance parameter information, and service tag information included in the acquired video surveillance device information, the corresponding standard video frame is selected from the standard library, and the video frame captured by the corresponding video surveillance device is compared with the standard video frame to check whether the video frame captured by the video surveillance device matches the standard video frame stored in the standard library. According to an embodiment of the present invention, the operations of identifying relevant information in the acquired video surveillance device information, selecting standard video frames from the standard library, and comparing the captured video frame with the standard video frame can be performed by a trained artificial intelligence model.

[0040] If the captured video footage does not match the standard video footage, it is determined that the corresponding video surveillance equipment capturing the footage may be malfunctioning, and the method proceeds to step S202. In step S202, a notification is sent to, for example, the maintenance personnel of the video surveillance equipment, instructing the corresponding video surveillance equipment and its captured footage to be adjusted and corrected.

[0041] If the captured video footage matches a standard video footage, the method proceeds to step S203. In step S203, the verified matching video stream is added to the sample library. Subsequently, the artificial intelligence model is trained using video samples from the sample library. The training may include step S204, where a manual method is used to determine whether the video footage content matches, and step S205, where the artificial intelligence model undergoes A / B testing and iterative training.

[0042] Back Figure 1 The video stream clustering step S104 is a step that selects and clusters video streams based on the input keywords and the acquired video information. The following will refer to... Figure 3 Describe the video stream clustering steps in detail.

[0043] Figure 3 This is a schematic diagram of the main process of video stream clustering step S104 in the anomaly detection method based on multidimensional information according to an embodiment of the present invention.

[0044] like Figure 3 As shown, in step S301, keywords input by the operator are received. The input keywords can be words or phrases describing the object being filmed by the video surveillance equipment. For example, in a logistics scenario, the input keywords could be words or phrases indicating the object being filmed by the video surveillance equipment ("flammable material") or indicating the location of the video surveillance equipment ("picking area"). Identification information, location information, and business marking information of the video surveillance equipment can also be used as keywords.

[0045] In step S302, based on the identification information, location information, and service tagging information of the video surveillance equipment associated with the video stream, a video stream matching the input keyword is selected from the sample library. An artificial intelligence model can be used to retrieve information fields such as identification information, location information, and service tagging information related to the video stream, determine whether their content is completely identical or semantically similar to the input keyword, and output the video stream matching the input keyword as the search result.

[0046] In step S303, the video quality of the selected video stream is determined based on an artificial intelligence model. The video quality assessment may include determining whether the video is clear, whether the image is occluded, whether there is backlighting, etc. This assessment can be performed by the artificial intelligence model. For example, the model can extract the Laplacian variance, high-frequency components, and texture features of the video image, calculate a sharpness score based on the extracted features, and compare the score with a threshold. If the score is higher than the threshold, the video is considered clear; otherwise, it is considered blurry. Another example is that the model can identify objects in an image based on a pre-prepared image dataset of various occlusions, calculate the proportion of occlusions in the total image area, and determine if the image is occluded if the proportion exceeds a threshold. Alternatively, the model can determine whether a key target is occluded. Yet another example is that the model can extract brightness distribution features from the image and calculate the proportion of highlight areas, average brightness, or contrast in the video image to determine if the video image is backlit.

[0047] If the video quality is deemed unacceptable, the method proceeds to step S304. In step S304, performance parameter information of the video surveillance device capturing the video stream is obtained, such as the device's resolution, viewing angle, ability to rotate, zoom capability, and night vision capability. Based on the obtained performance parameter information, an artificial intelligence model can automatically issue action commands to the video surveillance device, instructing it to adjust its focus, viewing angle, etc., thereby improving its video quality. Alternatively, a notification can be sent to, for example, maintenance personnel, instructing them that the corresponding video surveillance device is obstructed or that its captured image needs adjustment and correction.

[0048] If the video quality is determined to be substandard and cannot be improved by adjusting the video surveillance equipment, the method proceeds to step S305. In step S305, the corresponding video stream is discarded.

[0049] According to an embodiment of the present invention, when it is determined that multiple video streams simultaneously meet the keyword matching and video quality requirements, and the multiple video streams are shooting the same monitored object or the same location, the video stream with the best video quality and / or the best shooting angle can be selected for clustering and output.

[0050] If the video quality is deemed acceptable, the method proceeds to step S306. In step S306, a video stream clustering menu is generated, and the clustered video streams are output for polling.

[0051] According to embodiments of the present invention, an apparatus or server for performing an anomaly detection method according to embodiments of the present invention may have a user interface. The user interface may include one or more video output devices (e.g., a display), audio output devices (e.g., a speaker), or other types of devices capable of transmitting user-perceptible signals. The user interface may also include one or more input devices, such as a keyboard, mouse, microphone, etc. Operators can input keywords through the aforementioned user interface, view the generated video stream clustering menu, and perform corresponding selection and modification operations.

[0052] Back Figure 1 The video stream anomaly detection step S105 involves polling the clustered video streams to detect anomalies. Polling can be multi-screen polling. For this purpose, at least one display can be provided to the operator to show video frames from the clustered video streams one by one or simultaneously. According to the present invention, only the video streams clustered and output using keywords in step S104 are polled, rather than polling all video streams as in the prior art. According to embodiments of the present invention, an artificial intelligence model can be used to poll the clustered video streams and automatically detect anomalies. According to embodiments of the present invention, this step can be performed using the artificial intelligence model trained in step S102.

[0053] For example, when this invention is applied to fire monitoring in logistics parks, the artificial intelligence model can segment and detect video footage, and identify flames and smoke in the video footage based on pre-provided color, texture, and motion characteristics of flames and smoke. As another example, when this invention is applied to monitoring public places, the artificial intelligence model can identify human postures, target movement trajectories, crowd density, etc., in video footage, or detect the presence of specific objects, thereby determining whether an anomaly has occurred.

[0054] According to embodiments of the present invention, the clustered video streams can also be visually polled manually. Since only a limited number of filtered video streams are polled, rather than all video streams, manual costs and workload are reduced. In this case, the clustered video streams can be projected onto one or more displays, and operators can observe the video footage to detect anomalies.

[0055] The multidimensional information anomaly judgment step S106 is a step of judging abnormal events based on multidimensional information.

[0056] According to embodiments of the present invention, anomalies are identified not only based on alarm signals provided by alarm devices, but also additionally based on information provided by video surveillance devices associated with the alarm devices to collaboratively identify anomalies and set anomaly detection threshold. According to embodiments of the present invention, an artificial intelligence model can be used to determine anomalies based on pre-set risk level strategy rules.

[0057] For example, upon receiving an alarm signal from an alarm device, the system can retrieve the associated surveillance video stream based on multidimensional information and confirm the location of the anomaly. As another example, if multidimensional information indicates that the video surveillance device providing the video stream has thermal imaging capabilities, internal judgment logic can be invoked to compare the temperature displayed in the thermal imaging video frame with a fire risk temperature threshold to determine the fire risk level. Furthermore, if multidimensional information identifies the service type associated with the alarm device and the video surveillance device, different temperature thresholds can be set based on the item category and area function indicated by the service type to specifically determine fire risk.

[0058] The warning message sending step S107 is a step to issue a warning message to handle the anomaly when an anomaly is detected. According to an embodiment of the present invention, a warning message is issued in step S107 when an anomaly is detected by polling the clustered video stream (step S105) or by anomaly judgment based on multi-dimensional information (step S106). For example, in the case of fire monitoring in a logistics park, the present invention issues a fire alarm and initiates a fire extinguishing procedure according to the fire emergency plan. For example, information can be sent to a terminal device carried by security personnel. The sent information may include the location information of the alarm device related to the warning message, the video stream bound to the alarm device, and the corresponding emergency response strategy. For example, the sent video stream may include video taken at the alarm location 5 to 10 seconds before and after the time the anomaly is detected.

[0059] Figure 4 This is a schematic diagram of the main modules of an anomaly detection device based on multidimensional information according to an embodiment of the present invention. Figure 4 As shown, the anomaly detection device based on multidimensional information according to an embodiment of the present invention includes: an information acquisition module 401, a multidimensional information generation module 402, an artificial intelligence model training module 403, a video stream clustering module 404, a video stream anomaly judgment module 405, a multidimensional information anomaly judgment module 406, and a warning information sending module 407.

[0060] The information acquisition module 401 acquires video streams and related information from each of multiple video surveillance devices, and acquires alarm signals and related information from each of multiple alarm devices. The information acquired from the video surveillance devices includes: identification information for recognizing the video surveillance device, location information indicating the installation location of the video surveillance device, performance parameter information of the video surveillance device, and service tagging information related to the monitored objects of the video surveillance device. The information acquired from the alarm devices includes: identification information for recognizing the alarm device, location information indicating the installation location of the alarm device, performance parameter information of the alarm device, and service tagging information related to the monitored objects of the alarm device.

[0061] The multi-dimensional information generation module 402 is a module that binds information obtained from the alarm device with information obtained from the video surveillance device to generate multi-dimensional information. In this module, the video stream captured by the video surveillance device, the identification information, location information, performance parameter information of the video surveillance device, and service tagging information related to the monitored object are bound together. Furthermore, based on the acquired location information indicating the installation location of the video surveillance device and the location information indicating the installation location of the alarm device, the alarm device and nearby video surveillance devices are identified. Then, the alarm signal generated by the alarm device, the identification information, location information, performance parameter information of the alarm device, and service tagging information related to the monitored object are bound together with the video stream captured by at least one nearby video surveillance device. The bound information is stored as multi-dimensional information, for example, in the form of a mapping table.

[0062] The artificial intelligence model training module 403 is a module that uses information acquired from the video surveillance equipment in the information acquisition module to train the artificial intelligence model. In this module, the acquired information from the video surveillance equipment is compared with information stored in a standard library to determine if there are any anomalies in the corresponding video surveillance equipment that captured the video footage. Furthermore, the module uses video samples from a sample library consisting of video footage matched with standard video footage to train the artificial intelligence model.

[0063] The video stream clustering module 404 is a module that selects and clusters video streams based on input keywords and acquired video information. In this module, keywords are received by the operator; based on the identification information, location information, and service tag information of the video surveillance equipment associated with the video stream, video streams matching the input keywords are selected from the sample library; the video quality of the selected video streams is determined based on an artificial intelligence model; and the clustered video streams are output for polling.

[0064] The video stream anomaly detection module 405 is a module that polls the clustered video streams to detect anomalies. In this module, the content of the clustered video streams is checked using either an artificial intelligence model or manually to determine if any anomalies have occurred.

[0065] The multi-dimensional information anomaly judgment module 406 is a module that judges abnormal events based on multi-dimensional information. In the multi-dimensional information anomaly judgment module, anomalies are jointly identified and the anomaly level is determined based on the alarm signal provided by the alarm device and the information provided by the video surveillance device related to the alarm device.

[0066] The warning message sending module 407 is a module that issues warning messages to handle anomalies when they are detected. In the warning message sending module, warning messages are issued, for example, to terminal devices carried by security personnel, when an anomaly is detected in the video stream anomaly judgment module 405 or the multi-dimensional information anomaly judgment module 406.

[0067] Figure 5 An exemplary system architecture 500 is shown that can be applied to the anomaly detection method or device based on multidimensional information according to embodiments of the present invention.

[0068] like Figure 5 As shown, the system architecture 500 may include video surveillance equipment 501, alarm equipment 502, terminal equipment 503, network 504, and server 505. Network 504 serves as the medium for providing communication links between video surveillance equipment 501, alarm equipment 502, terminal equipment 503, and server 505. Network 504 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0069] Users can use terminal device 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal device 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0070] Terminal device 503 can be any electronic device with a display screen and web browsing capability, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0071] Server 505 can be a server that provides various services, such as a back-end management server that supports users in performing on-site equipment maintenance, fire extinguishing plans, or emergency response strategies through terminal device 503. The back-end management server can analyze and process received data such as warning messages, and feed back the processing results (such as on-site equipment maintenance instructions, fire extinguishing plans, or emergency response strategies) to the terminal device.

[0072] It should be noted that the anomaly detection method based on multidimensional information provided in the embodiments of the present invention is generally executed by server 505, and correspondingly, the anomaly detection device based on multidimensional information is generally set in server 505.

[0073] It should be understood that Figure 5 The number of video surveillance devices, alarm devices, terminal devices, networks, and servers shown is merely illustrative. Depending on the implementation requirements, any number of video surveillance devices, alarm devices, terminal devices, networks, and servers can be included.

[0074] The following is for reference. Figure 6 It shows a schematic diagram of the structure of a computer system 600 suitable for implementing a terminal device of the present invention. Figure 6 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0075] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the system 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0076] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0077] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined above in the system of this invention.

[0078] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0080] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including an information acquisition module, a multi-dimensional information generation module, an artificial intelligence model training module, a video stream clustering module, a video stream anomaly detection module, a multi-dimensional information anomaly detection module, and a warning information sending module. The names of these modules do not necessarily limit the module itself; for example, the information acquisition module may also be described as "a module that collects information from connected devices."

[0081] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: The information acquisition step involves obtaining video streams and related information from each of multiple video surveillance devices, and obtaining alarm signals and related information from each of multiple alarm devices. The information acquired from the video surveillance devices includes: identification information for recognizing the video surveillance device, location information indicating the installation location of the video surveillance device, performance parameter information of the video surveillance device, and service tagging information related to the monitored objects of the video surveillance device. Similarly, the information acquired from the alarm devices includes: identification information for recognizing the alarm device, location information indicating the installation location of the alarm device, performance parameter information of the alarm device, and service tagging information related to the monitored objects of the alarm device.

[0082] The multidimensional information generation step involves binding information obtained from an alarm device with information obtained from a video surveillance device to generate multidimensional information. In this step, the video stream captured by the video surveillance device, the identification information, location information, performance parameter information of the video surveillance device, and service tagging information related to the monitored object are bound together. Furthermore, based on the acquired location information indicating the installation location of the video surveillance device and the location information indicating the installation location of the alarm device, the alarm device and nearby video surveillance devices are identified. Then, the alarm signal generated by the alarm device, the alarm device's identification information, location information, performance parameter information, and service tagging information related to the monitored object are bound together with the video stream captured by at least one nearby video surveillance device. The bound information is stored as multidimensional information, for example, in the form of a mapping table.

[0083] The artificial intelligence model training step involves using information acquired from video surveillance equipment during the information acquisition step to train the AI ​​model. In this step, the acquired information from the video surveillance equipment is compared with information stored in a standard library to determine if the corresponding video surveillance equipment that captured the video footage exhibits any anomalies. Furthermore, the AI ​​model is trained using video samples from a sample library composed of video footage matched with standard video footage.

[0084] The video stream clustering step is a process of selecting and clustering video streams based on input keywords and acquired video information. In this step, keywords input by the operator are received; based on the identification information, location information, and service tagging information of the video surveillance equipment associated with the video streams, video streams matching the input keywords are selected from the sample library; the video quality of the selected video streams is determined based on an artificial intelligence model; and the clustered video streams are output for polling.

[0085] The video stream anomaly detection step involves polling the clustered video streams to detect anomalies. This step uses either an artificial intelligence model or manual methods to examine the content of the clustered video streams to determine if any anomalies have occurred.

[0086] The multi-dimensional information anomaly judgment step is a process of judging abnormal events based on multi-dimensional information. In this step, anomalies are collaboratively identified and their levels determined based on alarm signals provided by alarm devices and information provided by video surveillance devices associated with those devices.

[0087] The warning message sending step is the process of issuing a warning message to address an anomaly upon detection. This step may occur if an anomaly is detected during the video stream anomaly assessment step or the multidimensional information anomaly assessment step, for example, by sending a warning message to a terminal device carried by security personnel.

[0088] According to the technical solution of the present invention, by marking the video stream captured by the video surveillance equipment according to information such as the location of the video surveillance equipment, its monitored object and area type, and by retrieving, clustering and outputting the video stream with corresponding markings according to the keywords input by the user, the technical problems of low accuracy and poor timeliness in monitoring abnormal events such as fires in the prior art can be overcome, and the technical effect of timely and accurate monitoring of abnormal events in specific types of business functions can be achieved.

[0089] Furthermore, by binding the video stream output by the video surveillance equipment with the alarm signal output by the alarm equipment according to their locations, the technical problems of low accuracy in anomaly identification and poor adaptability and timeliness of emergency measures in the existing technology can be overcome. This can also achieve the technical effect of improving the timeliness and accuracy of anomaly identification, anomaly level confirmation, and handling strategy determination.

[0090] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An anomaly detection method based on multidimensional information, characterized in that, include: The information acquisition steps involve acquiring video streams and information related to the video surveillance devices from each of multiple video surveillance devices, and acquiring alarm signals and information related to the alarm devices from each of multiple alarm devices. The multidimensional information generation step involves associating the acquired information related to the video surveillance device with the video stream acquired by the corresponding video surveillance device, and associating the acquired alarm signal with at least one video stream based on the information related to the video surveillance device and the information related to the alarm device, thereby forming multidimensional information. The video stream clustering step involves retrieving information that matches keywords input from the outside from the information related to the video surveillance device included in the multidimensional information, clustering the video streams associated with the retrieved information in the multidimensional information, and outputting the clustered video streams. The video stream anomaly detection step involves polling the clustered video streams to detect anomalies. The multi-dimensional information anomaly judgment step detects anomalies based on the alarm signals and associated video streams included in the multi-dimensional information; The warning message sending step involves generating and sending a warning message based on the multidimensional information when an anomaly is detected.

2. The anomaly detection method according to claim 1, in, The information related to the video surveillance equipment includes: the identification information of the video surveillance equipment, the location information of the video surveillance equipment, the performance parameter information of the video surveillance equipment, and service tagging information related to the monitored object of the video surveillance equipment; and The information related to the alarm device includes: the identification information of the alarm device, the location information of the alarm device, the performance parameter information of the alarm device, and the business tag information related to the monitored object of the alarm device.

3. The anomaly detection method according to claim 2, in, In the multi-dimensional information generation step, based on the location information of the video surveillance device and the location information of the alarm device, the video surveillance device near the alarm device is identified, and the alarm signal of the alarm device is associated with the video stream captured by the identified video surveillance device.

4. The anomaly detection method according to claim 1, in, The video stream clustering step also includes determining the video quality of the clustered video streams; In response to the video quality of the clustered video stream being substandard, the corresponding video surveillance equipment is adjusted to improve the video quality based on the information related to the video surveillance equipment included in the multidimensional information.

5. The anomaly detection method according to claim 1, in, The multidimensional information anomaly judgment step further includes setting an anomaly detection threshold based on the video stream, information related to the video surveillance equipment, alarm signals, and information related to the alarm equipment included in the multidimensional information.

6. The anomaly detection method according to claim 1 further includes: The artificial intelligence model training step involves training an artificial intelligence model based on the video stream and information related to the video surveillance equipment obtained in the information acquisition step, using a standard library of pre-stored standard video images. The video stream clustering step, the video stream anomaly detection step, the multi-dimensional information anomaly detection step, and the warning information sending step are performed using a trained artificial intelligence model.

7. An anomaly detection device based on multidimensional information, characterized in that, include: The information acquisition module acquires video streams and information related to the video surveillance devices from each of multiple video surveillance devices, and acquires alarm signals and information related to the alarm devices from each of multiple alarm devices; The multi-dimensional information generation module associates the acquired information related to the video surveillance equipment with the video stream acquired by the corresponding video surveillance equipment, and associates the acquired alarm signal with at least one video stream based on the information related to the video surveillance equipment and the information related to the alarm equipment, thereby forming multi-dimensional information; The video stream clustering module retrieves information that matches keywords input from the outside from the information related to video surveillance equipment included in the multidimensional information, clusters the video streams associated with the retrieved information in the multidimensional information, and outputs the clustered video streams. The video stream anomaly detection module polls the clustered video streams to detect anomalies; The multi-dimensional information anomaly detection module detects anomalies based on the alarm signals and associated video streams included in the multi-dimensional information. The warning message sending module generates and sends a warning message based on the multidimensional information when an anomaly is detected.

8. A server for anomaly detection based on multidimensional information, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the anomaly detection method as described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the anomaly detection method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the anomaly detection method as described in any one of claims 1-6.

Images