A smart park abnormal event handling method and system based on a multi-modal model

Abstract

The application discloses a kind of wisdom park abnormal event handling methods and systems based on multi-modal model, and is related to the field of Internet of Things.The method is applied to the core control system, and the method comprises the following steps: acquiring multiple monitoring data; multiple monitoring data are identified for abnormal events respectively, to obtain multiple basic abnormal events; multiple basic abnormal events are cross combined, to obtain multiple abnormal event groups; multiple abnormal event groups are input into abnormal event reasoning model respectively, to obtain multiple reasoning abnormal events; multiple reasoning abnormal events are clustered, to obtain multiple reasoning abnormal event clustering clusters; according to the clustering center corresponding to each of multiple reasoning abnormal event clustering clusters, determine target clustering center; based on target clustering center, determine the handling priority of multiple reasoning abnormal events; according to the handling priority of multiple reasoning abnormal events, generate abnormal event handling strategy table. Thus, the problem that the abnormal event handling effect of the current wisdom park is not good is solved.

Description

Technical Field

[0001] This application relates to the Internet of Things (IoT) field, specifically to a method and system for handling abnormal events in smart parks based on a multimodal model. Background Technology

[0002] As the construction of smart parks continues to deepen, the requirements for real-time response and accurate decision-making in handling abnormal events are becoming increasingly stringent.

[0003] Currently, to meet real-time requirements, multiple distributed intelligent agents are deployed within the park, such as edge computing nodes, security robots, and terminal sensing and execution devices. Each agent autonomously formulates abnormal event handling decisions and executes corresponding actions by collecting its own monitoring data, thereby effectively shortening the decision-making cycle and ensuring rapid response to sudden abnormal events. However, this method of handling abnormal events in the park is only suitable for simple and singular abnormal events. For complex abnormal events, the handling decisions formulated by the various agents are prone to conflict, thus missing the optimal handling window and causing significant losses. Therefore, a core hub is also deployed in the smart park. This core hub integrates and analyzes the monitoring data and decisions uploaded by multiple agents, corrects the decisions of each agent in real time, and reallocates the response strategies of each agent to abnormal events, thereby effectively improving the accuracy of abnormal event handling.

[0004] However, in order to ensure the accuracy of decision-making, the above-mentioned abnormal event handling methods in smart parks require that the monitoring data of each intelligent agent be uploaded synchronously and be valid. However, due to varying degrees of dynamic interference in the monitoring environment of each intelligent agent, the quality of the synchronously uploaded data is inconsistent, thereby reducing the accuracy of the abnormal event handling decisions made by the core central hub. Summary of the Invention

[0005] To address the issue of ineffective handling of abnormal events in current smart parks, this application provides a method and system for handling abnormal events in smart parks based on a multimodal model.

[0006] Firstly, this application provides a method for handling abnormal events in a smart park based on a multimodal model, applied in a central control system, the method comprising:

[0007] Acquire various monitoring data uploaded synchronously by intelligent agents in multiple parks;

[0008] Anomaly events are identified for each of the various monitoring data to obtain a variety of basic anomaly events corresponding to each of the various monitoring data.

[0009] By cross-combining the various basic abnormal events corresponding to the various monitoring data, multiple different abnormal event groups are obtained;

[0010] The multiple groups of abnormal events are input into the abnormal event inference model to obtain multiple inference abnormal events;

[0011] Multiple inference anomaly events are clustered to obtain multiple inference anomaly event clusters;

[0012] The target cluster center is determined based on the cluster centers corresponding to each of the multiple clusters of inference anomalies.

[0013] Based on the target cluster center, determine the priority of handling multiple inference anomalies;

[0014] Based on the handling priorities of multiple inference anomalies, an anomaly handling strategy table is generated, and the anomaly handling strategy table is distributed to multiple park agents so that the multiple park agents can handle the anomalies.

[0015] Optionally, the step of identifying abnormal events for each of the various monitoring data to obtain various basic abnormal events corresponding to each of the various monitoring data specifically includes:

[0016] The first monitoring data is matched with a preset basic abnormal event signal table to obtain multiple matching abnormal events corresponding to the first monitoring data and the matching degree of the multiple matching abnormal events. The first monitoring data is any one of the multiple monitoring data.

[0017] The matching degree of multiple matching anomaly events is sorted from largest to smallest to obtain the reliability ranking result of multiple matching anomaly events;

[0018] Based on a preset matching degree threshold and a reliability ranking result threshold, multiple basic abnormal events corresponding to the first monitoring data are obtained by filtering multiple matching abnormal events corresponding to the first monitoring data.

[0019] Optionally, the abnormal event reasoning model is trained from the reasoning relationships, reasoning conclusions, and reasoning confidence among basic abnormal events.

[0020] Optionally, the step of inputting multiple groups of abnormal events into the abnormal event inference model to obtain multiple inference abnormal events specifically includes:

[0021] The first abnormal event group is input into the abnormal event inference model to obtain the inference abnormal event set of the first abnormal event group. The inference abnormal event set includes multiple inferable abnormal events and the inference confidence corresponding to the multiple inferable abnormal events. The first abnormal event group is any one of the multiple abnormal event groups.

[0022] The inference-possible abnormal event with the highest inference confidence in the inference abnormal event set of the first abnormal event group is selected as the inference abnormal event corresponding to the first abnormal event group.

[0023] Optionally, selecting the inference-possible abnormal event with the highest inference confidence from the inference abnormal event set of the first abnormal event group as the inference abnormal event corresponding to the first abnormal event group further includes:

[0024] Based on various monitoring data, the area where the abnormal event occurred was determined;

[0025] The abnormal event occurrence area is matched with a preset abnormal event confidence threshold table to obtain the abnormal event confidence threshold of the abnormal event occurrence area.

[0026] Determine whether the inference confidence of the inference anomaly corresponding to the first anomaly event group is greater than or equal to the anomaly event confidence threshold of the anomaly event occurrence area;

[0027] If not, then the reasoning exception corresponding to the first exception event group is determined to be an invalid exception event, and the first exception event group is removed.

[0028] Optionally, the step of clustering multiple inference anomaly events to obtain multiple inference anomaly event clusters specifically involves:

[0029] Based on the inference confidence dispersion coefficient, core word entropy value, and event size coefficient of multiple inference anomaly events, the clustering fit of multiple inference anomaly events is calculated.

[0030] The target clustering level is determined based on the clustering fit.

[0031] Based on the target clustering hierarchy, hierarchical clustering is used to cluster multiple inference anomaly events to obtain multiple inference anomaly event clusters. The target clustering hierarchy includes an inference confidence clustering layer, a core word clustering layer, and an event density clustering layer.

[0032] Optionally, determining the target cluster center based on the cluster centers corresponding to each of the multiple inference anomaly event clusters specifically includes:

[0033] Based on the multiple basic abnormal events corresponding to each of the multiple inference abnormal event clusters, calculate the information entropy of the multiple inference abnormal event clusters;

[0034] The cluster center weights are determined based on the information entropy of multiple clusters of the aforementioned inference anomaly events.

[0035] Based on the cluster center weights and cluster centers of the multiple inference anomaly event clusters, the target cluster centers of the multiple inference anomaly event clusters are calculated.

[0036] Secondly, this application provides a smart park abnormal event handling system based on a multimodal model. The system is a central control system, which includes a receiving module, a processing module, and a transmitting module, wherein:

[0037] The receiving module is used to acquire various monitoring data synchronously uploaded by multiple park intelligent agents; to identify abnormal events in the various monitoring data to obtain various basic abnormal events corresponding to each of the various monitoring data; and to cross-combine the various basic abnormal events corresponding to each of the various monitoring data to obtain multiple different abnormal event groups.

[0038] The processing module is used to input multiple groups of abnormal events into an abnormal event inference model to obtain multiple inference abnormal events; to cluster the multiple inference abnormal events to obtain multiple inference abnormal event clusters; to determine target cluster centers based on the cluster centers corresponding to each of the multiple inference abnormal event clusters; and to determine the processing priority of the multiple inference abnormal events based on the target cluster centers.

[0039] The sending module is used to generate an abnormal event handling strategy table according to the handling priority of multiple inference abnormal events, and to send the abnormal event handling strategy table to multiple park intelligent agents so that the multiple park intelligent agents can handle abnormal events.

[0040] Thirdly, this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of the first aspects.

[0041] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any one of the first aspects.

[0042] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0043] When an anomaly occurs, multiple pre-deployed intelligent agents within the park synchronously upload monitored data to the central control system. The central control system then identifies anomalies in the various monitoring data sets, uncovering all possible underlying anomalies reflected in each type of data. These underlying anomalies are then cross-combined to obtain multiple anomaly event groups. Each anomaly event group represents the event characteristics of a hypothetical anomaly event. These multiple anomaly event groups are then input into an anomaly event inference model to deduce multiple inferred anomalies and their confidence levels. Finally, based on the confidence levels, core word entropy values, and event density, the multiple inferred anomalies are hierarchically clustered. Multiple clusters of inference anomaly events are obtained. If the overall confidence of the inference anomaly event cluster is high, the core word entropy value is low, and the event density is high, it indicates that the anomaly events represented by the inference anomaly event cluster are more reliable. At this time, the consensus event description of the anomaly events represented by the inference anomaly event cluster is obtained by calculating the cluster center. Then, the target cluster center of multiple inference anomaly event clusters is calculated to determine the consensus event description of the global anomaly event. Finally, the distance between each inference anomaly event and the target cluster center is calculated. The closer the distance, the higher its reliability, and the higher its priority handling value, thereby improving the accuracy of anomaly event handling. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a method for handling abnormal events in a smart park based on a multimodal model, as provided in an embodiment of this application.

[0045] Figure 2 This is a schematic diagram of the structure of a smart park abnormal event handling system based on a multimodal model provided in this application embodiment.

[0046] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0047] Explanation of reference numerals in the attached drawings: 1. Receiving module; 2. Processing module; 3. Transmitting module; 300. Electronic device; 301. Processor; 302. Communication bus; 303. User interface; 304. Network interface; 305. Memory. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] This application provides a method for handling abnormal events in a smart park based on a multimodal model. This method is applied to a central control system, such as... Figure 1 As shown, the method includes steps S101 to S108, which are as follows:

[0050] S101. Acquire various monitoring data synchronously uploaded by multiple smart agents in the park.

[0051] In the above steps, multiple intelligent agents are deployed within the smart park to monitor the park's status in real time. These intelligent agents include, but are not limited to, edge computing nodes, security robots, environmental sensor clusters, and access control systems. When an intelligent agent detects an abnormal event, it sends an abnormal event detection signal to the central control system. The central control system then triggers an abnormal event handling procedure, sending monitoring data upload signals to other intelligent agents. Upon receiving these signals, the other intelligent agents simultaneously upload their real-time collected monitoring data to the central control system. This monitoring data includes, but is not limited to, video image streams, thermal imaging data, structured logs, audio signals, and IoT sensor time-series data.

[0052] S102. Perform abnormal event identification on various monitoring data to obtain various basic abnormal events corresponding to each monitoring data.

[0053] In the above steps, this application pre-establishes a basic abnormal event signal table, which stores the typical signal patterns corresponding to various basic abnormal events that may occur for each monitoring data mode. Then, for any first monitoring data among multiple monitoring data, the first monitoring data can be matched with the typical signal patterns of various basic abnormal events corresponding to it in the basic abnormal event signal table to obtain multiple matching abnormal events in the first monitoring data and the matching degree with these multiple matching abnormal events. Then, in order to remove irrelevant matching abnormal events, this application further sorts the matching degrees of the multiple matching abnormal events in descending order to obtain a reliability ranking result for the multiple matching abnormal events. The higher the reliability ranking result, the higher its reliability. Furthermore... If the matching degree of multiple matched abnormal events corresponding to the first monitoring data is low, it indicates that the first monitoring data is irrelevant data in this abnormal event. In this case, it will not be included in the subsequent abnormal event analysis, thereby reducing the interference of irrelevant data on the abnormal event analysis. Therefore, based on the preset matching degree threshold and reliability ranking result threshold, this application selects multiple matched abnormal events from the multiple matched abnormal events corresponding to the first monitoring data whose reliability ranking result is greater than or equal to the reliability ranking result threshold and whose matching degree is greater than or equal to the matching degree threshold, as the basic abnormal events corresponding to the first monitoring data. It should be noted that the basic abnormal events are abnormal events that do not involve correlation reasoning, such as: a sudden increase in electricity meter reading, video showing unauthorized personnel entering, abnormal activation of equipment, etc.

[0054] S103. Cross-combine the various basic abnormal events corresponding to various monitoring data to obtain multiple different abnormal event groups.

[0055] In the above steps, since the basic abnormal events in the single modality monitoring data are partial and isolated, it is not effective to deal with them individually. Therefore, this application cross-combines the various basic abnormal events corresponding to various monitoring data, systematically explores all possible composite event patterns, and obtains multiple different abnormal event groups.

[0056] S104. Input multiple abnormal event groups into the abnormal event inference model to obtain multiple inference abnormal events.

[0057] In the above steps, the basic abnormal event combinations in each abnormal event group can be roughly divided into interference items and reasoning items. The more reasoning items there are, the more unique the inferred abnormal events obtained by the abnormal event group will be, and the higher the confidence level will be. The more interference items there are, the more chaotic the reasoning logic of the abnormal event group will be, resulting in a large number of inferred abnormal events, but most of them are irrelevant inferred abnormal events with low confidence. Therefore, this application reduces the impact of irrelevant inferred abnormal events on the reasoning of subsequent real abnormal events. This application inputs multiple abnormal event groups into the abnormal event reasoning model to obtain multiple inferred abnormal events corresponding to each of the multiple abnormal event groups and the reasoning confidence levels corresponding to the multiple inferred abnormal events. The abnormal event reasoning model can be understood as a kind of knowledge graph model, which consists of the reasoning relationships between basic abnormal events and the reasoning conclusions. The model is trained using inference confidence, with nodes representing basic anomalous events and connecting edges representing the prior probabilities of inference between two nodes. When an anomalous event group is input into the anomalous event inference model, multiple consecutive inference paths are matched. The endpoint of each inference path can then be used to infer an anomalous event. Based on the Bayesian probabilistic inference model, the inference confidence of each inferable anomalous event is calculated. Finally, the model outputs multiple inferable anomalous events corresponding to the anomalous event group and their corresponding inference confidence. To remove irrelevant interference from the multiple inferable anomalous events corresponding to each anomalous event group, this application selects only the inferable anomalous event with the highest inference confidence among the multiple inferable anomalous events corresponding to each anomalous event group as the inference anomalous event corresponding to that anomalous event group, thereby improving the efficiency and accuracy of subsequent anomalous event handling and analysis.

[0058] In one possible implementation, since different areas within the park have significantly different tolerances and judgment criteria for abnormal events, for example, in a chemical plant area, even if the confidence level is low, signs of gas leaks should be accepted; while in an open office area, the same gas leak event, due to its extremely low probability, requires a higher confidence level to be accepted. This reduces the occurrence of underreporting in high-risk areas and false reporting in low-risk areas. Therefore, after obtaining the inferred abnormal events corresponding to multiple abnormal event groups, this application also determines one or more abnormal event occurrence areas based on the monitored areas of the park's intelligent agents corresponding to various monitoring data. At this time, the area with the highest risk level among the multiple abnormal event occurrence areas is selected as the core abnormal event occurrence area. Then, the abnormal event occurrence area is matched with a preset abnormal event confidence threshold table to obtain the abnormal event confidence threshold of the abnormal event occurrence area. The abnormal event confidence threshold table stores the minimum inference confidence threshold required for the abnormal event occurrence area to be valid. Then, the inference confidence of each of the multiple inference anomalies is compared with the anomaly confidence threshold of the current anomaly occurrence area. If the inference confidence of an inference anomaly is greater than or equal to the anomaly confidence threshold, the inference anomaly is determined to be a valid inference anomaly; otherwise, it is determined to be an invalid inference anomaly. In this case, the inference anomaly group is removed from the subsequent analysis.

[0059] S105. Cluster multiple inference anomaly events to obtain multiple inference anomaly event clusters.

[0060] S106. Determine the target cluster center based on the cluster centers corresponding to each of the multiple inference anomaly event clusters.

[0061] S107. Based on the target cluster center, determine the priority of handling multiple inference anomalies.

[0062] S108. Based on the handling priority of multiple inference abnormal events, generate an abnormal event handling strategy table and distribute the abnormal event handling strategy table to multiple park agents so that multiple park agents can handle abnormal events.

[0063] In steps S105 to S108 above, after obtaining multiple inference anomalies, due to the multi-dimensional heterogeneity of each inference anomaly in terms of credibility, semantic core, and scope of impact, directly allocating processing resources on demand would lead to inaccurate resource allocation, thereby affecting the overall processing efficiency and effectiveness. Therefore, this application improves the accuracy of processing resource allocation by constructing consensus attributes for multiple inference anomalies and using them as a benchmark to determine the processing priority of multiple inference anomalies. Specifically:

[0064] This application employs a hierarchical clustering algorithm to cluster multiple inference anomaly events, resulting in multiple inference anomaly event clusters. Each inference anomaly event cluster contains the same consensus attribute. The hierarchical clustering algorithm used in this application includes three clustering layers: an inference confidence clustering layer, a core word clustering layer, and an event density clustering layer. The consensus attribute of the inference confidence clustering layer is inference confidence, the consensus attribute of the core word clustering layer is core word similarity, and the consensus attribute of the event density clustering layer is event clustering region. Then, the cluster center of each inference anomaly event cluster is calculated. At this point, the cluster center represents the consensus center that integrates the event features in the inference anomaly event cluster. Then, the center point of multiple cluster centers (target cluster center) is calculated as the consensus center of all inference anomaly events. At this point, the target cluster center is used as a benchmark to calculate the distance between multiple inference anomaly events and the target cluster center. The closer the distance, the higher the handling value of the inference anomaly event, thus giving it a higher handling priority, thereby achieving a better handling effect for complex anomaly events.

[0065] In one possible implementation, when performing hierarchical clustering on multiple inference anomaly events, if the set of inference anomaly events is relatively concentrated, an excessive number of clustering layers will forcibly separate inference anomaly events that should belong to the same cluster, resulting in fragmented clustering results and making it difficult for the target cluster center to anchor the core anomaly. If the set of inference anomaly events is relatively dispersed, an insufficient number of clustering layers will mix different types of inference anomaly events, making it difficult for the cluster center of the inference anomaly event cluster itself to accurately reflect the consensus characteristics of the cluster, thereby affecting subsequent priority judgment. Therefore, in order to improve the clustering accuracy and thus improve the accuracy of the consensus center of each inference anomaly event cluster, this application first calculates the clustering fit of multiple inference anomaly events based on the inference confidence dispersion coefficient, core word entropy value, and event size coefficient. The inference confidence dispersion coefficient can be obtained by calculating the coefficient of variation of the confidence of all anomaly events, and the core word entropy value can be calculated using the Shannon entropy calculation formula, as follows:

[0066] Where H is the core word information entropy, and n is the total number of core words in multiple inference anomalies. The frequency of occurrence of the i-th core word is given.

[0067] The event size coefficient can be obtained by querying a predefined mapping relationship between the total number of regions covered by multiple inference anomaly events and the event size coefficient. Then, a weighted sum is calculated on the inference confidence dispersion coefficient, core word entropy value, and event size coefficient to obtain the clustering fit of multiple inference anomaly events.

[0068] At this point, the larger the coefficient of discrepancy of reasoning confidence, the more serious the mixing of high-confidence reasoning anomalies and low-confidence reasoning anomalies, requiring more layers for separation; the larger the core word entropy value, the more anomaly types the event covers, requiring more layers to aggregate similar semantics; the larger the event scale coefficient, the wider the distribution area of ​​reasoning anomalies, requiring more layers for refinement.

[0069] Then, the cluster fit is matched with the preset clustering layer table to obtain the target clustering layer for the current multiple inference anomaly events. It should be noted that the larger the cluster fit, the more complex the anomaly events occurring in the park, and thus the more clustering layers are required. In addition, the clustering order in this application is related to the magnitude of three clustering coefficients: inference confidence dispersion coefficient, core word entropy value, and event size coefficient. Specifically, the larger the clustering coefficient, the earlier the clustering order.

[0070] After determining the clustering hierarchy and clustering order, this application performs hierarchical clustering on multiple inference anomaly events to obtain multiple inference anomaly event clusters, thereby achieving the optimal clustering effect.

[0071] In one possible implementation, when calculating the target cluster center, since the information value within different inference anomaly event clusters differs, this application extracts all the basic anomaly events contained in each inference anomaly event cluster. Then, these basic event types are treated as random variables, and the frequency (i.e., probability distribution) of each basic event type within the inference anomaly event cluster is calculated. Based on the probability distribution of each basic event type, the information entropy of the inference anomaly event cluster is obtained. At this point, the higher the information entropy, the more types of events the inference anomaly event cluster consists of, the more complex its internal state, and the lower its information value; the lower the information entropy, the more singular and pure the inference anomaly event cluster consists of, the more focused its internal state, and the higher its information value. At this point, based on the information entropy of multiple inference anomaly event clusters, the cluster center weights of each inference anomaly event cluster are determined. Information entropy and cluster center weights are negatively correlated. Specifically, this application first normalizes the information entropy of each inference anomaly event cluster, then sorts the information entropy of each inference anomaly event cluster in descending order to obtain the information entropy ranking result. Next, the information entropy normalization result is sorted in descending order to obtain the information entropy normalization ranking result. Finally, the information entropy ranking result and the information entropy normalization ranking result are matched one by one according to the ranking order to determine the cluster center weights of each inference anomaly event cluster. Then, based on the cluster center weights of each inference anomaly event cluster, the weighted sum of the cluster centers of multiple inference anomaly event clusters is calculated to obtain the target cluster center. This allows the target cluster center to move closer to a more explicit and reliable inference anomaly event cluster, thereby improving the accuracy of subsequent anomaly event handling.

[0072] Finally, based on the handling priorities of multiple inference anomalies, an anomaly handling strategy table is generated and distributed to multiple park agents. This ensures that the multiple park agents handle the anomalies according to their respective priorities, thereby improving the accuracy of handling complex anomalies within the park.

[0073] Reference Figure 2 This application also provides a smart park abnormal event handling system based on a multimodal model. The system is a central control system, which includes a receiving module 1, a processing module 2, and a sending module 3, wherein:

[0074] The receiving module 1 is used to acquire various monitoring data synchronously uploaded by multiple park intelligent agents; to identify abnormal events in the various monitoring data to obtain various basic abnormal events corresponding to each monitoring data; and to cross-combine the various basic abnormal events corresponding to each monitoring data to obtain multiple different abnormal event groups.

[0075] Processing module 2 is used to input multiple abnormal event groups into the abnormal event inference model to obtain multiple inference abnormal events; to cluster the multiple inference abnormal events to obtain multiple inference abnormal event clusters; to determine the target cluster center based on the cluster center corresponding to each of the multiple inference abnormal event clusters; and to determine the processing priority of the multiple inference abnormal events based on the target cluster center.

[0076] The sending module 3 is used to generate an abnormal event handling strategy table based on the handling priority of multiple inference abnormal events, and to send the abnormal event handling strategy table to multiple park intelligent agents so that multiple park intelligent agents can handle abnormal events.

[0077] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0078] This application also discloses an electronic device. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0079] The communication bus 302 is used to enable communication between these components.

[0080] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0081] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0082] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0083] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. (Refer to...) Figure 3 The memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for handling abnormal events in a smart park based on a multimodal model.

[0084] exist Figure 3In the illustrated electronic device 300, the user interface 303 is mainly used to provide an input interface for the user and acquire user input data; while the processor 301 can be used to call an application program stored in the memory 305 that is a method for handling abnormal events in a smart park based on a multimodal model. When executed by one or more processors 301, the electronic device 300 performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0085] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0086] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0089] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0090] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0091] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for handling abnormal events in a smart park based on a multimodal model, characterized in that, When applied in a central control system, the method includes: Acquire various monitoring data uploaded synchronously by intelligent agents in multiple parks; Anomaly events are identified for each of the various monitoring data to obtain a variety of basic anomaly events corresponding to each of the various monitoring data. By cross-combining the various basic abnormal events corresponding to the various monitoring data, multiple different abnormal event groups are obtained; The multiple groups of abnormal events are input into the abnormal event inference model to obtain multiple inference abnormal events; Multiple inference anomaly events are clustered to obtain multiple inference anomaly event clusters; The target cluster center is determined based on the cluster centers corresponding to each of the multiple clusters of inference anomalies. Based on the target cluster center, determine the priority of handling multiple inference anomalies; Based on the handling priorities of multiple inference anomalies, an anomaly handling strategy table is generated, and the anomaly handling strategy table is distributed to multiple park agents so that the multiple park agents can handle the anomalies.

2. The method according to claim 1, characterized in that, The process of identifying abnormal events for each of the various monitoring data sets to obtain various basic abnormal events corresponding to each of the various monitoring data sets specifically includes: The first monitoring data is matched with a preset basic abnormal event signal table to obtain multiple matching abnormal events corresponding to the first monitoring data and the matching degree of the multiple matching abnormal events. The first monitoring data is any one of the multiple monitoring data. The matching degree of multiple matching anomaly events is sorted from largest to smallest to obtain the reliability ranking result of multiple matching anomaly events; Based on a preset matching degree threshold and a reliability ranking result threshold, multiple basic abnormal events corresponding to the first monitoring data are obtained by filtering multiple matching abnormal events corresponding to the first monitoring data.

3. The method according to claim 1, characterized in that, The abnormal event reasoning model is trained from the reasoning relationships, reasoning conclusions, and reasoning confidence among basic abnormal events.

4. The method according to claim 3, characterized in that, The step of inputting multiple groups of abnormal events into the abnormal event inference model to obtain multiple inference abnormal events specifically includes: The first abnormal event group is input into the abnormal event inference model to obtain the inference abnormal event set of the first abnormal event group. The inference abnormal event set includes multiple inferable abnormal events and the inference confidence corresponding to the multiple inferable abnormal events. The first abnormal event group is any one of the multiple abnormal event groups. The inference-possible abnormal event with the highest inference confidence in the inference abnormal event set of the first abnormal event group is selected as the inference abnormal event corresponding to the first abnormal event group.

5. The method according to claim 4, characterized in that, The step of selecting the inference-possible abnormal event with the highest inference confidence from the inference abnormal event set of the first abnormal event group as the inference abnormal event corresponding to the first abnormal event group specifically includes: Based on various monitoring data, the area where the abnormal event occurred was determined; The abnormal event occurrence area is matched with a preset abnormal event confidence threshold table to obtain the abnormal event confidence threshold of the abnormal event occurrence area; Determine whether the inference confidence of the inference anomaly corresponding to the first anomaly event group is greater than or equal to the anomaly event confidence threshold of the anomaly event occurrence area; If not, then the reasoning exception corresponding to the first exception event group is determined to be an invalid exception event, and the first exception event group is removed.

6. The method according to claim 1, characterized in that, The process of clustering multiple inference anomaly events to obtain multiple inference anomaly event clusters is specifically as follows: Based on the inference confidence dispersion coefficient, core word entropy value, and event size coefficient of multiple inference anomaly events, the clustering fit of multiple inference anomaly events is calculated. The target clustering level is determined based on the clustering fit. Based on the target clustering hierarchy, hierarchical clustering is used to cluster multiple inference anomaly events to obtain multiple inference anomaly event clusters. The target clustering hierarchy includes an inference confidence clustering layer, a core word clustering layer, and an event density clustering layer.

7. The method according to claim 1, characterized in that, The step of determining the target cluster center based on the cluster centers corresponding to each of the multiple inference anomaly event clusters specifically includes: Based on the multiple basic abnormal events corresponding to each of the multiple inference abnormal event clusters, calculate the information entropy of the multiple inference abnormal event clusters; The cluster center weights are determined based on the information entropy of multiple clusters of the aforementioned inference anomaly events. Based on the cluster center weights and cluster centers of the multiple inference anomaly event clusters, the target cluster centers of the multiple inference anomaly event clusters are calculated.

8. A smart park abnormal event handling system based on a multimodal model, characterized in that, The system is a central control system, which includes a receiving module (1), a processing module (2), and a transmitting module (3), wherein: The receiving module (1) is used to acquire multiple monitoring data uploaded synchronously by multiple park intelligent agents; to identify abnormal events in the multiple monitoring data respectively, and to obtain multiple basic abnormal events corresponding to each of the multiple monitoring data; and to cross-combine the multiple basic abnormal events corresponding to each of the multiple monitoring data to obtain multiple different abnormal event groups. The processing module (2) is used to input multiple abnormal event groups into the abnormal event inference model to obtain multiple inference abnormal events; to cluster the multiple inference abnormal events to obtain multiple inference abnormal event clusters; to determine the target cluster center according to the cluster center corresponding to each of the multiple inference abnormal event clusters; and to determine the processing priority of the multiple inference abnormal events based on the target cluster center. The sending module (3) is used to generate an abnormal event handling strategy table according to the handling priority of multiple inference abnormal events, and send the abnormal event handling strategy table to multiple park intelligent agents so that the multiple park intelligent agents can handle abnormal events.

9. An electronic device, characterized in that, The device includes a processor (301), a memory (305), a user interface (303), and a network interface (304). The memory (305) is used to store instructions. The user interface (303) and the network interface (304) are used to communicate with other devices. The processor (301) is used to execute the instructions stored in the memory (305) to cause the electronic device (300) to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1 to 7.

Images