Monitoring information identification method based on Internet of Things and server

Abstract

The invention provides a monitoring information identification method based on the Internet of Things and a server, and the method comprises the steps: firstly obtaining multi-modal perception data which are collected by a plurality of Internet of Things monitoring nodes in a target monitoring area in real time and comprise images, sounds and environment parameters, and then carrying out the cross-modal feature extraction and fusion processing of the multi-modal perception data; the method comprises the steps of generating a comprehensive spatio-temporal characteristic spectrum, determining abnormal event types and spatio-temporal distribution information in a region through dynamic mode matching based on a preset abnormal mode library, generating an adaptive regulation and control instruction set according to the abnormal event types and the spatio-temporal distribution information, and determining control parameters and execution priorities of different Internet of Things execution devices. And finally, distributing the instruction set to an associated Internet of Things execution device cluster, triggering the cluster to execute control parameters according to priorities, eliminating the influence of abnormal events, and realizing comprehensive and intelligent monitoring and exception handling of a target monitoring area.

Description

Technical Field

[0001] The present invention relates to the technical field of the Internet of Things, and in particular, to a method for identifying monitoring information based on the Internet of Things and a server. Background Art

[0002] With the rapid development of the Internet of Things technology, its applications in various fields are becoming increasingly widespread. Effective monitoring of the target area has become a key requirement in many scenarios, such as industrial production monitoring, urban public area safety monitoring, intelligent building environment management, etc.

[0003] In the related art, different types of data are often analyzed independently, without fully exploring the potential connections and complementarities between different modality data. This isolated data processing method leads to an inability to form an overall and comprehensive understanding of the target area, making it difficult to accurately determine whether there are abnormal situations in the area and the specific characteristics of the abnormal situations.

[0004] In terms of the identification of abnormal events, most traditional methods are based on fixed rules or simple threshold judgments. Facing complex and changeable actual monitoring scenarios, these methods lack sufficient flexibility and adaptability. Once the monitoring environment changes or new abnormal patterns appear, the original judgment rules may fail, resulting in misjudgment or missed judgment of abnormal events, and it is impossible to accurately determine the type of abnormal events and their distribution information in the spatio-temporal dimension. Summary of the Invention

[0005] In view of the above-mentioned problems, in combination with the first aspect of the present invention, embodiments of the present invention provide a method for identifying monitoring information based on the Internet of Things, and the method includes: Obtain multi-modal perception data collected in real time by multiple Internet of Things monitoring nodes in a target monitoring area, where the multi-modal perception data includes image monitoring data, sound monitoring data, and environmental parameter monitoring data; Perform cross-modal feature extraction and fusion processing on the multi-modal perception data to generate a comprehensive spatio-temporal feature map corresponding to the target monitoring area; Based on a preset abnormal pattern library, perform dynamic pattern matching on the comprehensive spatio-temporal feature map to determine at least one abnormal event type existing in the target monitoring area and its spatio-temporal distribution information; According to the abnormal event type and the spatio-temporal distribution information, generate a set of control commands adapted to the abnormal event type, where the set of control commands includes control parameters and execution priorities for different Internet of Things execution devices; Distribute the set of control commands to an Internet of Things execution device cluster associated with the target monitoring area, and trigger the Internet of Things execution device cluster to execute the control parameters according to the execution priorities to eliminate the impact of the abnormal event.

[0006] In another aspect, an embodiment of the present invention further provides a server, including a processor and a machine-readable storage medium. The machine-readable storage medium is connected to the processor. The machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0007] Based on the above aspects, in the embodiment of the present application, by obtaining multi-modal perception data of multiple Internet of Things monitoring nodes, covering image monitoring data, sound monitoring data and environmental parameter monitoring data, and then performing cross-modal feature extraction and fusion processing on the multi-modal perception data to generate a comprehensive spatio-temporal feature map, the complementary nature between different modal data is fully utilized, and complex internal associations between the data can be discovered, thereby generating a more representative and comprehensive feature map, which can more accurately reflect the true state of the target monitoring area. By performing dynamic pattern matching on the comprehensive spatio-temporal feature map based on a preset abnormal pattern library, accurate identification of the types of abnormal events and their spatio-temporal distribution information within the target monitoring area is achieved. It can adapt to different monitoring scenarios and real-time changes. Compared with traditional static matching or simple threshold judgment methods, it has higher flexibility and adaptability, can more accurately capture complex and changeable abnormal events, and greatly improves the accuracy and reliability of abnormal detection. According to the type of abnormal event and spatio-temporal distribution information, a set of adapted control instructions is generated, and the control parameters and execution priorities for different Internet of Things execution devices are clarified. It can customize precise control strategies for different execution devices according to specific abnormal situations, ensure that each device plays its maximum role at the appropriate time, and further optimize the resource utilization efficiency by reasonably setting the execution priorities, making the entire control process more efficient and orderly, and being able to quickly and effectively eliminate the impact of abnormal events. Finally, the set of control instructions is distributed to the associated Internet of Things execution device cluster and triggers it to execute the control parameters according to the priority, realizing the automated and intelligent closed-loop operation of the entire monitoring and control system, greatly improving the response speed and processing effect for abnormal events, reducing the need for manual intervention, reducing human errors and response delays, and significantly enhancing the overall efficiency of the Internet of Things monitoring system. Description of the Drawings

[0008] Figure 1 It is a schematic flowchart of the execution process of the monitoring information recognition method based on the Internet of Things provided by the embodiment of the present invention.

[0009] Figure 2 It is a schematic hardware architecture diagram of the server provided by the embodiment of the present invention. Detailed Embodiments

[0010] The present invention will be specifically described below in conjunction with the accompanying drawings of the specification. Figure 1It is a schematic flowchart of a monitoring information recognition method based on the Internet of Things provided by an embodiment of the present invention. The monitoring information recognition method based on the Internet of Things will be introduced in detail below.

[0011] Step S110: Obtain multi-modal perception data collected in real time by multiple Internet of Things monitoring nodes in the target monitoring area. The multi-modal perception data includes image monitoring data, sound monitoring data, and environmental parameter monitoring data.

[0012] Taking the industrial security monitoring scenario as an example, in a target monitoring area such as an industrial factory building, multiple Internet of Things monitoring nodes are deployed. These Internet of Things monitoring nodes have multiple perception capabilities and are used to obtain multi-modal perception data.

[0013] Specifically, for the image monitoring data, high-definition cameras installed at various key positions in the factory building continuously capture monitoring images. For example, there are cameras above the production lines in the production workshop, in the aisles of the warehouse, and at the entrances and exits of the factory building. The cameras can capture the operating status of production equipment, the operation of workers, the stacking and handling of goods, etc. By continuously shooting, real-time image monitoring data is generated.

[0014] The sound monitoring data is collected by sound sensors installed in different areas of the factory building. In a noisy industrial environment, the sound sensors can monitor the sound of machine operation, abnormal sounds during equipment failures, the sound of workers operating tools, and possible abnormal collision sounds. For example, when a certain component of a large stamping machine starts to wear, it may emit a sharp friction sound, which will be captured by the sound sensors.

[0015] For the environmental parameter monitoring data, temperature and humidity sensors are distributed in the factory building to monitor the temperature and humidity of the environment. In some production processes sensitive to temperature and humidity, such as the manufacturing workshop of electronic components, changes in temperature and humidity may affect product quality. In addition, smoke sensors monitor whether there is smoke generation, which is of great significance for preventing fire hazards. For example, if an electrical device short-circuits and smokes, the smoke sensor will immediately detect the change in smoke concentration.

[0016] The above different types of sensors jointly collect environmental parameter monitoring data, providing multi-faceted source data information for comprehensively monitoring the safety status of the industrial factory building.

[0017] Step S120: Perform cross-modal feature extraction and fusion processing on the multi-modal perception data to generate a comprehensive spatio-temporal feature map corresponding to the target monitoring area.

[0018] Specifically, first, multi-scale spatial feature extraction is performed on the image monitoring data to obtain the visual semantic feature vector within the target monitoring area. Taking a production workshop as an example, the image frame sequence captured by the camera is first divided at a preset time interval. Assuming that every 5 seconds is an interval, multiple image frame sequences are obtained. Then, these image frame sequences are denoised and subjected to illumination equalization processing to eliminate the noise generated by light changes and equipment interference. For example, the flickering of workshop lights or the vibration generated by machine operation may affect the image quality. After processing, a clear and stable image frame sequence can be obtained.

[0019] Next, a convolutional neural network is used to perform hierarchical feature extraction on the processed image frame sequence. In this process, neural networks at different levels can identify local texture feature maps and global scene feature maps at different scales. For example, for the products on the production line, the neural network can identify the minute texture features on the product surface, and at the same time, it can also grasp the global scene features such as the layout of the entire production line and the overall arrangement of equipment.

[0020] Then, through the spatial pyramid pooling layer, multi-resolution feature fusion is performed on the local texture feature map and the global scene feature map to generate a fused spatial semantic feature matrix. For example, for the products on the assembly line, information such as the texture features of different parts of the product and the positional relationship of the whole on the assembly line is fused into this matrix. Then, sliding window aggregation in the time dimension is performed on the spatial semantic feature matrix to extract the dynamic change trend features of the target monitoring area within a continuous time period. Assuming that the step size of the sliding window is 1 second and the window size is 10 seconds, then the dynamic trends such as the movement and processing state changes of the products on the production line within 10 seconds can be captured. Finally, the dynamic change trend features are concatenated with the spatial semantic feature matrix within the current time window to generate the visual semantic feature vector.

[0021] For the sound monitoring data, voiceprint spectrum decomposition is performed to extract the acoustic event feature vector within the target monitoring area. In an industrial plant, the sounds emitted by different devices have different voiceprint spectra. For example, the sound spectrum of a stamping machine is significantly different from that of a conveyor belt. Through voiceprint spectrum decomposition, the sound emitted by a specific device can be accurately identified. When the sound of the device is abnormal, this abnormal acoustic feature will be extracted.

[0022] Perform time series analysis on the environmental parameter monitoring data, extract the environmental fluctuation feature vectors within the target monitoring area, and construct a spatio-temporal correlation matrix corresponding to the environmental fluctuation feature vectors. Taking temperature and humidity as an example, temperature and humidity sensors continuously collect data. Through time series analysis, the fluctuation patterns of temperature and humidity within a day can be discovered. If the temperature and humidity suddenly increase or decrease abnormally within a certain period, such fluctuation features will be extracted, and a spatio-temporal correlation matrix will be constructed based on the changes in temperature and humidity at different positions in the factory building. For example, the changes in temperature and humidity in the area near the heating equipment may be different from those in the area far from the heating equipment.

[0023] Input the visual semantic feature vectors, acoustic event feature vectors, and spatio-temporal correlation matrix into the cross-modal fusion network. Through the attention weight assignment layer in the cross-modal fusion network, dynamically assign the contribution weights of each modal feature in the spatio-temporal dimension. For example, when judging whether there are potential equipment failure hazards, the image monitoring data may have a greater contribution weight to judging the appearance state of the equipment in the initial stage. When abnormal sounds start to appear in the equipment, the contribution weight of the sound monitoring data may increase. At the same time, if the temperature change in the environmental parameter monitoring data is related to the equipment failure, its weight will also be adjusted accordingly. Based on these contribution weights, perform weighted splicing on the visual semantic feature vectors, acoustic event feature vectors, and spatio-temporal correlation matrix to generate a comprehensive spatio-temporal feature map, which contains the spatial event distribution and environmental parameter evolution trend in the target monitoring area within a continuous time window, such as the spatial distribution of the operating state of the equipment in the factory building and the change trend of temperature and humidity over time.

[0024] Step S130, based on the preset abnormal pattern library, perform dynamic pattern matching on the comprehensive spatio-temporal feature map to determine at least one abnormal event type and its spatio-temporal distribution information existing in the target monitoring area.

[0025] In this embodiment, the preset abnormal pattern library is constructed based on historical monitoring data. For example, in past industrial security monitoring, an abnormal event occurred where a certain large-scale equipment suddenly stopped operating. The multi-modal perception data at that time included images of the equipment smoking captured by the camera, abnormal noises collected by the sound sensor, and a sharp increase in the temperature around the equipment detected by the temperature and humidity sensor, and the event type was marked as equipment failure, and the disposal record was that the equipment returned to normal after emergency repair by the maintenance personnel.

[0026] Load multiple abnormal event templates from the preset abnormal pattern library. Each abnormal event template includes the reference spatio-temporal features, event type labels, and influence range parameters corresponding to historical abnormal events. In this industrial factory building scenario, there may be different types of abnormal event templates such as equipment failure, fire hazard, and illegal intrusion.

[0027] Calculate the similarity matrix between the comprehensive spatio-temporal feature map and the reference spatio-temporal features of each abnormal event template. Suppose the current comprehensive spatio-temporal feature map shows that the temperature suddenly rises in a certain area and the device makes abnormal noises. By calculating the similarity matrix with the reference spatio-temporal features of the abnormal event template of equipment failure, it is found that there is a high similarity between the two. Based on the similarity matrix, a set of candidate abnormal event templates with similarity higher than the preset threshold is selected. For example, if the threshold is set to 0.8, and the calculated similarity exceeds 0.8, then the abnormal event template of equipment failure will be included in the set of candidate abnormal event templates.

[0028] Perform spatio-temporal alignment processing on each candidate abnormal event template in the set of candidate abnormal event templates, and map the timestamps of the comprehensive spatio-temporal feature map and the candidate abnormal event templates to a unified time coordinate system. For example, in the judgment of equipment failure, it is necessary to ensure that the time when the temperature rises and the abnormal noise appears in the comprehensive spatio-temporal feature map corresponds to the time in the candidate abnormal event template, so as to accurately judge whether it is the same type of equipment failure.

[0029] According to the feature difference degree after spatio-temporal alignment, determine the target abnormal event template that matches the comprehensive spatio-temporal feature map from the set of candidate abnormal event templates, and use the event type label and the influence range parameter of the target abnormal event template as the abnormal event type and spatio-temporal distribution information. If it is determined to be an abnormal event of the equipment failure type, the spatio-temporal distribution information may include the specific location (starting position) of the faulty equipment in the plant, the possible diffusion path of the fault impact (such as whether it will affect the adjacent production line), and the fault impact intensity attenuation curve (for example, whether the fault impact gradually expands or shrinks over time, etc.).

[0030] Step S140, generate a set of regulation instructions adapted to the abnormal event type according to the abnormal event type and the spatio-temporal distribution information, where the set of regulation instructions includes control parameters and execution priorities for different Internet of Things execution devices.

[0031] In this embodiment, according to the abnormal event type, an emergency response strategy associated with the abnormal event type can be called from the Internet of Things device strategy library. Taking equipment failure as an example, the emergency response strategies in the Internet of Things device strategy library include equipment linkage rules, parameter adjustment rules, and execution condition constraints. The equipment linkage rules may involve operations such as notifying maintenance personnel and stopping the operation of related equipment to prevent the expansion of the fault; the parameter adjustment rules may include adjusting the operating parameters of other equipment related to the faulty equipment, such as reducing the power of adjacent equipment to reduce the load on the faulty equipment; the execution condition constraints may stipulate that certain operations are only executed when the temperature around the faulty equipment reaches a certain threshold or the faulty equipment stops running.

[0032] Based on the influence intensity decay curve in the spatio-temporal distribution information, determine the optimal startup time and action duration of each execution device in the Internet of Things execution device cluster. If the influence intensity of the device failure gradually increases over time, then the optimal startup time of the maintenance device and related auxiliary devices should be as soon as possible, and the action duration should be long enough to ensure that the failure is completely repaired.

[0033] According to the device linkage rule and parameter adjustment rule, allocate control parameters to each execution device. For example, for a maintenance robot, its operating power should be adjusted according to the maintenance difficulty of the faulty device, the action direction should point to the specific location of the faulty device, and the coverage range should ensure that it can reach all possible fault points of the faulty device.

[0034] Combined with the execution condition constraints and the optimal startup time, calculate the estimated value of resource consumption of each execution device in the target monitoring area, and optimize and calibrate the control parameters based on the estimated value of resource consumption. For example, the energy consumption of a maintenance robot is different at different powers. If reducing the power can reduce the energy consumption under the premise of meeting the maintenance requirements, then optimize and calibrate its operating power.

[0035] According to the priority weight of the influence intensity decay curve and the estimated value of resource consumption, assign an execution priority to the control parameters of each execution device, and generate a set of regulation instructions. For example, assign a higher execution priority to the device that can quickly reduce the influence intensity of the failure. At the same time, considering the resource consumption situation, if a certain device is helpful for reducing the influence intensity of the failure but consumes too much resources, its execution priority may be reduced.

[0036] Step S150, distribute the set of regulation instructions to the Internet of Things execution device cluster associated in the target monitoring area, and trigger the Internet of Things execution device cluster to execute the control parameters according to the execution priority to eliminate the influence of the abnormal event.

[0037] In this embodiment, the set of regulation instructions can be divided into multiple instruction batches according to the execution priority, and each instruction batch is sent to the corresponding execution device sub-cluster in the order of batches. For example, first send the instructions related to the emergency stop of the faulty device as the first batch of instructions to the corresponding execution device sub-cluster (such as the device control system), and then send the instructions related to the preparation work of the maintenance device as the second batch of instructions to the maintenance device sub-cluster.

[0038] When sending each batch of instructions, the response status of the execution device sub-cluster is monitored in real time. If it is detected that the target execution device does not return a response confirmation signal within the preset time, a standby device replacement strategy is initiated. For example, if the maintenance robot does not return a response confirmation signal within the specified time, it indicates that it may malfunction and be unable to execute the instructions. A set of candidate standby devices with the same functional attributes as the target execution device (maintenance robot) is screened from the Internet of Things execution device cluster, such as other maintenance robots of the same model. The current working status, remaining resource capacity, and physical distance from the target monitoring area (location of the faulty device) of each candidate standby device are obtained. If a standby maintenance robot is currently in an idle state, has sufficient remaining resource capacity, and is close to the faulty device, its availability score will be relatively high. A comprehensive replacement priority list is generated based on these factors, and the candidate standby device with the highest score is selected as the target standby device. The control parameters corresponding to the target execution device are scaled by a preset ratio and then allocated to the target standby device. The scaled control parameters and the new execution priority are sent to the target standby device, and the device identification and parameter records in the regulation instruction set are updated.

[0039] After the execution device sub-cluster completes the parameter execution of the current batch of instructions, real-time feedback data of the target monitoring area is collected, and the change rate of the impact intensity of the abnormal event is evaluated based on the real-time feedback data. If it is a device failure, information such as the temperature around the device and the operating status of the device is detected again through sensors, and it is calculated whether the impact intensity of the failure is decaying as expected. If the change rate of the impact intensity does not reach the expected decay rate, the control parameters and execution priorities in the subsequent batches of instructions are dynamically adjusted according to the real-time feedback data. For example, if the impact intensity of the failure does not decrease as expected, it may be necessary to increase the operating power of the maintenance equipment or adjust the maintenance strategy, and accordingly increase the execution priorities of some execution devices.

[0040] After all batches of instructions are executed, a processing report of the abnormal event is generated and uploaded to the Internet of Things monitoring platform for archiving. Among them, the processing report includes information such as the type of the abnormal event, the time and location of its occurrence, the operation status of the execution device during the processing, and the final processing result of the abnormal event, so as to facilitate subsequent query and analysis and provide a reference basis for the processing of future similar abnormal events.

[0041] Based on the above steps, in the embodiment of the present application, multi-modal perception data of multiple Internet of Things monitoring nodes are obtained, including image monitoring data, sound monitoring data, and environmental parameter monitoring data. Then, cross-modal feature extraction and fusion processing are performed on the multi-modal perception data to generate a comprehensive spatio-temporal feature map. By fully utilizing the complementarity between different modal data, complex internal correlations between the data can be discovered, thereby generating a more representative and comprehensive feature map, which can more accurately reflect the true state of the target monitoring area. Dynamic pattern matching is performed on the comprehensive spatio-temporal feature map based on a preset abnormal pattern library, realizing the accurate identification of the types of abnormal events and their spatio-temporal distribution information in the target monitoring area. It can adapt to different monitoring scenarios and real-time changes. Compared with traditional static matching or simple threshold judgment methods, it has higher flexibility and adaptability, can capture complex and changeable abnormal events more accurately, and greatly improves the accuracy and reliability of abnormal detection. According to the types of abnormal events and spatio-temporal distribution information, an adapted set of control instructions is generated, and the control parameters and execution priorities for different Internet of Things execution devices are specified. It can customize precise control strategies for different execution devices according to specific abnormal situations, ensure that each device plays its maximum role at the appropriate time, and further optimize the resource utilization efficiency by reasonably setting the execution priorities, making the entire control process more efficient and orderly, and can quickly and effectively eliminate the impact of abnormal events. Finally, the set of control instructions is distributed to the associated Internet of Things execution device cluster and triggers it to execute the control parameters according to the priorities, realizing the automated and intelligent closed-loop operation of the entire monitoring and control system, greatly improving the response speed and processing effect for abnormal events, reducing the need for manual intervention, reducing human errors and response delays, and significantly enhancing the overall efficiency of the Internet of Things monitoring system.

[0042] In a possible implementation manner, step S120 includes: Step S121, performing multi-scale spatial feature extraction on the image monitoring data to obtain a visual semantic feature vector in the target monitoring area, and performing voiceprint spectrum decomposition on the sound monitoring data to extract an acoustic event feature vector in the target monitoring area.

[0043] For example, taking the equipment monitoring in a production workshop as an example, for multi-scale spatial feature extraction, the image monitoring data first needs to be preprocessed. Cameras installed at different positions in the workshop continuously collect image data, which contains information such as the appearance of the equipment, its operating status, and the surrounding environment. These image data are divided at certain preset time intervals. For example, an image frame sequence is formed with an interval of every 10 seconds. Since there may be factors such as uneven light and equipment vibration in the industrial environment, resulting in image noise and unstable lighting, each image frame sequence needs to be denoised and light equalized. The processed image frame sequence can more accurately reflect the actual situation in the workshop.

[0044] Next, a convolutional neural network is used for hierarchical feature extraction. In this process, different layers of the convolutional neural network can identify local texture feature maps and global scene feature maps at different scales. For large production equipment in the workshop, the local texture feature map may reflect microscopic features such as wear marks on the equipment surface and the connection status of components, while the global scene feature map can present macroscopic information such as the position of the entire equipment in the workshop, its relative layout with other equipment, and the surrounding personnel activities.

[0045] Then, through the spatial pyramid pooling layer, multi-resolution feature fusion is performed on the local texture feature map and the global scene feature map. Taking the product processing equipment on the production line as an example, different components of this product processing equipment may have different features at different resolutions. Through this fusion method, these features at different resolutions can be integrated into a fused spatial semantic feature matrix, which contains the semantic information of each component of the equipment and the whole in the workshop.

[0046] Furthermore, a sliding window aggregation in the time dimension is performed on the spatial semantic feature matrix to extract the dynamic change trend features of the target monitoring area in a continuous time period. Suppose the step size of the sliding window is 5 seconds and the window size is 30 seconds. During this time period, the production equipment may go through different states such as startup, stable operation, and shutdown. Through the sliding window aggregation, the dynamic changes of the equipment state during this time can be captured, such as whether the running speed of the equipment is stable and whether there are abnormal pauses. Finally, the dynamic change trend features are concatenated with the spatial semantic feature matrix within the current time window, so as to obtain the visual semantic feature vector of the target monitoring area, which can comprehensively reflect the visual features of the equipment and the environment in the workshop and their changes over time.

[0047] Meanwhile, perform voiceprint spectrum decomposition on the voice monitoring data to extract the acoustic event feature vectors. In an industrial plant, there are various sound sources. Different devices will emit sounds with specific voiceprint spectrums due to differences in their structures, operating principles, and working states. For example, a cutting device will emit a stable sound with specific frequencies and amplitudes during normal operation. When the cutting tool wears out, the frequency and amplitude of the sound will change, and the voiceprint spectrum will also change accordingly. Through voiceprint spectrum decomposition technology, the collected voice monitoring data can be decomposed into spectral components with different frequencies and amplitudes, thereby accurately extracting the feature vectors related to specific acoustic events. This acoustic event feature vector can reflect the characteristics of various sound events in the workshop, such as whether the equipment is operating normally, whether there are loose or collided parts, etc.

[0048] Step S122: Perform time series analysis on the environmental parameter monitoring data, extract the environmental fluctuation feature vectors in the target monitoring area, and construct a spatio-temporal correlation matrix corresponding to the environmental fluctuation feature vectors.

[0049] In an industrial plant, environmental parameters such as temperature, humidity, and smoke concentration have an important impact on the production process and equipment operation. Devices such as temperature and humidity sensors and smoke sensors continuously collect environmental parameter monitoring data. Taking temperature as an example, during a day's production process, due to equipment heat dissipation, the operation of the ventilation system, and the influence of the external environment, the temperature will show a certain fluctuation pattern. Through time series analysis, these temperature data that change over time can be converted into environmental fluctuation feature vectors. At the same time, due to the different distributions of equipment and ventilation conditions in different areas of the plant, there are also differences in temperature changes in space. Constructing a spatio-temporal correlation matrix corresponding to the environmental fluctuation feature vectors can describe the relationship between temperature at different times and spatial positions. For example, the temperature fluctuation may be larger in the area near large heat-generating equipment, while the temperature is relatively stable in the area far from the equipment. The spatio-temporal correlation matrix can accurately reflect this spatio-temporal temperature correlation relationship.

[0050] Step S123: Input the visual semantic feature vectors, the acoustic event feature vectors, and the spatio-temporal correlation matrix into a cross-modal fusion network, and dynamically allocate the contribution weights of each modal feature in the spatio-temporal dimension through the attention weight assignment layer in the cross-modal fusion network.

[0051] Step S124: Based on the contribution weights, perform weighted splicing on the visual semantic feature vectors, the acoustic event feature vectors, and the spatio-temporal correlation matrix to generate the comprehensive spatio-temporal feature map. Among them, the comprehensive spatio-temporal feature map contains the spatial event distribution and the environmental parameter evolution trend in the target monitoring area within a continuous time window.

[0052] In the cross-modal fusion network, the attention weight allocation layer plays a crucial role in dynamically allocating the contribution weights of features of each modality in the spatio-temporal dimension. In the monitoring scenario of industrial plants, different abnormal events may have different degrees of dependence on features of different modalities. For example, when judging whether there is external damage to equipment, the contribution weight of the visual semantic feature vector may be relatively large; while when judging whether there are abnormal sounds caused by internal faults in equipment, the contribution weight of the acoustic event feature vector will increase; if it is a fault related to environmental factors, such as equipment failure caused by too high temperature, then the weight of the spatio-temporal correlation matrix corresponding to the environmental fluctuation feature vector will be more important. After the attention weight allocation layer dynamically adjusts the weights of features of each modality according to the actual situation, the visual semantic feature vector, the acoustic event feature vector, and the spatio-temporal correlation matrix are weighted and concatenated based on these contribution weights, so as to generate a comprehensive spatio-temporal feature map, which contains the spatial event distribution and the evolution trend of environmental parameters in the target monitoring area within a continuous time window. For example, it can display information such as the operating state distribution of equipment in the workshop at different times, the activity trajectories of personnel, and the change trends of environmental parameters such as temperature and humidity.

[0053] In a possible implementation manner, step S130 includes: Step S131, loading a plurality of abnormal event templates from the preset abnormal pattern library, where each abnormal event template includes the reference spatio-temporal features, event type labels, and influence range parameters corresponding to historical abnormal events.

[0054] In this embodiment, the preset abnormal pattern library is constructed based on the historical monitoring data of industrial plants. During past industrial security monitoring, a large amount of abnormal event data has been recorded. For example, there was once a short-circuit fault in an electrical control cabinet. The monitoring data at that time included images showing the control cabinet smoking and flames flashing in the image monitoring data, the strong arc sound during the short circuit captured in the sound monitoring data, and the discovery of a sharp rise in temperature and a rapid increase in smoke concentration in the environmental parameter monitoring data in that area. This event was marked as an electrical equipment short-circuit fault type, and the disposal process was recorded, such as cutting off the power supply, using fire extinguishing equipment, and maintenance personnel for repair, etc.

[0055] The abnormal event template includes the reference spatio-temporal features, event type labels, and influence range parameters corresponding to various historical abnormal events. In the industrial plant scenario, the abnormal event template may include types such as equipment failures (such as mechanical failures, electrical failures), fire hazards, and dangerous gas leaks. The reference spatio-temporal features in each template are the feature summaries of historical abnormal events in different times and spaces. The event type label clarifies the type of the abnormal event, and the influence range parameter describes information such as the area range and the number of equipment that the abnormal event may affect.

[0056] Step S132: Calculate the similarity matrix between the comprehensive spatio-temporal feature map and the reference spatio-temporal features of each abnormal event template, and screen out a set of candidate abnormal event templates with similarity higher than a preset threshold based on the similarity matrix.

[0057] Suppose the current comprehensive spatio-temporal feature map shows that the temperature has risen, there is smoke, and the equipment makes abnormal noises in a certain area of the workshop. By calculating the similarity matrix between the reference spatio-temporal features of various abnormal event templates (such as fire hazard templates, equipment failure templates, etc.), the similarity degree between them can be quantified. Based on this similarity matrix, a set of candidate abnormal event templates with similarity higher than a preset threshold is screened out. For example, set the similarity threshold to 0.8. If the similarity with the fire hazard template reaches 0.85 and the similarity with the equipment failure template is 0.7, then the fire hazard template will be selected into the set of candidate abnormal event templates.

[0058] Step S133: Perform spatio-temporal alignment processing on each candidate abnormal event template in the set of candidate abnormal event templates, and map the time stamps of the comprehensive spatio-temporal feature map and the candidate abnormal event templates to a unified time coordinate system.

[0059] There may be differences in time and space scales when different historical abnormal events are recorded. For example, for a fire hazard event, the time in the historical record may be in minutes, while the time scale of the current comprehensive spatio-temporal feature map may be in seconds; spatially, the position in the historical record may be relative to an old coordinate system, while the current coordinate system may have been updated. Through spatio-temporal alignment processing, the time stamps of the comprehensive spatio-temporal feature map and the candidate abnormal event templates are mapped to a unified time coordinate system. For example, convert the current time stamp in seconds to a time scale corresponding to minutes in the historical record, and at the same time convert the current spatial coordinates to the same coordinate system as in the historical record to ensure comparison in the same spatio-temporal framework.

[0060] Step S134: Determine the target abnormal event template that matches the comprehensive spatio-temporal feature map from the set of candidate abnormal event templates according to the feature difference degree after spatio-temporal alignment, and use the event type label and influence range parameter of the target abnormal event template as the abnormal event type and the spatio-temporal distribution information.

[0061] Among them, the spatio-temporal distribution information includes the starting position, diffusion path, and influence intensity attenuation curve of the abnormal event in the target monitoring area.

[0062] For example, after spatio-temporal alignment, if it is found that the difference degree with the characteristics of the fire hazard template is the smallest, then it is determined that the abnormal event is of the fire hazard type. Its spatio-temporal distribution information includes the starting position of the fire hazard in the workshop, which may be an area where electrical equipment is concentrated; the diffusion path may be along the ventilation duct or in the direction close to the stacking of flammable materials; the influence intensity attenuation curve may show that over time, if no measures are taken, the influence range of the fire hazard will expand at a certain speed, such as the area spread per minute and other information. These spatio-temporal distribution information is crucial for subsequent targeted response measures.

[0063] In a possible implementation manner, step S140 includes: Step S141, according to the abnormal event type, call the emergency response strategy associated with the abnormal event type from the Internet of Things device policy library, and the emergency response strategy includes device linkage rules, parameter adjustment rules, and execution condition constraints.

[0064] Taking the occurrence of a fire hazard in an industrial plant as an example, the emergency response strategy in the Internet of Things device policy library includes device linkage rules, parameter adjustment rules, and execution condition constraints, etc. For the abnormal event type of fire hazard, the device linkage rules stipulate the collaborative operations between relevant devices. For example, the linkage relationship between the fire alarm system, fire extinguishing equipment (such as fire extinguishers, fire sprinklers), ventilation equipment, and production equipment related to the dangerous area. After the fire alarm system detects a fire hazard, it should immediately trigger the preparation work of the fire extinguishing equipment. At the same time, the ventilation equipment needs to adjust the ventilation mode according to the fire situation and smoke condition, which may be to increase the ventilation volume to discharge the smoke, but avoid fanning the flames. The production equipment related to the dangerous area needs to stop running according to the predetermined rules to prevent greater losses caused by the fire.

[0065] The parameter adjustment rules have different parameter settings for different devices in response to fire hazards. The operating power of the fire sprinkler needs to be adjusted according to the severity of the fire hazard. If the temperature in the fire hazard area rises rapidly and the smoke concentration is high, it may mean a large fire, then the operating power of the fire sprinkler should be increased accordingly to ensure that enough fire extinguishing agent is sprayed. The action direction of the fire sprinkler should accurately point to the starting position of the fire hazard and its possible diffusion path, and the coverage range should ensure that it can cover the area where the fire may spread. The operating power of the ventilation equipment also needs to be adjusted according to the diffusion situation of the smoke, and its action direction should be conducive to the discharge of the smoke, and the coverage range should include the entire area that may be affected by the smoke.

[0066] Execution condition constraints stipulate the conditions under which a device performs an operation. For example, the activation condition of a fire sprinkler may be that the fire alarm system detects that the temperature exceeds a certain threshold and the smoke concentration reaches a certain standard; the adjustment condition of a ventilation device may be to adjust according to the change of smoke concentration feedback by the smoke sensor within a certain time after a fire alarm.

[0067] Step S142: Based on the influence intensity decay curve in the spatio-temporal distribution information, determine the optimal start time and action duration of each execution device in the IoT execution device cluster.

[0068] For example, for a fire hazard event, the influence intensity decay curve reflects the development of the fire over time if no measures are taken or different measures are taken. Suppose the influence intensity decay curve shows that within the first 5 minutes after a fire breaks out, if no effective measures are taken, the fire will spread at a relatively fast rate. According to this influence intensity decay curve, the optimal start time of the fire extinguishing equipment should be as soon as possible after the fire is detected, for example, it should be started within 1 minute. The action duration of the fire extinguishing equipment should be determined according to the expected development of the fire. If it is expected that the fire can be effectively controlled within 10 minutes, then the action duration of the fire extinguishing equipment should be set to at least 10 minutes to ensure that the fire is completely extinguished. The optimal start time of the ventilation equipment should also be as early as possible to discharge the smoke as soon as possible, and its action duration should continue until the smoke concentration drops below the safety standard.

[0069] Step S143: According to the device linkage rule and the parameter adjustment rule, allocate the control parameters for each execution device, and the control parameters include operating power, action direction, and coverage range.

[0070] Taking a fire sprinkler as an example, its operating power is allocated according to the size of the fire. If the fire is in the initial stage, the operating power can be set at a relatively low level, for example, the amount of fire extinguishing agent sprayed per minute is 5 liters; when the fire develops to a certain extent, the operating power is increased to 10 liters per minute. The action direction of the fire sprinkler is adjusted according to the starting position and spreading path of the fire hazard. For example, if the fire starts in the southeast corner of the workshop and has a tendency to spread northward, then the action direction of the fire sprinkler should be adjusted to the southeast direction and cover the area extending northward from the southeast corner. Its coverage range should be set according to the area where the fire may spread, for example, a circular area with a coverage radius of 5 meters. For the ventilation equipment, the operating power is adjusted according to the smoke concentration. If the smoke concentration is high, the operating power is increased to 80% of the maximum ventilation volume, the action direction is towards the area with the thickest smoke, and the coverage range should cover the entire workshop.

[0071] Step S144: Combine the execution condition constraints and the optimal start time to calculate the estimated resource consumption of each execution device in the target monitoring area, and optimize and calibrate the control parameters based on the estimated resource consumption.

[0072] Taking a fire sprinkler as an example, if it is started within 1 minute of the optimal start time and runs at the set operating power for 10 minutes, the total amount of extinguishing agent consumed can be calculated, which is the estimated resource consumption of the fire sprinkler. At the same time, considering the energy supply of the fire sprinkler (if it is an electric sprinkler) or the reserve of extinguishing agent, if the estimated resource consumption is too high, it may lead to insufficient resources during the subsequent fire extinguishing process. Then it is necessary to optimize and calibrate the control parameters, such as reducing the operating power or adjusting the action duration. The same is true for ventilation equipment. According to its optimal start time, operating power and action duration, the estimated resource consumption of power consumption is calculated. If it exceeds the limit of the equipment's energy reserve or energy supply, it is necessary to adjust control parameters such as the operating power or action duration.

[0073] Step S145: According to the priority weight of the influence intensity decay curve and the estimated resource consumption, assign the execution priority to the control parameters of each execution device to generate the set of regulation instructions.

[0074] For example, for a fire hazard event, the fire extinguishing equipment is the most direct and crucial for controlling the fire and reducing the influence intensity, so its priority weight in the influence intensity decay curve is relatively high. If the estimated resource consumption of the fire extinguishing equipment is within an acceptable range, its execution priority will be set to the highest. Although the ventilation equipment is also very important, compared with the fire extinguishing equipment, its role in reducing the influence intensity is slightly less, and if the resource consumption is large, its execution priority may be slightly lower. Based on these factors, the execution priority is assigned to each execution device, thereby generating a set of regulation instructions. This set of regulation instructions includes the control parameters and execution priorities of each execution device, and is used to guide the IoT execution device cluster to respond to the fire hazard event.

[0075] Among them, step S150 includes: Step S151: According to the execution priority, divide the set of regulation instructions into multiple instruction batches, and send each instruction batch to the corresponding execution device sub-cluster in the order of batches.

[0076] For example, for a fire hazard event, first, send the instructions related to fire extinguishing equipment as the first batch of instructions to the fire extinguishing equipment sub-cluster because the execution priority of fire extinguishing equipment is the highest. This batch of instructions includes control parameters such as the start time, operating power, action direction, and coverage range of the fire extinguishing equipment. Then, send the instructions related to ventilation equipment as the second batch of instructions to the ventilation equipment sub-cluster, and the instructions include the control parameters of the ventilation equipment.

[0077] Step S152, when sending each batch of the instructions, monitor the response status of the execution equipment sub-cluster in real time. If it is detected that the target execution equipment does not return a response confirmation signal within the preset time, start the standby equipment replacement strategy, and re-allocate the control parameters corresponding to the target execution equipment to the standby execution equipment.

[0078] For example, after sending instructions to a certain fire sprinkler, if no response confirmation signal from the fire sprinkler is received within the preset 10 seconds, it indicates that the fire sprinkler may be faulty. At this time, start the standby equipment replacement strategy. Screen a set of standby fire sprinklers with the same functional attributes as the fire sprinkler from the Internet of Things execution equipment cluster. Obtain the current working status of each standby fire sprinkler (such as whether it is in an idle state), the remaining resource capacity (such as the amount of remaining fire extinguishing agent), and the physical distance from the target monitoring area (fire hazard area). If a standby fire sprinkler is currently in an idle state, has sufficient remaining fire extinguishing agent, and is close to the fire hazard area, then its availability score will be higher. Generate a comprehensive replacement priority list based on these factors, and select the standby fire sprinkler with the highest score in the comprehensive replacement priority list as the target standby equipment. Scale the control parameters corresponding to the faulty fire sprinkler by a preset ratio and allocate them to the target standby equipment. For example, if the original control parameter is to spray 10 liters of fire extinguishing agent per minute, according to the remaining resource capacity of the target standby equipment and other situations, scale the control parameter to spray 8 liters of fire extinguishing agent per minute. Send the scaled control parameters and the new execution priority to the target standby equipment, and update the device identification and parameter records in the regulation instruction set.

[0079] Step S153, when the execution equipment sub-cluster completes the parameter execution of the current batch of instructions, collect the real-time feedback data of the target monitoring area, and evaluate the change rate of the influence intensity of the abnormal event based on the real-time feedback data.

[0080] For example, for a fire hazard event, after the first batch of instructions is executed by the fire extinguishing equipment sub-cluster, real-time feedback data of the target monitoring area (fire hazard area) is collected through devices such as temperature sensors and smoke sensors. Calculate the change rate of the impact intensity of the fire, for example, measured by the rate of temperature drop, the rate of decrease in smoke concentration, etc. If the temperature does not drop significantly or the smoke concentration does not decrease as expected within a certain period after the instructions are executed by the fire extinguishing equipment, it indicates that the change rate of the impact intensity does not reach the expected attenuation rate.

[0081] Step S154, if the change rate of the impact intensity does not reach the expected attenuation rate, then dynamically adjust the control parameters and the execution priority in the subsequent instruction batches according to the real-time feedback data.

[0082] For example, if the impact intensity of the fire does not decrease as expected, it may be necessary to increase the operating power of the fire extinguishing equipment or adjust the action direction. For example, if it is found that the fire in a certain area is not effectively controlled, it may be necessary to increase the operating power of the fire sprinklers near that area from spraying 8 liters of fire extinguishing agent per minute to spraying 12 liters of fire extinguishing agent per minute. At the same time, since the role of the fire extinguishing equipment is more critical, it may be necessary to increase its execution priority to ensure that it can execute the control parameters more effectively. For the ventilation equipment, it may also be necessary to adjust control parameters such as its operating power and action direction according to the spread of the smoke.

[0083] Step S155, after all the instruction batches are executed, generate a processing report for the abnormal event, and upload the processing report to the Internet of Things monitoring platform for archiving.

[0084] For example, the processing report includes detailed information about the fire hazard event, such as the starting location of the fire hazard, the discovery time, the affected area, the emergency response strategies taken, the operation conditions of the execution devices (including the execution conditions of control parameters such as the start time, operating power, action direction, and coverage range of each execution device), and the final processing result of the abnormal event (such as whether the fire is extinguished, the extinguishing time, and whether there is equipment damage). Upload this processing report to the Internet of Things monitoring platform for archiving for future query and analysis, providing a reference basis for the handling of similar fire hazard events in the future.

[0085] In a possible implementation manner, step S121 includes: Step S1211, divide the image monitoring data into multiple image frame sequences at a preset time interval, and perform denoising and illumination equalization processing on each of the image frame sequences to obtain the processed image frame sequences.

[0086] For example, cameras installed at various locations in the factory building continuously collect image data, which contains a lot of information such as production equipment, staff, and goods. According to a specific preset time interval, for example, dividing the image data every 5 seconds, multiple image frame sequences are formed. In an industrial environment, due to factors such as the flickering of lighting equipment and the vibration generated by equipment operation, the images may have noise and uneven illumination. Professional image processing algorithms are used to denoise each image frame sequence, removing noise such as spots and miscellaneous colors generated by environmental interference, and at the same time performing illumination equalization processing to make the brightness and contrast of the image more balanced in different regions. The processed image frame sequence obtained in this way can more accurately reflect the actual situation in the factory building.

[0087] Step S1212, use a convolutional neural network to perform hierarchical feature extraction on the processed image frame sequence to obtain local texture feature maps and global scene feature maps at different scales.

[0088] For example, for the monitoring of a production workshop, different layers of the convolutional neural network can identify features at different scales. For example, when identifying production equipment, the shallower neural network layers can capture the local texture feature maps on the surface of the equipment, such as microscopic local texture features like small scratches on the equipment shell and wear of the markings. The deeper neural network layers, on the other hand, can obtain the global scene feature map, which includes macroscopic scene information such as the position layout of the equipment in the entire workshop, the relative position relationship with other equipment, and the distribution of surrounding staff.

[0089] Step S1213, perform multi-resolution feature fusion on the local texture feature map and the global scene feature map through a spatial pyramid pooling layer to generate a fused spatial semantic feature matrix.

[0090] In the actual monitoring of the factory building, features at different resolutions contain different semantic information. For example, for product inspection on a production line, the local detailed features of the product are clearer at high resolution, while its overall position relationship on the production line can be better reflected at low resolution. The spatial pyramid pooling layer fuses these local texture feature maps and global scene feature maps at different resolutions, integrating features at various scales into a fused spatial semantic feature matrix, which contains semantic information of various objects in the factory building from micro to macro, from local to whole, such as the state of equipment components and the spatial relationship of equipment in the factory building.

[0091] Step S1214, perform sliding window aggregation on the spatial semantic feature matrix in the time dimension to extract the dynamic change trend features of the target monitoring area in a continuous time period.

[0092] During the industrial production process, the situation in the factory building changes continuously over time. Set the step size and window size of the sliding window. For example, the step size is 2 seconds and the window size is 10 seconds. Within this 10-second time window, the production equipment may go through different states such as startup, acceleration, stable operation, deceleration, etc., and the staff may move between different areas. Through sliding window aggregation, the dynamic changes of these objects within this continuous time period can be captured, such as the changing trend of the operating speed of the equipment, the changing trend of the activity trajectory of the staff, and other dynamic change trend characteristics.

[0093] Step S1215, splice the dynamic change trend characteristics with the spatial semantic feature matrix within the current time window to generate the visual semantic feature vector.

[0094] In the monitoring scenario of the factory building, this visual semantic feature vector completely contains the visual information in the target monitoring area and its dynamic changes over time. For example, it contains both the spatial semantic information such as the appearance state and position relationship of the production equipment at a certain moment, and the information in the time dimension such as the changing trend of the operating state of the equipment in the previous period of time.

[0095] In a possible implementation manner, the method for constructing the preset abnormal mode library includes: Step S210, collect multiple historical case data marked as abnormal events in the historical monitoring data. Each historical case data includes multi-modal perception data at the time of the abnormality, an event type label, and a disposal record.

[0096] During the long-term operation of the industrial factory building, a large amount of historical monitoring data has been accumulated. For the construction of the preset abnormal mode library, first, multiple historical case data marked as abnormal events in the historical monitoring data need to be collected. These historical case data cover various possible abnormal situations in the factory building.

[0097] For example, there was once an electrical equipment short-circuit fault event in the production workshop. The multi-modal perception data at that time included images in the image monitoring data showing smoke and flashes of fire in the electrical control cabinet, sound monitoring data capturing strong arc sounds and abnormal buzzing sounds emitted by the equipment, and environmental parameter monitoring data showing a sharp increase in temperature and a rapid increase in smoke concentration in this area. This event was marked as an electrical equipment short-circuit fault type, and its disposal record included a series of operations. For example, the maintenance personnel arrived at the scene 3 minutes after receiving the alarm, first cut off the power supply of the equipment, then used a fire extinguisher to extinguish the flames that might cause a fire, and after 15 minutes of maintenance, replaced the short-circuited electrical components, and the equipment returned to normal operation.

[0098] Step S220: Extract cross-modal features from the multi-modal perception data in the historical case data to generate historical spatio-temporal feature maps corresponding to each piece of historical case data.

[0099] Taking the short-circuit fault of the electrical equipment mentioned above as an example, for the image monitoring data, visual semantic feature vectors are obtained through multi-scale spatial feature extraction. This vector contains visual feature information such as the location of smoke in the electrical control cabinet and the range of the fire. For the sound monitoring data, the acoustic event feature vector obtained after sound spectrogram decomposition can reflect the frequency and amplitude of the arc sound and the characteristics of abnormal buzzing sounds. For the environmental parameter monitoring data, the environmental fluctuation feature vector and spatio-temporal correlation matrix obtained through time series analysis can reflect the speed of temperature increase and the spatial diffusion of smoke concentration, etc. These feature vectors and matrices of different modalities are fused to generate the historical spatio-temporal feature map corresponding to the short-circuit fault event of the electrical equipment. This historical spatio-temporal feature map completely describes the multi-modal features of this abnormal event in a specific time and space.

[0100] Step S230: According to the event type label, cluster the historical spatio-temporal feature maps into different abnormal event categories and generate an initial abnormal event template for each abnormal event category.

[0101] In an industrial plant, there are various types of abnormal events, such as equipment failures (including mechanical failures, electrical failures, etc.), fire hazards, and hazardous gas leaks. All the historical spatio-temporal feature maps marked as short-circuit faults of electrical equipment are clustered into the abnormal event category of equipment failure. Then, for the category of equipment failure, analyze the common features in these historical spatio-temporal feature maps, such as the similarity of visual features like equipment smoking and fire in the image when the equipment failure occurs, the commonality of abnormal sounds in the sound, and the laws of temperature increase and smoke generation in the environmental parameters. Based on these common features, generate an initial abnormal event template. This initial abnormal event template includes the typical spatio-temporal features of the equipment failure type, the event type label (equipment failure - electrical short circuit), and the initially estimated impact range parameters, such as the number of adjacent equipment that may be affected and the area of the surrounding area.

[0102] Step S240: Based on the disposal effect evaluation indicators in the disposal record, calibrate the impact range parameters in the initial abnormal event template so that the calibrated impact range parameters have a negative correlation with the disposal effect evaluation indicators.

[0103] For example, the evaluation indicators for the disposal effect of an electrical equipment short - circuit fault event may include the repair time of the faulty equipment, whether there are secondary faults, the degree of impact on surrounding equipment and production, etc. If it is found during the disposal process that the actual number of adjacent equipment affected is less than the initial estimate, and the repair time is short, without causing a great impact on the surrounding production, it indicates that the impact range parameter in the initial abnormal event template may be too large. According to these evaluation indicators for the disposal effect, calibrate the impact range parameter to make it more accurately reflect the actual situation. For example, adjust the number of adjacent equipment that may be affected from 5 to 3, and adjust the affected area of the surrounding area from 20 square meters to 15 square meters.

[0104] Step S250, store the calibrated abnormal event templates in the preset abnormal pattern library, and establish a mapping relationship between the abnormal event templates and the emergency response strategies in the Internet of Things device policy library.

[0105] For example, in the preset abnormal pattern library, various calibrated abnormal event templates are stored, such as templates for different types like equipment failures, fire hazards, hazardous gas leaks, etc. At the same time, establish a mapping relationship with the emergency response strategies in the Internet of Things device policy library. Taking the short - circuit fault of electrical equipment as an example, there are corresponding emergency response strategies in the Internet of Things device policy library. For example, the device linkage rule stipulates that the short - circuit alarm of the electrical control cabinet should immediately trigger a partial cut - off of the main power supply to prevent more serious damage caused by excessive short - circuit current; the parameter adjustment rule sets the start parameters of the fire - extinguishing equipment and the adjustment parameters of the ventilation equipment; the execution condition constraint stipulates that maintenance personnel can only perform maintenance operations after the short - circuit current is stable or reduced to a certain level. Through this mapping relationship, when an abnormal event matching the short - circuit fault template of electrical equipment is detected during monitoring, the corresponding emergency response strategy can be quickly invoked.

[0106] In a possible implementation manner, the update method of the Internet of Things device policy library includes: Step S310, monitor the device operation logs and abnormal event disposal effect data after the Internet of Things execution device cluster executes the regulation instruction set.

[0107] For example, during the handling of an electrical equipment short - circuit fault event, the device operation logs record the operation conditions of each execution device. The device operation log of the fire - extinguishing equipment shows its start time, the change process of the operating power, and the consumption of the fire extinguishing agent; the device operation log of the ventilation equipment records the adjustment process of the ventilation volume and the continuous operation time; the device operation log of the maintenance equipment includes the start time of the maintenance operation, the parameter adjustment situation during the maintenance process, etc. The abnormal event disposal effect data reflects the results of the entire disposal process, such as whether the electrical equipment is completely repaired, the time taken for the repair, and whether it has caused additional impacts on surrounding equipment and production.

[0108] Step S320: Extract the parameter execution deviation value and the device energy consumption data from the device operation log, and calculate the anomaly elimination efficiency and resource utilization rate in the disposal effect data.

[0109] For example, for a fire extinguishing device, the parameter execution deviation value may be the difference between the actual operating power and the operating power set in the regulation instruction. If the set operating power is to spray 10 liters of fire extinguishing agent per minute, but due to equipment aging or other reasons, only 8 liters of fire extinguishing agent are sprayed per minute on average during actual operation, then the parameter execution deviation value is -2 liters / minute. The device energy consumption data records the energy consumption of the fire extinguishing device during the entire operation process, such as how much electricity is consumed or the reduction in the fire extinguishing agent reserve. In terms of the disposal effect data, the anomaly elimination efficiency can be measured by the time it takes for the electrical equipment to return to normal operation. If it should be repaired within 10 minutes under normal circumstances, but it actually takes 15 minutes, then the anomaly elimination efficiency is relatively low. The resource utilization rate considers the effective utilization degree of various resources (such as energy, equipment, manpower, etc.) during the disposal process. For example, whether the fire extinguishing agent of the fire extinguishing device is fully utilized and whether the working hours of the maintenance personnel are reasonably allocated.

[0110] Step S330: Optimize the device linkage rule and the parameter adjustment rule in the emergency response strategy according to the parameter execution deviation value and the anomaly elimination efficiency, and generate the updated device linkage rule and parameter adjustment rule.

[0111] If the parameter execution deviation value of the fire extinguishing device is large and the anomaly elimination efficiency is low, it may be necessary to adjust the device linkage rule. For example, originally it was stipulated that after the electrical equipment short - circuit alarm, the fire extinguishing device would start within 1 minute, but due to the actual start delay, the start time can be advanced to within 30 seconds. For the parameter adjustment rule, according to the actual operation situation and anomaly elimination efficiency of the fire extinguishing device, if it is found that the fire cannot be effectively controlled according to the original parameter setting, then the operating power range of the fire extinguishing device can be adjusted, and the maximum operating power can be increased from spraying 10 liters of fire extinguishing agent per minute to 12 liters of fire extinguishing agent to ensure more effective fire extinguishing operations in case of similar electrical equipment short - circuit faults.

[0112] Step S340: Dynamically adjust the resource allocation threshold in the execution condition constraint based on the device energy consumption data and the resource utilization rate, and recalculate the calibration coefficient of the resource consumption prediction value.

[0113] For ventilation equipment, if the equipment energy consumption data shows that the energy consumption is too high during the disposal of a short - circuit fault of an electrical equipment, it may be necessary to re - evaluate the resource allocation threshold in the execution condition constraints. For example, originally, the power resource allocation threshold for the ventilation equipment during fault disposal was set at 30% of the total power resources, but the actual consumption exceeded this threshold and reached 40%. Based on the equipment energy consumption data and resource utilization rate, the resource allocation threshold can be adjusted to 35%. At the same time, recalculate the calibration coefficient of the resource consumption prediction value. For example, when calculating the resource consumption prediction value of the ventilation equipment, considering the actual operating efficiency of the equipment, environmental factors, etc., adjust the calibration coefficient so that the resource consumption prediction value can more accurately reflect the actual situation.

[0114] Step S350, synchronize the updated device linkage rules, parameter adjustment rules, and the adjusted resource allocation threshold to the Internet of Things device policy library to replace the original policy.

[0115] For example, the optimized device linkage rules (such as advancing the start time of the fire - extinguishing equipment, adjusting the linkage relationship between the ventilation equipment and other equipment, etc.), parameter adjustment rules (such as adjusting the operating power range of the fire - extinguishing equipment, adjusting the ventilation volume adjustment parameters of the ventilation equipment, etc.), and the adjusted resource allocation threshold (such as adjusting the power resource allocation threshold of the ventilation equipment) can be synchronized to the Internet of Things device policy library to replace the original corresponding policies. In this way, when encountering similar abnormal events in the future, the Internet of Things devices can perform more efficient and accurate emergency response operations according to the updated policies.

[0116] In a possible implementation manner, step S152 includes: Step S1521, screen a set of candidate standby devices with the same functional attributes as the target execution device from the Internet of Things execution device cluster.

[0117] For example, in the ventilation system of a factory building, assume that the target execution device is a large axial - flow ventilator, and its function is to provide ventilation and air - change services for a specific area of the workshop. Then, in the Internet of Things execution device cluster, screen out all standby ventilators with the same functional attributes as this axial - flow ventilator. These standby ventilators are similar to the target ventilator in design, function, and application scenarios and can, to a certain extent, replace the work of the target ventilator.

[0118] Step S1522, obtain the current working status, remaining resource capacity, and physical distance from each candidate standby device to the target monitoring area.

[0119] For example, for these candidate backup ventilators, there may be differences in their current working states. Some may be in a completely idle state and can be put into use at any time; while some may be performing some auxiliary ventilation tasks, but still have some remaining working capacity available for replacing the target ventilator. The remaining resource capacity is also an important factor. For a ventilator, the remaining resource capacity can be reflected in aspects such as the remaining power supply capacity, the tolerable working duration, and the adjustment range of the ventilation volume. For example, if the power supply of a backup ventilator can still support it to operate at the maximum power for 2 hours and the ventilation volume can be adjusted within a large range, it indicates that it has a relatively high remaining resource capacity. In addition, the physical distance from the target monitoring area (i.e., the specific area of the workshop ventilated by the target ventilator) will also affect the feasibility of its replacement. If a backup ventilator is far from the target monitoring area, it may lead to complex connection of the ventilation ducts or reduced ventilation efficiency, while a backup ventilator closer to the target has an advantage in this regard.

[0120] Step S1523, calculate the availability scores of each candidate backup device according to the current working state and the remaining resource capacity, and generate a comprehensive replacement priority list in combination with the physical distance.

[0121] For each candidate backup ventilator, conduct a quantitative evaluation according to its current working state and remaining resource capacity. For example, if a backup ventilator is in a completely idle state and has a high remaining resource capacity, a relatively high basic score can be given; if it is performing some tasks but has limited remaining resource capacity, a relatively low score can be given. Then, combine this score with the physical distance factor. Assume that the closer a ventilator is to the target monitoring area, the higher its score in the physical distance factor. Through a specific algorithm, comprehensively calculate the basic score and the physical distance score to obtain the availability score of each candidate backup ventilator. Generate a comprehensive replacement priority list according to these availability scores, and sort the candidate backup ventilators in descending order of availability scores in this comprehensive replacement priority list.

[0122] Step S1524, select the candidate backup device with the highest score in the comprehensive replacement priority list as the target backup device, and allocate the control parameters corresponding to the target execution device to the target backup device after scaling by a preset ratio.

[0123] In the example of the above-mentioned ventilator, if a certain standby ventilator has the highest score in the comprehensive replacement priority list, it will be selected as the target standby device. For the control parameters of the target execution device (i.e., the axial flow ventilator with problems), they need to be adjusted according to the actual situation of the target standby device. For example, the original operating power of the target execution device is 10 kilowatts. Due to the remaining resource capacity or performance characteristics of the target standby device, the operating power may need to be scaled by a preset ratio. Suppose the preset ratio is 0.8, then the operating power allocated to the target standby device will be adjusted to 8 kilowatts. At the same time, other control parameters of the target execution device, such as the ventilation direction and the adjustment range of the ventilation volume, will also be adjusted accordingly according to the characteristics of the target standby device.

[0124] Step S1525, send the scaled control parameters and the new execution priority to the target standby device, and update the device identification and parameter records in the regulation instruction set.

[0125] In this embodiment, the adjusted control parameters such as the operating power and ventilation direction, as well as the new execution priority, can be sent to the target standby device so that it can work according to the new instructions. At the same time, in the regulation instruction set, replace the device identification originally related to the target execution device with the identification of the target standby device, and update the corresponding parameter records to ensure that the entire regulation instruction set matches the actual execution device situation, so that subsequent monitoring and management operations can be carried out accurately.

[0126] In a possible implementation manner, step S1214 includes: Step S1214-1, set the step size and window size of the sliding window, and divide the spatial semantic feature matrix into multiple overlapping feature sub-matrices in chronological order according to the step size and window size of the sliding window.

[0127] For example, in the monitoring of a production workshop, the spatial semantic feature matrix contains the semantic information of equipment, personnel, etc. in the workshop in space and their changes over time. Set the step size of the sliding window to 5 seconds and the window size to 30 seconds. According to this setting, starting from the starting time of the spatial semantic feature matrix, a 30-second window is divided every 5 seconds, and multiple overlapping feature sub-matrices will be obtained. Since the step size is 5 seconds, there will be an overlapping part of 25 seconds between adjacent feature sub-matrices. This overlapping setting helps to capture the subtle changes within continuous time.

[0128] Step S1214-2, perform a temporal convolution operation on each of the feature sub-matrices to extract the feature change gradients and directions of adjacent time segments.

[0129] For example, for each 30-second feature submatrix, the temporal convolution operation can analyze the feature changes of various monitored objects in the workshop (such as the operating status of equipment, the activities of personnel, etc.) between adjacent time segments (such as one segment every 5 seconds) within these 30 seconds. Taking an automated production equipment in the workshop as an example, through the temporal convolution operation, the change gradient of the operating speed of the equipment between adjacent time segments can be obtained, that is, whether the speed is increasing or decreasing, and whether the direction of change is positive (speed increasing) or negative (speed decreasing). For the activities of personnel, the change situations of features such as the moving speed and moving direction of personnel in the workshop between adjacent time segments can also be obtained.

[0130] Step S1214-3: Perform sequence modeling on the feature change gradient and direction through a long short-term memory network to generate trend prediction vectors of the target monitoring area in different time segments.

[0131] For example, in the long-term monitoring of a workshop, the operating status of equipment and the activity patterns of personnel often have certain temporal regularities. The long short-term memory network can model the sequences of feature change gradients and directions such as the change gradient of the operating speed of equipment and the change of the moving direction of personnel obtained through the temporal convolution operation before. For example, for production equipment, the long short-term memory network can predict whether the operating speed of the equipment will continue to be stable, gradually increase, or may experience a sudden drop in speed due to a malfunction in the next few time segments based on the change trend of the equipment operating speed in the past period of time, and generate corresponding trend prediction vectors. For the activities of personnel, trend prediction vectors such as the aggregation state of personnel in the workshop (such as whether they will gather in a specific area) and movement patterns (such as whether they will move along a fixed route) can also be predicted.

[0132] Step S1214-4: Concatenate the trend prediction vectors of each time segment in chronological order to form the dynamic change trend feature. Among them, the dynamic change trend feature is used to characterize the temporal regularities of the movement patterns, aggregation states, or morphological changes of the monitored objects in the target monitoring area.

[0133] For example, during the entire monitoring period of the workshop, there are corresponding trend prediction vectors for different time segments. Concatenating these trend prediction vectors in chronological order forms the dynamic change trend feature, which comprehensively reflects the temporal regularities of the movement patterns, aggregation states, or morphological changes of the monitored objects (equipment and personnel) in the target monitoring area (i.e., the production workshop). For example, through this dynamic change trend feature, it can be analyzed that during the production process of a day, the change rules of the operating status of equipment at different time periods, as well as the change situations of the activity patterns and aggregation states of personnel under different working links, providing a strong basis for the management of industrial plants and the prevention of abnormal events.

[0134] Figure 2 FIG. shows a schematic diagram of exemplary hardware and software components of a server 100 that can implement the ideas of the present application provided by some embodiments of the present application. For example, a processor 120 can be used on the server 100 and is used to execute the functions in the present application.

[0135] The server 100 can be a general-purpose server or a special-purpose server, both of which can be used to implement the Internet of Things-based monitoring information recognition method of the present application. Although only one server is shown in the present application, for convenience, the functions described in the present application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0136] For example, the server 100 can include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and different forms of storage media 140, such as disks, ROM, or RAM, or any combination thereof. Exemplarily, the server 100 can also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application can be implemented according to these program instructions. The server 100 also includes an input / output (I / O) interface 150 between the computer and other input / output devices.

[0137] For ease of explanation, only one processor is described in the server 100. However, it should be noted that the server 100 in the present application can also include multiple processors. Therefore, the steps executed by one processor described in the present application can also be jointly executed or separately executed by multiple processors. For example, if the processor of the server 100 executes step A and step B, it should be understood that step A and step B can also be jointly executed by two different processors or separately executed in one processor. For example, the first processor executes step A, the second processor executes step B, or the first processor and the second processor jointly execute steps A and B.

[0138] In addition, an embodiment of the present invention also provides a readable storage medium, in which computer-executable instructions are preset. When the processor executes the computer-executable instructions, the above-mentioned Internet of Things-based monitoring information recognition method is implemented.

[0139] It should be noted that, in order to simplify the description of the present invention disclosure and thus help the understanding of one or more embodiments of the present invention, in the previous description of the embodiments of the present invention, sometimes multiple features are merged into one embodiment, drawing, or description thereof.

Claims

1. A monitoring information identification method based on the Internet of Things, characterized in that: The method comprises: Acquire multimodal sensing data collected in real time by multiple IoT monitoring nodes in a target monitoring area, wherein the multimodal sensing data includes image monitoring data, sound monitoring data, and environmental parameter monitoring data; Performing cross-modal feature extraction and fusion processing on the multimodal sensing data to generate a comprehensive spatiotemporal feature map corresponding to the target monitoring area; Based on a preset abnormal pattern library, dynamic pattern matching is performed on the comprehensive spatiotemporal feature map to determine at least one abnormal event type and its spatiotemporal distribution information existing in the target monitoring area; Generate a control instruction set adapted to the abnormal event type according to the abnormal event type and the spatiotemporal distribution information, wherein the control instruction set includes control parameters and execution priorities for different IoT execution devices; The control instruction set is distributed to the associated IoT execution device cluster within the target monitoring area, triggering the IoT execution device cluster to execute the control parameters according to the execution priority to eliminate the impact of the abnormal event.

2. The monitoring information identification method based on the Internet of Things according to claim 1 is characterized in that: The cross-modal feature extraction and fusion processing of the multimodal sensing data to generate a comprehensive spatiotemporal feature map corresponding to the target monitoring area includes: Performing multi-scale spatial feature extraction on the image monitoring data to obtain a visual semantic feature vector within the target monitoring area, and performing voiceprint spectrum decomposition on the sound monitoring data to extract an acoustic event feature vector within the target monitoring area; Performing time series analysis on the environmental parameter monitoring data, extracting the environmental fluctuation characteristic vector in the target monitoring area, and constructing a spatiotemporal correlation matrix corresponding to the environmental fluctuation characteristic vector; Inputting the visual semantic feature vector, the acoustic event feature vector and the spatiotemporal association matrix into a cross-modal fusion network, and dynamically allocating the contribution weight of each modal feature in the spatiotemporal dimension through an attention weight allocation layer in the cross-modal fusion network; Based on the contribution weights, weighted concatenation is performed on the visual semantic feature vector, the acoustic event feature vector and the spatiotemporal association matrix to generate the comprehensive spatiotemporal feature map; Among them, the comprehensive spatiotemporal feature map includes the spatial event distribution and environmental parameter evolution trend of the target monitoring area in a continuous time window.

3. The monitoring information identification method based on the Internet of Things according to claim 2 is characterized in that: The method of performing dynamic pattern matching on the comprehensive spatiotemporal feature map based on a preset abnormal pattern library to determine at least one abnormal event type and its spatiotemporal distribution information existing in the target monitoring area includes: Loading multiple abnormal event templates from the preset abnormal pattern library, each of the abnormal event templates includes reference spatiotemporal features, event type labels, and impact range parameters corresponding to historical abnormal events; Calculating a similarity matrix between the comprehensive spatiotemporal feature map and the reference spatiotemporal features of each of the abnormal event templates, and screening out a set of candidate abnormal event templates having a similarity higher than a preset threshold based on the similarity matrix; Performing spatiotemporal alignment processing on each candidate abnormal event template in the candidate abnormal event template set, mapping the timestamp of the comprehensive spatiotemporal feature map and the timestamp of the candidate abnormal event template to a unified time coordinate system; According to the feature difference after spatiotemporal alignment, a target abnormal event template matching the comprehensive spatiotemporal feature map is determined from the candidate abnormal event template set, and the event type label and impact range parameter of the target abnormal event template are used as the abnormal event type and the spatiotemporal distribution information; The spatiotemporal distribution information includes the starting position, diffusion path and impact intensity attenuation curve of the abnormal event in the target monitoring area.

4. The monitoring information identification method based on the Internet of Things according to claim 3 is characterized in that: The step of generating a control instruction set adapted to the abnormal event type according to the abnormal event type and the spatiotemporal distribution information includes: According to the abnormal event type, calling the emergency response strategy associated with the abnormal event type from the IoT device policy library, the emergency response strategy including device linkage rules, parameter adjustment rules and execution condition constraints; Based on the impact intensity attenuation curve in the spatiotemporal distribution information, determining the optimal startup time and action duration of each execution device in the IoT execution device cluster; According to the device linkage rule and the parameter adjustment rule, the control parameters are allocated to each of the execution devices, wherein the control parameters include operating power, action direction and coverage range; In combination with the execution condition constraint and the optimal startup time, calculating the estimated resource consumption of each of the execution devices in the target monitoring area, and optimizing and calibrating the control parameters based on the estimated resource consumption; According to the priority weight of the impact intensity attenuation curve and the estimated resource consumption value, the execution priority is assigned to the control parameter of each execution device to generate the control instruction set; The step of distributing the control instruction set to an associated IoT execution device cluster within the target monitoring area, and triggering the IoT execution device cluster to execute the control parameter according to the execution priority, includes: According to the execution priority, the control instruction set is divided into a plurality of instruction batches, and each instruction batch is sent to a corresponding execution device sub-cluster in batch order; When sending each of the instruction batches, the response status of the execution device sub-cluster is monitored in real time. If it is detected that the target execution device does not return a response confirmation signal within a preset time, the backup device replacement strategy is activated to reallocate the control parameters corresponding to the target execution device to the backup execution device; After the execution device sub-cluster completes the parameter execution of the current instruction batch, real-time feedback data of the target monitoring area is collected, and the impact intensity change rate of the abnormal event is evaluated based on the real-time feedback data; If the impact intensity change rate does not reach the expected decay rate, dynamically adjusting the control parameters and the execution priority in subsequent instruction batches according to the real-time feedback data; After all the instruction batches are executed, a processing report of the abnormal event is generated, and the processing report is uploaded to the Internet of Things monitoring platform for archiving.

5. The monitoring information identification method based on the Internet of Things according to claim 2 is characterized in that: The performing multi-scale spatial feature extraction on the image monitoring data to obtain a visual semantic feature vector within the target monitoring area includes: Dividing the image monitoring data into a plurality of image frame sequences at preset time intervals, and performing denoising and illumination equalization processing on each of the image frame sequences to obtain a processed image frame sequence; Using a convolutional neural network to perform hierarchical feature extraction on the processed image frame sequence to obtain local texture feature maps and global scene feature maps at different scales; Performing multi-resolution feature fusion on the local texture feature map and the global scene feature map through a spatial pyramid pooling layer to generate a fused spatial semantic feature matrix; Perform sliding window aggregation on the spatial semantic feature matrix in the time dimension to extract dynamic change trend characteristics of the target monitoring area in a continuous time period; The dynamic change trend feature is concatenated with the spatial semantic feature matrix in the current time window to generate the visual semantic feature vector.

6. The monitoring information identification method based on the Internet of Things according to claim 4 is characterized in that: The method for constructing the preset abnormal pattern library includes: Collecting multiple historical case data marked as abnormal events in the historical monitoring data, each of the historical case data includes multimodal perception data, event type labels and disposal records when the abnormality occurs; Performing cross-modal feature extraction on the multimodal perception data in the historical case data to generate a historical spatiotemporal feature map corresponding to each of the historical case data; Clustering the historical spatiotemporal feature maps into different abnormal event categories according to the event type labels, and generating an initial abnormal event template for each abnormal event category; Based on the treatment effect evaluation index in the treatment record, calibrate the impact range parameter in the initial abnormal event template so that the calibrated impact range parameter is negatively correlated with the treatment effect evaluation index; The calibrated abnormal event templates are stored in the preset abnormal pattern library, and a mapping relationship between the abnormal event templates and the emergency response strategies in the Internet of Things device strategy library is established.

7. The monitoring information identification method based on the Internet of Things according to claim 4 is characterized in that: The method for updating the IoT device policy library includes: Monitor the device operation logs and abnormal event handling effect data after the IoT execution device cluster executes the control instruction set; Extracting parameter execution deviation values ​​and equipment energy consumption data from the equipment operation log, and calculating the abnormality elimination efficiency and resource utilization rate in the treatment effect data; According to the parameter execution deviation value and the abnormality elimination efficiency, the device linkage rule and the parameter adjustment rule in the emergency response strategy are optimized to generate updated device linkage rule and parameter adjustment rule; Based on the device energy consumption data and the resource utilization rate, dynamically adjust the resource allocation threshold in the execution condition constraint, and recalculate the calibration coefficient of the resource consumption estimate; The updated device linkage rules, parameter adjustment rules and adjusted resource allocation thresholds are synchronized to the IoT device policy library to replace the original policies.

8. The monitoring information identification method based on the Internet of Things according to claim 4 is characterized in that: The starting of the standby device replacement strategy to reallocate the control parameters corresponding to the target execution device to the standby execution device includes: Selecting a set of candidate backup devices having the same functional attributes as the target execution device from the IoT execution device cluster; Obtaining the current working status, remaining resource capacity and physical distance of each candidate backup device from the target monitoring area; Calculate the availability score of each candidate backup device according to the current working status and the remaining resource capacity, and generate a comprehensive replacement priority list in combination with the physical distance; Select the candidate standby device with the highest score in the comprehensive replacement priority list as the target standby device, and allocate the control parameters corresponding to the target execution device to the target standby device after scaling them according to a preset ratio; The scaled control parameters and the new execution priority are sent to the target standby device, and the device identification and parameter records in the control instruction set are updated.

9. The monitoring information identification method based on the Internet of Things according to claim 5 is characterized in that: The step of performing sliding window aggregation on the spatial semantic feature matrix in the time dimension to extract dynamic change trend characteristics of the target monitoring area in a continuous time period includes: Setting a step length and a window size of a sliding window, and dividing the spatial semantic feature matrix into a plurality of overlapping feature sub-matrices in chronological order according to the step length and the window size of the sliding window; Performing a time convolution operation on each of the feature sub-matrices to extract the feature change gradient and direction of adjacent time segments; The feature change gradient and direction are sequentially modeled by a long short-term memory network to generate trend prediction vectors of the target monitoring area in different time segments; The trend prediction vectors of each time segment are spliced ​​in chronological order to form the dynamic change trend feature; Among them, the dynamic change trend characteristics are used to characterize the temporal regularity of the movement mode, aggregation state or morphological change of the monitored objects in the target monitoring area.

10. A server, characterized in that: The server includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the monitoring information identification method based on the Internet of Things as described in any one of claims 1 to 9 above.

Images