Warehouse material management method and system based on intelligent inspection robot

Abstract

The application belongs to the technical field of data processing, and specifically discloses a warehouse material management method and system based on an intelligent inspection robot, wherein the method comprises the following steps: collecting environmental monitoring data through a data acquisition device, wherein the environmental monitoring data comprises temperature and humidity information, smoke concentration and an image sequence, and an original multi-modal signal set is obtained; a feature vector is extracted from the original multi-modal signal set by using a deep learning model, and a multi-dimensional feature representation is obtained for subsequent fusion processing; priority ranking is performed on the multi-dimensional feature representation for abnormality detection, a high-risk signal subset is determined to identify potential hidden dangers; and if the high-risk signal subset meets a triggering condition, image sequence analysis is performed; the application aims to solve the problem of inaccuracy in real-time identification of multi-source data complexity and abnormal events in environmental monitoring.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a warehouse material management method and system based on intelligent inspection robots. Background Technology

[0002] In the field of environmental monitoring, the core technical challenge lies in how to achieve accurate acquisition, analysis, and real-time response of multimodal data to ensure high accuracy and timeliness in identifying hidden dangers and controlling risks in complex environments.

[0003] This problem stems from the diversity and dynamism of environmental data. For example, multimodal signals such as temperature and humidity, smoke concentration, and image sequences may be limited by equipment accuracy, environmental interference, or data synchronization during acquisition, resulting in inconsistent raw data quality, which in turn affects the reliability of subsequent feature extraction and anomaly detection.

[0004] Meanwhile, the fusion and processing of multimodal data faces challenges of dimensional inconsistency and information redundancy. How to balance the weights of different modal signals and accurately identify high-risk signals in deep analysis has become an urgent problem to be solved.

[0005] Focusing further on business scenarios, suppose an environmental monitoring system in an industrial park needs to monitor the operating status of equipment and fire hazards in real time in a workshop filled with high temperature, high humidity, and smoke. The system not only has to deal with signal drift caused by the harsh environment, but also with the blurring of image sequences due to insufficient light or obstruction. The core of the problem is how to extract key abnormal features from multi-source heterogeneous data in such a complex scenario and quickly determine whether it is a real threat or a false alarm.

[0006] In addition to this major issue, there are also minor issues surrounding this problem, such as how to dynamically adjust data acquisition parameters to adapt to environmental changes and how to balance historical trends with the accuracy of current state predictions in real-time risk assessments. These minor issues all contribute to improving the overall accuracy and response speed of hazard identification and together constitute the full-chain technical challenges in environmental monitoring, from data to decision-making. Summary of the Invention

[0007] This invention provides a warehouse material management method and system based on intelligent inspection robots, aiming to solve the problems of complex multi-source data and inaccurate real-time identification of abnormal events in environmental monitoring.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0009] A warehouse material management method and system based on intelligent inspection robots, characterized by comprising: collecting environmental monitoring data through a data acquisition device, the environmental monitoring data including temperature and humidity information, smoke concentration, and image sequences, to obtain an original multimodal signal set; extracting feature vectors from the original multimodal signal set using a deep learning model to obtain multi-dimensional feature representations for subsequent fusion processing; prioritizing anomaly detection from the multi-dimensional feature representations to determine a high-risk signal subset for identifying potential hazards; if the high-risk signal subset meets the triggering conditions, performing image sequence analysis to determine abnormal events and obtain comprehensive anomaly labels; using a fusion mechanism to weight the comprehensive anomaly labels and dynamic responses to obtain a real-time risk assessment score; predicting hazard trends using a sequence prediction model based on the real-time risk assessment score, determining alarm conditions, generating a response sequence, and obtaining hazard identification results for environmental control; adjusting data acquisition parameters using a feedback mechanism based on the hazard identification results, updating the original multimodal signal set, and repeating the processing to obtain an optimized risk assessment score for continuous optimization.

[0010] In one aspect of this disclosure, the acquisition of environmental monitoring data via a data acquisition device, the environmental monitoring data including temperature and humidity information, smoke concentration, and image sequences, to obtain a raw multimodal signal set, includes:

[0011] Environmental monitoring data, including temperature and humidity values, smoke concentration, and image sequences, are continuously acquired through a data acquisition device to construct an initial multimodal signal set and obtain the original dataset.

[0012] Based on the original dataset, the temperature and humidity values ​​and smoke concentration were standardized. The mean was subtracted from each value and the standard deviation was divided. The standardized data were then preliminarily screened using a preset threshold range to determine whether there were any outliers.

[0013] If the temperature, humidity or smoke concentration exceeds the preset threshold range, the abnormal data points are marked and the marked dataset is obtained.

[0014] Using the labeled dataset, preprocessing operations are performed on the image sequence. A Gaussian filter is used to sharpen the image sequence, and a pixel-weighted average is calculated from each image frame to obtain the processed image data.

[0015] Based on the processed image data, combined with the labeled temperature and humidity values ​​and smoke concentration data, the support vector machine algorithm is applied to classify the multimodal signals, extract feature vectors from the combined data, and train the model to determine the environmental state category.

[0016] By classifying the environmental state categories, Pearson correlation coefficients are calculated for temperature and humidity values ​​and smoke concentration to obtain the comprehensive trend of environmental parameter changes.

[0017] Based on the overall trend of change, dynamic monitoring is carried out on the fluctuation of environmental parameters to determine whether there are any continuous abnormal signal characteristics, and the final monitoring results are obtained.

[0018] In one aspect of this disclosure, the step of extracting feature vectors from the original multimodal signal set using a deep learning model to obtain a multidimensional feature representation for subsequent fusion processing includes:

[0019] By processing multimodal signals, key information in the original set is obtained, and a preliminary data structure is derived.

[0020] Based on the preliminary data structure, a deep learning model is used to analyze the multimodal signals and determine the generation path of the feature vectors.

[0021] If the feature vector generation path meets the preset threshold conditions, the multidimensional feature representation is obtained through the extraction process, and it is determined whether it meets the fusion requirements.

[0022] If the representation of the multidimensional features meets the fusion requirements, then the subsequent fusion processing step is adopted to obtain the fused feature set and determine its consistency.

[0023] Based on the fused feature set, the final feature representation is obtained through further optimization of the processing steps, resulting in the integrated data result.

[0024] By integrating the data results and using a pre-established verification mechanism, we determine its applicability in multimodal signal analysis and obtain the verified feature output.

[0025] Based on the verified feature output, the integrity of the multimodal signal is determined by retrospective analysis of the signal source, and the final processing conclusion is obtained.

[0026] In one aspect of this disclosure, prioritizing anomaly detections from the multi-dimensional feature representations to determine a subset of high-risk signals for identifying potential hazards includes:

[0027] By extracting data from multi-dimensional features and using pre-established filtering rules, key dimension data is separated to obtain preliminary filtering results.

[0028] Based on the preliminary screening results, a priority ranking mechanism is applied to the key dimension data. If the indicator of a certain dimension exceeds the preset threshold, it is classified as a high-risk signal, and a set of high-risk signals is determined.

[0029] For high-risk signal sets, a signal classification method is used to divide them into different signal subsets and obtain the classified signal subset data;

[0030] From the classified subset of signal data, a subset related to anomaly detection is obtained. If the fluctuation pattern of a subset matches the preset anomaly pattern, it is marked as a potential hazard, and a subset of potential hazards is identified.

[0031] Based on the subset of potential hazards, and combined with the risk assessment model, the risk level of each subset is calculated to obtain the risk level distribution;

[0032] Based on the risk level distribution, a hidden Markov model is used to further identify potential hazards for a subset of high-risk individuals, thus determining the final set of hazard signals.

[0033] Obtain the final set of potential hazard signals, and combine data analysis methods to trace the source dimensions of the potential hazard signals and determine the root cause dimensions of the potential hazard signals.

[0034] In one aspect of this disclosure, the step of performing image sequence analysis, determining abnormal events, and obtaining comprehensive abnormal labels if the high-risk signal subset meets the triggering conditions includes:

[0035] If a data stream containing a subset of high-risk signals is received, a preliminary screening is performed by comparing it against a preset threshold to determine whether there is a signal combination that meets the triggering conditions, and a preliminary risk assessment result is obtained.

[0036] If the preliminary risk assessment results show that there are triggering conditions, the corresponding image sequence data is obtained from the repository, and the image sequence is preprocessed using a standardized processing method to determine the first image dataset after processing.

[0037] Based on the first image dataset, a pre-trained convolutional neural network model is invoked to extract features from the image sequence, obtain the visual feature vector of each frame image, and obtain the second image dataset after feature extraction.

[0038] For the second image dataset, if there are outliers in the feature vectors that deviate from a preset range, the relevant frames are marked as potential abnormal events, and the set of potential abnormal events is determined.

[0039] By performing time-series correlation analysis on the set of potential abnormal events, the continuous distribution of abnormal events in the time dimension is obtained, it is determined whether there are persistent abnormal events, and the final list of abnormal events is obtained.

[0040] Based on the final list of abnormal events and combined with predefined label mapping rules, corresponding comprehensive abnormal labels are generated to determine the final abnormal classification result.

[0041] If the final anomaly classification result needs further verification, the comprehensive anomaly label will be compared with historical data records to obtain the consistency verification result, and the final business judgment conclusion will be drawn.

[0042] In one aspect of this disclosure, the weighted processing of the integrated anomaly label and dynamic response using a fusion mechanism to obtain a real-time risk assessment score includes:

[0043] By using a fusion mechanism, key information is extracted from comprehensive anomaly labels and dynamic response data to obtain a preliminary set of anomaly features;

[0044] Based on the preliminary set of abnormal features, a weighted calculation method is used to rank the importance of each abnormal feature and determine the weighted feature weight distribution.

[0045] If the weight of an abnormal feature in the feature weight distribution exceeds a preset threshold, it is marked as a high-risk feature, and a subset of high-risk features is obtained.

[0046] For high-risk feature subsets, real-time calculations are performed using dynamic response data to determine whether persistent abnormal patterns exist.

[0047] If a persistent anomalous pattern is detected, the anomalous pattern is classified using a pre-established random forest model to obtain the classified risk category.

[0048] Based on the classified risk categories and the weighted calculation results, a final real-time risk assessment score is generated.

[0049] By mapping real-time risk assessment scores across intervals, the corresponding risk level classification is determined.

[0050] In one aspect of this disclosure, the step of predicting hazard trends using a sequence prediction model based on the real-time risk assessment score, determining alarm conditions, generating a response sequence, and obtaining hazard identification results for environmental control includes:

[0051] To obtain real-time risk assessment scores, environmental data is collected from multiple sensor nodes through a data acquisition system. Preprocessing techniques are used to clean and standardize the collected data to obtain a standardized risk score dataset.

[0052] For a standardized risk score dataset, a time series forecasting model is used to analyze the data's changing patterns, identify potential hidden danger trends, and determine the key time points of trend changes.

[0053] Based on the changing characteristics of the potential hazard trend, if the trend is detected to deviate from the preset threshold range, an alarm condition is triggered, and the corresponding alarm signal sequence is generated through the information processing module.

[0054] For the generated alarm signal sequence, a pre-established response rule base is used for matching to obtain the corresponding response sequence, and the priority and execution order of the response sequence are determined.

[0055] By analyzing the execution order of the response sequence, specific classification results for hazard identification are generated, determining the hazard category and its impact range.

[0056] Based on the classification results of hazard identification, an environmental control instruction set is generated. The instruction set is then sent to the control equipment via the data transmission module to obtain feedback on the adjustment of environmental parameters.

[0057] For the feedback on the adjustment of environmental parameters, information comparison technology is used to analyze the deviation between the feedback data and the expected target. If the deviation exceeds the preset range, the control instruction set is regenerated to determine the final environmental control result.

[0058] In one aspect of this disclosure, the step of adjusting data acquisition parameters based on the hazard identification results using a feedback mechanism, updating the original multimodal signal set, and repeating the processing to obtain an optimized risk assessment score for continuous optimization includes:

[0059] The initial multimodal signal set is obtained through the hazard identification module, and the signals are preliminarily analyzed using a pre-established classification model to obtain preliminary hazard classification results.

[0060] Based on the preliminary hazard classification results, the feedback mechanism module is triggered to adjust the data acquisition parameter configuration for abnormal signals in the classification results and determine the updated acquisition strategy.

[0061] The updated acquisition strategy is adopted to reacquire the multimodal signal set. For the newly acquired signal data, the support vector machine model is used to extract features to obtain the signal dataset after feature processing.

[0062] If there are outliers in the signal dataset after feature processing that deviate from the preset threshold, the outliers are filtered a second time through the loop processing module to determine whether there are potential hidden danger signals.

[0063] Based on the potential hazard signals after secondary filtering, the risk assessment module comprehensively scores the signal dataset to obtain an optimized risk assessment score.

[0064] By continuously improving the module, the optimized risk assessment score is compared with historical data. If the comparison results show that the accuracy improvement is insufficient, the weight parameters of feature extraction are adjusted to obtain the updated signal processing rules.

[0065] Based on the updated signal processing rules, the acquisition and analysis of the multimodal signal set are re-executed, and the process is iterated in a loop to continuously improve the evaluation accuracy.

[0066] In another aspect, this disclosure also relates to a warehouse material management system based on an intelligent inspection robot, the system comprising:

[0067] The data acquisition module is configured to acquire environmental monitoring data through the data acquisition device, the environmental monitoring data including temperature and humidity information, smoke concentration and image sequences, and to obtain the raw multimodal signal set.

[0068] The feature extraction module is configured to perform the step of extracting feature vectors from the original multimodal signal set using a deep learning model to obtain multidimensional feature representations for subsequent fusion processing;

[0069] The priority ranking module is configured to perform the step of prioritizing anomaly detections from the multi-dimensional feature representation to determine a subset of high-risk signals to identify potential hazards.

[0070] The image analysis module is configured to perform image sequence analysis, determine abnormal events, and obtain comprehensive abnormal labels if the high-risk signal subset meets the triggering conditions.

[0071] The fusion processing module is configured to perform weighted processing on the comprehensive anomaly label and dynamic response using a fusion mechanism to obtain a real-time risk assessment score.

[0072] The trend prediction module is configured to predict the trend of potential hazards based on the real-time risk assessment score through a sequence prediction model, determine alarm conditions and generate a response sequence, and obtain the hazard identification results for environmental control.

[0073] The parameter adjustment module is configured to adjust the data acquisition parameters using a feedback mechanism based on the hazard identification results, update the original multimodal signal set, and repeat the processing to obtain an optimized risk assessment score for continuous optimization.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] This invention extracts multi-dimensional feature representations using a deep learning model, prioritizes and filters high-risk signal subsets, combines image sequence analysis to identify abnormal events, and employs a fusion mechanism to generate real-time risk assessment scores. Furthermore, a sequence prediction model predicts hazard trends and generates response sequences. Simultaneously, a feedback mechanism is introduced to dynamically adjust data acquisition parameters and continuously optimize the risk assessment scores. Ultimately, this invention achieves accurate analysis of environmental monitoring data and real-time identification of hazards, effectively improving the response speed and accuracy of environmental control and providing technical support for safety management in complex environments. Attached Figure Description

[0076] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0077] Figure 1 This is one of the flowcharts for a warehouse material management method based on an intelligent inspection robot according to the present invention.

[0078] Figure 2 This is the second flowchart of a warehouse material management method based on an intelligent inspection robot according to the present invention. Detailed Implementation

[0079] The present invention will be further described below with reference to embodiments. These embodiments are merely some, not all, of the embodiments described. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the protection scope of the present invention.

[0080] Please see Figure 1 as well as Figure 2 As shown, this embodiment discloses a warehouse material management method based on an intelligent inspection robot;

[0081] In one embodiment, the method provided by this invention begins with the acquisition of environmental monitoring data, constructing a multimodal signal set to lay the foundation for subsequent analysis. The environmental monitoring data encompasses various types of information, acquired through different sensor devices, ensuring the comprehensiveness and diversity of the data.

[0082] Data acquisition devices can be deployed in various environments, such as industrial parks, office buildings, or residential communities, to monitor changes in environmental parameters in real time.

[0083] The collected data will serve as the raw input for subsequent processing and analysis, providing a basis for judging the environmental status.

[0084] Specific methods may include:

[0085] Step S1 involves collecting environmental monitoring data using a data acquisition device to construct a raw multimodal signal set. The environmental monitoring data includes temperature and humidity information, smoke concentration, and image sequences, which are continuously acquired through various sensor devices. The data acquisition device may include temperature and humidity sensors, smoke sensors, and cameras, which work together to capture different dimensions of information from the environment. The temperature and humidity sensors record temperature and humidity values, the smoke sensors detect the concentration of smoke particles in the air, and the cameras collect image sequence data from the environment. The data acquired by these devices forms a raw dataset containing multiple modal signals, providing a foundation for subsequent processing and analysis.

[0086] Step S1-1, specifically, involves the data acquisition device collecting data at a preset frequency during operation. For example, a temperature and humidity sensor can record values ​​every minute, a smoke sensor can detect concentration every 30 seconds, and a camera can capture one frame per second. The collected data is uploaded to the processing center in real time via a data transmission module, which can be a wired or wireless network, ensuring timely delivery after collection. During the acquisition process, the device automatically records timestamps and device identifiers for subsequent data traceability and classification. In constructing the original dataset, the system initially integrates different types of data in chronological order, forming a collection containing multimodal signals.

[0087] In steps S1-2, one possible implementation involves adjusting the parameters of the data acquisition device based on environmental characteristics. For example, in scenarios with drastic environmental changes, such as industrial parks, the sampling frequency of the smoke sensor can be increased to capture concentration changes more promptly; while in relatively stable scenarios like residential communities, the sampling frequency can be appropriately reduced to decrease energy consumption. The collected data undergoes preliminary verification to ensure data integrity, such as checking for missing data or abrupt changes. If missing data is detected, the system records the missing time period and completes it using subsequent interpolation methods. This approach ensures the quality of the original dataset, providing a reliable foundation for subsequent analysis.

[0088] Step S2: Standardize the temperature and humidity values ​​and smoke concentration based on the original dataset and perform preliminary anomaly screening. The temperature, humidity, and smoke concentration data in the original dataset typically have different dimensions and distribution characteristics. To facilitate subsequent analysis, they need to be standardized. The standardization process involves subtracting the mean from each data value and then dividing by the standard deviation, ensuring that the processed data conforms to a distribution with a mean of 0 and a standard deviation of 1. After standardization, the system will perform preliminary screening of the data based on a preset threshold range to determine if any outliers exist.

[0089] Step S2-1, specifically, standardization processing can be performed separately for different types of data. For example, the mean and standard deviation of temperature and humidity data can be calculated from historical data, while smoke concentration data can be determined based on equipment calibration values. After standardization, the system compares each data point with a preset threshold range, which can be pre-set according to the environmental scenario and equipment characteristics. If a data point exceeds the threshold range, such as a temperature value exceeding the upper limit of the normal range or a smoke concentration exceeding the safety standard, the system will mark it as a potential anomaly. This processing method can quickly filter out potentially problematic values, providing guidance for subsequent in-depth analysis.

[0090] Step S2-2: In one embodiment, outlier screening can also incorporate a time dimension. For example, the system checks if a data point experiences drastic fluctuations within a short period. If the temperature value jumps from the normal range to the abnormal range within 5 minutes, it will be marked as a potential outlier even if the value does not exceed the upper limit of the threshold. This approach captures dynamic changes in the data, avoiding missed detections caused by relying solely on static thresholds. Furthermore, the system can dynamically adjust the threshold range according to different environmental scenarios. For example, in hot and humid summers, the upper limit of the temperature threshold can be appropriately increased to reduce false alarms.

[0091] Step S3: If the temperature, humidity, or smoke concentration exceeds a preset threshold range, the abnormal data points are marked, and the marked dataset is obtained. Based on the initial screening, the system will further mark the data points that exceed the threshold range so that these abnormal points can be given priority in subsequent processing. The marked dataset will contain the original data values, the standardized data values, and the abnormal marking information, which will support subsequent multimodal signal analysis.

[0092] Step S3-1, specifically, the labeling of outlier data points can be achieved in several ways. For example, the system can add an outlier label to each data point exceeding a threshold. The label content can include the outlier type and the degree of outlier. The outlier type can be high temperature, low temperature, high humidity, or high smoke concentration, etc., while the degree of outlier can be determined by the degree of deviation of the data value from the threshold. If a temperature value exceeds the upper limit of the threshold by a large margin, its degree of outlier is marked as severe; if it only slightly exceeds it, it is marked as slight. The labeled dataset will be reorganized in chronological order to ensure that the distribution characteristics of outliers in the time dimension are preserved.

[0093] In step S3-2, one possible implementation is that anomaly labeling can also be combined with environmental context information. For example, in an industrial park scenario, if anomaly in smoke concentration occurs during peak equipment operation, the system will additionally record the equipment operating status during this period to facilitate subsequent analysis of whether the anomaly is related to equipment failure. This labeling method can provide more clues for the cause analysis of anomalies. Furthermore, the system can group consecutively occurring anomalies, for example, labeling multiple high-temperature points occurring within 5 consecutive minutes as an anomaly event group, so that the anomaly characteristics can be grasped holistically during subsequent analysis.

[0094] In step S3-3, in one embodiment, the labeled dataset can also be used for preliminary visualization of anomalies. For example, the system can plot the labeled temperature and humidity values ​​and smoke concentration data into a time series graph, with anomalies highlighted in different colors or symbols. This visualization method helps monitoring personnel quickly understand the distribution and frequency of anomalies, providing an intuitive basis for subsequent handling decisions. Simultaneously, the system can also generate statistical reports on anomalies, including information such as the number, type distribution, and time distribution of anomalies, enabling monitoring personnel to comprehensively understand the environmental status.

[0095] In steps S3-4, for further applications of anomaly marking, the system can also perform comparative analysis using historical data. For example, the system will compare currently marked anomalies with anomalies from the past month to determine if similar time or value patterns exist. If the current anomaly is found to be highly consistent with historical anomalies in terms of time or value, the system will indicate that there may be periodic anomalies requiring further attention. This comparative analysis can help identify hidden patterns of environmental change and provide a reference for subsequent dynamic monitoring.

[0096] In steps S3-5, one possible implementation is that anomaly labeling can also be combined with the specific characteristics of the environmental scenario. For example, in an office building scenario, if high-temperature anomalies frequently occur during the afternoon when sunlight is direct, the system will record this environmental characteristic and add relevant annotations to the labeling, so that the influence of external factors can be considered in subsequent analysis. This labeling method can improve the targeting of anomaly analysis and avoid misjudging normal fluctuations caused by changes in the external environment as abnormal events.

[0097] Steps S3-6 address the diverse processing of anomaly markers. The system can also classify and mark anomalies based on their duration. For example, if an anomaly in smoke concentration lasts only a few seconds, it may be a temporary disturbance and should be marked as low priority; if an anomaly lasts for several minutes or even longer, it may pose a real risk and should be marked as high priority. This classification and marking allows the system to allocate resources rationally in subsequent processing, focusing on high-priority anomalies and thus improving monitoring efficiency.

[0098] In one embodiment, steps S3-7, the labeled dataset can also be used for preliminary inference of the causes of anomalies. For example, the system analyzes the temporal and typological distribution of anomalies. If high-temperature anomalies and smoke concentration anomalies occur frequently within the same time period, the system will infer a potential fire hazard and generate a corresponding warning. While this inference is not a final conclusion, it can provide direction for subsequent in-depth analysis and reduce the randomness of the analysis.

[0099] In steps S3-8, the system can further customize the processing of the labeled dataset according to different monitoring targets and application scenarios. For example, in an industrial park scenario, the system will focus on anomalies in smoke concentration and prioritize pushing relevant data to the safety management department; in a residential community scenario, the system will focus on anomalies in temperature and humidity and push the data to the property management department so that timely control measures can be taken. Through this customized processing, the labeled dataset can better serve the actual needs of different scenarios.

[0100] In step S3-9, one possible implementation is that the labeled dataset can also be used for cross-modal association analysis of outliers. For example, the system checks whether temperature and humidity anomalies and smoke concentration anomalies are associated with certain features in the image sequence. If significant changes in lighting or object movement are found in the image when an anomaly occurs, the system records this association information for subsequent, more comprehensive analysis in conjunction with the image data. This cross-modal association analysis can provide more multi-dimensional data support for determining the causes of anomalies.

[0101] In step S3-10, regarding the storage and management of the labeled dataset, the system will classify and store the data according to time and type. For example, abnormal temperature and humidity data and abnormal smoke concentration data will be stored in different database partitions for quick retrieval later. Simultaneously, the system will also perform regular backups of the labeled dataset to ensure data security. This storage and management method improves data utilization efficiency and facilitates subsequent processing.

[0102] Step S3-11: In one embodiment, the labeled dataset can also be used for dynamic updating of outliers. For example, the system will re-evaluate the labeled outliers based on subsequently collected data. If an outlier recovers to normal within a subsequent time period, the system will update its labeling status and reduce its anomalousness. This dynamic updating method can avoid misjudgments caused by data fluctuations and improve the accuracy of outlier labeling.

[0103] In step S3-12, for further optimization of anomaly labeling, the system can also introduce a manual review mechanism. For example, the system will push high-priority anomalies to monitoring personnel for manual confirmation. The confirmation results will be fed back to the system to update the labeling status and adjust the threshold range. This combination of manual and automated methods can further improve the reliability of anomaly labeling and ensure the accuracy of subsequent analysis.

[0104] In step S3-13, one possible implementation is that the labeled dataset can also be used to train a prediction model for anomalies. For example, the system can use the labeled data as training samples to train a prediction model to predict potential future anomalies. During training, the system will focus on the temporal distribution and numerical characteristics of the anomalies so that the model can learn the patterns of anomaly occurrence. This training of the prediction model can provide technical support for subsequent dynamic monitoring and help identify potential risks in advance.

[0105] Step S3-14: For extended applications of the labeled dataset, the system can also combine external data sources for analysis. For example, the system can incorporate weather forecast data to analyze whether anomalies are related to weather changes. If it is found that high-temperature anomalies frequently occur during high-temperature weather warnings, the system will record this correlation information so that the influence of external factors can be comprehensively considered in subsequent analyses. This introduction of external data can improve the comprehensiveness of anomaly analysis and avoid biased judgments.

[0106] Step S3-15: In one embodiment, the labeled dataset can also be used for multi-dimensional statistical analysis of anomalies. For example, the system can statistically analyze the distribution of anomalies across different time periods, devices, and scenarios, generating a multi-dimensional statistical report. The report may include information such as the daily, weekly, and monthly distribution of anomalies, allowing monitoring personnel to understand the changing patterns of environmental conditions from different perspectives. This multi-dimensional statistical analysis can provide more comprehensive data support for environmental monitoring.

[0107] In step S3-16, regarding the sharing and collaboration of the labeled dataset, the system can also push the data to multiple relevant departments. For example, in an industrial park scenario, the labeled dataset can be pushed to both the safety management department and the equipment maintenance department simultaneously. The safety management department focuses on the potential risks caused by anomalies, while the equipment maintenance department focuses on whether the anomalies are related to equipment malfunctions. Through this data sharing and collaboration, the system can promote information exchange between different departments and improve the overall efficiency of environmental monitoring.

[0108] In step S3-17, one possible implementation is that the labeled dataset can also be used for long-term trend analysis of outliers. For example, the system analyzes the labeled outlier data from the past year to determine if there is a long-term upward or downward trend. If the frequency of outliers in smoke concentration is found to be increasing month by month, the system will indicate a potential risk of worsening environmental pollution, requiring further investigation. This long-term trend analysis can help identify hidden environmental problems and provide a basis for developing long-term prevention and control measures.

[0109] In step S3-18, for further applications of the labeled dataset, the system can also perform spatial analysis by combining geographic information. For example, the system will associate outliers with the geographical location of the data acquisition devices, analyzing whether the outliers are concentrated in certain specific areas. If the frequency of outliers in a certain area is found to be significantly higher than in other areas, the system will indicate that there may be local environmental problems that require close attention. Through this spatial analysis, the system can provide more accurate positioning support for environmental monitoring.

[0110] Step S3-19: In one embodiment, the labeled dataset can also be used to optimize the early warning rules for anomalies. For example, the system analyzes whether the existing threshold range is reasonable based on the labeled anomaly data. If a large number of anomalies are found to be concentrated near the threshold range, the system will suggest adjusting the threshold range to improve the sensitivity of anomaly detection or reduce false alarms. This optimization of early warning rules can continuously improve the system's monitoring capabilities and adapt to the needs of environmental changes.

[0111] In step S3-20, to provide diverse displays for the labeled dataset, the system can also generate dynamic monitoring dashboards. For example, the dashboard can display real-time information such as the number, type distribution, and time distribution of labeled anomalies. Monitoring personnel can quickly understand the current environmental status through the dashboard and take timely countermeasures based on the changing trends of anomalies. This dynamic display method improves the intuitiveness and convenience of monitoring, providing support for decision-making.

[0112] In step S3-21, one possible implementation is that the labeled dataset can also be used for multi-level classification of anomalies. For example, the system can classify anomalies into multiple levels based on their numerical, temporal, and spatial characteristics, with higher levels indicating more severe anomalies that require priority handling. Through this multi-level classification, the system can rationally allocate monitoring resources to ensure that high-risk anomalies receive timely attention.

[0113] In step S3-22, for further expansion of the labeled dataset, the system can also be optimized by incorporating user feedback. For example, when using the system, monitoring personnel can provide feedback on the labeled anomalies, and the system will adjust the labeling rules or threshold ranges based on the feedback to better meet actual needs. This introduction of user feedback continuously improves the system's functionality and enhances the practicality of anomaly labeling.

[0114] Step S3-23: In one embodiment, the labeled dataset can also be used for cross-temporal analysis of outliers. For example, the system analyzes the distribution patterns of outliers in different seasons and months. If it finds that high-temperature outliers are concentrated in summer, the system will suggest that this may be related to seasonal climate change and that monitoring efforts need to be strengthened in summer. Through this cross-temporal analysis, the system can provide more targeted recommendations for environmental monitoring.

[0115] In steps S3-24, regarding the application depth of the labeled dataset, the system can also perform comprehensive analysis by combining other environmental parameters. For example, the system will analyze whether outliers are related to parameters such as air quality index or wind speed. If it is found that smoke concentration anomalies occur more frequently on days with poor air quality, the system will record this correlation information so that the influence of multiple parameters can be comprehensively considered in subsequent analysis. This comprehensive analysis can improve the accuracy of anomaly cause judgment and provide a more comprehensive basis for subsequent processing.

[0116] In step S3-25, one possible implementation is that the labeled dataset can also be used to formulate automated rules for handling outliers. For example, the system can automatically generate handling rules based on the labeled outlier data. If the type and severity of an outlier reach a certain standard, the system will automatically trigger the corresponding processing flow, such as sending an early warning notification or activating emergency equipment. Through such automated handling rules, the system can reduce manual intervention and improve response speed.

[0117] In step S3-26, for continuous optimization of the labeled dataset, the system can also introduce machine learning techniques for anomaly pattern learning. For example, the system will use the labeled data as training samples to train a machine learning model to learn the distribution patterns and occurrence rules of anomalies. After the model is trained, it can be used to predict possible future anomalies and generate early warning information. This continuous optimization method can continuously improve the system's predictive capabilities and provide more intelligent support for environmental monitoring.

[0118] Step S3-27: In one embodiment, the labeled dataset can also be used for multi-scenario comparative analysis of outliers. For example, the system compares outlier data in different scenarios, analyzing the differences in the types and frequencies of outliers between industrial parks and residential communities. If it finds that there are significantly more outliers of smoke concentration in industrial parks than in residential communities, the system will indicate that it may be related to industrial activities and requires further investigation. Through this multi-scenario comparative analysis, the system can provide a reference for the formulation of monitoring strategies for different scenarios.

[0119] In step S3-28, for further applications of the labeled dataset, the system can also perform analysis in conjunction with device status data. For example, the system will check whether outliers are related to the operating status of the data acquisition equipment. If an outlier occurs when the equipment is in a faulty state, the system will mark the outlier as caused by a device malfunction and exclude its influence in subsequent analysis. Through this device status analysis, the system can improve the reliability of outliers and avoid misjudgments caused by device problems.

[0120] In step S3-29, in one possible implementation, the labeled dataset can also be used for multi-dimensional early warning of anomalies. For example, the system can generate multi-dimensional early warning information based on the type, severity, and distribution characteristics of the anomalies. This information can include detailed information about the anomalies, their potential impact range, and suggested countermeasures. Through this multi-dimensional early warning system, the system can provide monitoring personnel with more comprehensive decision support and improve the responsiveness of environmental monitoring.

[0121] In steps S3-30, regarding the extended functionality of the labeled dataset, the system can also integrate with mobile terminals for real-time push notifications. For example, the system will push information about labeled anomalies to the mobile terminals of monitoring personnel in real time. The push notification includes information such as the type, time, and location of the anomalies. Monitoring personnel can use their mobile terminals to understand the environmental status at any time and take timely action based on the pushed information. Through this real-time push notification function, the system can improve the timeliness of anomaly handling and ensure environmental safety.

[0122] The above content, through a detailed description of the steps such as the collection, standardization, and anomaly marking of environmental monitoring data, demonstrates the specific implementation of the method of the present invention in the initial stage of data processing, laying a solid foundation for subsequent multimodal signal analysis and hazard identification.

[0123] The embodiments of the present invention continue to describe the method for environmental monitoring and hazard identification in detail. Based on the processing results of the foregoing steps, the specific implementation of subsequent steps is further elaborated to demonstrate the application of the technical solution of the present invention in multimodal signal analysis, feature extraction and risk assessment.

[0124] Step S4 involves preprocessing the image sequences using the labeled dataset to obtain processed image data. In the preceding steps, the system standardized the temperature and humidity values ​​and smoke concentration data and marked anomalies. Next, the image sequences in the original dataset need to be preprocessed for subsequent comprehensive analysis in conjunction with other modal data. The main purpose of image sequence preprocessing is to improve the quality of the image data, reduce noise interference, and provide clear visual information for feature extraction and anomaly detection.

[0125] Step S4-1, specifically, involves preprocessing the image sequence using a Gaussian filter to sharpen the image frames. The Gaussian filter effectively removes noise points from the image by smoothing the pixels, while preserving the main structural features. During processing, the system reads the image sequence frame by frame, calculates a weighted average of the pixel values ​​in each frame, and generates smoothed image data. The processed image data is clearer in detail, providing a better foundation for subsequent feature extraction.

[0126] In step S4-2, one possible implementation involves adjusting the parameters of the image preprocessing based on the characteristics of the environmental scene. For example, in a dimly lit scene, the system enhances the smoothing intensity of the Gaussian filter to reduce noise interference caused by insufficient light; while in a well-lit scene, the smoothing intensity is appropriately reduced to preserve more image details. Through this adaptive adjustment, the system can ensure that the image preprocessing effect adapts to different environmental conditions, improving the accuracy of subsequent analysis.

[0127] Step S4-3, in one embodiment, image preprocessing may further include brightness and contrast adjustments. For example, the system analyzes the brightness distribution of each frame in the image sequence. If it finds that the brightness value is too low or the contrast is insufficient, it will automatically adjust the relevant parameters to make the target object in the image more prominent. This adjustment method can help the subsequent feature extraction process better identify key information in the image, especially in scenes with large changes in ambient light.

[0128] Step S4-4, for further optimization of image preprocessing, the system can also perform temporal smoothing on the image sequence. For example, the system analyzes pixel changes across multiple consecutive frames. If it finds that the pixel values ​​of a certain frame differ too much from those of the preceding and following frames, it will perform interpolation to reduce image jumps caused by device jitter or rapid object movement. This temporal smoothing improves the continuity of the image sequence, providing support for subsequent time-series analysis.

[0129] In steps S4-5, one possible implementation involves providing a preliminary visual representation of the image preprocessing results. For example, the system can compare the processed image data with the original image data, allowing monitoring personnel to intuitively understand the preprocessing effect through a visual interface. If significant noise or distortion is still found in the processed image, the system will prompt adjustments to the preprocessing parameters to ensure the image quality meets the requirements for subsequent analysis.

[0130] Step S5: Based on the processed image data, combined with the labeled temperature, humidity, and smoke concentration data, the Support Vector Machine (SVM) algorithm is applied to classify the multimodal signals and determine the environmental state category. After preprocessing the image sequence, the system integrates the processed image data with other modal data and uses a classification algorithm to make a preliminary judgment on the environmental state. The SVM algorithm, by constructing an optimal classification hyperplane, can effectively distinguish different environmental state categories, providing a basis for subsequent comprehensive analysis.

[0131] Step S5-1, specifically, involves the Support Vector Machine (SVM) algorithm's classification process first extracting feature vectors from the multimodal signals. For example, the system extracts texture and color features from processed image data, numerical features from labeled temperature and humidity values, and concentration change features from smoke concentration data. These feature vectors serve as input to the SVM algorithm for training and classification. During classification, the system determines whether the environmental state belongs to a normal, slightly abnormal, or severely abnormal category based on the distribution of the feature vectors.

[0132] In step S5-2, one possible implementation is that feature vector extraction can be optimized by incorporating the characteristics of the environmental scene. For example, in an industrial park scene, the system will focus on extracting smoke texture features and peak features of smoke concentration data from the image data to more accurately identify fire-related anomalies; in a residential community scene, it will focus on extracting fluctuation features of temperature and humidity data and object movement features from the image data to identify environmental comfort-related anomalies. Through this scenario-based feature extraction, the system can improve the targeting of classification.

[0133] Step S5-3: In one embodiment, the classification results of the support vector machine algorithm can be preliminarily verified. For example, the system compares the classification results with historical data. If the current classification result is consistent with the classification results of similar historical scenes, the classification is considered to have high reliability; if there is a large deviation, it will indicate that there may be improper feature extraction or classification parameter settings, requiring further adjustment. Through this verification method, the system can continuously optimize the classification effect.

[0134] Step S5-4, for further application of the classification process, the system can also generate an environmental status report based on the classification results. For example, the report includes information such as the current environmental status category, the distribution of key characteristics, and the classification confidence level. Monitoring personnel can quickly understand the environmental status through the report and take appropriate measures based on the classification results. This report generation method improves the intuitiveness of monitoring and the efficiency of decision-making.

[0135] In step S5-5, in one possible implementation, the Support Vector Machine (SVM) algorithm can also be combined with other classification methods for result fusion. For example, the system compares the classification results of the SVM algorithm with those of rule-based classification methods. If the results of the two methods are consistent, the reliability of the classification result is increased; if they are inconsistent, the result of the SVM algorithm is used first, and the inconsistency is recorded for subsequent analysis. Through this result fusion, the system can improve the robustness of classification.

[0136] Step S6 involves calculating the Pearson correlation coefficient between temperature, humidity, and smoke concentration based on the categorized environmental states, thereby obtaining the overall trend of environmental parameter changes. After determining the environmental state categories, the system needs to further analyze the correlation between different environmental parameters to grasp the overall pattern of environmental changes. The Pearson correlation coefficient, by calculating the linear correlation between two variables, reflects the relationship between temperature, humidity, and smoke concentration, providing a reference for subsequent dynamic monitoring.

[0137] Step S6-1, specifically, involves pairing temperature and humidity values ​​with smoke concentration data during the Pearson correlation coefficient calculation. For example, the system will pair temperature, humidity, and smoke concentration values ​​at the same time point in chronological order, and then calculate the correlation between these pairs. If the calculation shows a positive correlation between temperature and smoke concentration, it indicates that an increase in temperature may be accompanied by an increase in smoke concentration, requiring further attention to potential risk factors.

[0138] In step S6-2, one possible implementation involves combining correlation coefficient calculation with time windows. For example, the system might calculate correlation coefficients for the past hour, day, and week to understand the changing relationships of environmental parameters across different time scales. If a high correlation coefficient is found over a short period while a low correlation coefficient is found over a long period, it indicates a potential short-term correlation between environmental parameters, requiring close attention to short-term trends. Through this multi-time-scale analysis, the system can gain a more comprehensive understanding of environmental change patterns.

[0139] Step S6-3: In one embodiment, the correlation coefficient result can also be used for preliminary early warning of environmental parameters. For example, if the correlation coefficient between humidity and smoke concentration is found to be continuously increasing and reaching a certain threshold, the system will indicate that there may be a humid environment that restricts smoke diffusion, requiring enhanced ventilation or dehumidification measures. Through this early warning method, the system can detect potential problems in advance and prevent further escalation of risks.

[0140] In step S6-4, for further applications of correlation coefficient calculation, the system can also combine the classified environmental state categories for analysis. For example, if the environmental state category is "slightly abnormal," and the correlation coefficient between temperature / humidity values ​​and smoke concentration values ​​is high, the system will further analyze whether the abnormality is related to specific environmental parameters, in order to provide a more accurate direction for subsequent processing. Through this combined analysis, the system can improve the accuracy of determining the cause of the abnormality.

[0141] In step S6-5, one possible implementation is that the correlation coefficient results can also be visualized. For example, the system can plot the correlation coefficients between different environmental parameters as a heatmap, where the intensity of the color indicates the strength of the correlation. Monitoring personnel can intuitively understand the relationship between parameters through the heatmap and adjust monitoring priorities based on the strength of the correlation. This visualization method improves the intuitiveness and convenience of monitoring.

[0142] Step S7: Based on the overall trend, dynamically monitor the fluctuations of environmental parameters to determine if there are any persistent abnormal signal characteristics, and obtain the final monitoring results. After obtaining the overall trend of environmental parameters, the system needs to monitor their fluctuations in real time to promptly detect persistent abnormal signal characteristics. Dynamic monitoring, through continuous tracking of data changes, can support real-time assessment of environmental conditions.

[0143] Step S7-1, specifically, the dynamic monitoring process can be implemented based on time series analysis technology. For example, the system will continuously track the changes in temperature, humidity, and smoke concentration values ​​in chronological order. If a parameter is found to continuously exceed the normal range for a period of time, it will be marked as a persistent abnormal signal. The judgment of persistent abnormal signals can also be combined with features such as the slope and fluctuation amplitude of the trend to improve the accuracy of the judgment.

[0144] In step S7-2, one possible implementation is that dynamic monitoring can also be adjusted according to the characteristics of the environmental scenario. For example, in an industrial park scenario, the system will focus on monitoring fluctuations in smoke concentration. If the concentration value is found to rise continuously in a short period of time, an early warning message will be generated immediately. In a residential community scenario, the system will focus on monitoring fluctuations in temperature and humidity. If the temperature value is found to be consistently higher than the comfortable range, it will indicate that it may affect the living experience and that control measures need to be taken. Through this scenario-based monitoring, the system can improve the targeting of early warnings.

[0145] Step S7-3: In one embodiment, the results of dynamic monitoring can also be comprehensively analyzed in conjunction with the aforementioned classification results. For example, if dynamic monitoring detects continuous abnormal signals and the environmental state category is severely abnormal, the system will raise the warning level and push relevant information to relevant departments so that timely countermeasures can be taken. Through this comprehensive analysis, the system can ensure the comprehensiveness and reliability of the monitoring results.

[0146] Step S7-4: For further optimization of dynamic monitoring, the system can also introduce a duration threshold for abnormal signals. For example, the system can set a time threshold; only when the duration of an abnormal signal exceeds this threshold will it be identified as a continuous abnormal signal, avoiding false alarms caused by short-term fluctuations. By setting this time threshold, the system can improve the accuracy of monitoring.

[0147] In step S7-5, in one possible implementation, the results of dynamic monitoring can also be used for dynamic adjustment of the monitoring strategy. For example, if the system detects a high fluctuation frequency of a certain environmental parameter, it will automatically increase the monitoring frequency of that parameter to capture changes more promptly; if the fluctuation frequency is low, the monitoring frequency will be appropriately reduced to decrease resource consumption. Through this dynamic adjustment, the system can rationally allocate monitoring resources and improve monitoring efficiency.

[0148] Step S7-6, regarding the application expansion of dynamic monitoring, the system can also incorporate external environmental information for analysis. For example, the system can incorporate weather change data. If persistent abnormal signals are found to be highly correlated with weather changes—for instance, high-temperature abnormal signals often occur during periods of high temperatures—the system will record this correlation information so that the influence of external factors can be comprehensively considered in subsequent analysis. By introducing this external information, the system can improve the interpretability of the monitoring results.

[0149] In step S7-7, in one embodiment, the results of dynamic monitoring can also generate a real-time monitoring report. For example, the report may include information such as fluctuations in current environmental parameters, the distribution of persistent abnormal signals, and possible cause analyses. Monitoring personnel can quickly understand the environmental status through the report and formulate response strategies based on its content. This report generation enhances the system's decision support capabilities.

[0150] In steps S7-8, to provide diverse displays for dynamic monitoring, the system can also generate dynamic monitoring graphs. For example, the graphs will display the real-time trends of temperature, humidity, and smoke concentration values. Continuous abnormal signals are highlighted in different colors or marked on the graphs. Monitoring personnel can intuitively understand the fluctuations of environmental parameters through the graphs and take timely action based on trend changes. Through this dynamic display, the system can improve the intuitiveness and convenience of monitoring.

[0151] In steps S7-9, one possible implementation involves combining dynamic monitoring with multimodal signals for comprehensive judgment. For example, the system analyzes whether persistent abnormal signals are associated with certain features in the image sequence. If significant changes in light or object movement are detected in the image when abnormal temperature and humidity signals occur, the system records this association information to consider the synergistic effect of multimodal signals in subsequent comprehensive analysis. Through this comprehensive judgment, the system can improve the comprehensiveness of anomaly detection.

[0152] In step S8, for further applications of dynamic monitoring, the system can also generate environmental control recommendations based on the monitoring results. For example, if the system detects that the temperature value is consistently higher than the normal range, it recommends activating cooling equipment; if the smoke concentration value continues to rise, it recommends strengthening ventilation or activating smoke extraction equipment. Through the generation of such control recommendations, the system can provide practical support for environmental management and reduce the impact of abnormal signals on the environment.

[0153] Step S8-1: In one embodiment, the results of dynamic monitoring can also be used for predictive analysis of abnormal signals. For example, based on the current trend of persistent abnormal signals, the system can predict possible abnormal situations in the future. If the prediction results indicate that the abnormal signals may worsen, the system will generate early warning information so that monitoring personnel can take preventive measures in advance. Through this predictive analysis, the system can improve the foresight of monitoring.

[0154] In step S8-2, for continuous optimization of dynamic monitoring, the system can also introduce a feedback mechanism for parameter adjustment. For example, based on feedback on the accuracy of monitoring results, the system will adjust the judgment threshold or time threshold for continuous abnormal signals. If the false alarm rate is high, the threshold will be appropriately increased to reduce false alarms; if the missed alarm rate is high, the threshold will be decreased to increase sensitivity. Through this feedback mechanism, the system can continuously optimize the monitoring effect.

[0155] In step S8-3, one possible implementation is that the results of dynamic monitoring can also be used for multi-department collaboration. For example, the system can push information on continuous abnormal signals to the safety management department and the environmental control department. The safety management department assesses the risk level based on the abnormal signals, while the environmental control department formulates control plans based on the abnormal signals. Through this multi-department collaboration, the system can improve the overall efficiency of anomaly handling.

[0156] Step S8-4: For extended dynamic monitoring functions, the system can also integrate with mobile terminals for real-time notifications. For example, the system will push detailed information about persistent abnormal signals to the monitoring personnel's mobile terminals in real time. The push content includes information such as the type of abnormal signal, its duration, and potential impact. Monitoring personnel can use their mobile terminals to understand the environmental status at any time and take timely action based on the pushed information. Through this real-time notification function, the system can improve the timeliness of anomaly handling.

[0157] Step S8-5: In one embodiment, the results of dynamic monitoring can also be used for long-term trend analysis of environmental conditions. For example, the system analyzes the distribution patterns of persistent abnormal signals over a period of time. If it finds that the frequency of abnormal signals for a certain parameter increases month by month, the system will indicate a potential risk of exacerbating environmental problems, requiring further investigation. Through this long-term trend analysis, the system can provide a basis for formulating long-term prevention and control measures.

[0158] Step S8-6: For multi-scenario applications of dynamic monitoring, the system can also generate customized monitoring reports based on the characteristics of different scenarios. For example, in an industrial park scenario, the report will highlight the distribution and changing trends of abnormal smoke concentration signals; in a residential community scenario, the report will highlight the impact range of abnormal temperature and humidity signals and control recommendations. Through this customized reporting, the system can better serve the actual needs of different scenarios.

[0159] In step S8-7, one possible implementation is that the results of dynamic monitoring can also be used for cross-temporal analysis of anomalous signals. For example, the system can analyze the distribution patterns of persistent anomalous signals across different seasons and months. If it finds that high-temperature anomalous signals are concentrated in summer, the system will suggest that this may be related to seasonal climate change and that monitoring efforts should be strengthened in summer. Through this cross-temporal analysis, the system can provide more targeted recommendations for environmental monitoring.

[0160] Step S8-8, further expanding dynamic monitoring, allows the system to incorporate geographic information for spatial analysis. For example, the system can correlate persistent abnormal signals with the geographical location of the acquisition devices, analyzing whether the abnormal signals are concentrated in specific areas. If the frequency of abnormal signals in a certain area is significantly higher than in other areas, the system will indicate a potential local environmental problem requiring close monitoring. Through this spatial analysis, the system can provide more precise location support for environmental monitoring.

[0161] Steps S8-9: In one embodiment, the results of dynamic monitoring can also be used to evaluate the performance of the monitoring system. For example, the system will statistically analyze the detection rate and false alarm rate of persistent abnormal signals to assess the effectiveness of the current monitoring strategy. If the detection rate is found to be low or the false alarm rate is high, the system will suggest adjusting the monitoring parameters or introducing new analysis methods. Through this performance evaluation, the system can continuously improve its monitoring capabilities.

[0162] Step S9: For diverse applications of dynamic monitoring, the system can also generate multi-dimensional monitoring dashboards. For example, the dashboard can display real-time information such as the number, type distribution, and time distribution of continuous abnormal signals. Monitoring personnel can quickly understand the current environmental status through the dashboard and take timely countermeasures based on the changing trends of abnormal signals. This dynamic display method improves the intuitiveness and convenience of monitoring.

[0163] In step S9-1, one possible implementation is that the results of dynamic monitoring can also be used for multi-level classification of abnormal signals. For example, the system can classify abnormal signals into multiple levels based on their numerical, temporal, and spatial characteristics. Higher levels indicate more severe anomalies that require priority handling. Through this multi-level classification, the system can rationally allocate monitoring resources to ensure that high-risk abnormal signals receive timely attention.

[0164] Step S9-2, for further optimization of dynamic monitoring, the system can also introduce a manual review mechanism. For example, the system will push high-level continuous abnormal signals to monitoring personnel for manual confirmation. The confirmation result will be fed back to the system to update the classification status of the abnormal signals and adjust the monitoring strategy. Through this combination of manual and automatic methods, the system can further improve the reliability of abnormal signal judgment.

[0165] Step S9-3: In one embodiment, the results of dynamic monitoring can also be used to train a predictive model for abnormal signals. For example, the system uses data on continuous abnormal signals as training samples to train a predictive model to predict future abnormal signals. During training, the system focuses on the temporal distribution and numerical characteristics of the abnormal signals so that the model can learn the patterns of abnormal occurrence. Through this predictive model training, the system can provide technical support for subsequent dynamic monitoring and identify potential risks in advance.

[0166] In step S9-4, to further explore the application depth of dynamic monitoring, the system can also combine external data sources for comprehensive analysis. For example, the system can introduce external data such as air quality index or wind speed to analyze whether persistent abnormal signals are related to these parameters. If it is found that abnormal smoke concentration signals occur more frequently on days with poor air quality, the system will record this correlation information so that the influence of multiple parameters can be comprehensively considered in subsequent analyses. Through this comprehensive analysis, the system can improve the accuracy of determining the causes of anomalies.

[0167] In step S9-5, in one possible implementation, the results of dynamic monitoring can also be used for multi-dimensional early warning of abnormal signals. For example, the system can generate multi-dimensional early warning information based on the type, intensity, and distribution characteristics of persistent abnormal signals. This early warning information may include detailed information about the abnormal signals, their potential impact range, and suggested countermeasures. Through this multi-dimensional early warning system, the system can provide monitoring personnel with more comprehensive decision support and improve the responsiveness of environmental monitoring.

[0168] Step S9-6: For continuous improvement of dynamic monitoring, the system can also be optimized based on user feedback. For example, when using the system, monitoring personnel can provide feedback on the judgment results of continuous abnormal signals. The system will adjust the judgment rules or threshold ranges based on the feedback to better meet actual needs. Through the introduction of this user feedback, the system can continuously improve its monitoring functions and enhance the practicality of abnormal signal judgment.

[0169] Step S9-7, in one embodiment, the results of dynamic monitoring can also be used for the coordinated control of environmental control equipment. For example, the system will automatically generate control commands based on the type and severity of continuous abnormal signals. If a continuous temperature abnormality is detected, the cooling equipment will be activated; if a continuous smoke concentration abnormality is detected, the smoke extraction equipment will be activated. Through this coordinated control, the system can achieve rapid response to abnormal signals and reduce the impact of abnormalities on the environment.

[0170] In steps S9-8, for multi-departmental collaboration in dynamic monitoring, the system can also push information on continuous abnormal signals to multiple relevant departments. For example, in an industrial park scenario, the system will simultaneously push abnormal signal information to the safety management department and the equipment maintenance department. The safety management department focuses on the potential risks of the abnormal signals, while the equipment maintenance department focuses on whether the abnormal signals are related to equipment malfunctions. Through this data sharing and collaboration, the system can promote information exchange between different departments and improve the overall efficiency of environmental monitoring.

[0171] In step S9-9, in one possible implementation, the results of dynamic monitoring can also be used for long-term trend analysis of abnormal signals. For example, the system analyzes data on persistent abnormal signals over the past year to determine if there is a long-term upward or downward trend. If the frequency of abnormal smoke concentration signals is found to be increasing month by month, the system will indicate a potential risk of worsening environmental pollution, requiring further investigation. Through this long-term trend analysis, the system can help identify hidden environmental problems and provide a basis for developing long-term prevention and control measures.

[0172] Step S10: Further expanding dynamic monitoring, the system can also integrate with mobile terminals for real-time push notifications. For example, the system will push detailed information about continuous abnormal signals to the monitoring personnel's mobile terminal devices in real time. The push notification content includes information such as the type, time, and location of the abnormal signal. Monitoring personnel can use their mobile terminals to understand the environmental status at any time and take timely action based on the pushed information. Through this real-time push notification function, the system can improve the timeliness of anomaly handling and ensure environmental safety.

[0173] In the description of this invention, it should be understood that the terms "coaxial," "bottom," "one end," "top," "middle," "other end," "upper," "side," "top," "inner," "front," "center," "both ends," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0174] Furthermore, the terms “first,” “second,” “third,” and “fourth” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as “first,” “second,” “third,” or “fourth” may explicitly or implicitly include at least one of those features.

[0175] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "screw connection," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0176] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A warehouse material management method based on intelligent inspection robots, characterized in that, include: Environmental monitoring data is collected through a data acquisition device. The environmental monitoring data includes temperature and humidity information, smoke concentration, and image sequences to obtain a raw multimodal signal set. Based on the original multimodal signal set, a deep learning model is used to extract feature vectors to obtain multidimensional feature representations for subsequent fusion processing; Prioritize anomaly detection from the multi-dimensional feature representation to identify a subset of high-risk signals and potential hazards. If the high-risk signal subset meets the triggering conditions, image sequence analysis is performed to determine the abnormal event and obtain a comprehensive abnormal label. If the high-risk signal subset meets the triggering condition, image sequence analysis is performed to determine the abnormal event and obtain a comprehensive abnormal label, including: If a data stream containing a subset of high-risk signals is received, a preliminary screening is performed by comparing it against a preset threshold to determine whether there is a signal combination that meets the triggering conditions, and a preliminary risk assessment result is obtained. If the preliminary risk assessment results show that there are triggering conditions, the corresponding image sequence data is obtained from the repository, and the image sequence is preprocessed using a standardized processing method to determine the first image dataset after processing. Based on the first image dataset, a pre-trained convolutional neural network model is invoked to extract features from the image sequence, obtain the visual feature vector of each frame image, and obtain the second image dataset after feature extraction. For the second image dataset, if there are outliers in the feature vectors that deviate from a preset range, the relevant frames are marked as potential abnormal events, and the set of potential abnormal events is determined. By performing time-series correlation analysis on the set of potential abnormal events, the continuous distribution of abnormal events in the time dimension is obtained, it is determined whether there are persistent abnormal events, and the final list of abnormal events is obtained. Based on the final list of abnormal events and combined with predefined label mapping rules, corresponding comprehensive abnormal labels are generated to determine the final abnormal classification result. If the final anomaly classification result needs further verification, the comprehensive anomaly label will be compared with historical data records to obtain the consistency verification result and draw the final business judgment conclusion. A fusion mechanism is used to weight the comprehensive anomaly label and dynamic response to obtain a real-time risk assessment score. Based on the real-time risk assessment score, the sequence prediction model is used to predict the trend of potential hazards, determine the alarm conditions, generate a response sequence, and obtain the hazard identification results for environmental control. A feedback mechanism is used to adjust the data acquisition parameters based on the hazard identification results, update the original multimodal signal set, and repeat the processing to obtain an optimized risk assessment score for continuous optimization.

2. The warehouse material management method based on an intelligent inspection robot according to claim 1, characterized in that, The process involves collecting environmental monitoring data via a data acquisition device. This environmental monitoring data includes temperature and humidity information, smoke concentration, and image sequences, to obtain a raw multimodal signal set, including: Environmental monitoring data, including temperature and humidity values, smoke concentration, and image sequences, are continuously acquired through a data acquisition device to construct an initial multimodal signal set and obtain the original dataset. Based on the original dataset, the temperature and humidity values ​​and smoke concentration were standardized. The mean was subtracted from each value and the standard deviation was divided. The standardized data were then preliminarily screened using a preset threshold range to determine whether there were any outliers. If the temperature, humidity or smoke concentration exceeds the preset threshold range, the abnormal data points are marked and the marked dataset is obtained. Using the labeled dataset, preprocessing operations are performed on the image sequence. A Gaussian filter is used to sharpen the image sequence, and a pixel-weighted average is calculated from each image frame to obtain the processed image data. Based on the processed image data, combined with the labeled temperature and humidity values ​​and smoke concentration data, the support vector machine algorithm is applied to classify the multimodal signals, extract feature vectors from the combined data, and train the model to determine the environmental state category. By classifying the environmental state categories, Pearson correlation coefficients are calculated for temperature and humidity values ​​and smoke concentration to obtain the comprehensive trend of environmental parameter changes. Based on the overall trend of change, dynamic monitoring is carried out on the fluctuation of environmental parameters to determine whether there are any continuous abnormal signal characteristics, and the final monitoring results are obtained.

3. The warehouse material management method based on an intelligent inspection robot according to claim 1, characterized in that, The step of extracting feature vectors from the original multimodal signal set using a deep learning model to obtain multidimensional feature representations for subsequent fusion processing includes: By processing multimodal signals, key information in the original set is obtained, and a preliminary data structure is derived. Based on the preliminary data structure, a deep learning model is used to analyze the multimodal signals and determine the generation path of the feature vectors. If the feature vector generation path meets the preset threshold conditions, the multidimensional feature representation is obtained through the extraction process, and it is determined whether it meets the fusion requirements. If the representation of the multidimensional features meets the fusion requirements, then the subsequent fusion processing step is adopted to obtain the fused feature set and determine its consistency. Based on the fused feature set, the final feature representation is obtained through further optimization of the processing steps, resulting in the integrated data result. By integrating the data results and using a pre-established verification mechanism, we determine its applicability in multimodal signal analysis and obtain the verified feature output. Based on the verified feature output, the integrity of the multimodal signal is determined by retrospective analysis of the signal source, and the final processing conclusion is obtained.

4. The warehouse material management method based on an intelligent inspection robot according to claim 1, characterized in that, The step of prioritizing anomaly detections from the multi-dimensional feature representations to determine a subset of high-risk signals for identifying potential hazards includes: By extracting data from multi-dimensional features and using pre-established filtering rules, key dimension data is separated to obtain preliminary filtering results. Based on the preliminary screening results, a priority ranking mechanism is applied to the key dimension data. If the indicator of a certain dimension exceeds the preset threshold, it is classified as a high-risk signal, and a set of high-risk signals is determined. For high-risk signal sets, a signal classification method is used to divide them into different signal subsets and obtain the classified signal subset data; From the classified subset of signal data, a subset related to anomaly detection is obtained. If the fluctuation pattern of a subset matches the preset anomaly pattern, it is marked as a potential hazard, and a subset of potential hazards is identified. Based on the subset of potential hazards, and combined with the risk assessment model, the risk level of each subset is calculated to obtain the risk level distribution; Based on the risk level distribution, a hidden Markov model is used to further identify potential hazards for a subset of high-risk individuals, thus determining the final set of hazard signals. Obtain the final set of potential hazard signals, and combine data analysis methods to trace the source dimensions of the potential hazard signals and determine the root cause dimensions of the potential hazard signals.

5. The warehouse material management method based on an intelligent inspection robot according to claim 1, characterized in that, The method employs a fusion mechanism to weight the comprehensive anomaly label and dynamic response to obtain a real-time risk assessment score, including: By using a fusion mechanism, key information is extracted from comprehensive anomaly labels and dynamic response data to obtain a preliminary set of anomaly features; Based on the preliminary set of abnormal features, a weighted calculation method is used to rank the importance of each abnormal feature and determine the weighted feature weight distribution. If the weight of an abnormal feature in the feature weight distribution exceeds a preset threshold, it is marked as a high-risk feature, and a subset of high-risk features is obtained. For high-risk feature subsets, real-time calculations are performed using dynamic response data to determine whether persistent abnormal patterns exist; If a persistent anomalous pattern is detected, the anomalous pattern is classified using a pre-established random forest model to obtain the classified risk category. Based on the classified risk categories and the weighted calculation results, a final real-time risk assessment score is generated. By mapping real-time risk assessment scores across intervals, the corresponding risk level classification is determined.

6. The warehouse material management method based on an intelligent inspection robot according to claim 1, characterized in that, The process of predicting hazard trends using a sequence prediction model based on the real-time risk assessment score, determining alarm conditions, generating response sequences, and obtaining hazard identification results for environmental control includes: To obtain real-time risk assessment scores, environmental data is collected from multiple sensor nodes through a data acquisition system. Preprocessing techniques are used to clean and standardize the collected data to obtain a standardized risk score dataset. For a standardized risk score dataset, a time series forecasting model is used to analyze the data's changing patterns, identify potential hidden danger trends, and determine the key time points of trend changes. Based on the changing characteristics of the potential hazard trend, if the trend is detected to deviate from the preset threshold range, an alarm condition is triggered, and the corresponding alarm signal sequence is generated through the information processing module. For the generated alarm signal sequence, a pre-established response rule base is used for matching to obtain the corresponding response sequence, and the priority and execution order of the response sequence are determined. By analyzing the execution order of the response sequence, specific classification results for hazard identification are generated, determining the hazard category and its impact range. Based on the classification results of hazard identification, an environmental control instruction set is generated. The instruction set is then sent to the control equipment via the data transmission module to obtain feedback on the adjustment of environmental parameters. For the feedback on the adjustment of environmental parameters, information comparison technology is used to analyze the deviation between the feedback data and the expected target. If the deviation exceeds the preset range, the control instruction set is regenerated to determine the final environmental control result.

7. The warehouse material management method based on an intelligent inspection robot according to claim 1, characterized in that, The step involves adjusting data acquisition parameters based on the hazard identification results using a feedback mechanism, updating the original multimodal signal set, and repeating the processing to obtain an optimized risk assessment score for continuous optimization. This includes: The initial multimodal signal set is obtained through the hazard identification module, and the signals are preliminarily analyzed using a pre-established classification model to obtain preliminary hazard classification results. Based on the preliminary hazard classification results, the feedback mechanism module is triggered to adjust the data acquisition parameter configuration for abnormal signals in the classification results and determine the updated acquisition strategy. The updated acquisition strategy is adopted to reacquire the multimodal signal set. For the newly acquired signal data, the support vector machine model is used to extract features to obtain the signal dataset after feature processing. If there are outliers in the signal dataset after feature processing that deviate from the preset threshold, the outliers are filtered a second time through the loop processing module to determine whether there are potential hidden danger signals. Based on the potential hazard signals after secondary filtering, the risk assessment module comprehensively scores the signal dataset to obtain an optimized risk assessment score. By continuously improving the module, the optimized risk assessment score is compared with historical data. If the comparison results show that the accuracy improvement is insufficient, the weight parameters of feature extraction are adjusted to obtain the updated signal processing rules. Based on the updated signal processing rules, the acquisition and analysis of the multimodal signal set are re-executed, and the process is iterated in a loop to continuously improve the evaluation accuracy.

8. A warehouse material management system based on an intelligent inspection robot, configured as the warehouse material management method based on an intelligent inspection robot as described in any one of claims 1-7, characterized in that, The system includes: The data acquisition module is configured to acquire environmental monitoring data through a data acquisition device. The environmental monitoring data includes temperature and humidity information, smoke concentration, and image sequences, and the step of acquiring the raw multimodal signal set. The feature extraction module is configured to extract feature vectors from the original multimodal signal set using a deep learning model to obtain multidimensional feature representations for subsequent fusion processing. The priority ranking module is configured to prioritize anomaly detections from the multi-dimensional feature representations and determine a subset of high-risk signals to identify potential hazards. The image analysis module is configured to perform image sequence analysis, determine abnormal events, and obtain comprehensive abnormal labels if the high-risk signal subset meets the triggering conditions. If the high-risk signal subset meets the triggering condition, image sequence analysis is performed to determine the abnormal event and obtain a comprehensive abnormal label, including: If a data stream containing a subset of high-risk signals is received, a preliminary screening is performed by comparing it against a preset threshold to determine whether there is a signal combination that meets the triggering conditions, and a preliminary risk assessment result is obtained. If the preliminary risk assessment results show that there are triggering conditions, the corresponding image sequence data is obtained from the repository, and the image sequence is preprocessed using a standardized processing method to determine the first image dataset after processing. Based on the first image dataset, a pre-trained convolutional neural network model is invoked to extract features from the image sequence, obtain the visual feature vector of each frame image, and obtain the second image dataset after feature extraction. For the second image dataset, if there are outliers in the feature vectors that deviate from a preset range, the relevant frames are marked as potential abnormal events, and the set of potential abnormal events is determined. By performing time-series correlation analysis on the set of potential abnormal events, the continuous distribution of abnormal events in the time dimension is obtained, it is determined whether there are persistent abnormal events, and the final list of abnormal events is obtained. Based on the final list of abnormal events and combined with predefined label mapping rules, corresponding comprehensive abnormal labels are generated to determine the final abnormal classification result. If the final anomaly classification result needs further verification, the steps are to compare the comprehensive anomaly label with historical data records, obtain the consistency verification result, and draw the final business judgment conclusion. The fusion processing module is configured to use a fusion mechanism to weight the comprehensive anomaly label and dynamic response to obtain a real-time risk assessment score. The trend prediction module is configured to predict the trend of potential hazards based on the real-time risk assessment score through a sequence prediction model, determine alarm conditions and generate a response sequence, and obtain the hazard identification results for environmental control. The parameter adjustment module is configured to adjust the data acquisition parameters using a feedback mechanism based on the hazard identification results, update the original multimodal signal set, and repeat the processing to obtain an optimized risk assessment score for continuous optimization.

Images