Garbage classification transfer station supervision method, device and equipment and storage medium

Abstract

The invention discloses a garbage classification transfer station supervision method, apparatus and device, and a storage medium. The method comprises the steps of collecting multi-modal data of a garbage classification transfer station; carrying out primary processing and analysis on the preprocessed multi-modal data by utilizing local equipment; data encryption is carried out by utilizing a block chain network, secondary processing is carried out on the data by utilizing a cloud end, and comprehensive analysis and identification of abnormal conditions in the garbage classification transfer station are carried out by utilizing a multi-modal deep fusion network; generating an alarm event based on the fused comprehensive analysis result, and automatically controlling field equipment in the garbage classification transfer station to respond; a real-time video and multi-modal data are remotely viewed through a platform user, and related data fragments of an alarm event are retained. Through local and cloud combined processing, the real-time performance of processing is considered, and meanwhile, the high efficiency of processing is ensured. And meanwhile, the block chain technology is utilized to encrypt the data and generate a digital signature, so that the security of the data in the transmission and storage process is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of garbage classification, and more specifically to a method, device, equipment and storage medium for monitoring a garbage classification transfer station. Background Art

[0002] With the acceleration of urbanization and the improvement of environmental protection awareness, garbage sorting and transfer have become important tasks in urban management. In order to improve the efficiency and environmental protection of garbage disposal, intelligent monitoring systems are increasingly used in garbage sorting and transfer stations. These systems usually use multimodal data (such as video and audio data) for real-time monitoring and data analysis to optimize the garbage sorting and transfer process.

[0003] However, traditional monitoring systems have the following problems: Data security: The amount of multimodal data collected is large and sensitive, and it is vulnerable to attacks during transmission and storage, which poses a risk of data leakage and tampering. Traditional methods are difficult to effectively protect the integrity and privacy of data. Real-time data: Traditional monitoring systems usually rely on uploading large amounts of data to the cloud for processing, resulting in high bandwidth usage and high latency for data transmission, which affects the real-time and response speed of the system. In addition, the computing load of cloud processing is also large, making it difficult to effectively cope with high-concurrency real-time data processing needs. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method, device, equipment and storage medium for supervising a garbage classification transfer station.

[0005] To achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for supervising a garbage classification transfer station, comprising: Collecting multimodal data of the garbage sorting transfer station, wherein the multimodal data includes audio and video data; Preprocess the collected multimodal data; Use local equipment at the waste sorting transfer station to perform preliminary processing and analysis on the pre-processed multimodal data; Encrypt and digitally sign the preliminarily processed and analyzed multimodal data and upload it to the cloud blockchain network; Get uploaded data from the cloud blockchain network; The uploaded data is obtained and features are extracted from data of different modalities using a deep learning algorithm to form feature representations of each modality; The features of different modal data are integrated, and a multimodal deep fusion network is used to conduct comprehensive analysis and identify abnormal situations in garbage sorting transfer stations; Generate alarm events based on the integrated analysis results, and automatically control the on-site equipment in the waste sorting transfer station to respond; Through the platform, users can remotely view real-time video and multimodal data, and retain relevant data fragments of alarm events.

[0006] Furthermore, the local equipment of the garbage classification transfer station is used to perform preliminary processing and analysis on the pre-processed multimodal data, including: Use lightweight models to perform preliminary feature extraction on the data; Compress the processed data to reduce the amount of transmitted data.

[0007] Furthermore, the multimodal data after preliminary processing and analysis is encrypted and digitally signed, and uploaded to the cloud blockchain network, including: A hash algorithm is used to generate a hash value of the data; Generate public and private key pairs; Generate a digital signature using a signature algorithm; Pack the data, timestamp and hash value of the previous data block into blocks; Broadcast the packaged blocks to each node in the blockchain network; A consensus algorithm is used to ensure the consistency and reliability of blockchain data.

[0008] Furthermore, the obtaining of uploaded data from the cloud blockchain network includes: Query target data based on data hash value in the blockchain network; Use the private key to decrypt the queried data; Verify the integrity and authenticity of decrypted data and digital signatures; Temporarily store the decrypted and verified data in a cache; Convert multimodal data in different formats temporarily stored in the cache into the input format required by the deep learning model.

[0009] Furthermore, the uploaded data is obtained by using a deep learning algorithm to extract features from data of different modes to form feature representations of each mode, a convolutional neural network is used to extract features from video data in the garbage classification transfer station, and a recurrent neural network is used to extract features from audio data in the garbage classification transfer station, specifically including: Perform convolution operations on video data layer by layer to extract low-level to high-level image features; Concatenate the extracted features into a feature vector; Extract temporal features from audio data to capture long-term dependencies in audio signals.

[0010] Furthermore, the features of different modal data are fused, and a multimodal deep fusion network is used to conduct comprehensive analysis and identify abnormal situations in the garbage classification transfer station, including: Used to normalize the features of different modes to make them have the same scale; Assign weights to each modal feature and perform weighted averaging; Concatenate different modal features sequentially into a long feature vector; Classify the comprehensive features through the classifier; Anomalous data points are identified through isolation forest or density peak clustering methods.

[0011] Furthermore, the generating of alarm events based on the integrated analysis results after fusion and automatically controlling the on-site equipment in the garbage classification transfer station to respond include: Stores the rules and thresholds for alarm generation; Match the analysis results with the rules in the rule base to generate alarm events; When an alarm event is generated, the voice alarm device at the garbage classification transfer station is controlled to sound an alarm; Notify relevant personnel via SMS or email; Store and execute response strategies for different alarm events.

[0012] In a second aspect, the present invention also provides a monitoring device for a garbage classification transfer station, comprising: A collection unit, used to collect multimodal data of the garbage classification transfer station, wherein the multimodal data includes audio and video data; A preprocessing unit, used for preprocessing the collected multimodal data; An edge computing unit for preliminary processing and analysis of pre-processed multimodal data using local equipment at the waste sorting and transfer station; A blockchain data management unit, which is used to encrypt and generate digital signatures for the multimodal data after preliminary processing and analysis, and upload them to the cloud blockchain network; A cloud data extraction unit, used to obtain uploaded data from the cloud blockchain network; The cloud computing unit is used to extract features of data of different modalities using a deep learning algorithm to obtain uploaded data, thereby forming feature representations of each modality; Multimodal data fusion and analysis unit, used to fuse the features of data of different modalities, and use multimodal deep fusion network to conduct comprehensive analysis and identify abnormal situations in garbage classification transfer stations; The alarm generation and automatic response unit is used to generate alarm events based on the integrated analysis results and automatically control the on-site equipment in the waste sorting transfer station to respond; The remote monitoring and evidence collection unit allows platform users to remotely view real-time video and multimodal data, and retain relevant data fragments of alarm events.

[0013] In a third aspect, the present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the garbage sorting transfer station supervision method as described above when executing the computer program.

[0014] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the garbage sorting transfer station supervision method as described above.

[0015] Compared with the prior art, the present invention has the following beneficial effects: a method for supervising a garbage classification transfer station, comprising collecting multimodal data of a garbage classification transfer station, wherein the multimodal data includes audio and video data; preprocessing the collected multimodal data; using the local equipment of the garbage classification transfer station to preliminarily process and analyze the preprocessed multimodal data; encrypting and generating a digital signature for the preliminarily processed and analyzed multimodal data, and uploading it to a cloud blockchain network; obtaining uploaded data from the cloud blockchain network; extracting features of different modal data using a deep learning algorithm to form a feature representation of each modality; fusing the features of different modal data, and using a multimodal deep fusion network to perform comprehensive analysis and identify abnormal situations in the garbage classification transfer station; generating an alarm event based on the fused comprehensive analysis results, and automatically controlling the on-site equipment in the garbage classification transfer station to respond; remotely viewing real-time video and multimodal data through platform users, and retaining relevant data fragments of the alarm event. The present invention reduces the amount of data transmission and improves the real-time performance of the system by performing preliminary processing and analysis of data locally. At the same time, blockchain technology is used to encrypt data and generate digital signatures to ensure the security and non-tamperability of data during transmission and storage. In addition, deep learning algorithms are used in the cloud to perform in-depth feature extraction and comprehensive analysis of data of different modalities to form rich feature representations, thereby improving the accuracy and efficiency of abnormal situation identification.

[0016] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0018] Figure 1 A flowchart of a method for supervising a garbage classification transfer station provided by a specific embodiment of the present invention; Figure 2 A schematic block diagram of a garbage classification transfer station monitoring device provided by a specific embodiment of the present invention; Figure 3 A schematic block diagram of a computer device provided for a specific embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0021] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0022] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0023] like Figure 1 As shown, an embodiment of the present invention provides a method for supervising a garbage classification transfer station, comprising the following steps: S10-S90.

[0024] S10. Collect multimodal data of the garbage classification transfer station, wherein the multimodal data includes audio and video data.

[0025] Highly sensitive microphones installed in waste sorting transfer stations are used to collect sound data. The microphones can capture various sound signals in the transfer station, such as equipment operation noise, personnel conversations, abnormal sounds, etc. The audio data is transmitted to the local area in real time for pre-processing. The microphone can adopt an anti-noise design to reduce the impact of environmental noise on the collection results, thereby improving the accuracy of the data.

[0026] HD cameras are deployed at key locations of waste sorting transfer stations, such as waste delivery ports, sorting areas, and transfer exits. HD cameras collect real-time image data in the transfer station to monitor the waste sorting and processing process. The cameras can have night vision and autofocus functions to ensure clear video data can be collected in all environments.

[0027] S20: pre-processing the collected multimodal data.

[0028] The preprocessing process of audio data includes denoising, segmentation and feature extraction. Specifically, an adaptive filter is used to denoise the audio signal. The filter distinguishes normal signals from noise by setting a threshold, effectively reducing the impact of environmental noise. The audio signal is divided into several short-time frames (such as 20-40 milliseconds per frame), and the signal in each frame is regarded as a stationary signal. A fast Fourier transform (FFT) is performed on each frame to obtain its spectral features. The spectral features are represented as feature vectors for subsequent feature fusion.

[0029] The preprocessing process of video data includes frame rate adjustment, image enhancement and background subtraction. Specifically, the frame rate of video data is reduced to an appropriate level (such as 15 frames per second) to reduce data redundancy and increase processing speed. Histogram equalization technology is used to increase the contrast of the image, making important features in the image more prominent. The background subtraction algorithm is used to remove the static background in the video and only retain the dynamic targets. The Gaussian mixture model is used for background modeling and updating to ensure the accurate extraction of dynamic targets.

[0030] Through the above preprocessing steps, the environmental noise in the audio data is effectively suppressed, and the redundant information of the video data is greatly reduced. The processed audio and video data have clear feature representation, which helps to improve the accuracy of subsequent feature extraction and analysis.

[0031] S30. Use the local equipment of the garbage sorting transfer station to perform preliminary processing and analysis on the preprocessed multimodal data.

[0032] In one embodiment, step S30 specifically includes the following steps: S301 - S302 .

[0033] S301. Perform preliminary feature extraction on the data using a lightweight model.

[0034] Specifically, low-level image features are extracted by performing preliminary convolution operations on video frames; then the features extracted by the convolution layer are downsampled to reduce the size of the feature map, and the extracted features are connected into feature vectors for subsequent preliminary analysis and processing.

[0035] Specifically, low-level audio features are extracted by performing a preliminary convolution operation on the audio spectrum graph; the audio features extracted by the convolution layer are downsampled to reduce the size of the feature graph, and the extracted audio features are connected into a feature vector for subsequent preliminary analysis and processing.

[0036] S302: compress the processed data to reduce the amount of transmitted data.

[0037] The video data is compressed by reducing the frame rate to reduce the amount of video data, for example, by using coding technologies such as H.264 and HEVC to compress the video data.

[0038] The audio data is compressed by reducing the sampling rate to reduce the amount of audio data. For example, the audio data is compressed using encoding technologies such as AAC and MP3.

[0039] By performing preliminary processing and analysis on the pre-processed multimodal data locally, the delay of data transmission is reduced, the analysis results can be obtained in real time, and the response speed of the system is improved. At the same time, through preliminary processing and analysis, valuable information can be extracted from a large amount of raw data, reducing the burden of subsequent in-depth analysis.

[0040] S40, encrypting and digitally signing the preliminarily processed and analyzed multimodal data, and uploading it to the cloud blockchain network.

[0041] In one embodiment, step S40 specifically includes the following steps: S401 - S406 .

[0042] S401. Generate a hash value of the data using a hash algorithm.

[0043] Before data is uploaded to the blockchain, a hash algorithm is used to generate a hash value for the data. Common hash algorithms include SHA-256. This algorithm can generate fixed-length outputs, and slight changes to the input data will produce completely different outputs, ensuring the integrity of the data. The multimodal data after preliminary processing and analysis is hashed to generate a unique hash value, which is used for subsequent signatures and block packaging.

[0044] S402: Generate a public key and a private key pair.

[0045] The public and private keys are generated using the Elliptic Curve Cryptography (ECC) algorithm. The ECC algorithm provides higher security while requiring a relatively short key length. The private key is used for data signing and should be protected by security measures, while the public key can be publicly used to verify the legitimacy of the signature.

[0046] S403: Generate a digital signature using a signature algorithm.

[0047] The hash value is signed using the ECDSA algorithm. ECDSA combines the advantages of ECC and can ensure the efficiency of the signing process while providing high security. The hash value is signed using the private key to generate a digital signature. The digital signature is used to verify the source and integrity of the data.

[0048] S404: Pack the data, timestamp and hash value of the previous data block into a block.

[0049] The data, the generated digital signature, the current timestamp and the hash value of the previous block are packaged together to form a new data block. The timestamp ensures the timing of the data, and the hash value of the previous block ensures the continuity of the blockchain.

[0050] S405: Broadcast the packaged blocks to each node in the blockchain network.

[0051] Through the peer-to-peer network protocol, the block is broadcast to all nodes of the blockchain network. The nodes receive and verify the block information to ensure its legitimacy and integrity.

[0052] S406. Use consensus algorithm to ensure the consistency and reliability of blockchain data.

[0053] You can choose consensus algorithms such as Practical Byzantine Fault Tolerance (PBFT) or Proof of Stake (PoS) to make the best choice based on application requirements. Each node verifies the correctness of block information through the consensus algorithm and reaches a consensus to ensure the integrity and immutability of the blockchain.

[0054] For steps S401-S406, at the garbage sorting transfer station, the data processed and analyzed by local equipment is uploaded to the blockchain network after being encrypted and signed, ensuring that the data has not been tampered with. At the same time, the timeliness and traceability of the data are ensured, which facilitates post-audit and verification.

[0055] S50, obtaining uploaded data from the cloud blockchain network.

[0056] In one embodiment, step S50 specifically includes the following steps: S501 - S505 .

[0057] S501. Query target data in the blockchain network according to the data hash value.

[0058] The target data is queried in the blockchain network based on the unique hash value of the data. The hash value, as the unique identifier of the data, can accurately locate the required data. The query operation is performed on the blockchain node, and the blocks in the blockchain are traversed to find the data entry containing the target hash value. The decentralized storage characteristics of the blockchain are utilized to ensure efficient data query.

[0059] S502: Decrypt the queried data using the private key.

[0060] The queried data is decrypted using a symmetric encryption algorithm (such as AES). The key is decrypted by a pre-generated private key to ensure data security. The private key is stored and managed through a secure channel, and the decryption process is performed in a secure environment to ensure the security of the decryption operation and data privacy.

[0061] S503: Verify the integrity and authenticity of the decrypted data and digital signature.

[0062] Use the public key to verify the digital signature of the decrypted data to ensure the integrity and authenticity of the data. The public key is used in pairs with the private key of the data sender. First, the decrypted data is hashed to generate a data summary. Then the public key is used to verify the digital signature to ensure that the data has not been tampered with during transmission and that the source is legitimate and credible.

[0063] S504: temporarily storing the decrypted and verified data in a cache.

[0064] The decrypted and verified data is temporarily stored in the cloud server's cache (such as RAM) for further processing. The cache can provide fast read and write speeds and improve data processing efficiency. The verified data is written to the cache area, which uses advanced management algorithms (such as LRU) to ensure fast data access and management.

[0065] S505: uniformly convert the multimodal data of different formats temporarily stored in the cache into the input format required by the deep learning model.

[0066] Convert multimodal data in different formats into the input format required by deep learning models. Common conversion tools include OpenCV, Numpy, etc. Specifically, normalize image data, sample and extract features from audio data, and vectorize text data. All data are converted into matrix or tensor form to meet the input requirements of deep learning models.

[0067] S60: Using a deep learning algorithm to extract features from the uploaded data in different modes, to form feature representations of each mode.

[0068] Specifically, a convolutional neural network is used to extract features from video data in the garbage sorting transfer station, and a recurrent neural network is used to extract features from audio data in the garbage sorting transfer station.

[0069] Step S60 specifically includes the following steps: S601-S603.

[0070] S601, performing layer-by-layer convolution operations on the video data to extract low-level to high-level image features.

[0071] Convolutional neural networks are used to process video data. CNN performs well in extracting image features and can extract features from low-level to high-level layer by layer. Specifically, input image: First, decompose the video data into a series of static images by frame as the input of CNN. Convolution layer: Perform convolution operation on each frame of the image to extract low-level features (such as edges and textures). The size and number of convolution kernels are adjusted according to the specific application scenario. Pooling layer: Use maximum pooling or average pooling operations to reduce the size of the feature map, retain important features, and reduce computational complexity. Multi-layer convolution: Through multi-layer convolution operations, high-level features (such as shape and objects) are gradually extracted. The feature map after each convolution layer is nonlinearly transformed by an activation function (such as ReLU) to improve the expressive power of the model. The process of convolution operation includes matrix product and summation of the convolution kernel and the image pixel by pixel to generate a new feature map. Through multi-layer convolution, more complex and abstract features are gradually extracted.

[0072] S602: Connect the extracted features into a feature vector.

[0073] The feature maps extracted by multi-layer convolution operations are flattened and connected into a high-dimensional feature vector as the feature representation of the video data. Specifically, the multi-dimensional feature map output by the convolution layer is flattened into a one-dimensional vector, and the features of different convolution layers are connected to form a complete feature vector, retaining multi-level information. The generated feature vector can be used as the input for subsequent classification, detection and other tasks, improving the model's understanding and processing capabilities of video data.

[0074] S603: extracting time series features from the audio data to capture long-term dependencies in the audio signal.

[0075] Recurrent neural networks are used to process audio data, especially long short-term memory networks (LSTM) or gated recurrent units (GRU) to capture long-term dependencies in audio signals. Specifically, audio data is processed into a sequence of frames as input to the RNN, and each frame sequence represents a period of time in the audio signal. Through the recursive structure of the RNN, the audio data is processed frame by frame to capture timing information and dependencies. The LSTM / GRU unit can effectively alleviate the problem of long-term dependencies. When processing each frame of audio, the RNN unit updates its hidden state and passes the state information of the previous moment in sequence, so that the model can remember important information before. The output after RNN processing is used as a feature representation of the audio data, which includes the understanding of the entire audio signal and the extracted timing features.

[0076] For steps S601-S603, the combination of CNN and RNN can effectively extract the features of video and audio data and improve the data representation ability. Multi-level image features and audio features are integrated through feature vectors to provide high-quality input for subsequent machine learning tasks. The deep learning algorithm can automatically learn and optimize the feature extraction process when extracting features, improving the performance and generalization ability of the model.

[0077] By performing preliminary data processing and analysis locally, the amount of data transmission is reduced and real-time performance is improved. Secondary processing is then performed in the cloud to achieve efficient processing of complex tasks and improve overall performance. In summary, the combination of local and cloud processing takes into account both real-time processing and high efficiency.

[0078] S70. Fuse the features of data from different modalities and use a multimodal deep fusion network to conduct comprehensive analysis and identify abnormal situations in garbage sorting transfer stations.

[0079] In one embodiment, step S70 specifically includes the following steps: S701 - S705 .

[0080] S701 is used to normalize the features of different modes so that they have the same scale.

[0081] Standardization or normalization methods are used to process different modal features so that they have the same scale. This can include standard deviation normalization, maximum and minimum normalization, etc.

[0082] Specifically, the mean of each modal feature is subtracted and divided by the standard deviation, so that the feature data presents a standard normal distribution. The feature data is scaled to the interval [0,1] to facilitate subsequent feature fusion. Normalization can eliminate the scale differences between different modal features and improve the effectiveness of data fusion.

[0083] S702: Assign a weight to each modal feature and perform weighted averaging.

[0084] Assign weights to each modality feature based on experience or through training models. The weights reflect the importance of each modality in the overall fusion.

[0085] Specifically, the normalized features are weighted averaged to obtain the fused feature representation. Weighted average can integrate features according to the importance of each modality and improve the representativeness of the fused features.

[0086] S703: Concatenate different modal features in sequence into a long feature vector.

[0087] Specifically, the features of each modality are connected into a long feature vector in a predetermined order as the input of the multimodal deep fusion network. According to the order of different modalities, the feature vectors are spliced ​​into a complete vector, and the position of each modal feature in the long vector must be fixed to facilitate subsequent model processing. Integrating all modal information into one vector improves the model's ability to process multimodal data.

[0088] S704: Classify the comprehensive features through a classifier.

[0089] Select a suitable classifier (such as support vector machine, decision tree or neural network) to classify the cascaded comprehensive features. Specifically, use the labeled data to train the classifier and learn the classification boundaries of different feature patterns. Classify and predict real-time data to identify abnormal situations. The classifier can effectively identify different patterns in multimodal features and improve the accuracy of abnormality identification.

[0090] S705. Identify abnormal data points by using an isolation forest or density peak clustering method.

[0091] Select algorithms such as Isolation Forest or Density Peak Clustering to identify anomalies in the data. Specifically, use comprehensive feature vectors for anomaly detection. Apply anomaly detection algorithms to identify potential anomalies, such as garbage classification errors, equipment failures, etc. Anomaly detection algorithms can identify situations in the data that do not conform to normal patterns, improving the security and stability of the system.

[0092] For steps S701-S705, the rationality of multimodal feature fusion is improved through normalization and weighted averaging. The classification accuracy of comprehensive features is improved through the use of classifiers, and abnormal situations can be effectively identified. Using advanced anomaly detection algorithms, abnormal situations can be discovered and handled in a timely manner, improving the reliability of the system.

[0093] S80. Generate an alarm event based on the integrated analysis results, and automatically control the on-site equipment in the garbage classification transfer station to respond.

[0094] In one embodiment, step S80 specifically includes the following steps: S801-S805.

[0095] S801. Store alarm generation rules and thresholds.

[0096] Specifically, a rule base is established that contains various abnormal situations and their corresponding thresholds. The rule base includes but is not limited to the following: Garbage accumulation detection rules: For example, the garbage accumulation height exceeds a certain threshold (for example, 2 meters). Equipment failure detection rules: For example, the equipment operation status is abnormal or the downtime exceeds a certain threshold (for example, 30 minutes). Abnormal behavior detection rules: For example, the classification error rate exceeds a certain threshold (for example, 10%).

[0097] The rule base is stored in the system's central database to facilitate subsequent rule matching and alarm generation. The rules and thresholds for alarm generation are stored in advance to ensure that abnormal situations can be quickly and accurately identified and alarms can be generated.

[0098] S802: Match the analysis results with the rules in the rule base to generate an alarm event.

[0099] Specifically, the matching process is as follows: Obtain analysis results: Obtain comprehensive analysis results from the multimodal deep fusion network. Rule matching: Traverse the rule base and compare the analysis results with each rule. Alarm generation: If the match is successful, an alarm event is generated, including information such as alarm type, location, time, and severity.

[0100] Generate alarm events through rule matching to ensure that the alarm information is accurate and reliable, which helps to respond quickly to abnormal situations.

[0101] S803: When an alarm event is generated, the voice alarm device at the garbage classification transfer station is controlled to sound an alarm.

[0102] Specifically, the control process is as follows: Alarm trigger: After an alarm event is generated, the system sends a control command to the voice alarm device. Device response: After receiving the command, the voice alarm device emits a preset alarm sound or prompt message.

[0103] The timely response of voice alarm equipment can quickly attract the attention of on-site personnel and improve safety and emergency handling capabilities.

[0104] S804. Notify relevant personnel via SMS or email.

[0105] Specifically, the notification process is as follows: Contact information: The system stores the contact information of relevant personnel (such as mobile phone number, email address, etc.) Notification sending: The system automatically sends notification information to relevant personnel based on the type and severity of the alarm event.

[0106] Notify relevant personnel through multiple channels to ensure that warning information can be conveyed in a timely manner and improve emergency response speed.

[0107] S805: Store and execute response strategies for different alarm events.

[0108] Specifically, the strategy is implemented as follows: Strategy selection: Select the corresponding response strategy from the response strategy library according to the generated alarm event. Strategy execution: The system automatically executes the selected strategy, including adjusting the equipment operation status, starting the backup equipment, etc.

[0109] Store and execute response strategies to ensure that appropriate response measures can be taken when different alarm events occur, thereby improving the automation and intelligence level of the system.

[0110] For steps S801-S805, the response strategy is stored and executed to ensure that appropriate response measures can be taken when different alarm events occur, thereby improving the automation and intelligence level of the system.

[0111] S90. Through the platform users, real-time video and multimodal data can be viewed remotely, and relevant data fragments of alarm events can be retained.

[0112] Specifically, users can remotely view real-time video and multimodal data through the platform. When an alarm event is detected, the platform automatically retains relevant video and multimodal data fragments, provides an alarm event playback function, supports user viewing and analysis, and archives and manages data fragments related to the alarm event.

[0113] Systematic data management ensures the integrity and security of historical event data and improves information traceability capabilities.

[0114] The embodiment of the present invention also provides a garbage classification transfer station supervision device, which is used to perform the steps in any embodiment of the above-mentioned garbage classification transfer station supervision method. Figure 2 , Figure 2 A schematic block diagram of a garbage classification transfer station monitoring device 100 provided in an embodiment of the present application is shown. The garbage classification transfer station monitoring device 100 specifically includes: The collection unit 110 is used to collect multimodal data of the garbage classification transfer station, and the multimodal data includes audio and video data. The preprocessing unit 120 is used to preprocess the collected multimodal data. The edge computing unit 130 is used to use the local equipment of the garbage classification transfer station to perform preliminary processing and analysis on the preprocessed multimodal data. The blockchain data management unit 140 is used to encrypt and generate digital signatures for the multimodal data after preliminary processing and analysis, and upload them to the cloud blockchain network. The cloud data extraction unit 150 is used to obtain uploaded data from the cloud blockchain network. The cloud computing unit 160 is used to use a deep learning algorithm to extract features of different modal data from the uploaded data to form feature representations of each modality. The multimodal data fusion and analysis unit 170 is used to fuse the features of different modal data, and use a multimodal deep fusion network to perform comprehensive analysis and identify abnormal situations in the garbage classification transfer station. The alarm generation and automatic response unit 180 is used to generate alarm events based on the integrated analysis results and automatically control the on-site equipment in the garbage classification transfer station to respond. The remote monitoring and evidence collection unit 190 allows platform users to remotely view real-time video and multimodal data and retain relevant data fragments of alarm events.

[0115] In one embodiment, the edge computing unit 130 includes: The preliminary feature extraction module is used to extract preliminary features of the data using a lightweight model. The data compression module is used to compress the processed data to reduce the amount of transmitted data.

[0116] In one embodiment, the blockchain data management unit 140 includes: The hash algorithm generation module is used to generate the hash value of the data using the hash algorithm. The key generation module is used to generate the public key and private key pair. The signature algorithm module is used to generate the digital signature using the signature algorithm. The data structure module is used to package the data, timestamp and the hash value of the previous data block into blocks. The network transmission module is used to broadcast the packaged blocks to each node in the blockchain network. The consensus algorithm module is used to ensure the consistency and reliability of the blockchain data using the consensus algorithm.

[0117] In one embodiment, the cloud data extraction unit 150 includes: The target data query module is used to query the target data according to the data hash value in the blockchain network; the decryption module is used to decrypt the queried data using the private key; the decryption verification module is used to verify the integrity and authenticity of the decrypted data and digital signature; the temporary storage module is used to temporarily store the decrypted and verified data in the cache; the format conversion module is used to uniformly convert the multimodal data of different formats temporarily stored in the cache into the input format required by the deep learning model.

[0118] In one embodiment, the cloud computing unit 160 uses a convolutional neural network to extract features from the video data in the garbage classification transfer station, and uses a recurrent neural network to extract features from the audio data in the garbage classification transfer station, specifically including: The convolution processing module is used to perform layer-by-layer convolution operations on video data to extract low-level to high-level image features. The feature vector generation module is used to connect the extracted features into feature vectors. The audio feature extraction module is used to extract time series features from audio data to capture long-term dependencies in audio signals.

[0119] In one embodiment, the multimodal data fusion and analysis unit 170 includes: The normalization processing module is used to normalize the features of different modes to make them have the same scale. The weight assignment module is used to assign weights to each modal feature and perform weighted averaging. The cascade module is used to sequentially cascade the features of different modes into a long feature vector. The classification module is used to classify the comprehensive features through a classifier. The anomaly identification module is used to identify abnormal data points through the isolation forest or density peak clustering method.

[0120] In one embodiment, the alarm generation and automatic response unit 180 includes: The alarm rule storage module is used to store the rules and thresholds for alarm generation. The rule matching module is used to match the analysis results with the rules in the rule base to generate alarm events. The alarm trigger module is used to control the voice alarm equipment at the garbage classification transfer station to sound an alarm when an alarm event is generated. The notification module is used to notify relevant personnel via SMS or email. The strategy storage module is used to store and execute response strategies for different alarm events.

[0121] It should be noted that technical personnel in the relevant field can clearly understand that the specific implementation process of the above-mentioned garbage sorting transfer station supervision device 100 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.

[0122] The above-mentioned garbage classification transfer station supervision device can be implemented in the form of a computer program, which can be used in Figure 3 Runs on the computer device shown.

[0123] See also Figure 3 , Figure 3 700 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 700 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0124] like Figure 3 As shown, the computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for supervising a garbage sorting transfer station as described above are implemented.

[0125] The computer device 700 may be a terminal or a server. The computer device 700 includes a processor 720 , a memory, and a network interface 750 connected via a system bus 710 , wherein the memory may include a non-volatile storage medium 730 and an internal memory 740 .

[0126] The non-volatile storage medium 730 can store an operating system 731 and a computer program 732. When the computer program 732 is executed, the processor 720 can execute a method for supervising a garbage classification transfer station.

[0127] The processor 720 is used to provide computing and control capabilities and support the operation of the entire computer device 700.

[0128] The internal memory 740 provides an environment for the operation of the computer program 732 in the non-volatile storage medium 730. When the computer program 732 is executed by the processor 720, the processor 720 can execute the garbage classification transfer station supervision method.

[0129] The network interface 750 is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will appreciate that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the computer device 700 to which the present application scheme is applied. The specific computer device 700 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. The processor 720 is used to run the program code stored in the memory to implement the garbage classification transfer station supervision method.

[0130] Those skilled in the art will understand that Figure 3 The embodiments of the computer device shown in the figure do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structure and function of the memory and the processor are the same as those of the embodiment of the present invention. Figure 3 The embodiments shown are consistent and will not be described again here.

[0131] It should be understood that in the embodiment of the present application, the processor 720 may be a central processing unit (CPU), and the processor 720 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0132] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the method for supervising a garbage classification transfer station disclosed in the embodiment of the present invention is implemented.

[0133] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0134] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. Units with the same function may also be combined into one unit. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), disk or optical disk and other media that can store program codes.

[0138] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for supervising a garbage classification transfer station, characterized in that: include: Collecting multimodal data of the garbage sorting transfer station, wherein the multimodal data includes audio and video data; Preprocess the collected multimodal data; Use local equipment at the waste sorting transfer station to perform preliminary processing and analysis on the pre-processed multimodal data; Encrypt and digitally sign the preliminarily processed and analyzed multimodal data and upload it to the cloud blockchain network; Get uploaded data from the cloud blockchain network; The uploaded data is obtained and features are extracted from data of different modalities using a deep learning algorithm to form feature representations of each modality; The features of different modal data are integrated, and a multimodal deep fusion network is used to conduct comprehensive analysis and identify abnormal situations in garbage sorting transfer stations; Generate alarm events based on the integrated analysis results, and automatically control the on-site equipment in the waste sorting transfer station to respond; Through the platform, users can remotely view real-time video and multimodal data, and retain relevant data fragments of alarm events.

2. The method for supervising a waste classification transfer station according to claim 1, characterized in that: The method of using the local equipment of the garbage classification transfer station to preliminarily process and analyze the preprocessed multimodal data includes: Use lightweight models to perform preliminary feature extraction on the data; Compress the processed data to reduce the amount of transmitted data.

3. The method for supervising a garbage classification transfer station according to claim 1, characterized in that: The multimodal data after preliminary processing and analysis is encrypted and digitally signed, and uploaded to the cloud blockchain network, including: A hash algorithm is used to generate a hash value of the data; Generate public and private key pairs; Generate a digital signature using a signature algorithm; Pack the data, timestamp and hash value of the previous data block into blocks; Broadcast the packaged blocks to each node in the blockchain network; A consensus algorithm is used to ensure the consistency and reliability of blockchain data.

4. The method for supervising a waste classification transfer station according to claim 1, characterized in that: The obtaining of uploaded data from the cloud blockchain network includes: Query target data based on data hash value in the blockchain network; Use the private key to decrypt the queried data; Verify the integrity and authenticity of decrypted data and digital signatures; Temporarily store the decrypted and verified data in a cache; Convert multimodal data in different formats temporarily stored in the cache into the input format required by the deep learning model.

5. The method for supervising a waste classification transfer station according to claim 1, characterized in that: The uploaded data is obtained by using a deep learning algorithm to extract features from data of different modes to form feature representations of each mode, a convolutional neural network is used to extract features from video data in the garbage classification transfer station, and a recurrent neural network is used to extract features from audio data in the garbage classification transfer station, specifically including: Perform convolution operations on video data layer by layer to extract low-level to high-level image features; Concatenate the extracted features into a feature vector; Extract temporal features from audio data to capture long-term dependencies in audio signals.

6. The method for supervising a waste classification transfer station according to claim 1, characterized in that: The features of different modal data are fused, and a multimodal deep fusion network is used to conduct comprehensive analysis and identify abnormal situations in the garbage classification transfer station, including: Used to normalize the features of different modes to make them have the same scale; Assign weights to each modal feature and perform weighted averaging; Concatenate different modal features sequentially into a long feature vector; Classify the comprehensive features through the classifier; Anomalous data points are identified through isolation forest or density peak clustering methods.

7. The method for supervising a waste classification transfer station according to claim 1, characterized in that: The generating of alarm events based on the integrated analysis results after fusion and automatically controlling the on-site equipment in the garbage classification transfer station to respond include: Stores the rules and thresholds for alarm generation; Match the analysis results with the rules in the rule base to generate alarm events; When an alarm event is generated, the voice alarm device at the garbage classification transfer station is controlled to sound an alarm; Notify relevant personnel via SMS or email; Store and execute response strategies for different alarm events.

8. A monitoring device for a garbage classification transfer station, characterized in that: include: A collection unit, used to collect multimodal data of the garbage classification transfer station, wherein the multimodal data includes audio and video data; A preprocessing unit, used for preprocessing the collected multimodal data; An edge computing unit for preliminary processing and analysis of pre-processed multimodal data using local equipment at the waste sorting and transfer station; The blockchain data management unit is used to encrypt and generate digital signatures for the multimodal data after preliminary processing and analysis, and upload them to the cloud blockchain network; A cloud data extraction unit, used to obtain uploaded data from the cloud blockchain network; The cloud computing unit is used to extract features of data of different modalities using a deep learning algorithm to obtain uploaded data, thereby forming feature representations of each modality; Multimodal data fusion and analysis unit, used to fuse the features of data of different modalities, and use multimodal deep fusion network to conduct comprehensive analysis and identify abnormal situations in garbage classification transfer stations; The alarm generation and automatic response unit is used to generate alarm events based on the integrated analysis results and automatically control the on-site equipment in the waste sorting transfer station to respond; The remote monitoring and evidence collection unit allows platform users to remotely view real-time video and multimodal data, and retain relevant data fragments of alarm events.

9. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the garbage sorting transfer station supervision method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the garbage sorting transfer station supervision method as described in any one of claims 1 to 7.

Images