A fault-tolerant control method and system of a self-service recycling terminal, a terminal and a medium

Abstract

This invention relates to the field of IoT control, specifically providing a fault-tolerant control method, system, terminal, and medium for a self-service recycling terminal. The method includes: after the delivery compartment door is closed, acquiring visual data from an internal camera and logic state signals from overflow sensors; processing the visual data using a pre-trained visual feature extraction model to obtain structured features of the scene, comparing these features with expected features based on sensor signal mapping, and generating a state mismatch event when the difference exceeds a threshold; then querying a preset diagnostic decision matrix to obtain and execute the corresponding atomic control command sequence to cover or correct the original control logic. This invention, through multi-source information fusion verification from vision and sensors, can automatically identify and correct sensor missed alarms, false alarms, and other faults, achieving intelligent self-healing of the business flow and improving the operational reliability, continuity, and user experience of the equipment.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) control, specifically to a fault-tolerant control method, system, terminal, and medium for a self-service recycling terminal. Background Technology

[0002] Currently, smart recycling bins, waste recycling machines, and other self-service recycling terminals are widely used in communities and public places. These terminals integrate functions such as weighing, payment, overflow detection, and automatic compression, enabling convenient 24-hour unattended recycling services.

[0003] The existing self-service recycling terminals rely on preset physical state sensors, especially overflow sensors to determine whether the recycling containers are full. Their standard workflow is as follows: when the sensor detects an "overflow" signal, the system stops accepting deliveries, may trigger compression, and waits for collection. However, this operational logic makes sensor reliability a major bottleneck affecting device availability and user experience.

[0004] First, the diverse materials, irregular shapes, and loose stacking of recyclables can cause sensors to malfunction, resulting in "overfilling" issues where the container is actually full but the system still allows delivery. This creates difficulties for collection and may cause odors or leaks. Second, dust, condensation, obstruction by foreign objects, or sensor drift or malfunction can cause false "overfill" signals when the container is not full, leading to unexplained equipment shutdowns, delivery failures, and user complaints. Third, if a sensor signal is abnormal, the system can only follow a fixed, pre-set logic, lacking the ability to verify the actual physical state. This necessitates remote manual monitoring or on-site maintenance, resulting in slow response times and high costs.

[0005] Related solutions attempt to improve the problem by increasing the number of sensors or improving their accuracy, but this increases hardware costs and complexity, and fails to address the inherent risk of misjudgment by sensors in complex field environments. Other solutions propose using cameras for object recognition or overflow detection, but these are mostly intended to replace sensor functions, are complex, are greatly affected by lighting and occlusion, and lack effective collaboration and cross-verification mechanisms with existing sensor systems. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a fault-tolerant control method, system, terminal, and medium for self-service recycling terminals. Based on multi-source sensing state comparison and intelligent decision-making, it achieves self-healing fault tolerance for sensor failures in self-service recycling terminals by automatically diagnosing and reshaping the control process, thereby improving equipment reliability, continuity, and user experience.

[0007] In a first aspect, the technical solution of the present invention provides a fault-tolerant control method for a self-service recycling terminal, the method comprising the following steps: After the delivery compartment door is closed, visual data representing the physical scene inside the recycling container is acquired through a camera, as well as logical state signals output by the physical state sensor. Visual data is processed by a pre-trained visual feature extraction model to obtain scene structured features; these scene structured features are compared with the expected scene features obtained by mapping logical state signals; when the difference between the two exceeds a preset threshold, a state mismatch event containing a specific type of encoding is generated; the type of encoding is determined by the current logical state and the comparison result. The system takes the type encoding of the state mismatch event as input and queries a preset diagnostic decision matrix, which stores the mapping relationship between the state mismatch event type and the control command. Based on the query result, the system sends the corresponding control command sequence to the actuator to cover or correct the original control logic.

[0008] Secondly, the technical solution of the present invention provides a fault-tolerant control system for a self-service recycling terminal, comprising: The data signal acquisition module is used to acquire visual data representing the physical scene inside the recycling container through a camera after the delivery compartment door is closed, as well as to acquire logical state signals output by the physical state sensor. The event generation module is used to process visual data through a pre-trained visual feature extraction model to obtain scene structured features; compare the scene structured features with the expected scene features obtained by mapping logical state signals; when the difference between the two exceeds a preset threshold, generate a state mismatch event containing a specific type code; the type code is determined by the current logical state and the comparison result. The control adjustment module takes the type code of the state mismatch event as input, queries a preset diagnostic decision matrix, which stores the mapping relationship between the state mismatch event type and the control command; based on the query result, it sends the corresponding control command sequence to the actuator to cover or correct the original control logic.

[0009] Thirdly, the technical solution of the present invention provides a terminal, comprising: Memory, used to store the fault-tolerant control program of the self-service recycling terminal; The processor is used to implement the steps of the fault-tolerant control method for the self-service recycling terminal when executing the fault-tolerant control program of the self-service recycling terminal.

[0010] Fourthly, the present invention provides a computer-readable storage medium storing a fault-tolerant control program for a self-service recycling terminal, wherein the fault-tolerant control program for the self-service recycling terminal, when executed by a processor, implements the steps of the fault-tolerant control method for the self-service recycling terminal described above.

[0011] As can be seen from the above technical solutions, this application has the following advantages: By using visual perception as an independent information source, the physical state of the recycling container is captured and analyzed after the delivery compartment door is closed. By comparing the structured features of the scene obtained from visual analysis with the expected scene features based on sensor signal mapping in real time, the mismatch between sensor signals and visual facts can be effectively identified. This changes the risk of misjudgment that may be caused by relying solely on sensors and improves the overall reliability of the system's state judgment. When a state mismatch is detected, the system automatically generates and executes a corresponding sequence of control commands by querying a pre-defined diagnostic decision matrix. For example, it automatically performs compression to make room when a sensor misses a report, and forcibly overwrites the erroneous state to restore service when a sensor false alarm occurs. This allows the system to correct various common sensor anomalies that cause business process errors without manual intervention, ensuring continuous equipment operation and reducing delivery failure rates and user complaints caused by false faults. It utilizes existing or required hardware in self-service recycling terminals, eliminating the need for expensive dedicated sensing equipment. The solution has low implementation costs, is easy to integrate into existing products, and has high practicality and promotional value. Attached Figure Description

[0012] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic flowchart of a fault-tolerant control method for a self-service recycling terminal provided in an embodiment of the present invention.

[0014] Figure 2 This is a schematic block diagram of a fault-tolerant control system for a self-service recycling terminal provided in an embodiment of the present invention.

[0015] Figure 3 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0018] Figure 1 This is a schematic flowchart illustrating a fault-tolerant control method for a self-service recycling terminal provided in an embodiment of the present invention. Figure 1 The executing entity can be a fault-tolerant control system for a self-service recycling terminal. The fault-tolerant control method for the self-service recycling terminal provided in this embodiment of the invention is executed by a computer device; correspondingly, the fault-tolerant control system for the self-service recycling terminal runs on the computer device. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0019] like Figure 1 As shown, the method includes the following steps.

[0020] S1, after the delivery compartment door is closed, acquires visual data representing the physical scene inside the recycling container through a camera, as well as logical state signals output by the physical state sensor.

[0021] After a single delivery is completed at the self-service recycling terminal and the delivery compartment door is confirmed to be closed, the intelligent fault-tolerant control process of this invention is initiated. The purpose of this step is to simultaneously collect two types of independent and mutually verifying sensing data at key decision points that determine the subsequent processes of this delivery (such as whether to compress or settle accounts), as the basis data for subsequent state fusion and diagnosis.

[0022] It should be noted that the significance of "after the delivery compartment door is closed" as a business event node is as follows: after the compartment door is closed, the delivery action is completed, and the state of the items in the recycling container tends to be still, avoiding motion blur caused by user operation and ensuring image quality; the image acquired at this time reflects the final change in the state of the container caused by this delivery, and can be used to determine "whether it is overflowing" and "whether there are any abnormal items".

[0023] Visual data acquisition is accomplished by the integrated business camera within the terminal, and the acquired visual data consists of one or more frames of digital images. The image field of view fully covers the effective storage area of ​​the recycling container, presenting the surface, outline, and compression plate of the material accumulation inside the container. In practice, the camera can be triggered to take a high-resolution static photo, or a short video stream can be recorded and the clearest frame selected for processing.

[0024] The acquisition of the logic status signal is accomplished by the terminal's inherent physical status sensor. In this embodiment, the physical status sensor specifically refers to an overflow sensor, such as an ultrasonic ranging sensor or an infrared photoelectric sensor. The overflow sensor outputs a discrete logic status signal. For example, when the material surface is detected to reach a preset height, it outputs a high level or a specific digital code, representing an "overflow" state; otherwise, it outputs a low level or another digital code, representing a "non-overflow" state. This signal directly determines the original, unverified business logic of the terminal's main controller; that is, overflow will stop service and may trigger compression business logic.

[0025] S2, Visual data is processed by a pre-trained visual feature extraction model to obtain scene structured features; The scene structured features are compared with the expected scene features obtained by mapping logical state signals. When the difference between the two exceeds a preset threshold, a state mismatch event containing a specific type of encoding is generated; The type of encoding is determined by the current logical state and the comparison result.

[0026] The visual feature extraction model in this embodiment is trained based on an initial neural network model. The initial neural network model adopts an encoder-decoder architecture. The encoder is a deep convolutional neural network used to extract shared visual features. The decoder includes multiple parallel task-specific subnetworks, which are respectively configured to regress the stacking height value, the material distribution uniformity index, and detect the position coordinates of the compression mechanism.

[0027] The initial neural network model specifically includes the following layers.

[0028] Input layer: Receives a fixed-size RGB image I. Optionally, a corresponding depth map D can be input simultaneously to form a 4-channel input [R, G, B, D].

[0029] Shared feature encoder: A deep convolutional neural network pre-trained on the dataset serves as the backbone network. This can be ResNet, EfficientNet, or ConvNeXt. The encoder is responsible for extracting multi-level visual features from the input image.

[0030] Multi-task predictive decoder head: Following the output feature map of the shared encoder, multiple lightweight sub-networks ("heads") are connected in parallel, each head responsible for predicting the ground truth of a specific physical state.

[0031] Stacked height regression head: This can consist of several convolutional and fully connected layers, ultimately outputting a scalar value H_pred, which is the predicted stacked height. Mean squared error loss is used.

[0032] Uniformity Index Regression Head: Similar in structure to the height regression head, it outputs a scalar value U_pred, which is the predicted value of the material distribution uniformity index. It uses mean squared error or Huber loss.

[0033] Compression Mechanism Keypoint Detection Head: Dense Prediction Head. A small deconvolutional or upsampling network is used to upsample the low-resolution feature map output by the encoder to the original image size. For each predefined keypoint (e.g., the four corners of the compression plate), this head outputs a heatmap of the same size as the original image; the location of the heatmap peak corresponds to the keypoint's location. A pixel-wise weighted mean square error loss is used. Finally, by calculating argmax or centered moments on the heatmap, the pixel coordinates of the keypoints are obtained, and P_pred, the predicted coordinate of the compression mechanism's position, can be calculated.

[0034] The total loss when training the initial neural network model is the weighted sum of the losses from each task.

[0035] In this embodiment, the training process of the visual feature extraction model includes the following steps.

[0036] Step 1: Obtain an image sample set containing the scene inside the recycling container, and label the physical state truth value for each sample. The physical state truth value includes the stacking height value, the material distribution uniformity index, and the position coordinates of the compression mechanism.

[0037] We collected a large number of images of recycling containers in different states, including empty containers, half-full containers, overflowing containers, and the compression mechanism in different positions. We then performed feature annotation on these images, that is, we labeled each image with a multi-dimensional regression vector [stack height estimate, distribution uniformity score, and compression plate center coordinates].

[0038] Stack height estimate: refers to the vertical distance of the highest point of the surface of the delivered items in the recycling container relative to a predefined reference plane at the bottom of the container, used to quantify the degree of container fullness.

[0039] A pixel-level depth map D(x, y) is obtained using a depth camera, where (x, y) are the pixel coordinates and D is the depth value. An image segmentation algorithm is then used to identify the mask M_material(x, y) for the "material region" from the RGB image, where a value of 1 represents a material pixel and 0 represents a non-material pixel.

[0040] In the empty container state, an average depth value D_base representing the bottom of the container is determined through the depth map. The material mask is applied to the depth map to obtain a set of depth values ​​for the material area. The stacking height value can be calculated as the distance from the highest point of the material surface to the reference surface.

[0041] Material distribution uniformity index: A scalar indicator used to quantify the uniformity of material distribution on a horizontal plane within a recycling container. Poor uniformity may indicate that items are piled up at an angle, stuck in corners, or contain large items, affecting the accuracy of the overflow sensor or the compression effect.

[0042] The 3D material point cloud or depth map is vertically projected onto a horizontal plane to obtain a 2D "occupancy grid" or "height profile". The horizontal projection area of ​​the recycling bin opening is divided into N×M regular grid cells. The average height of the material points in each grid cell is calculated. Then, the statistical characteristics of the height values ​​of all non-empty grid cells are calculated.

[0043] First, calculate the uniformity index of the height variance. ,in, It is the standard deviation of the average height of the grid cells. It is its mean. The closer to 1, the more uniform the height distribution.

[0044] Next, the uniformity index based on spatial entropy is calculated. ,in, The total number of grid cells. Let be the probability that there is material in the i-th grid, i.e. =Number of material pixels in the i-th grid / Total number of pixels in that grid; The actual Shannon entropy is the value of the material distribution. The larger the entropy value, the more "chaotic" or "unpredictable" the material distribution is, which means that the distribution is more dispersed and more uniform. Theoretically, the maximum possible entropy is the Shannon entropy when the material is perfectly uniformly distributed across all grids.

[0045] Calculate the comprehensive index, namely the material distribution uniformity index. ,in and The weighting coefficients combine information from two dimensions: high volatility and spatial dispersion.

[0046] Compression mechanism position coordinates: used to describe the position of the compression mechanism in space.

[0047] The bounding box of the compression mechanism is defined in the RGB image using an object detection model, and multiple predefined feature points on the compression plate are located using a keypoint detection model, such as the four corners of the compression plate and the endpoints of the hydraulic rods. The pixel coordinates of the keypoints are obtained as the position coordinates of the compression mechanism. For scenarios where the focus is mainly on lifting, the position coordinates can be simplified to a scalar, namely the vertical height of the center point of the lower edge of the compression plate, used to determine whether it has fully retracted or has sunk abnormally.

[0048] Step 2: Using image samples as input and the true physical state as the supervision signal, the initial neural network model is trained by minimizing the loss function between the predicted features and the true physical state.

[0049] The preprocessed image samples are input into the network. The encoder first extracts features from the image, generating a rich intermediate feature representation. This intermediate feature is simultaneously fed into three prediction heads: the height regression head outputs a scalar value H_pred as a prediction of the stacking height; the uniformity regression head outputs a scalar value U_pred as a prediction of the distribution uniformity; and the keypoint detection head outputs a series of heatmaps, where the peak position of each heatmap corresponds to the predicted pixel coordinates of a keypoint.

[0050] Calculate the error between the output value of each prediction head and the corresponding physical state true value: use mean squared error loss to calculate the difference between H_pred and the labeled height H_gt; use mean squared error or Huber loss to calculate the difference between U_pred and the labeled uniformity U_gt; use pixel-by-pixel weighted mean squared error loss to calculate the difference between the predicted heatmap and the Gaussian heatmap generated centered on the true value coordinates.

[0051] The three losses are summed according to preset weights to obtain the total loss. This total loss is then propagated from the network output to the input using the backpropagation algorithm, and the gradient of each parameter in the network is calculated. Using the Adam optimization algorithm, all parameters in the network are updated based on the calculated gradients. This process of forward propagation, loss calculation, backpropagation, and parameter update is repeated iteratively on a large number of image samples.

[0052] Training is complete when the model's prediction error on the validation set converges to the preset target.

[0053] It should be noted that after the initial neural network model is trained, its feature encoding portion is retained as a visual feature extraction model to extract scene structured features from the input visual data. During the model application phase, the trained model is used to extract high-quality scene structured features from new images for state comparison. Only the shared encoder portion of the initial neural network model is used; real-time images are input into the encoder, and the activation values ​​of a certain layer or the last layer are the required scene structured features, which contain information related to overflow, uniformity, and device location.

[0054] This embodiment processes visual data based on a pre-trained visual feature extraction model to obtain scene structured features. Then, the scene structured features are compared with the expected scene features obtained by mapping logical state signals. When the difference between the two exceeds a preset threshold, a state mismatch event containing a specific type of encoding is generated. Specifically, the following steps are included.

[0055] S2.1, Based on the currently received logic state signal, index and call the corresponding steady-state expected features from the pre-built logic state-expected feature mapping library; wherein, the logic state-expected feature mapping library contains the mapping relationship between each logic state and the corresponding steady-state expected features, and the expected features are obtained by processing the corresponding visual data through a pre-trained visual feature extraction model.

[0056] During the deployment or maintenance phase of the self-service recycling terminal, a logical state-expected feature mapping library is established in advance.

[0057] First, when the device is in a known and defined logical business state, it captures multiple frames of images using its internal camera. These logical business states include: confirming the recycling container is empty and the overflow sensor outputs a "not overflowing" signal, and the overflow sensor being triggered by a reliable object and outputting an "overflowing" signal. Then, a pre-trained visual feature extraction model is used to process these images, obtaining multiple sets of scene structured features. These features are then statistically analyzed to form a steady-state expected feature vector representing the logical state, which is stored in the mapping library and associated with its corresponding logical state code.

[0058] During the diagnostic process, the expected scenario features obtained by mapping the logic state signals are specifically as follows: based on the currently received logic state signals, the corresponding steady-state expected feature vector is indexed and called from the logic state-expected feature mapping library.

[0059] In this embodiment, the logic state includes an overflow state and a non-overflow state. When the logic state is an overflow state, the steady-state expectation feature is the overflow expectation feature; when the logic state is a non-overflow state, the steady-state expectation feature is the empty expectation feature.

[0060] Overflow Status: The logic signal status output by the overflow sensor when it detects that the material accumulation height in the recycling container has reached or exceeded its preset trigger threshold. For example, the sensor outputs a high level, a specific digital code (such as 1), or a message (such as "FULL"). In the original control logic of the terminal, this status means "stop receiving delivery" or "trigger compression".

[0061] Non-overflow state: The logic signal state output when the overflow sensor detects that the material accumulation height is below its trigger threshold. For example, the sensor outputs a low level, another digital code (such as 0), or a message (such as "NOT_FULL"). In the original logic, this state means "delivery is allowed to continue".

[0062] Under controllable conditions, the recycling container is filled to the point where the overflow sensor is triggered using standard materials. In this state, multiple frames of images are captured by an internal camera to form an overflow state image set. A pre-trained visual feature extraction model is used to process each frame of the image set to obtain a set of overflow state feature vectors. Statistical analysis is performed on this set of vectors, and the resulting aggregate vector is defined as the overflow expectation feature, which represents the visually full state pattern.

[0063] The recycling container is completely emptied, and the overflow sensor is confirmed to output a stable "non-overflow" signal. In this state, multiple frames of images are acquired to form an empty state image set. Similarly, features are extracted from the image set and the mean vector is calculated to obtain the expected empty features, which represent the visually empty state pattern.

[0064] When the current logic state signal is read as "overflow", a pre-built overflow expectation feature vector is retrieved from storage as the comparison benchmark. When the current logic state signal is read as "non-overflow", a pre-built empty expectation feature vector is retrieved from storage as the comparison benchmark.

[0065] S2.2, using a predetermined similarity metric function or distance metric function, calculate the difference between the scene's structured features and the steady-state expected features.

[0066] If a similarity measurement function is used, the difference = 1 - (cosine similarity between the scene's structured features and the steady-state expected features).

[0067] If a distance metric function is used, the difference is the Euclidean distance between the scene's structured features and the steady-state expected features.

[0068] S2.3, compare the difference degree with the threshold corresponding to the current logical state. If the comparison result meets the expectations, it is determined that the current physical scene is consistent with the logical state and the process continues. Otherwise, it is determined that a state mismatch has occurred and a state mismatch event is generated. This event includes a type encoding field and a difference degree value field.

[0069] Specifically, it includes the following two comparison scenarios. A decision threshold is preset for each comparison scenario, which is determined through statistical analysis of historical or experimental data.

[0070] a) When the logical state is not overflowing, the difference between the scene's structured features and the expected overflow features is... Greater than the first threshold If the current physical scene is consistent with the logical state, the process continues. If the difference between the scene's structured features and the overflow expected features is less than or equal to the first threshold, a state mismatch is determined to have occurred, and a state mismatch event is generated. The type encoding of this event is the first type encoding.

[0071] This situation is used to detect false negatives. This indicates that the current scene looks very different from the "overflow state". The visual evidence supports the sensor's "non-overflow" status, so it is determined that the state is consistent and the original business process continues.

[0072] like This indicates that the current scene looks very similar to an "overflow state," which contradicts the sensor's "non-overflow" state. A state mismatch is determined to have occurred, and a first-type encoded mismatch event is generated, which represents "the sensor may have missed a report, and the visual appearance is suspected to be overflow."

[0073] b) When the logical state is overflowing, if the difference between the scene's structured features and the expected empty features is... Greater than the second threshold If the current physical scene is consistent with the logical state, the process continues. If the difference between the scene's structured features and the empty expected features is less than or equal to the second threshold, a state mismatch is determined to have occurred, and a state mismatch event is generated. The type encoding of this event is the second type encoding.

[0074] This method is used to detect false alarms, typically because the risk of false alarms is higher. Less than The requirement is that the visual appearance "looks more like an empty bucket" before correction is triggered, in order to improve the reliability of decision-making.

[0075] like This indicates that the current scene looks very different from the "empty state," and the visual evidence supports the "overflow" of the sensor, so it is determined that the state is consistent.

[0076] like This indicates that the current scene looks very similar to an "empty state," which contradicts the "overflow" of the sensor. A state mismatch is determined, and a mismatch event of type 2 encoding is generated, which represents "the sensor may be falsely reporting, and the scene appears to be empty."

[0077] The generated mismatch event is a structured data object containing the following fields: event_type: Event type encoding; discrepancy_value: The calculated discrepancy value; timestamp: The timestamp when the event occurred.

[0078] S3 takes the type code of the state mismatch event as input, queries the preset diagnostic decision matrix, which stores the mapping relationship between the state mismatch event type and the control command; based on the query result, sends the corresponding control command sequence to the actuator to cover or correct the original control logic.

[0079] The diagnostic decision matrix is ​​a predefined mapping table or dictionary structure stored in the system, which establishes a one-to-one correspondence between state mismatch event types and atomic control instruction sequences.

[0080] Each row or entry in the matrix defines a response strategy for a specific type of fault. The key is the type code for the state mismatch event and serves as a unique index for the query. The value is a pre-arranged sequence of atomic control instructions corresponding to that type code.

[0081] The type encoding field is extracted from the received state mismatch event object. Using this type encoding as the key, a search is performed in the diagnostic decision matrix. The retrieved atomic instruction sequence, combined with the target parameters, is encapsulated into a specific control command package. The generated command package is marked as a system-level high-priority instruction and sent to the corresponding actuator driver via the device control bus. This high-priority instruction package interrupts or overrides the regular business process instructions currently being generated or to be executed by the terminal master controller based on the raw sensor signals. For example, when the matrix instruction requires "ignore the overflow signal and continue settlement," this instruction will force the master controller's state machine to jump to the settlement state, ignoring its internal pause logic caused by the overflow signal.

[0082] For the first type of coded state mismatch event, the instruction sequence obtained from the query matrix will drive the compression mechanism to perform a compression operation. This aims to compact the material, create space, and attempt to automatically restore the equipment to a deliverable state, preventing overfilling due to sensor failure. The corresponding control instruction sequence includes: starting the compression motor to rotate forward, and continuing operation. Seconds, stop the compressor motor, delay. Seconds later, the state is reassessed.

[0083] State reassessment refers to re-capturing the compressed image through the internal camera and performing state fusion and diagnosis, which means re-executing step S2 to verify the compression effect and update the system state.

[0084] Among them, compression time Dynamically adjusted based on the severity of visual deviation, as shown below:

[0085] In the formula, Based on time compression, This is the time gain coefficient. The larger the value, the higher the calculated value. The longer the duration, the more severe the suspected overflow. In this case, a longer compression period is implemented to make room for maneuver and improve the accuracy and efficiency of the fault-tolerant action.

[0086] For the second type of coded state mismatch event, the instruction sequence obtained from the query matrix will send a forced state overwrite signal to the main controller, resetting its internal logic state from "overflow" to "non-overflow," and allowing subsequent settlement, payment, and other processes to continue, thereby automatically correcting false alarms and restoring service to users. The corresponding control instruction sequence includes: sending a "forced state overwrite" signal to the main controller, setting the internal logic state to "non-overflow," allowing subsequent business processes, and generating sensor reliability alarm logs. After the state is overwritten, the interrupted subsequent processes, such as item weighing and settlement, paying points to users, and preparing to open the warehouse door, will be immediately resumed, ensuring that the user's delivery experience is not disrupted by a single erroneous sensor alarm.

[0087] Among them, the confidence weight of reliability alarm logs Used to quantify the confidence level of false alarms, for reference in backend maintenance, and represented as:

[0088] In other words, The smaller the value, the more it resembles emptiness visually, and the higher the confidence level that it is a false alarm.

[0089] In some optional implementations, to further enhance equipment safety and prevent users from damaging the compression mechanism or recycling bin by throwing hard, incompressible objects such as bricks and stones, weight sensing and morphological analysis are used to detect and protect against high-risk items. Specifically, in step S1, in addition to acquiring image data and overflow status, the weight data of the currently thrown item is also acquired. Then, in subsequent steps, the following process is executed: the visual data is analyzed using an image processing algorithm to obtain the item's morphological characteristics, and the item's volume is calculated based on these characteristics, and it is determined whether abnormal morphological characteristics are detected; the unit volume weight is calculated based on the weight data and item volume. If the unit volume weight exceeds the weight risk threshold and abnormal morphological characteristics are detected, an item abnormality event containing a third type of code is generated; the type code of the item abnormality event is used as input to query a preset diagnostic decision matrix to obtain control commands. This matrix also stores the mapping relationship between the third type of code and the control commands.

[0090] Specifically, after the delivery compartment door is closed, the weighing data output by the weighing sensor during this delivery process is recorded, and the difference between this and the weighing data before this delivery is made to obtain the weighing data of the item being delivered.

[0091] Image processing algorithms are applied to the post-delivery status image to extract the morphological features of the item and estimate its volume. First, using background subtraction or color-based thresholding, the region in the image belonging to the "newly delivered item" is separated from the original background material inside the container, resulting in a binary mask image of the item. In the item mask image, its minimum bounding rectangle is calculated to obtain the item's pixel width and height. Using the known calibration relationship between the actual container size and the image pixel size, the pixel size is converted to physical size. Based on a simplified geometric model (e.g., treating the item as a cuboid), its volume is estimated. Morphological features are then calculated for the segmented item region. Calculate the roundness, rectangularity, or aspect ratio of the region; regular, rigid objects (such as bricks) typically have high rectangularity and low roundness. The contour is extracted using an edge detection algorithm, and the proportion of straight line segments in the contour is analyzed by detecting straight lines using Hough transform; the contours of rigid man-made objects often contain a large number of straight line segments. If all of the above feature values ​​exceed the corresponding preset threshold, it is determined that abnormal morphological features have been detected.

[0092] Divide the current weight of the delivered item by its volume to obtain the weight per unit volume. If the weight per unit volume exceeds the weight risk threshold, which can be set to be much higher than the typical density of common recyclables such as plastic and cardboard, and the image analysis detects abnormal morphological features, then the risk determination is established and an item abnormality event containing a third type of code is generated.

[0093] Using this third type of encoding as input, the diagnostic decision matrix is ​​queried, which contains pre-defined response strategies for this type of event. The resulting sequence of control instructions mainly includes: Send a "lock" signal to the compression control unit: immediately and forcibly prohibit any compression operation from starting during this business cycle; this is the highest priority equipment protection action; Normal execution of subsequent business processes: To avoid conflicts with users and complete transactions, continue to execute processes such as weighing settlement and points payment; Marking and Reporting: Record this transaction as "high-risk delivery" in the background, and package the obtained weighing data, estimated volume, morphological analysis results and on-site images together and report them to the cloud management platform for subsequent manual verification and processing.

[0094] Control instructions coded in the third type have a higher priority than those coded in the first and second types. Specifically, fixed execution priorities are set for control instructions triggered by different types of mismatch events, with the principle that device safety protection takes precedence over business process correction. Control instructions corresponding to the third type of code have the highest priority, those corresponding to the first type of code have medium priority, and those corresponding to the second type of code have lower priority.

[0095] If a "suspected overflow" (Type 1) and a "hard object risk" (Type 3) occur simultaneously, the Type 1 instruction requires compression to be performed, while the Type 3 instruction requires compression to be disabled. According to safety principles, only the highest priority Type 3 instruction is executed, and the Type 1 instruction is blocked or ignored.

[0096] If a "sensor false alarm" (Type 2) and a "hard object risk" (Type 3) occur simultaneously, and there is no direct instruction conflict between the two, the Type 2 instruction aims to restore service, while the Type 3 instruction aims to protect the device. Compatible instructions are executed in parallel. The system will execute the Type 3 instruction to disable compression, while simultaneously executing the Type 2 instruction to cover the overflow state and continue the settlement process. The final result is that the user completes the delivery and receives the cashback, but the device will not perform compression during this cycle. The system generates a log to record this compound event.

[0097] The first type and the second type are logically mutually exclusive and will not cause concurrent conflicts.

[0098] The above text provides a detailed description of an embodiment of a fault-tolerant control method for a self-service recycling terminal. Based on the fault-tolerant control method for a self-service recycling terminal described in the above embodiment, this invention also provides a fault-tolerant control system for a self-service recycling terminal corresponding to the method.

[0099] Figure 2 This is a schematic block diagram of a fault-tolerant control system for a self-service recycling terminal provided in an embodiment of the present invention. In this embodiment, the fault-tolerant control system 200 of the self-service recycling terminal can be divided into multiple functional modules according to the functions it performs. A module, as referred to in this invention, is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory.

[0100] The data signal acquisition module 210 is used to acquire visual data representing the physical scene inside the recycling container through a camera after the delivery compartment door is closed, as well as to acquire logical state signals output by the physical state sensor.

[0101] The event generation module 220 is used to process visual data through a pre-trained visual feature extraction model to obtain scene structured features; compare the scene structured features with the expected scene features obtained by mapping logical state signals; when the difference between the two exceeds a preset threshold, generate a state mismatch event containing a specific type code; the type code is determined by the current logical state and the comparison result.

[0102] The control adjustment module 230 is used to take the type code of the state mismatch event as input, query the preset diagnostic decision matrix, which stores the mapping relationship between the state mismatch event type and the control command; according to the query result, it sends the corresponding control command sequence to the actuator to cover or correct the original control logic.

[0103] In some optional implementations, the data signal acquisition module 210 is further configured to: acquire the weighing data of the currently delivered item. The event generation module 220 is further configured to: analyze the visual data using an image processing algorithm to obtain the item's morphological features, and calculate the item's volume and determine whether abnormal morphological features are detected based on the item's morphological features; calculate the unit volume weight based on the weighing data and the item's volume; if the unit volume weight exceeds the weight risk threshold and abnormal morphological features are detected, then generate an item abnormality event containing a third type code. The control adjustment module 230 is further configured to: take the type code of the item abnormality event as input, query a preset diagnostic decision matrix to obtain control commands, and this matrix also stores the mapping relationship between the third type code and the control commands.

[0104] The fault-tolerant control system of the self-service recycling terminal in this embodiment is used to implement the aforementioned fault-tolerant control method of the self-service recycling terminal. Therefore, the specific implementation of this system can be found in the embodiment section of the fault-tolerant control method of the self-service recycling terminal mentioned above. Thus, the specific implementation can be referred to the description of the corresponding embodiments, and will not be elaborated here.

[0105] Furthermore, since the fault-tolerant control system of the self-service recycling terminal in this embodiment is used to implement the aforementioned fault-tolerant control method of the self-service recycling terminal, its function corresponds to the function of the above method, and will not be repeated here.

[0106] Figure 3 This is a schematic diagram of the structure of a terminal 300 provided in an embodiment of the present invention, including: a processor 310, a memory 320, and a communication unit 330. The processor 310 is used to implement the fault-tolerant control program of the self-service recycling terminal stored in the memory 320, and to implement the process steps of the above-described embodiment of the fault-tolerant control method for the self-service recycling terminal.

[0107] This invention also provides a computer storage medium, which may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The computer storage medium stores a fault-tolerant control program for a self-service recycling terminal. When the fault-tolerant control program for the self-service recycling terminal is executed by a processor, it implements the process steps of the above-described embodiment of the fault-tolerant control method for the self-service recycling terminal.

[0108] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fault-tolerant control method for a self-service recycling terminal, characterized in that, The method includes the following steps: After the delivery compartment door is closed, visual data representing the physical scene inside the recycling container is acquired through a camera, as well as logical state signals output by the physical state sensor. Visual data is processed by a pre-trained visual feature extraction model to obtain scene structured features; these scene structured features are compared with the expected scene features obtained by mapping logical state signals; when the difference between the two exceeds a preset threshold, a state mismatch event containing a specific type of encoding is generated; the type of encoding is determined by the current logical state and the comparison result. The system takes the type encoding of the state mismatch event as input and queries a preset diagnostic decision matrix, which stores the mapping relationship between the state mismatch event type and the control command. Based on the query result, the system sends the corresponding control command sequence to the actuator to cover or correct the original control logic.

2. The fault-tolerant control method for the self-service recycling terminal according to claim 1, characterized in that, The training process of a visual feature extraction model includes: Acquire an image sample set containing the scene inside the recycling container, and label each sample with the physical state truth value, which includes the stacking height value, the material distribution uniformity index, and the position coordinates of the compression mechanism; Using image samples as input and the true physical state as the supervision signal, the initial neural network model is trained by minimizing the loss function between the predicted features and the true physical state. The initial neural network model adopts an encoder-decoder architecture. The encoder is a deep convolutional neural network used to extract shared visual features. The decoder includes multiple parallel task-specific subnetworks, which are configured to regress the stacking height value, the material distribution uniformity index, and detect the position coordinates of the compression mechanism, respectively. After the initial neural network model is trained, the feature encoding part of the initial neural network model is retained as a visual feature extraction model, which is used to extract the scene structured features from the input visual data.

3. The fault-tolerant control method for the self-service recycling terminal according to claim 2, characterized in that, The structured features of the current scene are compared with the expected scene features obtained by mapping logical state signals. When the difference between the two exceeds a preset threshold, a state mismatch event containing a specific type of encoding is generated, specifically including: Based on the currently received logical state signal, the corresponding steady-state expected features are indexed and called from the pre-built logical state-expected feature mapping library. The logical state-expected feature mapping library contains the mapping relationship between each logical state and the corresponding steady-state expected features. The expected features are obtained by processing the corresponding visual data through a pre-trained visual feature extraction model. The logical states include overflow states and non-overflow states. The difference between the structured features of the scene and the steady-state expected features is calculated using a predetermined similarity metric function or distance metric function. The difference is compared with the threshold corresponding to the current logical state. If the comparison result meets the expectations, the current physical scene is determined to be consistent with the logical state, and the process continues. Otherwise, a state mismatch is determined to have occurred, and a state mismatch event is generated. This event includes a type encoding field and a difference value field.

4. The fault-tolerant control method for the self-service recycling terminal according to claim 3, characterized in that, When the logical state is overflowing, the steady-state expected characteristic is the overflow expected characteristic; When the logical state is not overflowing, the steady-state expected characteristic is the empty expected characteristic; The difference is compared with the threshold corresponding to the current logical state, specifically including: When the logical state is not overflowing, the difference between the scene's structured features and the expected overflow features is... Greater than the first threshold If the current physical scene is consistent with the logical state, the process continues. If the difference between the scene's structured features and the overflow expected features is less than or equal to the first threshold, then a state mismatch is determined to have occurred, and a state mismatch event is generated. The type encoding of this event is the first type encoding. When the logical state is overflowing, if the difference between the scene's structured features and the expected empty features is... Greater than the second threshold If the current physical scene is consistent with the logical state, the process continues. If the difference between the scene's structured features and the empty expected features is less than or equal to the second threshold, a state mismatch is determined to have occurred, and a state mismatch event is generated. The type encoding of this event is the second type encoding.

5. The fault-tolerant control method for the self-service recycling terminal according to claim 4, characterized in that, The control command sequence corresponding to the first type of encoding includes, in sequence: start the compressor motor to rotate forward, and continue operation. Seconds, stop the compressor motor, delay. Seconds, triggering a reassessment of the state; where compression time is included. Dynamically adjusted based on the severity of visual deviation, as shown below: In the formula, Based on time compression, This refers to the time gain coefficient; The control instruction sequence corresponding to the second type of encoding includes: sending a "forced state overwrite" signal to the main controller, setting the internal logic state to "non-overflow", allowing subsequent business processes, and generating sensor reliability alarm logs.

6. The fault-tolerant control method for a self-service recycling terminal according to claim 5, characterized in that, The method also includes: Obtain the weight data of the currently delivered item; Visual data is analyzed using image processing algorithms to obtain the shape features of objects, and the volume of objects is calculated and whether abnormal shape features are detected based on the shape features. The unit volume weight is calculated based on the weighing data and the volume of the item. If the unit volume weight exceeds the weight risk threshold and abnormal morphological features are detected, an item abnormality event containing a third type of code is generated. The type code of the abnormal event of the item is used as input, and the control command is obtained by querying the preset diagnostic decision matrix. The matrix also stores the mapping relationship between the third type code and the control command.

7. The fault-tolerant control method for a self-service recycling terminal according to claim 6, characterized in that, The control instructions corresponding to the third type of code include: sending a "lock" signal to the compression control unit to prohibit compression operations during this business cycle, executing subsequent settlement and payment processes normally, marking the business record as a high-risk delivery, and reporting the item's shape characteristics and weight data to the cloud. Control instructions encoded in the third type have a higher priority than those encoded in the first type and the second type.

8. A fault-tolerant control system for a self-service recycling terminal, characterized in that, include: The data signal acquisition module is used to acquire visual data representing the physical scene inside the recycling container through a camera after the delivery compartment door is closed, as well as to acquire logical state signals output by the physical state sensor. The event generation module is used to process visual data through a pre-trained visual feature extraction model to obtain scene structured features; compare the scene structured features with the expected scene features obtained by mapping logical state signals; when the difference between the two exceeds a preset threshold, generate a state mismatch event containing a specific type code; the type code is determined by the current logical state and the comparison result. The control adjustment module is used to take the type code of the state mismatch event as input and query the preset diagnostic decision matrix, which stores the mapping relationship between the state mismatch event type and the control command. Based on the query results, the corresponding control command sequence is sent to the actuator to overwrite or correct the original control logic.

9. A terminal, characterized in that, include: Memory, used to store the fault-tolerant control program of the self-service recycling terminal; The processor is configured to implement the steps of the fault-tolerant control method for the self-service recycling terminal as described in any one of claims 1 to 7 when executing the fault-tolerant control program of the self-service recycling terminal.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a fault-tolerant control program for a self-service recycling terminal, which, when executed by a processor, implements the steps of the fault-tolerant control method for a self-service recycling terminal as described in any one of claims 1 to 7.

Images