Garbage disposal image recognition system based on visual detection
By analyzing the micro-vibration frequency field on the surface of waste and using a multilayer sensor network for identification, the problem of high-precision identification and grasping of heterogeneous waste under the cover of mud and sludge was solved, and a stable waste sorting process was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JINZHAI HAICHUANG ENVIRONMENTAL ENG CO LTD
- Filing Date
- 2026-03-13
- Publication Date
- 2026-06-09
AI Technical Summary
Existing technologies struggle to achieve high-precision material identification and grasping of heterogeneous waste surfaces when they are obscured by mud and visual overlap, leading to positioning errors and grasping mistakes.
By acquiring a continuous video stream of the target object under mechanical vibration, spatiotemporal filtering and phase clustering are performed to extract micro-vibration frequency field data. Material properties are identified using a multilayer perceptron network, and kinematic decoupling instructions are generated to guide the robotic arm to grasp the object.
It achieves high-precision identification and grasping of heterogeneous waste in complex environments, avoiding breakage and falling, and has the ability to update the model online to adapt to changes in different types of waste.
Smart Images
Figure CN122164661A_ABST
Abstract
Description
Technical Field
[0001] This embodiment relates to the field of waste sorting and machine vision recognition technology, specifically to a waste processing image recognition system based on visual detection. Background Technology
[0002] In the current solid waste sorting environment, heterogeneous waste mixtures usually move on conveyor belts with mechanical noise and vibration; due to the complex working environment, the surface of heterogeneous waste is often completely covered by mud and dirt, and waste with different physical properties is prone to overlap and close arrangement.
[0003] To visually inspect and classify this waste, existing solutions generally adopt traditional image recognition architectures based on the surface texture, color, or two-dimensional shape of objects. Although such solutions have certain processing capabilities under ideal observation scenarios, they are highly dependent on the visible optical characteristics of the object's surface and will fail significantly when encountering severe mud or dirt obscuring the surface. At the same time, when faced with visually overlapping but physically independent objects, existing two-dimensional image segmentation technology is difficult to accurately segment the boundaries. This not only produces serious positioning errors but also causes the external robotic arm to be unable to dynamically match the appropriate gripping torque when performing gripping, which in turn leads to misoperations such as the target object breaking and becoming contaminated or falling due to insufficient gripping force.
[0004] Therefore, how to accurately invert the physical material properties of a target and achieve high-precision boundary segmentation in complex scenes where the object surface is severely occluded and there is visual aliasing has become an urgent technical problem to be solved. Summary of the Invention
[0005] The purpose of this invention is to provide a visual detection-based waste disposal image recognition system to solve the following technical problems:
[0006] It avoids positioning errors caused by the complete obscuring of the surface of heterogeneous waste by mud and dirt and visual overlap, and makes it easier to use mechanical vibration excitation to identify the material of the target object, thereby guiding the external robotic arm to perform precise grasping actions that prevent the target object from breaking and contaminating, and also avoid insufficient grasping force leading to falling.
[0007] The objective of this invention can be achieved through the following technical solutions:
[0008] A visual detection-based waste management image recognition system includes:
[0009] The data acquisition module is used to acquire a continuous video stream of the target object moving on a dynamic bearing surface with mechanical vibration excitation, and to stack the continuous video stream into an initial spatiotemporal pixel tensor containing spatial and temporal dimensions.
[0010] The spatiotemporal filtering and amplification module is used to perform spatial frequency band separation and temporal filtering on the initial spatiotemporal pixel tensor, extract pixel brightness change data, and amplify the pixel brightness change data based on the preset amplification factor to generate micro vibration frequency field data containing the spatial distribution of vibration frequency and amplitude of each pixel.
[0011] The phase clustering module is used to calculate the temporal vibration frequency covariance and phase covariance of adjacent pixels in the micro-vibration frequency field data, and construct a dynamic graph with pixels as nodes and edges representing the covariance relationship based on the temporal vibration frequency covariance and phase covariance to generate a target object mask.
[0012] The attribute inversion module is used to input the vibration features corresponding to the target object mask into a preset multilayer perceptron network, and combine it with a preset material response dictionary that represents the mapping relationship between vibration features and physical properties to output physical property classification results and boundary polygon data.
[0013] The closed-loop control module is used to generate kinematic decoupling instructions based on the physical property classification results and boundary polygon data, and to store the abnormal vibration characteristics that meet the preset abnormal conditions into the preset long-tail material library to update the preset material response dictionary.
[0014] Furthermore, the spatiotemporal filtering and amplification module includes:
[0015] Spatial decomposition unit is used to apply Laplacian pyramid decomposition to each frame of the image in the initial spatiotemporal pixel tensor to separate image bands with different spatial frequencies;
[0016] The temporal extraction unit is used to filter the image band using a preset bandpass filter in the time dimension to extract the pixel brightness change data of each pixel.
[0017] Furthermore, when constructing a dynamic graph based on the time-domain vibration frequency covariance and the phase covariance, the phase clustering module executes the following judgment logic:
[0018] If the combined weighted value of the time-domain vibration frequency covariance and the phase covariance is greater than the preset covariance threshold, then a connection is generated between adjacent pixels, and the adjacent pixels are assigned to the same target object cluster.
[0019] If the overall weighted value is less than the preset covariance threshold, then the generation of edges between adjacent pixels is rejected, and the adjacent pixels are assigned to different target object clusters.
[0020] If the overall weighted value is equal to the preset covariance threshold, then adjacent pixels are marked as undetermined boundary pixels, and boundary verification request data is generated.
[0021] Furthermore, the vibration characteristics corresponding to the target object mask include dominant frequency data and attenuation rate data;
[0022] Among them, the main frequency data represents the resonant frequency of the target object under mechanical vibration excitation, and the attenuation rate data represents the damping coefficient of the target object.
[0023] Furthermore, the physical property classification results include material confidence data. When generating kinematic decoupling instructions, the closed-loop control module executes the following judgment logic:
[0024] If the material confidence data is greater than the preset confidence threshold, a kinematic decoupling instruction is generated based on the physical property classification results and the boundary polygon data.
[0025] If the material confidence data is less than the preset confidence threshold, the vibration feature is determined to meet the preset abnormal conditions, and the corresponding abnormal vibration waveform is extracted as an abnormal vibration feature and stored in the preset long-tail material library.
[0026] If the material confidence data equals the preset confidence threshold, a re-acquisition command is generated and sent to the data acquisition module.
[0027] Furthermore, the kinematic decoupling command includes the centroid coordinates of the target object, the physical property classification results, and the boundary polygon data;
[0028] Among them, the kinematic decoupling command is used to send to the external actuator to guide the external actuator to perform physical grasping action.
[0029] Furthermore, the closed-loop control module is also used for:
[0030] Extract abnormal vibration features from a pre-defined long-tail material library as baseline data;
[0031] The preset multilayer perceptron network is iteratively trained based on baseline data to update the weight parameters of the preset multilayer perceptron network and the preset material response dictionary.
[0032] Furthermore, the system is applied to waste sorting scenarios;
[0033] The target object is a mixture of heterogeneous waste, and the dynamic bearing surface is a waste conveyor belt with motor noise and vibration.
[0034] The physical property classification results include high-frequency, low-damping glass and metal materials, mid-frequency, high-damping plastic materials, and zero-response wet paper and clay materials.
[0035] The beneficial effects of this invention are:
[0036] 1. This invention extracts the micro-vibration frequency field data of garbage under mechanical vibration excitation, transforming traditional surface visual recognition into deep physical feature inversion; even if the surface of an object is completely covered by mud and dirt, the material properties can be accurately identified through its inherent resonant frequency and damping characteristics; this non-contact sensing method effectively solves the problem of classification failure caused by the lack of optical features in heterogeneous garbage, and greatly improves the robustness of recognition in complex environments.
[0037] 2. This invention addresses the segmentation challenge caused by overlapping and closely packed waste. The system utilizes the temporal vibration frequency and phase covariance between adjacent pixels to construct a dynamic graph, defining the object mask through the consistency of physical motion. As long as there is a difference in the physical stiffness of adjacent waste, the system can achieve precise segmentation at the pixel level, fundamentally solving the positioning deviation caused by visual aliasing and providing high-precision boundary data for subsequent automated sorting.
[0038] 3. The kinematic decoupling instructions generated by this invention integrate physical properties and geometric features, which can guide the external robotic arm to dynamically adjust the gripping torque according to the target material; for example, a larger gripping force is applied to glass and metal, and pressure correction is applied to materials such as plastic; this mechanism avoids falling due to insufficient gripping force, and also prevents the target object from breaking and causing environmental pollution due to excessive torque, ensuring the stability and efficiency of the sorting process.
[0039] 4. This invention achieves automatic capture of abnormal vibration characteristics and incremental model updates through a closed-loop control module and a long-tail material library. When encountering unknown materials or low-confidence scenarios, the system can automatically trigger high-precision resampling or store the data in the library for iterative training. This self-evolving design enables the system to have online correction capabilities, continuously adapt to the ever-changing types of waste, and reduce long-term maintenance costs. Attached Figure Description
[0040] The invention will now be further described with reference to the accompanying drawings.
[0041] Figure 1 This is a block diagram of a visual detection-based waste management image recognition system in an embodiment of this application. Detailed Implementation
[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0043] Please see Figure 1As shown, the waste disposal image recognition system based on visual detection includes: a data acquisition module, used to acquire a continuous video stream of a target object moving on a dynamic bearing surface with mechanical vibration excitation, and to stack the continuous video stream into an initial spatiotemporal pixel tensor containing spatial and temporal dimensions;
[0044] The spatiotemporal filtering and amplification module is used to perform spatial frequency band separation and temporal filtering on the initial spatiotemporal pixel tensor, extract pixel brightness change data, and amplify the pixel brightness change data based on the preset amplification factor to generate micro vibration frequency field data containing the spatial distribution of vibration frequency and amplitude of each pixel.
[0045] The phase clustering module is used to calculate the temporal vibration frequency covariance and phase covariance of adjacent pixels in the micro-vibration frequency field data, and construct a dynamic graph with pixels as nodes and edges representing the covariance relationship based on the temporal vibration frequency covariance and phase covariance to generate a target object mask.
[0046] The attribute inversion module is used to input the vibration features corresponding to the target object mask into a preset multilayer perceptron network, and combine it with a preset material response dictionary that represents the mapping relationship between vibration features and physical properties to output physical property classification results and boundary polygon data.
[0047] The closed-loop control module is used to generate kinematic decoupling instructions based on the physical property classification results and boundary polygon data, and to store the abnormal vibration characteristics that meet the preset abnormal conditions into the preset long-tail material library to update the preset material response dictionary.
[0048] The data acquisition module uses an industrial area array camera to acquire continuous video streams of the target object moving on a dynamic bearing surface subject to mechanical vibration excitation. The dynamic bearing surface is defined as a transmission device with constant or random minor vibrations, and the mechanical vibration excitation originates from the noise floor of the drive motor to ensure the construction of original observation data containing time dimension information. The data acquisition module stacks the continuous video streams on the time axis to generate an initial spatiotemporal pixel tensor. ; Let be the initial spatiotemporal pixel tensor with dimension . ; and The spatial resolution of the image; The number of consecutive sampling frames;
[0049] The spatiotemporal filtering and amplification module applies the initial spatiotemporal pixel tensor Spatial frequency band separation is performed, and pixel brightness variation data within a specific frequency range is extracted using a time-domain bandpass filter, and then amplified by a preset factor. The pixel brightness variation data is amplified to generate micro-vibration frequency field data. Specifically, micro-vibration frequency field data The formula for generating it is:
[0050]
[0051] in, The output micro-vibration frequency field data tensor has the same dimensions as the initial spatiotemporal pixel tensor. Consistent, all are ; Represents the spatial coordinate index of the image. The frame index represents the time dimension;
[0052] The linear amplification factor is specifically derived by statistically analyzing the signal-to-noise ratio relationship between the visible amplitude and the background noise amplitude of heterogeneous waste in a large number of historical samples. The value is set between 10 and 30 to ensure that the weak vibration characteristics are enhanced while suppressing background noise. In this embodiment, the value is 15.
[0053] To represent the time-domain bandpass filtering operation, its input is The output is a filtered tensor of the same dimension. This operation is an ideal bandpass filtering process that is performed independently in the time dimension for each pixel in the initial spatiotemporal pixel tensor.
[0054] In order to obtain micro-vibration frequency field data Extracting instantaneous frequency In this embodiment, Hilbert transform is used to construct an analytical signal for the time series of each pixel. The instantaneous frequency sequence that changes with time is obtained by calculating the phase derivative of the analytical signal. At the same time, Fourier transform is performed on the time series to extract the phase corresponding to the maximum energy frequency component as the main frequency phase angle.
[0055] Based on this, the phase clustering module calculates the temporal vibration frequency covariance between adjacent pixels. With phase covariance Among them, the time-domain vibration frequency covariance The calculation method is as follows:
[0056]
[0057] in, The time-domain vibration frequency covariance; and Adjacent pixels and The instantaneous frequency; and The average frequency is calculated by applying the instantaneous frequency sequence in the time dimension. Take the arithmetic mean above; The value is a preset minimum positive constant, which is taken as the value in this embodiment. This is used to ensure the stability and continuity of covariance calculation; phase covariance. The calculation method is as follows:
[0058]
[0059] in, For phase covariance; and Adjacent pixels and The dominant frequency phase angle; the phase clustering module constructs a dynamic graph based on the above covariance, with the specific rule being: when the combined weighted value of the time domain vibration frequency covariance and the phase covariance is greater than the preset covariance threshold. When an edge is generated between adjacent pixel nodes, an edge is generated; otherwise, an edge is not generated.
[0060] Graph-based clustering algorithms are used to extract sets of pixels with the same vibration rhythm as the target object mask. By using the consistency of physical motion to segment object boundaries, the problem of visual aliasing can be solved.
[0061] The attribute inversion module will mask The vibration characteristics within the region are input into a pre-defined multilayer perceptron network. The multilayer perceptron network acts as a feature encoder, and its penultimate layer output is defined as a high-dimensional feature vector. ,calculate Material Response Dictionary Chinese standard material feature vectors The cosine similarity is calculated using the following formula:
[0062]
[0063] in, It is a high-dimensional feature vector; It is a material response dictionary, which stores a database of the correlation between the resonant frequency and damping characteristics of known materials under standard excitation;
[0064] The standard material feature vector; Cosine similarity; To determine the output, select the standard material category with the highest similarity. For the target object mask;
[0065] The boundary polygon data consists of a finite sequence of vertex coordinates, obtained by targeting a mask. The edge pixels are obtained by performing connected component extraction and polygon approximation algorithms; This represents the Euclidean norm, used for normalizing the magnitude of a vector.
[0066] The closed-loop control module is based on the classification results. and polygons Generate kinematic decoupling commands to drive an external robotic arm to perform sorting, while simultaneously responding to the detection of abnormal vibration characteristics that do not conform to known features. Store it in the preset long-tail material library Used for subsequent material response dictionaries Online updates;
[0067] Based on this, to enhance the physical interpretability of attribute inversion, the multilayer perceptron network specifically adopts a four-layer architecture including an input layer, two hidden layers each containing 128 neurons, and an output layer, and uses the ReLU function as the activation layer to extract deep nonlinear material features; the graph theory clustering algorithm identifies pixel clusters that meet the covariance threshold condition as independent object masks by constructing an adjacency matrix and performing connected component search.
[0068] In the scenario of a waste conveyor belt, the system in this embodiment introduces the physical dimension of micro-vibration. Even if the surface of heterogeneous waste is completely covered by mud, the system can still identify its material through the inherent resonant frequency of the object. At the same time, by using phase consistency for non-heuristic clustering, it can effectively distinguish heterogeneous objects that overlap but are physically independent.
[0069] In this embodiment, the spatiotemporal filtering and amplification module includes: a spatial decomposition unit, used to apply Laplacian pyramid decomposition to each frame of the initial spatiotemporal pixel tensor to separate image bands with different spatial frequencies; and a temporal extraction unit, used to filter the image bands in the time dimension using a preset bandpass filter to extract pixel brightness change data for each pixel.
[0070] Spatial decomposition unit for initial spatiotemporal pixel tensor Each frame of the image is decomposed using Laplacian pyramid decomposition. Through recursive Gaussian smoothing and differencing operations, the image is decomposed into multiple spatial bands from low frequency to high frequency. To achieve multi-scale feature extraction; the temporal extraction unit uses a preset bandpass filter to process each spatial band term in the time dimension, filtering out ambient light interference and locking in vibration signals; and extracts pixel brightness variation data. The calculation logic is as follows:
[0071]
[0072] Among them, the bandpass filter operator is derived from the preset configuration. Used for frequency truncation of time series, its specific mathematical form is the time-domain convolution of an ideal bandpass filter, that is:
[0073]
[0074] in, For the impulse response, its calculation formula is:
[0075]
[0076] in, Pi is a constant. For continuous-time variables; to solve when The data overflow problem caused by the denominator being zero is in Using L'Hôpital's rule to find the limit, Defined separately This ensures that the filter impulse response is calculated continuously in the time domain without singularities; among which, It is an impulse response The value at zero is used to characterize the center weight of the bandpass filter in the discrete-time convolution process. This special definition is used to establish... With filtering results The direct computational correlation ensures the convergence of the filtering calculation; This refers to pixel brightness variation data; It is a bandpass filter operator derived from a preset configuration;
[0077] For the filter impulse response in The defined value at that location; The low cutoff Hz is derived from the lowest operating frequency of the conveyor belt motor. In this embodiment, the specific value is 0.5Hz, which is used to filter out low-frequency trend interference generated by the uniform motion of the conveyor belt. The high cutoff Hz is derived from the upper limit of the Nyquist sampling theorem. In this embodiment, the specific value is 25Hz, which is used to meet the anti-aliasing filtering requirements of the system at a sampling rate of 60fps.
[0078] Based on this, the spatiotemporal filtering and amplification module achieves cascaded enhancement of the signal-to-noise ratio of weak signals through the combination of Laplace pyramid and bandpass filtering. In the garbage conveyor belt scenario, spatial decomposition can lock the optimal observation scale for heterogeneous garbage of different sizes, while temporal filtering eliminates constant illumination changes and low-frequency displacement interference, ensuring the signal purity of subsequent amplification processing and verifying the robustness of this technical solution in complex illumination and vibration coupling environments.
[0079] In this embodiment, when constructing a dynamic graph based on the temporal vibration frequency covariance and the phase covariance, the phase clustering module executes the following judgment logic: if the combined weighted value of the temporal vibration frequency covariance and the phase covariance is greater than a preset covariance threshold, then an edge is generated between adjacent pixels, and the adjacent pixels are assigned to the same target object cluster; if the combined weighted value is less than the preset covariance threshold, then the generation of an edge between adjacent pixels is rejected, and the adjacent pixels are assigned to different target object clusters; if the combined weighted value is equal to the preset covariance threshold, then the adjacent pixels are marked as undetermined boundary pixels, and boundary verification request data is generated.
[0080] The phase clustering module uses a comprehensive weighting value The topological connectivity between pixels is determined by the following formula:
[0081]
[0082] in, This is a comprehensive weighted value; The weighting factor is set to 0.7; This refers to the time-domain vibration frequency covariance mentioned earlier; This refers to the phase covariance mentioned earlier; To preset the covariance threshold, during the system initialization phase, the background noise vibration signal of the empty conveyor belt is collected in real time, the average covariance between background pixels is calculated and multiplied by 1.5 as an empirical threshold.
[0083] Response to the overall weighted value Greater than the preset covariance threshold The phase clustering module generates edges between adjacent pixels; in response to the comprehensive weighting value Less than the preset covariance threshold The phase clustering module refuses to generate edges between adjacent pixels;
[0084] Based on this, the response is that the overall weighted value equals the preset covariance threshold. The phase clustering module marks adjacent pixels as undetermined boundary pixels and generates boundary verification request data, triggering higher-precision resampling of local areas;
[0085] The specific operation is as follows: the instruction data acquisition module temporarily increases the sampling frequency of the continuous video stream from the preset 60fps to 120fps for a local pixel area, and recalculates the comprehensive weighted value of that area. ;
[0086] definition The weighted stability difference before and after resampling is calculated using the following formula:
[0087]
[0088] in, The result calculated above is in the preset... Comprehensive weighted value at sampling frequency , The weighted value after resampling;
[0089] like Less than or equal to the preset stability threshold , The value is a dimensionless empirical constant, which is set to 0.02 in this embodiment. Therefore, the vibration characteristics of the pixel point are determined to have reached a steady state.
[0090] Calculate the boundary slope under steady-state conditions. , Characterizes the combined weighted value between adjacent pixels The spatial rate of change, in this embodiment, is determined by the resampling region. The distribution is obtained by performing a first-order spatial difference operation.
[0091] like If the gradient value is greater than the preset gradient threshold, it is confirmed as a true boundary formed by a sudden change in physical properties;
[0092] The recommended range for the preset gradient threshold is between 0.15 and 0.35. Specifically, this is achieved by performing boundary scanning on standard samples and extracting the comprehensive weighted value at points of material abrupt change. The spatial first derivative mean is determined by combining it with the system noise margin;
[0093] Among them, the boundary slope Characterizes the combined weighted value between adjacent pixels The spatial rate of change, through the resampling region The distribution is obtained by taking the first-order spatial derivative.
[0094] In the scenario of garbage conveyor belts, the phase clustering module provides a non-heuristic edge segmentation method through covariance weighted logic. As long as the physical stiffness of two pieces of garbage is different, resulting in a slight difference in vibration phase, the system can segment them at the pixel level, thus solving the positioning deviation caused by visual aliasing.
[0095] In this embodiment, the vibration characteristics corresponding to the target object mask include dominant frequency data and attenuation rate data; wherein, the dominant frequency data represents the resonant frequency of the target object under mechanical vibration excitation, and the attenuation rate data represents the damping coefficient of the target object.
[0096] The vibration feature extraction process corresponding to the target object mask performs dimensionality reduction characterization of the physical response, and the main frequency data... The extraction formula is:
[0097]
[0098] in, The main frequency data characterizes the natural resonant frequency of the target object under mechanical vibration excitation; The Fourier transform result of the vibration signal; symbol Operators that concatenate matrices or vectors;
[0099] The attenuation rate data λ is defined as the damping coefficient of the target object to reflect the energy dissipation characteristics of the material. Specifically, the formula for extracting the attenuation rate data λ is as follows:
[0100]
[0101] in, This is the attenuation rate data; and For the vibration signal at time and The envelope peak amplitude; The moment when the target object is just subjected to mechanical vibration excitation and the amplitude reaches the global maximum peak value; The time corresponding to a local peak in the envelope of the vibration signal of the target object within the observation time window. For a small noise floor bias value, in this embodiment, the value is taken as [value]. The noise level is determined based on the root mean square noise of the image sensor at zero input, which is used to prevent data overflow.
[0102] Specifically, the system automatically identifies vibration sequences within the observation time window. and The point with the largest absolute value of the amplitude across the entire sequence is determined as the global maximum peak moment. Then from Searching backwards along the time axis, the first coordinate point satisfying both zero first derivative and negative second derivative is identified as a local maximum moment. ;
[0103] Specifically, the vibration signal at time envelope peak amplitude An analytical signal is constructed by performing a Hilbert transform on the original vibration sequence:
[0104]
[0105] And take its mold length Obtain; among them, The peak amplitude of the envelope; To analyze the signal; The imaginary unit satisfies ; The original vibration signal sequence of pixel brightness changing over time within the masked area of the target object;
[0106] In the scenario of garbage conveyor belts, extracting the main frequency data and attenuation rate data establishes a mapping bridge from pixel fluctuations to material properties, realizing material matching based on low-dimensional frequency domain features and reducing the computing power overhead of the back-end computing unit.
[0107] In this embodiment, the physical property classification result includes material confidence data. When generating kinematic decoupling instructions, the closed-loop control module executes the following judgment logic: if the material confidence data is greater than the preset confidence threshold, then the kinematic decoupling instructions are generated based on the physical property classification result and the boundary polygon data.
[0108] If the material confidence data is less than the preset confidence threshold, the vibration feature is determined to meet the preset abnormal conditions, and the corresponding abnormal vibration waveform is extracted as an abnormal vibration feature and stored in the preset long-tail material library; if the material confidence data is equal to the preset confidence threshold, a re-acquisition command is generated and sent to the data acquisition module.
[0109] After receiving the physical property classification results, the closed-loop control module extracts the material confidence data. ; Responding to material confidence data Greater than the preset confidence threshold The closed-loop control module drives the actuator to perform normal grasping; in response to material confidence data. Less than the preset confidence threshold The closed-loop control module determines that the vibration characteristic meets the preset abnormal conditions, extracts the corresponding abnormal vibration waveform, and stores it in the preset long-tail material library. ;
[0110] For material confidence data, the maximum probability value is obtained from the probability distribution vector of the multilayer perceptron network output layer after transformation by the Softmax layer.
[0111] The default confidence threshold is set to 0.85, taking into account both the fault tolerance of the robotic arm end effector and the physical variability of the waste material.
[0112] Responding to material confidence data Equal to the preset confidence threshold The closed-loop control module generates a re-acquisition command and sends it to the data acquisition module, requiring an increase in the frame rate for secondary observation. In the garbage conveyor scenario, the closed-loop control module establishes a self-evolving data closed loop, enabling the system to have the ability to assess confidence boundaries, effectively avoiding erroneous operations of the robotic arm under low confidence.
[0113] In this embodiment, the kinematic decoupling instruction includes the centroid coordinate data of the target object, the physical attribute classification results, and the boundary polygon data; wherein, the kinematic decoupling instruction is used to send to the external actuator to guide the external actuator to perform physical grasping action.
[0114] The generation process of kinematic decoupling commands integrates multi-dimensional spatial and physical information, and extracts centroid coordinate data. Classification results of physical attributes Encapsulated in instructions, it is used by external actuators to dynamically adjust the gripping torque; specifically, it establishes material categories. With pressure correction factor The linear mapping relationship:
[0115]
[0116] in, The centroid coordinate data is derived from the target object mask. The first moment is calculated; k: physical attribute classification result; The basic gripping force of the robotic arm is preset based on the rated load of the actuator; For different materials The pressure correction factor is 1.5 for high-frequency, low-damping metals, 0.8 for mid-frequency, high-damping plastics, and 0.4 for zero-response wet paper. To output gripping force; This is boundary polygon data, used for planning obstacle avoidance paths;
[0117] In the context of a waste conveyor belt, kinematic decoupling commands ensure seamless integration between visual recognition results and physical actuators. The external robotic arm can dynamically adjust the gripping pressure of the end effector based on the physical property classification results, preventing both breakage and contamination of the target object and avoiding drops due to insufficient gripping force. Kinematic decoupling commands refer to separating and calculating the material physical property parameters and spatial geometric gripping parameters of the target object, and converting them into control commands for the independent motion components of each joint of the robotic arm.
[0118] In this embodiment, the closed-loop control module is also used to: extract abnormal vibration features from a preset long-tail material library as baseline data; and iteratively train a preset multilayer perceptron network based on the baseline data to update the weight parameters of the preset multilayer perceptron network and the preset material response dictionary.
[0119] The closed-loop control module periodically extracts abnormal vibration features from a pre-defined long-tail material library as baseline data. It then uses an unsupervised density clustering algorithm to aggregate the similarity of the baseline data to generate pseudo-labels. Based on the clustering results, it temporarily assigns category identifiers to unlabeled abnormal data. These labels serve as the target classification constraint, and the module incrementally learns a pre-defined multilayer perceptron network using stochastic gradient descent. The module iteratively updates the weight parameters of the pre-defined multilayer perceptron network through training. Material Response Dictionary ;
[0120] In this process, for multilayer perceptron networks, neurons are dynamically added to the output layer, and the weights of the newly added nodes are initialized according to a mapping formula.
[0121]
[0122] in, For the weight parameters of the multilayer perceptron network; D: preset material response dictionary; The weight of the newly added node; The original weights; It is a linear mapping operator; For the first The feature center centroid vector of a new material cluster;
[0123] By directly mapping cluster centers to the weight space, the classifier can quickly identify new material categories and reduce the number of training convergence steps. In the waste conveyor belt scenario, the closed-loop control module realizes the system's online incremental evolution and dynamic mismatch repair. The system can use abnormal data on the production line to perform incremental learning to update model parameters.
[0124] In this embodiment, the system is applied to a waste sorting scenario; the target object is a heterogeneous waste mixture, and the dynamic bearing surface is a waste conveyor belt with motor noise vibration; the physical property classification results include high-frequency low-damping glass and metal material categories, mid-frequency high-damping plastic material categories, and zero-response wet paper and mud block material categories.
[0125] The system is deployed on the dynamic bearing surface of a garbage conveyor belt with motor noise and vibration, and continuously observes the heterogeneous garbage mixture as the target object; when the heterogeneous garbage mixture passes by, the system extracts its micro-vibration characteristics and maps them to the physical property classification results;
[0126] For objects exhibiting high-frequency, low-damping characteristics, the system classifies them into the glass-metal material category; based on this, for objects exhibiting mid-frequency, high-damping characteristics, they are classified into the plastic material category; for objects exhibiting zero response or extremely low-frequency characteristics due to non-rigid connections and high energy absorption properties, they are classified into the wet paper / mud block material category.
[0127] In the scenario of a waste conveyor belt, the system utilizes the background vibration of the conveyor belt as an excitation source to achieve non-contact material sensing.
[0128] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.
Claims
1. A waste disposal image recognition system based on visual inspection, characterized in that, include: The data acquisition module is used to acquire a continuous video stream of the target object moving on a dynamic bearing surface with mechanical vibration excitation, and to stack the continuous video stream into an initial spatiotemporal pixel tensor containing spatial and temporal dimensions. The spatiotemporal filtering and amplification module is used to perform spatial frequency band separation and temporal filtering on the initial spatiotemporal pixel tensor, extract pixel brightness change data, and amplify the pixel brightness change data based on a preset amplification factor to generate micro-vibration frequency field data containing the spatial distribution of vibration frequency and amplitude of each pixel. The phase clustering module is used to calculate the temporal vibration frequency covariance and phase covariance of adjacent pixels in the micro-vibration frequency field data, and construct a dynamic graph with pixels as nodes and edges representing the covariance relationship based on the temporal vibration frequency covariance and the phase covariance to generate a target object mask. The attribute inversion module is used to input the vibration features corresponding to the target object mask into a preset multilayer perceptron network, and combine it with a preset material response dictionary that characterizes the mapping relationship between vibration features and physical attributes to output physical attribute classification results and boundary polygon data. The closed-loop control module is used to generate kinematic decoupling instructions based on the physical property classification results and the boundary polygon data, and to store abnormal vibration characteristics that meet the preset abnormal conditions into a preset long-tail material library to update the preset material response dictionary.
2. The waste disposal image recognition system based on visual detection according to claim 1, characterized in that, The spatiotemporal filtering and amplification module includes: The spatial decomposition unit is used to apply Laplacian pyramid decomposition to each frame of the initial spatiotemporal pixel tensor to separate image bands with different spatial frequencies. The temporal extraction unit is used to filter the image band in the time dimension using a preset bandpass filter to extract the pixel brightness change data of each pixel.
3. The waste disposal image recognition system based on visual detection according to claim 1, characterized in that, When constructing a dynamic graph based on the time-domain vibration frequency covariance and the phase covariance, the phase clustering module executes the following judgment logic: If the combined weighted value of the time-domain vibration frequency covariance and the phase covariance is greater than a preset covariance threshold, then a connection is generated between the adjacent pixels, and the adjacent pixels are assigned to the same target object cluster. If the overall weighted value is less than the preset covariance threshold, then the generation of a connection between the adjacent pixels is rejected, and the adjacent pixels are assigned to different target object clusters. If the overall weighted value is equal to the preset covariance threshold, then the adjacent pixels are marked as undetermined boundary pixels, and boundary verification request data is generated.
4. The waste disposal image recognition system based on visual detection according to claim 1, characterized in that, The vibration characteristics corresponding to the target object mask include dominant frequency data and attenuation rate data; The dominant frequency data represents the resonant frequency of the target object under the mechanical vibration excitation, and the attenuation rate data represents the damping coefficient of the target object.
5. The waste disposal image recognition system based on visual detection according to claim 1, characterized in that, The physical property classification result includes material confidence data. When generating the kinematic decoupling command, the closed-loop control module executes the following judgment logic: If the material confidence data is greater than the preset confidence threshold, then the kinematic decoupling instruction is generated based on the physical property classification result and the boundary polygon data; If the material confidence data is less than the preset confidence threshold, the vibration feature is determined to meet the preset abnormal conditions, and the corresponding abnormal vibration waveform is extracted as the abnormal vibration feature and stored in the preset long-tail material library. If the material confidence data is equal to the preset confidence threshold, a re-acquisition instruction is generated and sent to the data acquisition module.
6. The waste disposal image recognition system based on visual detection according to claim 5, characterized in that, The kinematic decoupling command includes the centroid coordinates of the target object, the physical attribute classification results, and the boundary polygon data; The kinematic decoupling command is used to send to an external actuator to guide the external actuator to perform a physical grasping action.
7. The waste disposal image recognition system based on visual detection according to claim 1, characterized in that, The closed-loop control module is also used for: Extract the abnormal vibration features from the preset long-tail material library as baseline data; The preset multilayer perceptron network is iteratively trained based on the baseline data to update the weight parameters of the preset multilayer perceptron network and the preset material response dictionary.
8. The waste disposal image recognition system based on visual detection according to claim 1, characterized in that, The system is applied to waste sorting scenarios; The target object is a heterogeneous waste mixture, and the dynamic bearing surface is a waste conveyor belt with motor noise vibration. The physical property classification results include high-frequency, low-damping glass and metal material categories, mid-frequency, high-damping plastic material categories, and zero-response wet paper and clay block material categories.