Automatic recognition system for recyclable components of decoration garbage based on multi-modal feature fusion

Abstract

The present application relates to the technical field of decoration garbage intelligent sorting, and discloses a decoration garbage recyclable component automatic identification system based on multi-modal feature fusion. The system comprises a multi-modal data acquisition module, a cross-modal feature fusion module, a recyclable component discrimination module and a self-adaptive identification strategy optimization module. The multi-modal data acquisition module is based on a sample library construction framework, synchronously acquires visible light images, near-infrared spectra and three-dimensional point cloud data and outputs a feature set; the cross-modal feature fusion module compares the feature set with a standard library in time domain, frequency domain and spatial dimension to generate a component-level feature correlation matrix; the recyclable component discrimination module inputs the matrix into a spatial semantic analysis network, generates a spatial heat distribution map in combination with spatial distribution parameters and position information and locates an enrichment area; and the self-adaptive module configures parameters accordingly, enables multi-modal synchronous acquisition for high-probability areas and applies feature disturbance tests to adjacent category samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent sorting of decoration waste, in particular to an automatic identification system for recyclable components of decoration waste based on multi-modal feature fusion. BACKGROUND

[0002] Under the background of continuous expansion of urban construction scale, the amount of decoration waste is increasing year by year, and the complexity of its components brings great challenges to recycling. A large number of recyclable resources such as waste metals, plastic pipes, and wooden boards are mixed in this kind of waste, but for a long time, the recycling rate of these resources has been at a low level. At present, in the process of decoration waste treatment, the identification of recyclable components has obvious shortcomings, which has become the main bottleneck restricting resource utilization.

[0003] The traditional identification method is mainly manual sorting, and the sorting personnel need to observe and judge by naked eye and experience to select recyclable materials from mixed waste. This method is limited by individual cognitive differences, and different sorting personnel may have deviations in judging the same kind of waste, resulting in poor stability of the identification results. At the same time, decoration waste is often accompanied by dust, sharp objects and chemical residues, and the working environment of sorting personnel is poor, which can easily cause health problems. In addition, the efficiency of manual sorting is very low, and it is difficult to achieve comprehensive and detailed sorting in the face of hundreds of tons of waste treatment per day, and a large amount of recyclable components are treated as waste, causing unnecessary consumption of resources.

[0004] With the development of intelligent technology, some automatic identification equipment has begun to be applied in the field of waste sorting, but existing equipment mostly relies on single identification technology. For example, the identification equipment based on visible light image mainly judges through color and contour features, but plastic and rubber, wood and paperboard in decoration waste are highly similar in appearance, and it is difficult to distinguish them only by visual features; near-infrared spectrum identification equipment identifies through the spectral characteristics of substances, but the spectral signals of different components in decoration waste often overlap, especially in wet or contaminated state, the spectral characteristics will shift, resulting in increased identification error; three-dimensional point cloud identification equipment can obtain the spatial structure information of objects, but it cannot identify the internal material of the object, and it is difficult to accurately distinguish recyclable materials with similar shapes but different materials, such as aluminum alloy and galvanized steel plate.

[0005] Another prominent problem of existing recognition systems is the lack of dynamic adjustment capability. When the stacking form and ingredient proportion of garbage change, the system's recognition parameters cannot automatically adapt and still use fixed recognition mode. For example, in the scenario where recyclable ingredients are dispersedly distributed, the system still performs comprehensive scanning at a fixed frequency, causing waste of computing power; while in the area where recyclable ingredients are highly concentrated, insufficient sampling may lead to missed detection. In addition, the feature boundaries of different categories of garbage are often ambiguous, such as the overlapping area of the spectral features of PVC plastic and PE plastic, and existing systems lack effective feature differentiation mechanisms, which can easily confuse the two and affect the accuracy of recognition.

[0006] At the same time, a unified recyclable feature standard has not been established in the industry, and different systems use different feature parameter systems. In the feature comparison process, most of them are limited to single-dimensional matching, such as only comparing spectral similarity or shape consistency, ignoring the relevance between different dimensional features. For example, the texture feature of a certain plastic has a high matching degree with the standard library, but the spectral feature has a difference, and single-dimensional comparison may lead to misjudgment. This one-sided feature analysis method greatly reduces the reliability of the recognition result and makes it difficult to meet the needs of large-scale recycling processing. SUMMARY

[0007] The purpose of the present application is to provide an automatic recognition system for recyclable ingredients of decoration garbage based on multi-modal feature fusion to solve the problems raised in the background art.

[0008] To achieve the above-mentioned purpose, the present application provides an automatic recognition system for recyclable ingredients of decoration garbage based on multi-modal feature fusion, which comprises:

[0009] A multi-modal data acquisition module: a multi-source data acquisition framework is constructed based on a decoration garbage sample library, visible light image data, near-infrared spectral data and three-dimensional point cloud data of the target garbage are synchronously acquired, and a multi-modal feature set is output through the multi-source data acquisition framework;

[0010] A cross-modal feature fusion module: multi-dimensional feature comparison is performed between the multi-modal feature set and a standard recyclable feature library, the multi-dimensional feature comparison includes time domain texture consistency, frequency domain spectral matching degree and spatial point cloud coincidence degree, and a feature correlation matrix at the ingredient level is generated;

[0011] A recyclable ingredient discrimination module: the feature correlation matrix is input into a spatial semantic analysis network, combined with garbage category spatial distribution parameters and target garbage location information, a spatial thermal distribution map of recyclable ingredient recognition probability is generated, and a recyclable ingredient enrichment area is located;

[0012] An adaptive recognition strategy optimization module: recognition parameters are configured according to the spatial thermal distribution map, including:

[0013] Multimodal synchronous acquisition mode is enabled for high-probability enrichment areas;

[0014] Test by applying feature perturbations to samples from adjacent categories.

[0015] Preferably, the multimodal data acquisition module specifically includes:

[0016] Multi-source data synchronous acquisition: Multimodal decomposition processing is performed on the data of the decoration waste sample library, including: using adaptive Gaussian filtering to extract the pixel ratio of surface texture features and edge contour features of image data, establishing a correlation map between near-infrared spectral data and material composition through spectral band screening, and using spatiotemporal alignment algorithm to match the coordinates of three-dimensional point cloud data from different perspectives;

[0017] Multimodal feature set construction: The processed sample database data is input into a hybrid feature extraction network, which includes:

[0018] Image semantic-based convolutional feature extraction unit is used to generate basic image feature vectors. A fully connected network with embedded position attention mechanism is used to correct feature deviations caused by point cloud noise. Based on spectral feature enhancer, feature weights are dynamically adjusted according to real-time acquired near-infrared spectral data.

[0019] The image data is simultaneously captured by a heterogeneous sensor array deployed on the edge side, including brightness uniformity and contrast fluctuation parameters, reflectance of characteristic bands of the near-infrared spectrum and half-width distribution, density gradient of three-dimensional point cloud and coordinate distribution entropy.

[0020] Multimodal feature set output: Input the real-time acquired data into the multi-source data acquisition framework to obtain a multimodal feature set.

[0021] Preferably, the multimodal feature set output includes the following:

[0022] Adaptive denoising based on waste composition type eliminates acquisition noise caused by ambient light.

[0023] The spatiotemporal feature fusion algorithm integrates the associated features of images, spectra, and point clouds;

[0024] The output includes a multimodal feature set of typical component fluctuation ranges, which is dynamically adjusted as the sample library is updated.

[0025] Preferably, the cross-modal feature fusion module specifically includes:

[0026] Temporal texture consistency calculation: The multimodal feature set and the standard feature library are compared by sliding within a preset time window. The feature sequences acquired asynchronously are aligned by a dynamic time warping algorithm. The texture similarity within each window is calculated to generate a temporal texture vector.

[0027] Frequency domain spectral matching degree detection: the near-infrared spectral data of the multi-modal feature set and the standard feature library are decomposed by a frequency domain decomposition algorithm, the spectral reflectivity ratio is calculated, the spectral matching index of each wave band is extracted, and a frequency domain spectral vector is constructed;

[0028] Spatial point cloud coincidence degree evaluation: based on a structure matching algorithm, the point cloud coordinate distribution of the multi-modal feature set and the standard feature library is matched, the position synchronization error of the point cloud density mutation point is calculated, the KL divergence of the point cloud distance distribution is quantified, and a spatial point cloud vector is generated;

[0029] Feature correlation matrix generation: the time domain texture vector, the frequency domain spectral vector, and the spatial point cloud vector are spliced by tensor, the dimension difference is eliminated by normalized processing of feature importance weighting, and a three-order feature correlation matrix with a dimension of [component number x timestamp x feature type] is output.

[0030] Preferably, the frequency domain spectral matching degree detection specifically includes:

[0031] In the frequency domain analysis stage, first, the corresponding near-infrared spectral data in the multi-modal feature set and the standard feature library is extracted, wavelet packet decomposition is used to analyze each group of spectral signals in multiple scales, the reflectivity distribution characteristics in the pre-set sensitive wave band interval are extracted, the sensitive wave band interval is selected to cover the characteristic absorption range of typical recyclable materials, after the extraction is completed, the reflectivity density of the multi-modal feature set and the standard feature library in the sensitive wave band interval is quantitatively calculated, and the matching index of each wave band is extracted based on the relative matching degree of the two, the spectral matching results of all wave bands are summarized, and a frequency domain spectral vector is constructed.

[0032] Preferably, in the spatial point cloud coincidence degree evaluation, the structure matching algorithm adopts a cosine similarity algorithm to match the position of the mutation point coordinates in the two point cloud sets and identify the position synchronization error.

[0033] Preferably, the recyclable component discrimination module specifically includes:

[0034] Spatial semantic modeling: a spatial distribution topology graph of each category is constructed according to the position information of the target garbage, the spatial adjacency parameters between each category are labeled, the prior constraint condition of the standard distribution of the recyclable component is superimposed in the topology graph, and a semantic topology model including an adjacency matrix and a category confidence matrix is generated;

[0035] Identification probability deduction: the feature correlation matrix is mapped to the corresponding category node of the spatial semantic topology model; based on the graph neural network, the identification probability deduction is performed, and the calculation of the identification probability deduction includes:

[0036] i. calculating the attenuation factor of feature propagation according to the category adjacency parameters;

[0037] ii. Capture cross-category feature correlation features through multi-head attention mechanism;

[0038] iii. Simulate the diffusion path of recyclable component features in the topology network using the Monte Carlo method;

[0039] Probability distribution generation: count the frequency of each category in the simulation propagation, calculate the recyclable component recognition probability value combined with the category adjacency parameter, generate a spatial heat distribution map covering the entire sample, and label the suspicious category set with a probability value exceeding the preset recognition confidence probability threshold;

[0040] Physical area positioning: perform spatial clustering analysis on the spatial heat distribution map to identify the recognition probability aggregation area; according to the target garbage location and spatial semantic connection relationship, the recyclable component rich physical boundary is drawn.

[0041] Preferably, the recyclable component discrimination module further comprises output including suspicious component identification, feature propagation main path, wherein:

[0042] The suspicious component identification is based on the spatial semantic nodes connected by the suspicious category set, and the spatial semantic nodes are bound with the actual recyclable components to form a suspicious component identification set, indicating potential recyclable components or interfering components;

[0043] The feature propagation main path is recorded by recording the node path and its order experienced in each round of propagation in the feature diffusion process simulated by Monte Carlo simulation, and the path sequence with the highest cumulative frequency of occurrence is selected as the feature propagation main path. The output feature propagation main path sequence is a structured and ordered node list reflecting the main propagation trajectory of feature information in the spatial semantics.

[0044] Preferably, the adaptive recognition strategy optimization module specifically comprises:

[0045] When the recognition probability value of a certain area in the spatial heat distribution map exceeds the preset recognition confidence probability threshold, the collection mode switching instruction is issued to the collection terminal to which the area belongs, and the following is executed:

[0046] i. Increase the image / spectrum sampling frequency to 5-10 times the original frequency;

[0047] ii. Synchronously enable real-time tracking mode of feature waveband, capture feature waveband reflectivity mutation;

[0048] iii. Deploy feature anomaly detector on the edge side, record the point cloud density sudden change segment;

[0049] Feature disturbance test execution: apply multi-band feature disturbance to the adjacent category samples with the largest recognition probability gradient change in the spatial heat distribution map.

[0050] Preferably, the multi-band feature perturbation comprises:

[0051] i. injecting a visible-near infrared sweep test signal by a controllable light source;

[0052] Theoretical feature response calculation: based on the semantic model and the adjacency matrix, the theoretical response feature spectrum is calculated;

[0053] Measured feature response acquisition: record the features of each category after the injection of the test signal to obtain the measured response feature spectrum;

[0054] iii. Abnormal offset determination: calculate the Euclidean distance between the theoretical response feature spectrum and the measured response feature spectrum;

[0055] iv. Compare the Euclidean distance between the measured feature spectrum and the theoretical feature spectrum, calculate the abnormal category offset percentage, and when the abnormal category offset percentage exceeds the second threshold, mark it as a "feature abnormal association category".

[0056] Compared with the prior art, the beneficial effects of the present application are:

[0057] The decoration waste recyclable component automatic identification system based on multi-modal feature fusion significantly improves the comprehensive performance of decoration waste recyclable component identification through the cooperative work of multiple modules. Compared with the traditional manual identification method, the system realizes the comprehensiveness of information acquisition with the help of the multi-modal data acquisition module. When manually identifying, the sorting personnel can only rely on limited perception methods such as vision and touch, and it is difficult to capture multiple features such as appearance texture, material spectrum and spatial form of the waste at the same time. However, the system can record the feature information of the waste from multiple dimensions by synchronously collecting visible light images, near-infrared spectra and three-dimensional point cloud data, and build a multi-source data acquisition framework, which covers the external form and internal properties of the material and creates conditions for subsequent accurate identification.

[0058] In the feature processing link, the cross-modal feature fusion module breaks through the limitations of single modal identification. Existing single technologies such as visible light recognition can only process surface features, near-infrared recognition is limited to the spectral response range of the material, and three-dimensional point cloud recognition cannot involve material information. However, the system can generate a feature association matrix at the component level by comparing the time-domain texture consistency, frequency-domain spectral matching degree and spatial point cloud coincidence degree with the standard recyclable material feature library after deep comparison. This multi-dimensional association analysis can find the internal relationship between different features, such as the correspondence between the texture features and spectral features of a certain plastic, thereby avoiding misjudgment caused by single feature analysis, and significantly improving the comprehensiveness and accuracy of feature matching.

[0059] The recyclable component discrimination module introduces a spatial semantic analysis network to realize spatial presentation of the recognition result. Traditional recognition methods can only output the type and proportion of recyclable components, and cannot determine their specific position in the garbage. The module combines garbage category spatial distribution parameters and target position information to generate a spatial heat distribution map that can clearly show the distribution density and probability of recyclable components and accurately mark the enrichment area. This spatialized recognition result enables the recycling equipment to directly locate the high-value area, avoiding blind sorting and improving the relevance and effectiveness of recycling operations.

[0060] The adaptive recognition strategy optimization module gives the system the ability to dynamically adjust, further improving the recognition performance in complex scenarios. For different regional features reflected in the spatial heat distribution map, the system enables a multi-modal synchronous acquisition mode in high-probability enrichment areas, increases the density and dimensionality of data acquisition, and ensures that the feature information of key areas is fully captured. By fine-tuning feature parameters and observing changes in recognition results, the system can effectively distinguish different categories with similar features, such as different types of metal materials. This flexible parameter configuration method enables the system to adjust the recognition strategy according to the actual situation of the garbage, maintaining stable recognition performance in mixed and unevenly distributed scenarios, and avoiding the problem of insufficient adaptability in the fixed parameter mode.

[0061] In addition, the standard recyclable feature library built by the system provides a unified reference benchmark for the recognition process, solving the problem of mutual incompatibility of recognition results caused by inconsistent feature standards in existing different systems. The fusion analysis of multi-modal features forms a complete feature system covering appearance, material, and spatial form, which is more comprehensive than single feature analysis in reflecting the essential properties of substances, making the recognition result closer to the actual situation. In summary, through multi-dimensional feature fusion, spatial discrimination, and adaptive optimization, the system effectively solves the problems of low efficiency, poor accuracy, and weak adaptability of traditional recognition methods, reduces the dependence on manual work, and reduces resource waste, having significant application value in the field of decoration waste resource processing. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 A working principle diagram of the decoration waste recyclable component automatic recognition system based on multi-modal feature fusion described in the present application;

[0063] Figure 2 A flowchart of the multi-modal data acquisition module;

[0064] Figure 3 A flowchart of the cross-modal feature fusion module;

[0065] Figure 4 A flowchart of the frequency domain spectrum matching degree detection;

[0066] Figure 5 Flowchart of the recyclable component identification module. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the present application.

[0068] Please refer to Figures 1-5 The present application provides an automatic identification system for recyclable components of decoration waste based on multi-modal feature fusion, and the specific implementation steps are as follows:

[0069] The multi-modal data acquisition module constructs a multi-source data acquisition framework based on a decoration waste sample library, synchronously acquires visible light image data, near-infrared spectral data and three-dimensional point cloud data of the target waste, and outputs a multi-modal feature set through the multi-source data acquisition framework. The cross-modal feature fusion module performs multi-dimensional feature comparison between the multi-modal feature set and a standard recyclable feature library, including time domain texture consistency, frequency domain spectral matching degree and spatial point cloud coincidence degree, and generates a component-level feature correlation matrix. The recyclable component identification module inputs the feature correlation matrix into a spatial semantic analysis network, combines waste category spatial distribution parameters and target waste location information, generates a spatial heat distribution map of recyclable component recognition probability, and locates a recyclable component enrichment area. The adaptive recognition strategy optimization module configures recognition parameters according to the spatial heat distribution map, including enabling a multi-modal synchronous acquisition mode for a high-probability enrichment area, and applying feature disturbance tests to adjacent category samples.

[0070] Embodiment 1:

[0071] In the multi-source data synchronous acquisition stage, multi-modal decomposition processing is carried out on the decoration garbage sample library data. Adaptive Gaussian filter is used to process image data. This filtering method can dynamically adjust the filtering parameters according to the texture complexity of different regions in the image, effectively smooth the noise area while retaining the surface texture feature details, and then extract the pixel proportion of surface texture features and edge contour features respectively. Through spectral band screening processing of near-infrared spectrum data, combined with the characteristic absorption characteristics of common decoration garbage recyclable materials (such as wood, plastic, metal, etc.) in the near-infrared region, the wave bands closely related to the material composition are screened from the full spectrum data, and the correlation atlas between near-infrared spectrum data and material composition is established, and the material types corresponding to different spectral curve forms are determined. The three-dimensional point cloud data is processed by using the space-time alignment algorithm. This algorithm analyzes the spatial coordinate relationship of point cloud data under different angles, calculates the conversion matrix of point cloud under different angles, realizes the accurate matching of three-dimensional point cloud data coordinates under different angles, and eliminates the spatial position deviation caused by the difference of collection angle.

[0072] In the multi-modal feature set construction process, the sample library data after decomposition processing is input into the mixed feature extraction network. The mixed feature extraction network includes three core units: the convolution feature extraction unit based on image semantics, which is composed of multiple convolution layers and pooling layers, which extracts features from low to high level from image data through layer-by-layer convolution operation, and finally generates a basic image feature vector, which contains information such as color distribution, texture pattern, shape feature, etc. of the image; the fully connected network embedded with position attention mechanism, which learns the spatial position weight of each point in the point cloud data, corrects the feature deviation caused by point cloud noise (such as holes caused by occlusion in the collection process, redundant points caused by sensor errors), and enhances the stability of point cloud features; the spectral feature enhancer, which can monitor the signal intensity of each wave band in the collected near-infrared spectrum data in real time, dynamically adjust the feature weight according to the contribution of different wave bands to material identification, so that the spectral features strongly related to material composition occupy a higher proportion in the feature set. At the same time, the heterogeneous sensor array deployed on the edge works synchronously, which includes visible light camera, near-infrared spectrometer, three-dimensional laser scanner and other sensors, which cooperatively capture three key parameters: brightness uniformity parameter (reflecting the overall brightness consistency of the image) and contrast fluctuation parameter (reflecting the brightness difference change of different regions in the image) of image data, characteristic wave band reflectivity (light reflection ratio of specific wave band) and half-width distribution (spectral peak width feature) of near-infrared spectrum, density mutation gradient (rate of sudden change of point cloud density) and coordinate distribution entropy value (degree of disorder of point cloud coordinate distribution) of three-dimensional point cloud.

[0073] The multi-modal feature set output link inputs the real-time collected data into the multi-source data collection framework, and obtains the multi-modal feature set after a series of processing. Before output, adaptive denoising processing based on the garbage component type is performed. According to the characteristics that different components of the decoration garbage (such as gypsum, wood, glass, etc.) are affected by environmental light during the collection process, a differentiated denoising strategy is adopted. For example, for the metal garbage with strong reflection, the noise in the highlight area is mainly eliminated; for the dark building garbage, the dark details are enhanced while the noise is suppressed. The correlation features of images, spectra and point clouds are integrated through a spatio-temporal feature fusion algorithm. The algorithm analyzes the collection synchronization in the time dimension and the corresponding relationship in the space dimension of the three kinds of data, and correlates and maps the texture features of the image, the component features of the spectrum and the structure features of the point cloud to form a complementary feature combination. The adaptive denoising processing based on the garbage component type is performed in the following manner:

[0074] Component classification: According to the visible light reflectance and spectral signal intensity counted by the sample library, the garbage components are divided into three categories: high-reflective type (reflectance ≥ 60%, spectral signal ≥ 80 dB, such as metal, glass), high-texture type (reflectance 20%-60%, spectral signal 40-80 dB, such as wood, paperboard), and low-signal type (reflectance < 20%, spectral signal < 40 dB, such as plastic film).

[0075] Differentiated denoising: The high-reflective type adopts 5x5 window bilateral filtering with a spatial domain standard deviation of 1.5 and a gray domain standard deviation of 30, which eliminates the spot noise while retaining the reflection characteristics; the high-texture type adopts non-local mean denoising with a 15x15 search window and a 7x7 similar window, with a denoising intensity coefficient of 0.1 to avoid blurring the texture details; the low-signal type adopts wavelet threshold denoising with a db4 wavelet basis and 3 layers of decomposition, and the soft threshold calculation formula is as follows:

[0076]

[0077] wherein, is the number of pixels, is the noise standard deviation, and the weak spectral signal is recovered.

[0078] The spatio-temporal feature fusion algorithm is specifically implemented as follows: the time dimension fusion takes 0.5 seconds as the time interval, takes the multi-modal features of the continuous 5 time stamps, and assigns the weights according to the Gaussian distribution, and the formula is as follows:

[0079]

[0080] wherein, is the window center time stamp, and the time dimension feature is obtained by weighted fusion. The spatial dimension fusion maps the image pixel coordinates, the spectral collection point coordinates and the point cloud three-dimensional coordinates to the same coordinate system, and the formula for calculating the spatial coincidence degree is as follows:

[0081]

[0082] wherein, is an image feature space, is a spectral feature space, is a point cloud feature space, the coincidence degree is ≥ 80%, when the coincidence degree is 50%-80%, the features are weighted and summed according to the image 0.3, the spectrum 0.4, and the point cloud 0.3, and when the coincidence degree is < 50%, the original features are retained. The spatio-temporal joint fusion performs tensor point multiplication operation on the time and space dimension features to generate a spatio-temporal fusion feature tensor of [space grid number x time window number x feature type], and completes feature integration.

[0083] The final output multi-modal feature set contains the fluctuation range of various typical components, such as the texture feature variation range of different wood samples and the spectral feature fluctuation range of the same plastic in different states, and the feature set will be dynamically adjusted with the update of the decoration waste sample library. When new sample data of a certain material is added to the sample library, the feature set will automatically include the feature range of the material. When the original sample data in the sample library is verified to have deviation, the feature set will be corrected accordingly. Through the above process, the multi-modal data acquisition module can comprehensively and accurately obtain the multi-dimensional features of the decoration waste.

[0084] Example 2:

[0085] In the time-domain texture consistency calculation process, a preset time window is set to slide and compare the multi-modal feature set and the standard feature library. The size of the time window is determined according to the movement speed of the decoration waste in the transmission or processing process and the feature change frequency, to ensure that the window contains enough feature information to reflect the continuity of the texture. The dynamic time warping algorithm is used to align the non-synchronous collected feature sequences. This algorithm calculates the similarity between two sequences to find the optimal matching path, thereby solving the problem of asynchronous feature sequences caused by differences in response speed of the collection equipment or changes in the movement state of the waste. In each time window, the texture similarity is calculated. By comparing the image texture features in the multi-modal feature set with the texture features of the corresponding category in the standard feature library, the similarity of the texture direction, texture spacing, and texture thickness, etc. is calculated, and a time-domain texture vector is generated. This vector contains the texture matching results in different time windows, and can reflect the change law of the texture features of the target waste over time.

[0086] In the frequency domain spectral matching detection, first, the corresponding near-infrared spectral data is extracted from the multi-modal feature set and the standard feature library. Wavelet packet decomposition is used to analyze each group of spectral signals at multiple scales. Wavelet packet decomposition can decompose spectral signals into different frequency bands, achieving fine division of spectral signals and more clearly displaying the characteristics of spectral signals in different frequency ranges. A preset sensitive band interval is set, which covers the characteristic absorption range of typical recyclable materials (such as plastics, metals, wood, glass, etc.). For example, plastics will have absorption peaks due to molecular vibration at certain near-infrared bands, and metals will have strong reflection characteristics at certain bands. After extracting the reflectivity distribution characteristics in the sensitive band interval, the reflectivity density of the multi-modal feature set and the standard feature library in this interval is quantitatively calculated. The reflectivity density reflects the distribution probability of the reflectivity value in the interval. Based on the relative matching degree of the two, the matching index of each band is extracted, such as calculating the overlapping area of the reflectivity density curve of each band and the peak position deviation, etc. The spectral matching results of all bands are summarized and arranged in the order of the bands to form a frequency domain spectral vector. This vector can reflect the matching degree of the target garbage and the standard recyclable materials in the spectral characteristics.

[0087] In the spatial point cloud coincidence degree evaluation, the point cloud coordinate distribution of the multi-modal feature set and the standard feature library is matched based on the structure matching algorithm. The structure matching algorithm uses the cosine similarity algorithm to calculate the cosine value of the mutation point coordinate vector in the two point cloud sets to determine the similarity between the mutation points and realize the position matching of the mutation point coordinates. Mutation points refer to points where the point cloud density changes significantly, usually corresponding to the edges of garbage or the junctions of different materials. In the matching process, the position synchronization error, i.e. the actual coordinate difference between the matched point pairs, is identified. The position synchronization error of the point cloud density mutation points is calculated, and the distribution of the error in the three coordinate axes directions is counted. The KL divergence of the point cloud distance distribution is quantified. The KL divergence is used to measure the difference between two probability distributions. By calculating the KL divergence between the point cloud distance distributions of the multi-modal feature set and the standard feature library, the difference degree of the two in the point cloud spatial distribution is evaluated, and a spatial point cloud vector is generated. This vector contains information such as mutation point matching results, position synchronization error and KL divergence, and can reflect the coincidence degree of the target garbage and the standard recyclable materials in the spatial structure.

[0088] During the generation of the feature correlation matrix, the time domain texture vector, the frequency domain spectrum vector, and the spatial point cloud vector are tensor spliced. Tensor splicing can combine three vectors of different dimensions into a higher-dimensional tensor, retaining the original information of each vector and the correlation between them. Through normalization processing of feature importance weighting, different features are given corresponding weights according to their contribution to recyclable component identification. For example, for plastic identification, the weight of the spectral feature may be higher than that of the texture feature. Normalization processing converts feature values of different dimensions to the same numerical range, eliminating the influence caused by different feature units. Finally, a three-order feature correlation matrix with dimensions of [component number x timestamp x feature type] is output, where the component number corresponds to different recyclable component categories, the timestamp corresponds to different collection time windows, and the feature type covers features in the time domain, frequency domain, and spatial domain. This matrix can comprehensively reflect the correlation between each component of the target garbage and the standard recyclable material in different time and feature dimensions.

[0089] Example 3:

[0090] During spatial semantic modeling, a spatial distribution topology graph of each component category is constructed according to the position information of the target garbage. First, the three-dimensional coordinates of the target garbage in the collection area are obtained through the positioning system, and different spatial units are divided based on this, with each spatial unit corresponding to one or more possible garbage components. In the topology graph, each node represents a garbage category, and the edges between nodes represent the spatial adjacency relationship between categories, with spatial adjacency parameters between categories marked, including the distance between adjacent categories in space, the contact area ratio, etc. At the same time, the prior constraint conditions of the standard distribution of recyclable components are superimposed in the topology graph. These constraint conditions are determined based on the typical distribution rules of common construction waste recyclable components, for example, metal parts are usually adjacent to wood, plastic, etc., and glass fragments are mostly distributed in the surface or edge area of construction waste. Through the above operations, a semantic topology model including an adjacency matrix and a category confidence matrix is generated. The semantic topology model is a directed weighted graph structure, which consists of the following: the node layer has 28 nodes, each node corresponds to a construction waste recyclable component (such as aluminum alloy, PVC plastic, etc.), and the node attributes include category ID, component density (unit g / cm³), feature dimension number, and standard identification confidence (correct identification probability based on sample library statistics); the edge layer represents the spatial adjacency relationship between components, and the weight is the adjacency probability formula as follows:

[0091]

[0092] wherein, is the number of times component appears adjacent to component is the number of times component ​Total occurrence, edge direction from high occurrence frequency component to low occurrence frequency component; constraint layer superimposes prior constraint matrix, including spatial position constraint (such as glass is mostly distributed in the upper layer, weight 0.7) and component proportion constraint (such as metal proportion ≤30%, weight 0.3), and constraint parameters are obtained by statistics of 1000 groups of samples. After model construction, the graph structure verification algorithm is used to check the node attribute integrity and edge weight rationality to ensure effectiveness. The adjacency matrix records the connection strength between different category nodes, and the category confidence matrix reflects the initial probability of the category represented by each node appearing in the current spatial position.

[0093] In the recognition probability deduction process, the feature association matrix is mapped to the corresponding category node of the spatial semantic topology model. The mapping method is based on the correspondence between the component number in the feature association matrix and the category node in the topology model, and the association value of each component in the matrix under different time stamps and feature types is assigned to the corresponding node. Based on the graph neural network, the recognition probability deduction is performed. First, the attenuation factor of feature propagation is calculated according to the category adjacency parameters. The factor decreases with the increase of the distance between adjacent categories and increases with the increase of the contact area ratio, so that the feature is more likely to propagate between categories that are close in space and have close contact. The graph neural network adopts a mixed architecture of two layers of GCN and one layer of GAT, and the specific operation process is as follows:

[0094] The first layer of GCN inputs [28x64] node feature vector and 28x28 adjacency matrix, and updates the feature through the formula as follows:

[0095]

[0096] wherein, , is degree matrix, is [64x128] weight matrix, is [1x128] bias, and the activation function is ReLU, and the output is [28x128] feature matrix.

[0097] The second layer of GCN inputs the output of the first layer, and updates is [128x256] weight matrix, and the formula is as follows:

[0098]

[0099] wherein, is [128x256] weight matrix, is [1x256] bias, and the activation function is LeakyReLU, and the output is [28x256] feature matrix.

[0100] GAT layer: 8 attention mechanisms are adopted to calculate the attention coefficient:

[0101]

[0102] wherein is a [256x64] weight matrix, is a single-layer fully connected attention function, and the output after Softmax normalization is [28x512] final node features, which are used for subsequent probability deduction. The multi-head attention mechanism is used to capture cross-category feature correlation features. This mechanism focuses on different feature dimensions and category relationships through multiple attention heads, and the outputs of multiple attention heads are combined to obtain more comprehensive cross-category correlation information. The Monte Carlo method is used to simulate the diffusion path of recyclable component features in the topology network. During the simulation process, the probability of feature propagation from one node to another node is determined according to the decay factor and cross-category correlation features, and multiple repeated simulations are performed to generate a large number of possible propagation paths. In the identification probability deduction, the calculation of the feature propagation probability is involved, and the formula is:

[0103]

[0104] wherein, represents the probability of feature propagation from category node to category node ; is a decay factor, which is determined by the spatial adjacency parameter between category and category ; is the feature similarity between category and category , which is calculated by the multi-head attention mechanism; is the connection weight between category and category , which is set based on the co-occurrence frequency of the two in historical data.

[0105] The probability distribution generation link is to count the frequency of feature appearance of each category in the simulation propagation, i.e., the number of times each category node is passed by the feature in all simulation paths. The recyclable component identification probability value is calculated in combination with the category adjacency parameter. When converting the frequency to a probability value, the influence of different adjacency parameters on the probability needs to be considered, for example, the node adjacent to the high-confidence category will get a probability bonus. A spatial heat distribution map covering all samples is generated, and the color depth of each spatial unit in the map represents the identification probability value of the recyclable component in the unit. At the same time, a suspicious category set with a probability value exceeding a preset identification confidence probability threshold is labeled, and these categories are potential recyclable components or components that need to be further verified.

[0106] During physical area location, spatial clustering analysis is performed on the spatial heat map. Clustering algorithms group spatially adjacent units with similar identification probabilities into a single region, identifying areas of high probability where recyclable components are likely to be concentrated. Based on the target waste location and spatial semantic connectivity, physical boundaries for recyclable component enrichment are delineated. Boundary delineation references the spatial distribution range of category nodes and the contours of clustered regions in the topology map to ensure accurate enclosure of high-probability recyclable component areas. Simultaneously, suspicious component identifiers and feature propagation main paths are output. Suspicious component identifiers are based on the spatial semantic nodes connected to the suspicious category set, mapping each semantic node to a potentially recyclable component to form an identifier set, thus indicating which components require focused attention. The main feature propagation path is determined by recording the nodes traversed and their order in each round of feature diffusion during the Monte Carlo simulation. The frequency of each path is counted across all simulated paths, and the most frequently occurring path sequence is selected as the main feature propagation path. This path sequence is output as a structured and ordered list of nodes, demonstrating the main propagation direction and the category nodes traversed by feature information in the spatial semantic network. The spatial semantic analysis network adopts a three-layer cascaded architecture, specifically implemented as follows:

[0107] The input layer flattens the dimension of the third-order feature correlation matrix with dimensions [component number × timestamp × feature type], converting it into a two-dimensional matrix of [(component number × timestamp) × feature type]. At the same time, it normalizes the spatial distribution parameters of waste categories (such as the average area occupied by each category and the spatial overlap coefficient) and the three-dimensional coordinates (X,Y,Z) of the target waste, mapping them uniformly to the [0,1] interval to eliminate the difference in dimensions.

[0108] The feature association layer operation calculates the semantic correlation coefficient between each feature dimension and the spatial distribution parameter, as shown in the following formula;

[0109]

[0110] in For the first The first feature dimension and the first feature dimension The correlation coefficient of each spatial parameter For the first The feature in the first Feature values ​​of each sample For the first The sample mean of each feature, For the first The spatial parameter in the th... The parameter values ​​for each sample, For the first The sample mean of each spatial parameter, The sample size is used as the coefficient. A semantic association weight matrix is ​​constructed based on this coefficient. (element ), weighting the flattened feature matrix, and outputting a spatially correlated feature matrix.

[0111] The probability output layer calculates: input the spatially correlated feature matrix into a multi-classification logistic regression model, taking recyclable categories such as metal, plastic, and wood as the prediction target, and outputting the identification probability of each (component number x timestamp); combined with the target three-dimensional coordinates, divide the collection area into a 10cm x 10cm x 5cm spatial grid, and calculate the average probability value of each grid to generate a spatial heat map.

[0112] Example 4:

[0113] When the identification probability value of a certain area in the spatial heat map exceeds the preset identification confidence probability threshold, the system issues a collection mode switching instruction to the collection terminal to which the area belongs, and starts the multi-modal synchronous collection mode. After receiving the instruction, the collection terminal increases the image / spectrum sampling frequency to 5-10 times the original frequency, for example, the original sampling frequency is 30 frames of image data per second, and after increasing, it becomes 150-300 frames per second, so as to increase the density of data collection and obtain more detailed feature change information. The feature waveband real-time tracking mode is enabled synchronously, which focuses on monitoring the feature wavebands closely related to recyclable components in near-infrared spectrum data, and continuously captures the reflectivity changes of these wavebands. Once a sudden increase or decrease in reflectivity is found, the time point and specific value of the mutation are immediately recorded. At the same time, a feature anomaly detector is deployed on the edge, which is specially designed for analyzing three-dimensional point cloud data, and identifies and records the segments where the point cloud density suddenly increases or decreases by calculating the density change rate of the point cloud in real time. These segments often correspond to the boundaries or structural mutations of garbage components.

[0114] When the feature perturbation test is performed, the system first identifies the adjacent class samples with the largest identification probability gradient change in the spatial thermal distribution map, that is, the region where the two samples are adjacent in space and the change rate of the identification probability value is the highest, for example, one side of a certain region is a high-probability plastic identification region, and the adjacent side is a low-probability wood identification region, and the probability difference of the two changes significantly in a short distance. For these adjacent class samples, multi-band feature perturbation is applied, and the specific process is as follows: a controllable light source is used to inject a visible-near-infrared sweep test signal to the target region, the signal covers multiple continuous wavelength ranges, and is swept from the visible light band to the near-infrared band step by step, to ensure that the characteristic response of different material components can be excited. Based on the semantic model and the adjacency matrix, the theoretical response characteristic spectrum is calculated, the semantic model is a signal parameter-component attribute-response characteristic mapping model, and the specific implementation is as follows: input and output parameters: input includes sweep wavelength range (400-1100nm), signal intensity (50-100dB), sweep rate (10-50nm / s), component class ID, and component temperature (20-30℃); and the output is the theoretical response characteristic spectrum (horizontal coordinate wavelength interval 5nm, vertical coordinate reflectivity 0-100%), peak position, and half-width.

[0115] The core calculation steps are as follows: the feature database is constructed based on 10000 groups of signal-response samples, and the measured response spectra of various components under different parameters are stored.

[0116] Similar sample matching: the Euclidean distance formula of the input parameters and the database samples is as follows:

[0117]

[0118] wherein, represents the parameter number, represents the parameter value to be calculated, represents the parameter value of the database sample, and the minimum 5 samples are selected. Segmented linear interpolation: 140 intervals are divided according to the 5nm wavelength interval, and the theoretical reflectivity is calculated by the weight formula in each interval

[0119]

[0120]

[0121] wherein, represents the Euclidean distance between the i-th similar sample and the parameter to be calculated, represents the weight summation number, and the theoretical reflectivity

[0122]

[0123] wherein,​​ Indicates the first The weights of similar samples, Indicates the first Similar samples at wavelength The measured reflectance at that location.

[0124] Temperature correction: Reflectivity is corrected using a formula, as follows:

[0125]

[0126] in, Using the input temperature (25°C) as the standard temperature, the final theoretical response characteristic spectrum is generated. The semantic model provides the typical response patterns of different materials under specific wavebands, while the adjacency matrix reflects the mutual influence between adjacent sample categories. Combining the two, the characteristic spectrum shape that each sample category should exhibit under the action of the test signal can be derived.

[0127] In the measured feature response acquisition stage, the system synchronously records the multimodal feature changes of each category of samples after the test signal injection through a multimodal data acquisition module. This includes changes in image texture, near-infrared spectral reflectance fluctuations, and the structural response of 3D point clouds. These changes are integrated to obtain the measured response feature spectrum. The Euclidean distance between the theoretical and measured response feature spectra is calculated, which measures the degree of difference between the two. The Euclidean distance between the measured and theoretical feature spectra is compared, and the percentage of anomaly category offset is calculated based on the distance value, i.e., the proportion of deviation from the theoretical value. When the percentage of anomaly category offset exceeds a second threshold, the category is marked as a "feature anomaly associated category." These categories may have component misjudgment or feature confusion, requiring further optimization of the identification parameters.

[0128] Throughout the adaptive identification strategy optimization process, the system dynamically adjusts the parameter settings for multimodal data acquisition and the weight allocation for feature fusion based on real-time data collected from high-probability enriched areas and the results of feature perturbation tests. For example, for areas marked as "feature anomaly associated categories," the system increases the sampling frequency and feature extraction dimensions for those areas to obtain richer data for subsequent identification optimization. Simultaneously, by continuously monitoring changes in the spatial heat map, the system constantly updates the range of high-probability enriched areas and the locations of adjacent category samples, ensuring that the adaptive identification strategy is always optimized for the most critical areas and samples, thereby improving the overall system's adaptability and accuracy in identifying recyclable components of construction waste.

[0129] Example 5:

[0130] In the multi-source data synchronous acquisition process of the multi-modal data acquisition module, the application of adaptive Gaussian filtering needs to be dynamically adjusted in combination with the actual characteristics of image data during multi-modal decomposition processing of the decoration waste sample library data. For areas with dense textures in the image, such as wood surface patterns and plastic patterns, the filter parameter is set to a small standard deviation to retain more detailed information. For areas with sparse or smooth textures, such as metal plate surfaces, a larger standard deviation is used to reduce noise interference. Through this differentiated processing, the pixel proportion of the extracted surface texture features and edge contour features can better reflect the true properties of the materials. In the spectral band selection link, the range of selected bands is determined based on the characteristic absorption peak positions of different recyclable materials in the near-infrared spectral region. For example, plastics have strong absorption peaks due to molecular vibration in specific near-infrared bands, and metals exhibit high reflectivity in certain bands. Based on these characteristics, the selected bands can more accurately establish the correlation atlas between near-infrared spectral data and material composition. In the spatial-temporal alignment algorithm, a dynamic weight factor is introduced when matching the coordinates of three-dimensional point cloud data under different viewing angles. For areas with high point cloud density and distinct features, such as the corners and protruding parts of the garbage, a higher matching weight is assigned. For areas with sparse point clouds and ambiguous features, such as the shadow parts of the garbage, the weight is reduced. This improves the accuracy of coordinate matching under different viewing angles.

[0131] During the operation of the hybrid feature extraction network, the image semantics-based convolution feature extraction unit uses a multi-layer convolution structure, and the size and number of convolution kernels in each layer are adjusted according to the feature maps output by the previous layer. The bottom convolution layer uses a small convolution kernel to extract low-level features such as edges and colors of the image. The high-level convolution layer uses a larger convolution kernel to integrate low-level features to form high-level semantic features such as texture and shape, and the generated basic image feature vector contains multi-dimensional image information. The fully connected network with embedded position attention mechanism analyzes the spatial coordinate relationships of points in the point cloud data to calculate the influence weight of each point on surrounding points. For isolated points or abnormal points that may be caused by noise, the weight in feature extraction is reduced to correct the feature bias. Based on the spectral feature enhancer, the weight is dynamically adjusted according to the signal intensity of the characteristic band in the real-time acquired near-infrared spectral data. When the signal intensity of a certain band is high and stable, such as the characteristic absorption peak band of plastic, the weight of this band in the feature set is increased. For bands with weak signals and large fluctuations, the weight is reduced, so that the feature set focuses more on the spectral information that is more critical for material identification.

[0132] The edge side deployed heterogeneous sensor array sets multiple groups of acquisition parameter combinations for different environment light conditions when synchronously capturing related parameters. In strong light environment, the exposure time and gain of the image sensor are reduced to avoid image overexposure; in weak light environment, the exposure time is increased and the light supplement device is turned on to ensure the brightness uniformity of the image. Through this adaptive adjustment, the captured brightness uniformity and contrast fluctuation parameters can more truly reflect the quality of the image. In the capture process of the characteristic band reflectivity and half-width distribution of near-infrared spectrum, a high-precision spectrometer is used for data acquisition, and multiple measurements are performed in each characteristic band range. The average value is calculated to reduce the influence of random error on the calculation of reflectivity and half-width. The calculation of the density mutation gradient and coordinate distribution entropy value of the three-dimensional point cloud sets a reasonable calculation window combined with the acquisition accuracy and resolution of the point cloud data. For high-density point cloud areas, a smaller calculation window is used to capture subtle density changes; for low-density areas, the window range is expanded to ensure that the density mutation gradient and coordinate distribution entropy value can accurately reflect the spatial distribution characteristics of the point cloud.

[0133] With the continuous updating of the decoration garbage sample library, the multi-modal feature set will also be adjusted accordingly, new material features will be included, and the bias of the original features will be corrected, so that the entire data acquisition process can continuously adapt to different types and states of decoration garbage.

[0134] It should be noted that, in the present document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0135] While embodiments of the present application have been shown and described with reference to particular embodiments thereof, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the application as defined by the appended claims and their equivalents.

Claims

1. A multi-modal feature fusion-based automatic identification system for recyclable components of decoration waste, characterized in that, Comprise: A multi-modal data acquisition module: based on the construction of a multi-source data acquisition framework of a decoration garbage sample library, visible light image data, near-infrared spectral data and three-dimensional point cloud data of the target garbage are synchronously acquired, and a multi-modal feature set is output through the multi-source data acquisition framework; A cross-modal feature fusion module: multi-dimensional feature comparison is performed between the multi-modal feature set and a standard recyclable material feature library, the multi-dimensional feature comparison includes time domain texture consistency, frequency domain spectral matching degree and spatial point cloud coincidence degree, and a component-level feature correlation matrix is generated; A recyclable component discrimination module: the feature correlation matrix is input into a spatial semantic analysis network, garbage category spatial distribution parameters and target garbage position information are combined, a spatial thermal distribution map of recyclable component recognition probability is generated, and a recyclable component enrichment area is located; An adaptive recognition strategy optimization module: according to the spatial thermal distribution map, recognition parameters are configured, including: Enabling a multi-modal synchronous acquisition mode for a high-probability enrichment area; Applying feature disturbance tests to adjacent category samples; The cross-modal feature fusion module specifically comprises: Time domain texture consistency calculation: a preset time window is used to slide and compare the multi-modal feature set and the standard feature library, a dynamic time warping algorithm is used to align the non-synchronous collected feature sequences, the texture similarity in each window is calculated, and a time domain texture vector is generated; Frequency domain spectral matching degree detection: the near-infrared spectral data of the multi-modal feature set and the standard feature library are decomposed by a frequency domain decomposition algorithm, the spectral reflectance ratio is calculated, the spectral matching index of each waveband is extracted, and a frequency domain spectral vector is constructed; Spatial point cloud coincidence degree evaluation: based on a structure matching algorithm, the point cloud coordinate distribution of the multi-modal feature set and the standard feature library is matched, the position synchronization error of the point cloud density mutation point is calculated, the KL divergence of the point cloud distance distribution is quantified, and a spatial point cloud vector is generated; Feature correlation matrix generation: the time domain texture vector, the frequency domain spectral vector and the spatial point cloud vector are spliced to form a three-order feature correlation matrix with a dimension of [component number × time stamp × feature type] by normalization processing with feature importance weighting to eliminate dimension differences; The recyclable component discrimination module specifically comprises: Spatial semantic modeling: a category spatial distribution topology graph is constructed according to the target garbage position information, spatial adjacency parameters between categories are labeled, prior constraint conditions of recyclable component standard distribution are superimposed in the topology graph, and a semantic topology model including an adjacency matrix and a category confidence matrix is generated; Recognition probability deduction: the feature correlation matrix is mapped to the corresponding category node of the spatial semantic topology model; based on a graph neural network, recognition probability deduction is performed, and the calculation of the recognition probability deduction includes: i. Calculate the attenuation factor of feature propagation according to the category adjacency parameters; ii. Capture cross-category feature correlation features through a multi-head attention mechanism; iii. Use the Monte Carlo method to simulate the diffusion path of recyclable component features in the topology network; Probability distribution generation: count the occurrence frequency of each category in the simulation propagation, calculate the recyclable component identification probability value based on the category adjacency parameter, generate a spatial heat distribution map covering the entire sample, and label a suspicious category set with a probability value exceeding a preset identification confidence probability threshold; Physical region positioning: performing spatial clustering analysis on the spatial heat distribution map to identify an identification probability aggregation area; and according to the target garbage location and the spatial semantic connection relationship, the recyclable component rich physical boundary is drawn; The recyclable component identification module further includes outputting a suspicious component identifier and a feature propagation main path, wherein: The suspicious component identifier is based on the spatial semantic nodes connected by the suspicious category set, and the spatial semantic nodes are bound with the actual recyclable components to form a suspicious component identifier set, indicating potential recyclable components or interfering components; The feature propagation main path records the node path and its order experienced in each round of propagation in the feature diffusion process of Monte Carlo simulation, and in all simulation paths, the occurrence frequency of each path is counted, and the path sequence with the highest cumulative occurrence frequency is selected as the feature propagation main path. The output feature propagation main path sequence is a structured and ordered node list, reflecting the main propagation trajectory of the feature information in the spatial semantics.

2. The automatic identification system for recyclable components of decoration garbage based on multi-modal feature fusion according to claim 1, characterized in that, The multi-modal data acquisition module specifically includes: Multi-source data synchronous acquisition: multi-modal decomposition processing is performed on the decoration garbage sample library data, including: using adaptive Gaussian filtering to extract the pixel proportion of surface texture features and edge contour features of image data, establishing an association atlas of near-infrared spectral data and material components through spectral band screening, and matching three-dimensional point cloud data coordinates under different perspectives using a space-time alignment algorithm; Multi-modal feature set construction: inputting the processed sample library data into a mixed feature extraction network, the mixed feature extraction network includes: Convolution feature extraction unit based on image semantics, used to generate basic image feature vectors, fully connected network embedded with position attention mechanism, used to correct feature deviation caused by point cloud noise, and spectral feature enhancer, used to dynamically adjust feature weights according to real-time collected near-infrared spectral data; The heterogeneous sensor array deployed on the edge synchronously captures the brightness uniformity and contrast fluctuation parameters of image data, the feature band reflectivity and half-width distribution of near-infrared spectrum, and the density mutation gradient and coordinate distribution entropy value of three-dimensional point cloud; Multi-modal feature set output: inputting real-time acquisition data into the multi-source data acquisition framework to obtain a multi-modal feature set.

3. The automatic identification system for recyclable components of decoration garbage based on multi-modal feature fusion according to claim 2, characterized in that, In the multi-modal feature set output, the following operations are performed: Adaptive denoising processing based on garbage component type, to eliminate acquisition noise caused by environmental light; Integrating the associated features of images, spectra and point clouds through a space-time feature fusion algorithm; Outputting a multi-modal feature set including a typical component fluctuation interval, and dynamically adjusting the multi-modal feature set according to the sample library update state.

4. The automatic identification system for recyclable components of decoration garbage based on multi-modal feature fusion according to claim 1, characterized in that, The frequency domain spectral matching degree detection specifically includes: In the frequency domain analysis stage, first, the multi-modal feature set and the corresponding near-infrared spectrum data in the standard feature library are extracted, and wavelet packet decomposition is used to analyze each group of spectrum signals in multiple scales and frequency bands, and the reflectivity distribution characteristics in the preset sensitive band interval are extracted, the sensitive band interval is selected to cover the characteristic absorption range of typical recyclable materials, after the extraction, the reflectivity density of the multi-modal feature set and the standard feature library in the sensitive band interval is quantitatively calculated, and the matching index of each band is extracted based on the relative matching degree of the two, the spectrum matching results of all bands are summarized, and the frequency domain spectrum vector is constructed.

5. The automatic identification system for recyclable components of decoration garbage based on multi-modal feature fusion according to claim 1, characterized in that, In the spatial point cloud coincidence degree evaluation, the structure matching algorithm adopts a cosine similarity algorithm to match the coordinates of the mutation points in the two point cloud sets and identify the position synchronization error.

6. The automatic identification system for recyclable components of decoration garbage based on multi-modal feature fusion according to claim 1, characterized in that, The adaptive recognition strategy optimization module specifically includes: When the recognition probability value of a region in the spatial thermal distribution map exceeds the preset recognition confidence probability threshold, a collection mode switching instruction is issued to the collection terminal to which the region belongs, and the following is executed: i. The image / spectrum sampling frequency is increased to 5-10 times the original frequency; ii. Synchronously enable the feature band real-time tracking mode to capture the reflectivity mutation of the feature band; iii. Deploy a feature anomaly detector on the edge to record the point cloud density sudden change segment; Feature disturbance test execution: apply multi-band feature disturbance to the adjacent class samples with the largest recognition probability gradient change in the spatial thermal distribution map.

7. The automatic identification system for recyclable components of decoration garbage based on multi-modal feature fusion according to claim 6, characterized in that, The multi-band feature disturbance includes: i. Inject a visible-near-infrared sweep frequency test signal through a controllable light source; Theoretical feature response calculation: calculate the theoretical response feature spectrum based on the semantic model and the adjacency matrix; Measured feature response acquisition: record the features of each class after the test signal injection to obtain the measured response feature spectrum; iii. Abnormal offset amount determination: calculate the Euclidean distance between the theoretical response feature spectrum and the measured response feature spectrum; iv. Compare the Euclidean distances of the measured feature spectrum and the theoretical feature spectrum to calculate the abnormal class offset percentage, and when the abnormal class offset percentage exceeds the second threshold, mark it as a "feature anomaly associated class".

Images