A method and system for intelligent classification of household garbage based on the Internet of Things

Abstract

The application discloses a kind of intelligent classification method and system of domestic waste based on Internet of Things, is specifically relevant to domestic waste processing technical field, by setting up high-speed camera acquisition device below intelligent garbage can delivery mouth, short video stream is collected before, middle and after the impact baffle of delivery object;Local region of interest is extracted after image pre-processing, obtains visual elasticity feature, optical discrimination feature and edge texture feature, constructs multimodal visual feature representation, combines pre-training identification model and visual atlas dictionary and outputs garbage class and fragility grade, and generates flexible damping control parameter control buffer baffle action.The application utilizes dynamic visual information in the process of impact to identify, compared with the scheme only relying on static appearance, can more effectively distinguish garbage categories such as transparent glass, transparent plastic and other similar appearance;Improve garbage classification accuracy, reduce the breakage risk of fragile garbage in the process of shunting in the box.

Description

Technical Field

[0001] This invention relates to the field of municipal solid waste treatment technology, and in particular to a smart classification method and system for municipal solid waste based on the Internet of Things. Background Technology

[0002] With the promotion and application of household waste sorting, smart trash cans, smart recycling terminals, and other equipment are gradually being used in residential communities, office areas, public places, and commercial settings. Existing smart waste sorting equipment typically uses methods such as static image recognition, weight detection, material sensing, or QR code recognition to classify waste.

[0003] However, existing technologies have at least the following problems: First, some items being disposed of have a high degree of similarity in static visual appearance. For example, transparent glass bottles, transparent plastic bottles, and some transparent packaging containers are similar in color, outline, and overall appearance, making it difficult to accurately distinguish them based on static images alone, thus affecting the accuracy of waste category identification.

[0004] Secondly, existing solutions typically focus only on the waste category itself, lacking the identification of the fragile characteristics of the materials being disposed of. For fragile waste such as glass products and ceramic shards, without targeted buffering control during disposal, sorting, and transfer, secondary collisions and breakage are likely to occur, which not only affects the sorting effect but also leads to problems such as equipment pollution, odor diffusion, and difficulties in subsequent recycling and processing.

[0005] Third, the existing internal actuators of garbage bins mostly adopt fixed action modes, such as fixed rotation speed, fixed angle, or fixed buffer duration of the baffle action mode. They cannot be adaptively adjusted according to the actual fragility and quality differences of the deposited materials, making it difficult to balance the requirements of identification accuracy and buffer protection.

[0006] Therefore, there is an urgent need for a smart waste sorting method and system that can combine dynamic visual information of the object during the impact process to identify the type and fragility level of the waste, and generate flexible damping control parameters based on the identification results, so as to improve the sorting accuracy, reduce the risk of breakage of fragile waste during the diversion process, and improve the overall reliability and adaptability of smart waste sorting equipment. Summary of the Invention

[0007] The main objective of this invention is to provide an intelligent sorting method and system for household waste based on the Internet of Things, in order to solve the problems in the prior art that it is difficult to effectively distinguish between similar-looking but different materials and brittleness, and that it is difficult to achieve adaptive buffer control based on the characteristics of the materials.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A smart waste sorting method based on the Internet of Things includes the following steps: Step S1: Install a high-speed camera acquisition device near the impact baffle below the smart trash can's delivery port. When the device detects that the object is in contact with the impact baffle, it triggers the high-speed camera acquisition device to continuously acquire images of the object's impact process, resulting in a short video stream containing the object before, during, and after the impact. Step S2: Perform image preprocessing on the short video stream, and extract the motion pixel region of the delivery object during the impact phase based on background subtraction and connected component analysis, so as to determine and track the local region of interest of the delivery object in multiple consecutive frames of images. Step S3: For the continuous frame image sequence corresponding to the local region of interest, extract the temporal change features of the outer contour boundary of the delivery object, and extract the visual elastic features that characterize the deformation response of the delivery object based on the contour changes before and after the impact. Step S4: For the local region of interest, extract the reflective optical features of the surface of the delivery object and the light transmission features of the backlit area to obtain optical discrimination features characterizing the optical response of the delivery object surface; Step S5: Fuse the visual elasticity feature, the optical discrimination feature, and the edge texture feature of the delivery object to construct a multimodal visual feature representation of the delivery object; Step S6: Input the multimodal visual feature representation into the pre-trained recognition model for deep feature extraction to obtain a feature vector with deep representation, and perform dimensionality reduction and / or noise reduction on the feature vector; Step S7: Match the processed features with the preset fragile waste visual atlas dictionary, and output the waste category label and corresponding fragility level parameters of the delivered items; Step S8: Based on the brittleness level parameter and the mass coefficient estimated from the area of ​​the object image, generate flexible damping control parameters through a preset visual control mapping function, and output the flexible damping control parameters to the bottom controller of the trash can to control the end diversion buffer baffle in the trash can to adaptively intercept and divert the object.

[0009] Preferably, the high-speed camera acquisition device in step S1 is a global shutter high-speed camera, the impact baffle is an inclined impact baffle, and the high-speed camera is located to the side or below the impact baffle to acquire transient collision images when the delivered object comes into contact with the impact baffle; triggering the high-speed camera acquisition device to continuously acquire images of the delivery object's impact process includes using an impact sensing signal, an image frame difference abrupt change signal, or a combination of both as triggering conditions to start the high-speed camera acquisition device to continuously acquire images.

[0010] Preferably, the image preprocessing in step S2 includes at least one or more of grayscale processing, background modeling, noise filtering, and binarization segmentation; the step of determining and tracking the local region of interest of the delivery object in multiple consecutive frames of images includes: extracting the foreground motion region based on the background subtraction result; filtering the target region based on the area of ​​the connected component, the bounding rectangle, the centroid position, and the shape constraint; and establishing the target association relationship based on the positional continuity and area continuity of the target region in adjacent frames to form a sequence of local regions of interest of the delivery object.

[0011] Preferably, the visual elasticity feature in step S3 includes at least one or more of the following parameters: the temporal change rate of the pixel coordinates of the outer contour boundary of the delivery object, the contour compression ratio at the moment of impact, the contour recovery rate after impact, and the peak value of the contour deformation; wherein, the contour compression ratio is determined by the change in contour size before and after the delivery object impact, the contour recovery rate is determined by the degree of contour rebound after the delivery object impact, and the visual elasticity feature is used to characterize the visual deformation response characteristics of the delivery object during the collision process.

[0012] Preferably, the optical discrimination features in step S4 include at least one or more of the following parameters: the area ratio of the specular reflection highlight area on the surface of the delivery object, the brightness distribution, the change in boundary curvature, the morphological distortion rate, and the grayscale transmission distribution or shadow transmittance of the backlit area of ​​the delivery object; the optical discrimination features are used to distinguish between transparent glass, transparent plastic and other delivery object categories with similar static visual appearance.

[0013] Preferably, the pre-trained recognition model in step S6 is a residual network model; the dimensionality reduction and / or denoising processing is implemented using at least one of principal component analysis, feature selection, autoencoder compression, or subspace mapping; the matching in step S7 involves performing distance metric comparison and / or classification probability comparison between the processed features and the standard feature templates in the preset fragile waste visual atlas dictionary to determine the waste category label and the fragility level parameter.

[0014] Preferably, the flexible damping control parameters in step S8 include at least one or more of the following: driving torque parameters, buffer duration parameters, and baffle action level parameters; the visual control mapping function generates a corresponding flexible damping control parameter matrix based on the brittleness level parameters and the mass coefficient, which is used to control the action state of the end diversion buffer baffle so that the end diversion buffer baffle can adaptively intercept and divert the delivered material.

[0015] A waste identification and flexible damping control system for the above method, characterized in that it comprises: A high-speed camera acquisition module is used to capture a short video stream containing the impact process when the delivered object comes into contact with the impact barrier; The image preprocessing module is used to perform background subtraction, connected component analysis, and local region of interest extraction on the short video stream; The visual feature extraction module is used to extract the visual elasticity features, optical discrimination features, and edge texture features of the delivered object. The feature fusion and recognition module is used to construct multimodal visual feature representations and use pre-trained recognition models to complete deep feature extraction, dimensionality reduction and / or noise reduction, as well as matching and recognition with a preset fragile waste visual atlas dictionary; The control parameter generation module is used to generate flexible damping control parameters based on the identified waste category label, brittleness level parameter, and mass coefficient. The control output module is used to output the flexible damping control parameters to the bottom controller of the garbage bin, so as to control the end diversion buffer baffle to adaptively intercept and divert the delivered materials.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention overcomes the limitations of traditional waste sorting schemes that primarily rely on static appearance information by acquiring continuous images during the contact between the waste and the impact barrier, and extracting the waste's visual elasticity features, optical discrimination features, and edge texture features. For waste such as transparent glass, transparent plastic, and other waste with similar static visual appearances, this invention can comprehensively distinguish them by combining their contour changes, reflectivity, and light transmission characteristics during the impact process, thus improving the accuracy, stability, and robustness of waste category identification.

[0017] 2. This invention, while identifying the type of waste being disposed of, further outputs a fragility level parameter. Combined with a mass coefficient estimated from the area of ​​the waste image, a flexible damping control parameter is generated through a visual control mapping function. This parameter is used to control the end-of-pipe diversion buffer baffle within the waste bin to adaptively intercept and redirect the waste. Therefore, this invention not only achieves intelligent waste sorting but also provides differentiated buffer protection for fragile waste, reducing the risk of breakage of items such as glass products during diversion and descent within the bin, thus improving the safety and practicality of the equipment. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall structure of the present invention. Detailed Implementation

[0019] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the linguistic context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0020] It should be understood that the terms "system," "device," "unit," and / or "module" as used in this specification are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0021] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0022] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0023] The following describes in detail, with reference to the accompanying drawings, the IoT-based intelligent waste sorting method and system provided in the embodiments of this specification.

[0024] Example 1

[0025] This embodiment provides a waste identification and flexible damping control system for implementing the above-described method. The system can be installed inside a smart waste bin and mainly includes a high-speed camera acquisition module, an image preprocessing module, a visual feature extraction module, a feature fusion and recognition module, a control parameter generation module, and a control output module.

[0026] The high-speed camera acquisition module is positioned near the impact baffle below the waste bin's disposal opening to capture a short video stream containing the impact process when the waste comes into contact with the baffle. An image preprocessing module, connected to the high-speed camera acquisition module, performs background suppression, target separation, and local region of interest (ROI) extraction on the short video stream. A visual feature extraction module extracts the visual elasticity features, optical discrimination features, and edge texture features of the waste from continuous images corresponding to the ROI. A feature fusion and recognition module fuses these multiple features and uses a pre-trained recognition model to perform deep feature extraction and matching recognition, outputting the waste category label and fragility level parameters of the waste. A control parameter generation module generates flexible damping control parameters based on the waste category label, fragility level parameters, and mass coefficient. The control output module then sends these flexible damping control parameters to the waste bin's bottom-level controller to control the end-of-pipe diversion buffer baffle to perform corresponding interception, diversion, and buffering actions.

[0027] In this embodiment, the impact baffle is preferably configured as an inclined structure. This serves two purposes: firstly, it ensures a relatively stable collision process when the object enters the trash can, facilitating the high-speed camera acquisition module to capture continuous images including those before, during, and after the impact; secondly, it helps to standardize the object's motion posture, reducing recognition errors caused by random flipping. The high-speed camera acquisition module preferably employs a global shutter high-speed camera and is positioned to the side or below the impact baffle to ensure that the camera's field of view covers the main motion areas of the object approaching the baffle, undergoing collision deformation, and continuing to fall after the collision. In practical applications, the camera's sampling frame rate, exposure time, and resolution can be set according to equipment cost, bin space, and recognition accuracy requirements, as long as the contour changes and optical response of the object during the impact process are clearly recorded.

[0028] To improve the accuracy of data acquisition timing, in this embodiment, the high-speed camera acquisition module can be triggered by impact sensing, image frame difference, or a combination of both. Specifically, an impact sensor can be installed on the impact baffle. When the mechanical vibration or impact signal generated by the contact between the delivered object and the impact baffle exceeds a preset threshold, the high-speed camera is triggered to start continuous acquisition. Alternatively, inter-frame difference detection can be performed on the preview video stream, and acquisition can be triggered when the frame difference abrupt change reaches a preset condition. In the preferred embodiment, a combined triggering method using impact sensing signals and frame difference abrupt change signals is adopted to reduce the probability of false triggering and improve the effectiveness of image acquisition.

[0029] Example 2

[0030] This embodiment provides a method for intelligent sorting of household waste using the above-described system. See also... Figure 1 The method mainly includes the following steps.

[0031] 1. Image acquisition during the impact process When a user deposits an item into the smart trash can, the item falls along the disposal path and contacts the impact barrier as it passes below the disposal opening. When a trigger condition is detected, the high-speed camera acquisition module continuously captures images of the impact process, forming a short video stream. This short video stream includes at least several frames before the item contacts the barrier, continuous images during the contact process, and several frames after the contact, to fully preserve the dynamic changes of the item during the collision.

[0032] Compared with traditional methods that only collect static images, this invention can obtain dynamic information such as contour compression, rebound, reflection changes, and light transmission changes of the delivered object under stress by collecting short video streams of the impact process, providing richer discrimination criteria for subsequent identification.

[0033] 2. Image preprocessing and determination of local regions of interest After the short video stream is captured, the image preprocessing module preprocesses each frame. The preprocessing process may include grayscale conversion, background modeling, noise filtering, and binarization segmentation. For the interior environment of a trash can with a relatively stable background, a background model can be built first without any trash, and then the foreground moving region can be extracted using background subtraction. Scattered noise or local interference in the captured images can be suppressed using median filtering and morphological opening / closing operations to obtain a more continuous and clear target area.

[0034] After obtaining the foreground region, the target region corresponding to the delivered object is further determined through connected component analysis. Specifically, candidate regions can be screened based on the area of ​​the connected components, the size of the circumscribed rectangle, the position of the centroid, and shape features, eliminating false targets caused by light spots, shadows, or background disturbances. For target regions in multiple consecutive frames, the positional continuity, area continuity, and overlap relationship of regions in adjacent frames can be combined to establish target associations, thereby forming a sequence of local regions of interest for the delivered object in multiple consecutive frames.

[0035] It should be understood that the local region of interest can be represented by a bounded rectangle, a contour region, or a pixel mask. The purpose is to focus subsequent feature extraction processing on the area where the delivery object is located, so as to reduce background interference and improve computational efficiency.

[0036] 3. Visual elasticity feature extraction After obtaining the sequence of local regions of interest, the visual feature extraction module first extracts the visual elastic features of the delivery object during the impact process. Specifically, the outer contour boundary of the delivery object can be obtained through edge detection and contour tracking, and a pre-impact reference frame, an impact peak frame, and a post-impact recovery frame are selected from consecutive frames. The pre-impact reference frame refers to the frame image when the delivery object is about to contact the impact barrier but has not yet undergone significant deformation; the impact peak frame refers to the frame image where the contour change of the delivery object is most significant or the degree of compression is greatest; and the post-impact recovery frame refers to the frame image when the contour of the delivery object tends to stabilize after the impact.

[0037] Based on this, visual elasticity features including, but not limited to, the following can be extracted: the temporal rate of change of pixel coordinates of the outer contour boundary of the delivery object, the contour compression ratio at the moment of impact, the contour recovery rate after impact, and the peak value of contour deformation. The contour compression ratio can be determined based on the contour size change between the reference frame before impact and the impact peak frame; the contour recovery rate can be determined based on the degree of contour size recovery between the recovered frame after impact and the reference frame before impact; and the peak value of contour deformation can be determined based on the maximum change in contour area, width, height, or perimeter in consecutive frames.

[0038] Because different materials exhibit varying deformation responses upon impact, the aforementioned visual elasticity characteristics can effectively reflect the brittleness of the object. For example, glass and ceramic products typically show relatively small overall compression and weak contour recovery upon impact, while plastic containers often exhibit more pronounced localized deformation and rebound after impact. Therefore, these visual elasticity characteristics can serve as an important basis for distinguishing between highly brittle and less brittle objects.

[0039] 4. Optical discrimination features and edge texture feature extraction In addition to visual elasticity features, this embodiment further extracts optical discrimination features and edge texture features of the delivery object from the local region of interest. Specifically, for the bright areas on the surface of the delivery object, its specular reflection highlight areas can be identified, and the area ratio, brightness distribution, boundary curvature changes, and morphological distortion degree of the highlight areas can be analyzed. For transparent or translucent delivery objects, the grayscale transmission distribution and shadow transmission under backlight conditions can be analyzed to obtain characteristic parameters characterizing its light transmittance.

[0040] In practical identification, although transparent glass and transparent plastic may appear similar in static appearance, they typically differ in highlight morphology, edge reflection characteristics, and light transmission distribution. Glass-based deliverables generally exhibit more concentrated highlight boundaries and relatively stable light transmission characteristics, while plastic deliverables may show more pronounced boundary distortion and uneven light transmission distribution. Therefore, extracting reflective optical features and light transmission features can help improve the ability to distinguish between similar-looking categories.

[0041] Furthermore, edge texture features of the delivered object can be extracted. These edge texture features can be obtained through local binary mode, gray-level co-occurrence matrix, edge density statistics, or other commonly used texture description methods, and are used to characterize the surface texture details and edge distribution patterns of the delivered object. These features can, to some extent, supplement the deficiencies of dynamic deformation features and optical features, thereby improving the overall recognition stability.

[0042] 5. Multimodal visual feature fusion and deep feature extraction After extracting visual elasticity features, optical discrimination features, and edge texture features, these features are fused to construct a multimodal visual feature representation of the delivered object. The fusion method can be implemented using feature vector concatenation, weighted concatenation, or other commonly used fusion methods in the field, depending on specific needs. Before fusion, various features can be normalized to reduce the impact of different units and numerical ranges on subsequent recognition results.

[0043] Subsequently, the multimodal visual feature representation is input into a pre-trained recognition model for deep feature extraction. In this embodiment, a residual network model is preferably used as the pre-trained recognition model, but it is not limited to this; other deep learning models with good feature representation capabilities can also be used as needed. Through the pre-trained recognition model, the deep representation features of the delivery object in the multimodal feature space can be further extracted, thereby improving the ability to distinguish complex targets and boundary samples.

[0044] After obtaining the deep representation features, dimensionality reduction and / or denoising can be performed as needed. For example, principal component analysis, feature selection, autoencoder compression, or subspace mapping can be used to compress high-dimensional features to reduce redundant information and improve subsequent matching efficiency and recognition stability.

[0045] 6. Dictionary matching and recognition, and fragility level output. The features processed as described above will be input into the matching and recognition stage. To achieve stable determination of the type and fragility level of the waste, this invention pre-establishes a visual atlas dictionary for fragile waste. This visual atlas dictionary can be understood as a set of standard feature templates that store feature representations corresponding to different waste types and fragility levels. This dictionary can be obtained through statistical analysis, clustering, or template extraction of the multimodal visual features and deep representation features of training samples.

[0046] During identification, the processed features of the delivery object to be identified are matched against standard feature templates in the visual atlas dictionary. Matching can be achieved using distance metrics and / or classification probability comparison. For example, the similarity between the current sample and each category template can be evaluated based on Euclidean distance, cosine similarity, Mahalanobis distance, or classifier output probability, with the category having the smallest distance, the largest similarity, or the highest classification probability being used as the final identification result.

[0047] While outputting the waste category label, the system further outputs the fragility level parameter corresponding to the waste. The fragility level parameter can be represented in a graded manner, such as divided into low fragility, medium fragility, and high fragility, or it can be represented as a continuous parameter. This invention does not limit the specific form of the fragility level parameter, as long as it reflects the relative level of the waste's breakage risk.

[0048] 7. Estimation of mass coefficient and generation of flexible damping control parameters After identifying the waste category and fragility level, the control parameter generation module generates flexible damping control parameters based on the fragility level parameters and the quality coefficient estimated from the image area of ​​the waste. The quality coefficient can be estimated from the image area of ​​the waste in a local region of interest and can be corrected by combining category empirical coefficients to reflect the quality differences between different categories of waste. For example, under the same image area conditions, the quality coefficient of glass waste is usually higher than that of plastic waste.

[0049] The flexible damping control parameters may include at least one or more of the following: driving torque parameters, buffer duration parameters, and baffle action level parameters. The control parameters can be generated through a preset visual control mapping function. This mapping function can be implemented using a lookup table, rule table, piecewise function, or explicit mathematical function; its essence lies in establishing the correspondence between "brittleness level—quality coefficient—control action."

[0050] For example, when the identification result indicates that the delivery item is of a high brittleness level and has a large mass coefficient, the system can generate control parameters with a lower impact velocity, a longer buffer time, and a higher level of flexible interception, so that the end diversion buffer baffle can minimize secondary impacts when receiving the delivery item; when the delivery item is of low brittleness, a relatively conventional diversion action strategy can be adopted to improve the equipment processing efficiency.

[0051] 8. Control output and execution process After the control parameters are generated, the control output module sends the flexible damping control parameters to the bottom controller of the waste bin. The bottom controller controls the operation of the end-diversion buffer baffle according to the parameters, including but not limited to the baffle's opening angle, rotation speed, dwell time, and return method. The end-diversion buffer baffle can be driven by a servo motor, stepper motor, electromagnetic actuator, or other suitable actuators.

[0052] Preferably, the underlying controller receives control parameters and sends out actions within a predetermined time window after the identification result is generated, so as to ensure that the end diversion buffer baffle completes its preparation action before the delivered item reaches the target diversion position. For highly brittle delivered items, the baffle preferably adopts a slow-speed, low-impact, and flow-guiding action mode; for low-brittle delivered items, a more conventional fast diversion method can be used.

[0053] Example 3

[0054] The working process of the present invention will be further explained below with reference to a specific application scenario.

[0055] In this embodiment, the intelligent trash can is equipped with a disposal port, a tilting impact baffle, a high-speed camera acquisition module, an image preprocessing module, a visual feature extraction module, a feature fusion and recognition module, a control parameter generation module, a control output module, and an end-diversion buffer baffle. The high-speed camera acquisition module is located to the side and below the impact baffle, and its field of view covers the main motion area of ​​the disposed object from its approach to the impact baffle to its continued descent after impact. An impact sensor is installed on the impact baffle, and a bottom-level controller connected to the end-diversion buffer baffle is also installed inside the trash can.

[0056] When a user deposits a transparent glass bottle into the trash can through the disposal slot, the bottle falls along the disposal path and comes into contact with an inclined impact baffle. At the instant the bottle contacts the baffle, the impact sensor outputs an impact trigger signal, and simultaneously, the high-speed camera acquisition module detects a significant change in image frame difference. Based on this, it initiates continuous image acquisition of the glass bottle's impact process, obtaining a short video stream containing images at multiple moments before, during, and after the impact. This short video stream is then sent to the image preprocessing module.

[0057] The image preprocessing module first performs grayscale conversion and background subtraction on the short video stream to separate the foreground motion region corresponding to the glass bottle from the background. Then, it performs denoising and connected component analysis on the foreground region to remove irrelevant regions formed by reflections, shadows, or local background disturbances. Based on the area of ​​the connected components, positional continuity, and changes in the circumscribed rectangle, it determines the target region corresponding to the glass bottle. Furthermore, the image preprocessing module establishes a tracking relationship between the target regions in multiple consecutive frames, thereby forming a sequence of local regions of interest (ROIs) of the glass bottle during the impact process, and outputs this sequence to the visual feature extraction module.

[0058] The visual feature extraction module extracts visual elastic features, optical discrimination features, and edge texture features of the glass bottle based on the local region of interest sequence. Specifically, for the pre-impact reference frame, impact peak frame, and post-impact recovery frame, the outer contour boundary of the glass bottle is extracted, and its contour compression ratio, contour recovery rate, and peak contour deformation are calculated. Simultaneously, considering the reflection of the glass bottle surface, the area ratio of its highlight region, brightness distribution, and highlight boundary morphology changes are extracted; considering the grayscale changes in the backlight region, its light transmission distribution features are extracted; and its edge texture features are further extracted. In this embodiment, since the glass bottle typically exhibits relatively small overall contour compression, limited post-impact recovery deformation, relatively concentrated highlight areas, and significant light transmission characteristics during impact, the above feature combination can effectively reflect its glass material and brittleness.

[0059] Subsequently, the feature fusion and recognition module fuses the extracted visual elasticity features, optical discrimination features, and edge texture features to construct a multimodal visual feature representation corresponding to the glass bottle. This multimodal visual feature representation is then input into a pre-trained recognition model for deep feature extraction, yielding the corresponding deep representation feature vector. After dimensionality reduction and / or denoising processing, the deep representation feature vector is matched with standard feature templates in a pre-defined fragile waste visual atlas dictionary. The matching results show that the item has the highest similarity to the glass template, and its fragility level corresponds to the high fragility level. Therefore, the waste category label is output as glass, and the fragility level parameter is high fragility level.

[0060] After the identification result is output, the control parameter generation module further generates corresponding flexible damping control parameters based on the brittleness level parameter of the glass bottle and the mass coefficient estimated from the area of ​​the local region of interest. Since the object is determined to be of a high brittleness level and its mass coefficient is at a medium or high level, the flexible damping control parameters generated by the control parameter generation module correspond to a lower impact velocity, a longer buffer time, and a higher level of flexible interception action. Subsequently, the control output module sends the flexible damping control parameters to the underlying controller.

[0061] After receiving the flexible damping control parameters, the bottom-level controller controls the end-of-line diversion buffer baffle to enter a ready state in advance. When the glass bottle reaches the predetermined diversion position, the baffle unfolds in a gradually changing speed to receive and guide the glass bottle. Compared to a fixed action mode, in this embodiment, the end-of-line diversion buffer baffle experiences less impact when contacting the glass bottle and adjusts the bottle's movement posture more smoothly, thereby effectively reducing the probability of breakage of the glass bottle during subsequent diversion and discharge processes.

[0062] Furthermore, in this embodiment, when the delivery object is replaced with a transparent plastic bottle, although its appearance is somewhat similar to that of a transparent glass bottle, the transparent plastic bottle typically has a higher profile compression ratio and recovery rate during impact, and its highlight boundary and light transmission distribution characteristics differ from those of a glass bottle. Therefore, the feature fusion and recognition module can identify it as a plastic delivery object and output a relatively low brittleness level parameter. At this time, the flexible damping control parameters generated by the control parameter generation module can correspond to a shorter buffer duration or a conventional diversion action. Thus, this invention can not only distinguish between similarly appearing transparent delivery objects but also generate different control strategies based on their brittleness differences, demonstrating good recognition accuracy and control adaptability.

[0063] In summary, this embodiment demonstrates that by collecting short video streams during the impact process of the delivered items and comprehensively utilizing the dynamic deformation characteristics, optical response characteristics, and texture characteristics of the delivered items for identification, the present invention can accurately output waste category labels and fragility level parameters. Simultaneously, the flexible damping control parameters generated based on the identification results can effectively guide the end-of-pipe diversion buffer baffle to implement adaptive interception and flow, thereby improving the accuracy of intelligent sorting of household waste and reducing the risk of breakage of fragile waste during the diversion process within the bin.

[0064] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A smart waste sorting method based on the Internet of Things, characterized in that, Includes the following steps: Step S1: Install a high-speed camera acquisition device near the impact baffle below the smart trash can's delivery port. When the device detects that the object is in contact with the impact baffle, it triggers the high-speed camera acquisition device to continuously acquire images of the object's impact process, resulting in a short video stream containing the object before, during, and after the impact. Step S2: Perform image preprocessing on the short video stream, and extract the motion pixel region of the delivery object during the impact phase based on background subtraction and connected component analysis, so as to determine and track the local region of interest of the delivery object in multiple consecutive frames of images. Step S3: For the continuous frame image sequence corresponding to the local region of interest, extract the temporal change features of the outer contour boundary of the delivery object, and extract the visual elastic features that characterize the deformation response of the delivery object based on the contour changes before and after the impact. Step S4: For the local region of interest, extract the reflective optical features of the surface of the delivery object and the light transmission features of the backlit area to obtain optical discrimination features characterizing the optical response of the delivery object surface; Step S5: Fuse the visual elasticity feature, the optical discrimination feature, and the edge texture feature of the delivery object to construct a multimodal visual feature representation of the delivery object; Step S6: Input the multimodal visual feature representation into the pre-trained recognition model for deep feature extraction to obtain a feature vector with deep representation, and perform dimensionality reduction and / or noise reduction on the feature vector; Step S7: Match the processed features with the preset fragile waste visual atlas dictionary, and output the waste category label of the delivered item and the corresponding fragility level parameter; Step S8: Based on the brittleness level parameter and the mass coefficient estimated from the area of ​​the object image, generate flexible damping control parameters through a preset visual control mapping function, and output the flexible damping control parameters to the bottom controller of the trash can to control the end diversion buffer baffle in the trash can to adaptively intercept and divert the object.

2. The method according to claim 1, characterized in that, The high-speed camera acquisition device in step S1 is a global shutter high-speed camera, the impact baffle is an inclined impact baffle, and the high-speed camera is set to the side or below the impact baffle to acquire transient collision images when the delivered object comes into contact with the impact baffle; triggering the high-speed camera acquisition device to continuously acquire images of the delivery object's impact process includes using an impact sensing signal, an image frame difference abrupt change signal, or a combination of both as triggering conditions to start the high-speed camera acquisition device to continuously acquire images.

3. The method according to claim 1, characterized in that, The image preprocessing in step S2 includes at least one or more of the following: grayscale conversion, background modeling, noise filtering, and binarization segmentation. The process of determining and tracking the local regions of interest (ROIs) of the delivery object in multiple consecutive frames includes: extracting the foreground motion region based on the background subtraction results; filtering the target region based on the area of ​​the connected components, the bounding rectangle, the centroid position, and the shape constraints; and establishing target association relationships based on the positional and area continuity of the target regions in adjacent frames to form a sequence of ROIs for the delivery object.

4. The method according to claim 1, characterized in that, The visual elasticity feature in step S3 includes at least one or more of the following parameters: the temporal change rate of the pixel coordinates of the outer contour boundary of the delivery object, the contour compression ratio at the moment of impact, the contour recovery rate after impact, and the peak value of the contour deformation; wherein, the contour compression ratio is determined by the change in contour size before and after the delivery object impact, the contour recovery rate is determined by the degree of contour rebound after the delivery object impact, and the visual elasticity feature is used to characterize the visual deformation response characteristics of the delivery object during the collision process.

5. The method according to claim 1, characterized in that, The optical discrimination features in step S4 include at least one or more of the following parameters: the area ratio of the specular reflection highlight area on the surface of the delivery object, the brightness distribution, the change in boundary curvature, the morphological distortion rate, and the grayscale transmission distribution or shadow transmittance of the backlit area of ​​the delivery object; the optical discrimination features are used to distinguish between transparent glass, transparent plastic and other delivery object categories with similar static visual appearance.

6. The method according to claim 1, characterized in that, The pre-trained recognition model in step S6 is a residual network model; the dimensionality reduction and / or denoising processing is implemented using at least one of principal component analysis, feature selection, autoencoder compression, or subspace mapping; the matching in step S7 involves performing distance metric comparison and / or classification probability comparison between the processed features and the standard feature templates in the preset fragile waste visual atlas dictionary to determine the waste category label and the fragility level parameter.

7. The method according to claim 1, characterized in that, The flexible damping control parameters in step S8 include at least one or more of the following: driving torque parameters, buffer duration parameters, and baffle action level parameters; the visual control mapping function generates a corresponding flexible damping control parameter matrix based on the brittleness level parameters and the mass coefficient, which is used to control the action state of the end diversion buffer baffle so that the end diversion buffer baffle can adaptively intercept and divert the delivered material.

8. A waste identification and flexible damping control system for implementing the method according to any one of claims 1 to 7, characterized in that, include: A high-speed camera acquisition module is used to capture a short video stream containing the impact process when the delivered object comes into contact with the impact barrier; The image preprocessing module is used to perform background subtraction, connected component analysis, and local region of interest extraction on the short video stream; The visual feature extraction module is used to extract the visual elasticity features, optical discrimination features, and edge texture features of the delivered object. The feature fusion and recognition module is used to construct multimodal visual feature representations and use pre-trained recognition models to complete deep feature extraction, dimensionality reduction and / or noise reduction, as well as matching and recognition with a preset fragile waste visual atlas dictionary; The control parameter generation module is used to generate flexible damping control parameters based on the identified waste category label, brittleness level parameter, and mass coefficient. The control output module is used to output the flexible damping control parameters to the bottom controller of the garbage bin, so as to control the end diversion buffer baffle to adaptively intercept and divert the delivered materials.

Images