Domestic garbage intelligent classification identification and question-answering system based on deep learning

Abstract

The invention discloses a household garbage intelligent classification identification and question answering system based on deep learning. The system comprises an image acquisition module, a garbage detection module, a classification mapping module, an intelligent question answering module, a data storage module, an interactive question answering module and a user interface module. The image acquisition module supports picture uploading and camera real-time acquisition; the garbage detection module adopts a YOLO model to realize garbage image real-time detection, and the false detection rate is reduced through stability filtering; the classification mapping module is used for mapping detection categories into standard four classifications; the intelligent question and answer module integrates a large language model and returns structured classification guidance; the data storage module realizes persistent management and tracing of detection records; the interactive answering module provides a garbage classification knowledge learning function; and the user interface module adapts to screens with different resolutions. The system is high in recognition efficiency, good in interaction experience and traceable in detection result, supports multi-source input, can effectively improve the intelligent level of garbage classification, and is suitable for multiple fields of environmental protection, smart cities and the like.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computer vision technology, specifically involving the application of deep learning-based image recognition technology in the scenario of household waste sorting. It can be widely used in the intelligent management of waste sorting in industries such as environmental protection, smart cities, and campus management. Background Technology

[0002] With increasing environmental awareness and the implementation of waste sorting policies, the standardized sorting of household waste has become a key focus of public attention. Current waste sorting methods primarily rely on manual identification and classification, which presents several significant problems:

[0003] The identification efficiency is low. Traditional manual sorting requires users to have professional knowledge of waste sorting. For users who are not familiar with the sorting standards, it is easy to make sorting errors. In addition, manual sorting is slow and cannot meet the needs of large-scale waste sorting and processing.

[0004] The user experience is poor. Most existing garbage classification query systems only support keyword search and cannot understand users' natural language questions. The interaction methods are simple and inconvenient, resulting in a poor user experience.

[0005] The test results lack traceability. Existing systems generally lack the function of managing test history records. Users cannot view and manage past test data, which is not convenient for subsequent statistical analysis and traceability.

[0006] The existing system lacks support for multiple input sources and typically only supports a single image input method, which cannot simultaneously meet the diverse usage scenarios of user image upload detection and real-time camera detection.

[0007] Furthermore, while some waste sorting systems have attempted to introduce intelligent recognition technology, they still have shortcomings in areas such as detection stability, structured answer output, and historical data management, making it difficult to fully meet users' needs for intelligent, efficient, and convenient waste sorting. Therefore, there is an urgent need for an intelligent waste sorting system that can solve the above problems. Summary of the Invention

[0008] This invention aims to overcome the shortcomings of existing waste sorting technologies, such as low recognition efficiency, poor interactive experience, lack of traceability of detection results, and insufficient support for multi-source input. It provides a deep learning-based intelligent waste sorting and identification system with question-and-answer capabilities, enabling automatic identification and classification of waste images, intelligent question-and-answer interaction, detection history management, and interactive learning of waste sorting knowledge, thereby improving the level of intelligence in waste sorting and the user experience.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A deep learning-based intelligent classification, identification, and question-answering system for household waste includes an image acquisition module, a waste detection module, a classification mapping module, an intelligent question-answering module, a data storage module, an interactive question-answering module, and a user interface module. These modules work together to complete the intelligent classification, identification, and related interactive functions of household waste.

[0011] 1. Image Acquisition Module: Used to acquire garbage images to be detected, supporting both image upload and real-time camera acquisition. Image upload detection allows users to select local image files via a file dialog box, and the system uses OpenCV to read the image data. Real-time camera acquisition uses a separate acquisition thread to capture video frames. Camera parameters are configured as follows: resolution 1280×720, frame rate 30fps, buffer size 1, automatic exposure disabled, and fixed exposure and gain set. A frame buffer queue (length 1) is used to achieve asynchronous decoupling of acquisition and processing, outputting images of a fixed size of 640×640 pixels to ensure display consistency.

[0012] 2. Waste Detection Module: This module uses a YOLO deep learning model to detect waste images and outputs the waste category and confidence score. It employs the Ultralytics YOLO model framework, supports loading pre-trained weight files (.pt format), and has an input size of 640×640 pixels. Image preprocessing includes scaling the input image while maintaining aspect ratio, adjusting the image to the standard input size using edge padding with a fill color of (114, 114, 114), converting BGR format images to RGB format and normalizing them (pixel values ​​divided by 255), and converting image data to PyTorch tensor format. The system provides an adjustable confidence threshold slider (range 0.01-0.99), allowing users to adjust the detection sensitivity according to their needs. This module also includes a detection result stability filtering mechanism. It maintains a detection history queue (5 frames long) for temporal filtering, calculates the IOU (Intersection over Union) between the current detection and historical frames, sets the IOU threshold to 0.3, and judges target consistency in consecutive frames, requiring the target to be consistent within 2 consecutive frames. Only when a detection occurs at a frame or higher is it considered a valid detection, effectively reducing the false detection rate.

[0013] 3. Classification Mapping Module: This module maps detected waste categories (such as Plastic Bottle, Battery, Can, etc.) to the standard four categories (recyclable waste, hazardous waste, kitchen waste, and other waste) and generates corresponding processing suggestions. This module has a built-in waste classification knowledge base to provide data support for classification mapping.

[0014] 4. Intelligent Question Answering Module: Integrating a large language model, this module receives user questions in natural language and returns structured guidance on waste sorting. It communicates with the locally deployed Ollam service via HTTP and supports multiple large language models, including deepseek-R1:1.5B. The system pre-sets expert prompts for waste sorting, including four categories of waste sorting standards, special rules (such as electronic waste disposal), and standardized answer format requirements. The module also includes a post-response processing mechanism that automatically filters thought process text (such as "I need," "Let me think," etc.) from the large language model output, extracts and formats numbered answer content (classification judgment, processing suggestions, classification reasons, environmental protection suggestions), and cleans up redundant blank lines and spaces. It uses QThread worker threads for API requests to avoid blocking the user interface and supports both streaming and normal response modes.

[0015] 5. Data Storage Module: Saves detection records to the database, supporting historical queries and management; uses a MySQL database, creating the `detection_history` table, containing the following fields: `id` (primary key, int type), `image_path` (original image path, varchar type), `image_data` (original image binary data, longblob type), `result_image_path` (result image path, varchar type), `result_image_data` (result image binary data with detection boxes, longblob type), `detection_results` (detection result JSON data, json type), `detection_time` (detection time, datetime type), `confidence_scores` (confidence score grouping array, json type), `processing_time` (processing time, float type), and `source_type` (source type, enum type); supports paginated queries, filtering by source type, and time sorting, records the last update time, supports incremental retrieval of new records, has an optional automatic refresh function, and also supports exporting detected images to local files.

[0016] 6. Interactive Quiz Module: Provides interactive learning functions for garbage classification knowledge, with 100 built-in multiple-choice questions on garbage classification, covering four categories: recyclable waste, hazardous waste, kitchen waste, and other waste. The quiz mechanism randomly selects 30 questions each time, without repetition, with a time limit of 30 seconds per question, a countdown display, and real-time statistics of the number of correct and incorrect answers. The final score is displayed after the quiz ends.

[0017] 7. User Interface Module: Provides a graphical user interface to display detection results and interactive Q&A; builds a cross-platform desktop application interface based on the PyQt5 framework, uses the QSplitter splitter to achieve a draggable and resizable adaptive layout, and supports adaptive display on screens with different resolutions; uses 1600×900 as the base size and scales font, margins, rounded corners, and other style parameters proportionally according to the actual window size; uses QStackedWidget to switch between five functional modules: homepage, recognition and detection, interactive Q&A, Q&A assistant, and detection history.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] 1. High recognition efficiency: The system uses the YOLO deep learning model to achieve real-time detection of garbage images. The detection time per frame is in the millisecond range, which significantly improves efficiency compared to manual recognition. The system supports the automatic recognition of 7 common types of garbage and can automatically map them to four garbage classification standards to meet the needs of rapid classification.

[0020] 2. Excellent interactive experience: The system integrates a large language model to achieve natural language question answering. Users can ask questions about garbage classification in everyday language, and the system will return structured classification judgments, processing suggestions, reasons for classification, and environmental protection suggestions. The interaction method is user-friendly and convenient. At the same time, an interactive question-and-answer module is set up to enhance users' interest and enthusiasm in learning about garbage classification.

[0021] 3. Traceable test results: All test records are automatically saved to a MySQL database, including complete information such as original images, test result images, test time, and confidence level. Historical records can be queried, filtered, exported, and deleted, facilitating subsequent traceability and statistical analysis for users.

[0022] 4. Multi-source input support: It supports both image upload and real-time camera detection input methods. The camera detection adopts an independent acquisition thread and buffering mechanism to ensure smooth detection and meet the needs of users in different usage scenarios.

[0023] 5. Strong interface adaptability: It adopts a responsive layout design and proportional scaling mechanism, which automatically adjusts the size of controls, font size and layout ratio when the window size changes, adapting to screens with different resolutions and improving the user's visual experience.

[0024] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0025] In the attached diagram:

[0026] Figure 1 This is a diagram of the overall system architecture of the present invention;

[0027] Figure 2 This is a flowchart of the garbage image detection process of the present invention;

[0028] Figure 3 This is a flowchart of the intelligent question-answering process of the present invention;

[0029] Figure 4 This is a screenshot of the homepage of the system of the present invention;

[0030] Figure 5 This is a schematic diagram of the image upload detection interface of the present invention;

[0031] Figure 6 This is a schematic diagram of the interface for detecting historical records according to the present invention;

[0032] Figure 7 This is a schematic diagram of the real-time detection interface of the camera in this invention;

[0033] Figure 8 This is a schematic diagram of the AI ​​question-and-answer function interface of the present invention. Detailed Implementation

[0034] Example 1: Image Upload Detection Implementation Method

[0035] Hardware requirements: The computer must be equipped with an Intel Core i5 or higher processor, 8GB or more of RAM, 100GB or more of available space, and an optional NVIDIA GTX 3060 graphics card for GPU acceleration.

[0036] Software environment: Operating system is Windows 10 / 11, Python version 3.8 and above, the main dependent libraries include PyQt5==5.15.11 (graphical interface), opencv-python==4.10.0.84 (image processing), ultralytics==8.3.34 (YOLO model), mysql-connector-python==8.2.0 (database connection), requests==2.32.3 (HTTP requests).

[0037] Operating steps:

[0038] Step 1: The user launches the application, the system loads the main window (MainWindow), and displays the home screen;

[0039] Step 2: The user clicks the "Recognition and Detection" button in the sidebar to switch to the detection page (DetectionPage);

[0040] Step 3: The user clicks the "Select Model" button and selects the pre-trained YOLO model file (.pt format) through the file dialog box. The system calls YOLO (model_path) to load the model.

[0041] Step 4: The user confirms that the detection type drop-down box selects "Image Detection" and adjusts the confidence threshold slider to the desired value (default 50%).

[0042] Step 5: The user clicks the "Start Image Recognition and Detection" button, and the system pops up a file selection dialog box. The user selects the image to be detected.

[0043] Step 6: The system creates a DetectionWorker thread to preprocess the selected image (maintaining aspect ratio scaling, edge padding, format conversion, normalization, and tensor conversion), and then inputs it into the YOLO model for single-frame detection, using a detection history queue for stability filtering.

[0044] Step 7: The main thread receives the detection results, maps them to standard four-class classification through the classification mapping module, and generates processing suggestions. The original image is displayed on the left side of the interface, and the result image with the detection box is displayed on the right side. At the same time, the detection information (including English name, Chinese name, classification result, quantity, confidence, processing suggestions, etc.) is displayed.

[0045] Step 8: The system automatically saves the detection record to the MySQL database, including the original image, the result image, the detection result JSON, the confidence level, the detection time, the processing time, the source type, and other information. Users can query and export this record in the detection history module.

[0046] Example 2: Real-time camera detection implementation method

[0047] Hardware environment: In addition to meeting the computer hardware requirements of Example 1, a USB camera or built-in camera is also required, and the camera must support a resolution of 1280×720.

[0048] Software environment: consistent with the software environment of Example 1.

[0049] Operating steps:

[0050] Step 1: The user launches the application and switches to the DetectionPage, loads the pre-trained YOLO model file, and adjusts the confidence threshold slider to the desired value (default 50%).

[0051] Step 2: The user selects "Camera Detection" from the detection type drop-down box, and the system automatically displays the camera control buttons (Start Camera, Stop Camera) and the exposure adjustment slider;

[0052] Step 3: When the user clicks the "Start Camera" button, the system creates a WebcamWorker thread, starts the camera and captures video frames according to preset parameters (resolution 1280×720, frame rate 30fps, auto exposure off), and stores the captured video frames in the frame buffer queue.

[0053] Step 4: The detection thread retrieves frames from the buffer queue, performs image preprocessing (scaling while maintaining aspect ratio, edge padding, format conversion, normalization, and tensor conversion), and then inputs them into the YOLO model for object detection;

[0054] Step 5: The system performs stability filtering on the detection results, requiring the target to appear in more than 2 consecutive frames and IOU>0.3 to be considered a valid detection;

[0055] Step 6: The classification mapping module maps the effectively detected waste categories to the standard four categories and generates processing suggestions. The original camera footage is displayed in real time on the left side of the interface, and the result screen with the detection box is displayed on the right side, while the detection information is also displayed.

[0056] Step 7: Users can click the "Save to Detection History" button to manually save the current detection screen and related detection records to the MySQL database. The record can be queried and exported from the detection history module later.

[0057] Step 8: After the user completes the detection, click the "Stop Camera" button to turn off the camera and end the detection thread.

[0058] Example 3: Implementation of AI Question Answering Function

[0059] Hardware environment: Same as the computer hardware requirements in Example 1.

[0060] Software environment: In addition to meeting the software environment requirements of Example 1, the Ollam service (default port 11434) needs to be run locally, and the deepseek-r1:1.5b large language model (approximately 1.5GB) needs to be deployed.

[0061] Operating steps:

[0062] Step 1: The user launches the application, clicks the "Q&A Assistant" button in the sidebar, and enters the KnowledgeBaseWindow interface;

[0063] Step 2: The system automatically tests the connection status with the Ollam service and displays "Connected" or "Not Connected" on the interface;

[0064] Step 3: Users can click the FAQ button on the left (such as "What kind of waste are plastic bottles?" "How should batteries be disposed of?" etc.) to ask questions quickly, or enter custom waste sorting questions in the input box;

[0065] Step 4: After the user clicks the "Send" button, the system creates a ChatWorker thread, which concatenates the preset garbage classification expert system prompts with the user's question to construct an API request;

[0066] Step 5: Send the API request to the local Ollam service via the HTTP protocol and receive the response from the large language model;

[0067] Step 6: The system performs post-processing on the AI ​​response, filters the thought process text, extracts the content in the sequence format (classification judgment, processing suggestions, classification reasons, environmental protection suggestions), and cleans up extra blank lines and spaces;

[0068] Step 7: The interface displays the question and answer records in the form of chat bubbles. The AI ​​answers are presented in a 1-4 number format for easy viewing by the user.

[0069] Example 4: Implementation of Interactive Question-Answering Function

[0070] Hardware environment: Same as the computer hardware requirements in Example 1.

[0071] Software environment: consistent with the software environment of Example 1.

[0072] Operating steps:

[0073] Step 1: The user launches the application, clicks the "Interactive Quiz" button in the sidebar, and enters the quiz interface;

[0074] Step 2: The user clicks the "Start Answering Questions" button, and the system randomly selects 30 non-repeating questions from the built-in database of 100 multiple-choice questions on garbage sorting;

[0075] Step 3: The system displays the questions and options one by one, with a time limit of 30 seconds for each question, and a countdown timer is displayed on the screen;

[0076] Step 4: Users select and submit answers within the specified time. The system judges the correctness of the answers in real time, counts the number of correct and incorrect answers, and displays them on the interface.

[0077] Step 5: After all 30 questions are completed or the user clicks the "End Questions" button, the system displays the final score (number of correct questions, number of incorrect questions, and accuracy rate). The user can choose to "Retake Questions" to draw questions again for practice.

Claims

1. A deep learning-based intelligent classification, identification, and question-answering system for household waste, characterized in that, include: The image acquisition module is used to acquire images of the garbage to be detected, and supports both image upload and real-time acquisition by camera. The garbage detection module uses the YOLO deep learning model to detect objects in garbage images and outputs the garbage category and confidence score. The classification mapping module maps the detected waste categories to the standard four categories (recyclable waste, hazardous waste, kitchen waste, and other waste). The intelligent question-answering module integrates a large language model, receives users' natural language questions, and returns structured garbage sorting guidance; The data storage module saves detection records to the database, supporting historical queries and management; The interactive quiz module has a built-in question bank on waste sorting and provides interactive learning functions; The user interface module provides a graphical user interface that displays test results and provides interactive Q&A.

2. The intelligent classification, identification, and question-answering system for household waste based on deep learning according to claim 1, characterized in that, The image acquisition module uses an independent acquisition thread and a frame buffer mechanism for real-time acquisition of the camera. The acquisition thread continuously reads camera frames and stores them in a buffer queue, while the detection thread retrieves frames from the queue for processing. The camera parameters are configured as follows: resolution 1280×720, frame rate 30fps, buffer size 1, automatic exposure off and fixed exposure value and gain set, outputting an image with a fixed size of 640×640 pixels.

3. The intelligent classification, identification, and question-answering system for household waste based on deep learning according to claim 1, characterized in that, The garbage detection module uses the Ultralytics YOLO model framework and supports loading pre-trained weight files (.pt format). The model input size is 640×640 pixels. The image preprocessing process includes aspect ratio-preserving scaling, edge padding (fill color 114, 114, 114), BGR to RGB conversion, normalization (pixel value / 255), and tensor transformation. The system provides a confidence threshold adjustment slider in the range of 0.01-0.

99. It also includes a detection result stability filtering mechanism, which maintains a detection history queue of 5 frames, calculates the IOU between the current detection and the historical frames (threshold 0.3), and retains only targets that appear in more than 2 consecutive frames as valid detection results.

4. The intelligent classification, identification, and question-answering system for household waste based on deep learning according to claim 1, characterized in that, The intelligent question-answering module communicates with the locally deployed Ollam service via the HTTP protocol and supports large language models such as deepseek-R1:1.5B; it also includes preset expert role prompts containing garbage classification standards, special rules, and answer format requirements. It includes a post-response processing mechanism, automatically filters the thought process text, extracts and formats numbered responses (categorization judgment, processing suggestions, categorization reasons, environmental protection suggestions); it uses QThread worker threads to make API requests, supporting streaming and normal responses.

5. The intelligent classification, identification, and question-answering system for household waste based on deep learning according to claim 1, characterized in that, The data storage module uses a MySQL database and creates a table called detection_history, which includes fields such as id, image_path, image_data, result_image_path, result_image_data, detection_results, detection_time, confidence_scores, processing_time, and source_type. It supports paginated queries, filtering by source type, sorting by time, incremental updates, automatic refresh, and exporting detected images.

6. The intelligent classification, identification, and question-answering system for household waste based on deep learning according to claim 1, characterized in that, The interactive quiz module contains 100 multiple-choice questions covering four types of waste. Each quiz randomly selects 30 non-repeating questions, with a time limit of 30 seconds per question. The number of correct and incorrect answers is counted in real time, and the final score is displayed after the quiz ends.

7. The intelligent classification, identification, and question-answering system for household waste based on deep learning according to claim 1, characterized in that, The user interface module is built on the PyQt5 framework and uses the QSplitter to achieve adaptive layout, scaling proportionally with a base size of 1600×900. The QStackedWidget is used to switch between five functional modules: homepage, recognition and detection, interactive question answering, question and answer assistant, and detection history.

8. A method for intelligent classification, identification, and question answering of household waste based on deep learning, characterized in that, Includes the following steps: S1: Obtain the garbage image to be detected through the image acquisition module (image upload or real-time acquisition by camera); S2: The garbage detection module preprocesses the image (maintaining aspect ratio scaling, edge padding, format conversion, normalization, tensor conversion); S3: Input the preprocessed image into the YOLO model for object detection; S4: Perform stability filtering on the detection results (maintain a 5-frame historical queue, calculate IOU, and retain targets that appear in more than 2 consecutive frames). S5: The classification mapping module maps the detected waste categories to the standard four categories and generates processing suggestions; S6: The user interface module displays the detection results, and the data storage module saves the detection records to the database; S7: The intelligent question answering module receives user questions in natural language, constructs a request containing system prompts, sends it to the large language model, performs post-processing on the response, and returns structured guidance. S8: The interactive quiz module provides practice questions and answers on garbage sorting, with random questions, time limits for answering, and statistics on scores.

9. The method for intelligent classification, identification, and question answering of household waste based on deep learning according to claim 8, characterized in that, In step S1, the camera uses an independent acquisition thread and a frame buffer queue to achieve asynchronous decoupling during real-time acquisition. The acquisition parameters are 1280×720 resolution, 30fps frame rate, and automatic exposure is turned off.

10. The method for intelligent classification, identification, and question answering of household waste based on deep learning according to claim 8, characterized in that, The post-processing of the intelligent question-and-answer module in step S7 includes filtering the thought process text, extracting the formatted sequence number content, cleaning up redundant blank lines and spaces, and displaying the question-and-answer record in the form of chat bubbles.

Images