Dustbin cover state detection system and method based on deep learning

Abstract

The invention discloses a garbage can cover state detection system and method based on deep learning, and relates to the technical field of deep learning, and the method comprises the steps: constructing a multi-layer image pyramid, calculating a scaling ratio and a standardized image set, generating a corrected image set through the combination of gray conversion and perspective correction, and generating a gradient field through space and time gradients. A fractional order time gradient and sub-lattice stress are introduced to calculate an optical flow component, FAST and Harris algorithms are used to extract a refining corner point, a YOLOv8n model is used to identify the barrel cover state probability, and the classification accuracy is optimized through an FLNN model; according to the method, optical flow analysis and fractional gradient are fused, the precision and robustness of garbage can cover state detection in fast motion and complex environments are improved, and the accuracy of state detection and the adaptability to different visual angles and shielding scenes are improved through combination of a multi-scale mask and YOLOv8n and FLNN models.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a deep learning-based system and method for detecting the state of trash can lids. Background Technology

[0002] With the acceleration of urbanization and the widespread implementation of waste sorting policies, the demand for intelligent management of trash cans is becoming increasingly prominent. Traditional trash can status detection mainly relies on mechanical sensors or manual inspections. With the advancement of computer vision and deep learning technologies, image-based detection methods have gradually become a research hotspot. Real-time target detection methods based on YOLOv8n can effectively identify the area and status of trash cans, while optical flow analysis technology further improves the accuracy of status detection by capturing motion information from dynamic images. Image preprocessing technology and feature extraction methods also provide strong support for image analysis in complex environments, significantly improving the automation level of trash can status detection.

[0003] However, existing technologies still have shortcomings. In dynamic scenes, although traditional optical flow algorithms can capture motion information, they are sensitive to noise, which leads to unstable gradient calculations under changes in illumination or background interference, thus affecting the accuracy of state classification. Although traditional corner detection algorithms are used for feature extraction, they ignore brightness correction and constraint residual analysis, which limits the adaptability of the system.

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a deep learning-based trash can lid state detection system and method, which solves the problem that in dynamic scenes, although traditional optical flow algorithms can capture motion information, they are sensitive to noise, resulting in unstable gradient calculation under changes in illumination or background interference, which in turn affects the accuracy of state classification. Although traditional corner detection algorithms are used for feature extraction, they ignore brightness correction and constraint residual analysis, which limits the adaptability of the system.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a deep learning-based trash can lid status detection system, which includes, The image preprocessing module is used to collect RGB images and preprocess them to obtain smooth images. Based on the smooth images, a multi-layer image pyramid is constructed to obtain a pyramid image set. The scaling ratio of each layer of the image is calculated, the width and height of the image are scaled using linear interpolation, and then filled. A normalized image set is generated, and grayscale conversion and perspective correction are performed to obtain a corrected image set. The image correction and gradient calculation module is used to calculate the spatial and temporal gradients of the corrected image set, generate the gradient field, obtain the fractional-order temporal gradient by combining the fractional order, calculate the sub-grid stress and scalar transport constraints of the corrected image set, obtain the initial constraint residual, calculate the preprocessed optical flow component of the coarsest layer, and obtain the stable optical flow component through iterative optimization. The optical flow processing and mask generation module is used to transfer stable optical flow components through bilinear interpolation, fuse multiple stable optical flow components to obtain fused optical flow components, calculate optical flow amplitude, construct an optimization function objective, use the AntLion algorithm for iterative optimization to obtain the optimal threshold, generate a static mask in combination with optical flow amplitude, calculate local consistency weight, adjust the fusion to obtain a multi-scale mask, and generate a background suppression image. The corner detection and image correction module is used to calculate candidate corners of the background suppression mask using the FAST algorithm, refine them using the Harris algorithm to obtain the cropped image, calculate the local brightness mean of the cropped image, and correct it using the CLAHE algorithm with AOA optimization constraint factor to obtain the optimized image. The state detection and storage module is used to input the optimized image into the YOLOv8n model to predict the state probability and label it. The final state and its confidence level are obtained through the FLNN model optimized by the APSO algorithm, recorded in the local database and uploaded to the cloud platform for storage.

[0007] As a preferred embodiment of the deep learning-based trash can lid state detection system of the present invention, wherein: the generation of a standardized image set, followed by grayscale conversion and perspective correction, to obtain a corrected image set includes: Based on the smoothed image, a multi-layer image pyramid is constructed to obtain the pyramid image set. The scaling ratio of each layer of the image is calculated. The width and height of the image are scaled by linear interpolation, and the width and height of the image are filled to obtain the normalized image set. The normalized image set is then subjected to grayscale conversion and perspective correction to obtain the corrected image set.

[0008] As a preferred embodiment of the deep learning-based trash can lid state detection system of the present invention, the step of calculating the preprocessed optical flow component of the coarsest layer and obtaining a stable optical flow component through iterative optimization includes: Spatial and temporal gradients are calculated for the calibrated image set to generate a gradient field. Fractional temporal gradients are obtained by combining fractional gradients. Sub-grid stress and scalar transport constraints of the calibrated image set are calculated to obtain initial constraint residuals. The preprocessed optical flow components of the coarsest layer are calculated and iteratively optimized to obtain stable optical flow components, which are then smoothed using a Gaussian kernel function.

[0009] As a preferred embodiment of the deep learning-based trash can lid state detection system of the present invention, the step of calculating local consistency weights, adjusting and fusing to obtain a multi-scale mask, and generating a background suppression image includes: The stable optical flow component of the coarsest layer is transferred to other layers through bilinear interpolation. The stable optical flow components of multiple layers are fused to obtain the fused optical flow component. The optical flow amplitude is calculated, an optimization function objective is constructed, the motion threshold is optimized to obtain the optimal threshold, a static mask is generated, its local consistency weight is calculated, and it is adjusted and fused to obtain a multi-scale mask and a background suppression image.

[0010] As a preferred embodiment of the deep learning-based trash can lid state detection system of the present invention, the step of calculating the local brightness mean of the cropped image and correcting it using the CLAHE algorithm with AOA optimization constraint factor to obtain an optimized image includes: For the background suppression mask, the FAST algorithm is used to calculate candidate corner points, and the Harris algorithm is used to refine them to obtain the refined corner point coordinates. The background suppression mask is then cropped to obtain the cropped image. The local brightness mean is calculated and corrected using the CLAHE algorithm. Finally, the constraint factor in the CLAHE algorithm is optimized using AOA to obtain the optimized image.

[0011] As a preferred embodiment of the deep learning-based trash can lid state detection system of the present invention, wherein: the FLNN model optimized by the APSO algorithm obtains the final state and its confidence level, including: The optimized image is pixel normalized, and its state probability is obtained through the YOLOv8n model. The determined state is obtained by filtering. The feature vector of the determined state is input into the FLNN model to obtain the final state and its confidence level. This is then recorded in the local database and uploaded to the cloud platform for storage.

[0012] As a preferred embodiment of the deep learning-based trash can lid state detection system of the present invention, the step of collecting RGB images and preprocessing them to obtain smooth images includes: A high-definition camera is deployed vertically above the top of the trash can to collect RGB images and calculate the image sharpness score. If the image sharpness score is less than the image sharpness score threshold, it is marked as a blurry image, and RGB images are collected again. Gaussian filtering is used to denoise the RGB images to obtain a smooth image.

[0013] Secondly, this invention provides a deep learning-based method for detecting the state of a trash can lid, including: Collect RGB images and preprocess them to obtain smooth images. Construct a multi-layer image pyramid based on the smooth images to obtain a pyramid image set. Calculate the scaling ratio of each layer of the image, scale the image width and height using linear interpolation, and fill the image to generate a normalized image set. Perform grayscale conversion and perspective correction to obtain a corrected image set. The spatial and temporal gradients of the corrected image set are calculated to generate a gradient field. The fractional-order temporal gradient is obtained by combining the fractional-order gradients. The sub-grid stress and scalar transport constraints of the corrected image set are calculated to obtain the initial constraint residuals. The preprocessed optical flow components of the coarsest layer are calculated, and the stable optical flow components are obtained through iterative optimization. The stable optical flow component is passed through bilinear interpolation and fused with multiple stable optical flow components to obtain the fused optical flow component. The optical flow amplitude is calculated and an optimization function objective is constructed. The Ant Lion algorithm is used for iterative optimization to obtain the optimal threshold. A static mask is generated by combining the optical flow amplitude, the local consistency weight is calculated, and the fusion is adjusted to obtain a multi-scale mask to generate a background suppression image. The candidate corner points of the background suppression mask are calculated using the FAST algorithm, refined using the Harris algorithm to obtain the cropped image, the local brightness mean of the cropped image is calculated, and the CLAHE algorithm with AOA optimization constraint factor is used for correction to obtain the optimized image. The optimized image is input into the YOLOv8n model to predict the state probability and is labeled. The final state and its confidence are obtained through the FLNN model optimized by the APSO algorithm, recorded in the local database and uploaded to the cloud platform for storage.

[0014] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the deep learning-based trash can lid state detection method described in the present invention.

[0015] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the deep learning-based trash can lid state detection method as described in the present invention.

[0016] The beneficial effects of this invention are as follows: This invention constructs a multi-layer image pyramid, calculates the scaling ratio and normalizes the image set, generates a corrected image set by combining grayscale conversion and perspective correction, generates a gradient field using spatial and temporal gradients, introduces fractional-order temporal gradients and subgrid stress to calculate optical flow components, extracts and refines corner points using FAST and Harris algorithms, identifies the lid state probability using the YOLOv8n model, and optimizes the classification accuracy using the FLNN model; it improves the accuracy and robustness of trash can lid state detection in fast-moving and complex environments, and enhances the accuracy of state detection and adaptability to different viewpoints and occlusion scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the deep learning-based trash can lid status detection system in Example 1.

[0019] Figure 2 This is a flowchart of the deep learning-based trash can lid state detection method in Example 1. Detailed Implementation

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a deep learning-based trash can lid status detection system, including the following steps: S1, Image Preprocessing Module, is used to collect RGB images and preprocess them to obtain smooth images. Based on the smooth images, a multi-layer image pyramid is constructed to obtain a pyramid image set. The scaling ratio of each layer of the image is calculated, the width and height of the image are scaled using linear interpolation and filled, a normalized image set is generated, and grayscale conversion and perspective correction are performed to obtain a corrected image set. Specifically, RGB images are collected and preprocessed to obtain smoothed images, including: A high-definition camera is deployed vertically above the top of the trash can to collect RGB images and calculate an image sharpness score using the following formula: , in, Score the image sharpness. For pixels grayscale values, of which The row index of the pixel. The column index for pixels. For the Laplacian operator in pixels The response value, Total number of pixels; Construct an image coordinate system, with the top left corner of the image as the origin, the X-axis pointing horizontally to the right, and the Y-axis pointing vertically downwards, with the unit being pixels; If the image sharpness score is less than the image sharpness score threshold, it is marked as a blurry image. The RGB image is then collected again, and Gaussian filtering is used to denoise the RGB image to obtain a smooth image.

[0024] By introducing an image sharpness scoring mechanism, adaptive evaluation of data quality is achieved. This score, based on statistical analysis of the Laplacian operator response value, can quantify the edge sharpness and overall focus consistency of the image, thereby effectively eliminating blurred frames caused by environmental factors such as vibration or light fluctuations. This not only improves the robustness of subsequent processing but also reduces the waste of computing resources. The standardized operation of constructing the image coordinate system provides a unified geometric reference framework for subsequent multi-scale analysis, avoiding coordinate drift problems caused by camera installation deviations. Through Gaussian filtering for noise reduction, the interference of high-frequency noise on gradient calculation is further suppressed, enhancing the system's anti-interference capability and forming a closed-loop feedback mechanism, which helps to achieve higher real-time performance and adaptability in actual deployment.

[0025] Furthermore, a standardized image set is generated, and grayscale conversion and perspective correction are performed to obtain a corrected image set, including: Based on the smoothed image, a multi-layer image pyramid is constructed to obtain the pyramid image set, using the following formula: , in, For layer Smooth image, and To smooth the x and y coordinates of image pixels, For the number of channels, For layer Smooth image, The pixel region is obtained by dividing a smooth image into a grid. Based on the pyramid image set, the scaling ratio of each layer is calculated. The width and height of the images are scaled using linear interpolation, and padding is applied to the width and height to obtain a standardized image set. The formula is as follows: , , , , , in, For layer Scaling ratio, For the target resolution, and For layer Width and height, and For scaled layers Width and height, and For layer The amount of fill in width and height; A corrected image set is obtained by performing grayscale conversion and perspective correction on the standardized image set.

[0026] By constructing and standardizing a multi-layer image pyramid, a multi-resolution representation of the input image is achieved. This pyramid structure, based on grid partitioning and channel-level smoothing, can capture hierarchical features from global structure to local details, thus significantly improving the sensitivity to variations in the bucket lid state. The linear interpolation and padding mechanism of the scaling ratio ensures the size consistency of each layer of the image, avoiding the information loss problem of traditional single-scale methods when processing images with uneven scales. This method draws on the multi-scale theory in the field of computer vision and creatively introduces the target resolution as an anchor parameter, further optimizing computational efficiency. The subsequent grayscale conversion and perspective correction steps generate a cleaner set of corrected images by eliminating color redundancy and geometric distortion. This not only improves the accuracy of subsequent optical flow estimation but also reduces the influence of artifacts caused by viewpoint deviation, enhances the system's generalization ability, and provides high-quality input for deep learning models.

[0027] S2, Image Correction and Gradient Calculation Module, is used to calculate the spatial and temporal gradients of the corrected image set, generate the gradient field, obtain the fractional-order temporal gradient by combining the fractional order, calculate the subgrid stress and scalar transport constraints of the corrected image set, obtain the initial constraint residual, calculate the preprocessed optical flow component of the coarsest layer, and obtain the stable optical flow component through iterative optimization. Specifically, the preprocessed optical flow components of the coarsest layer are calculated, and stable optical flow components are obtained through iterative optimization, including: The spatial and temporal gradients are calculated on the corrected image set to generate the gradient field, as shown in the formula: , , , in, For layer of Spatial gradient of orientation, For layer of Spatial gradient of orientation, For layer Time time gradient, For layer Time The corrected image; Based on the gradient field, and combined with the fractional time gradient (defined based on empirical rules), the fractional time gradient is obtained, as follows: , in, For layer Time fractional time gradient, It is of fractional order. It is the difference order. For the first fractional order The coefficient; The formulas for calculating the subgrid stress and scalar transport constraints of the corrected image set are as follows: , , , , in, For layer The strain rate tensor, For filtering scale, and For layer The initial optical flow, expressed as the initial horizontal and vertical velocity components, is obtained based on the Horn-Schunck model. layer Time eddy viscosity coefficient, For reference density, This is Smagorinsky's constant. For layer The sublattice stress tensor, For layer scalar transport constraints Let Reynolds number be 1. For the Schmitt number, These are partial derivatives; Based on scalar transport constraints, the initial constraint residual is calculated using the following formula: , in, For layer fractional order The initial constraint residuals; Based on the initial constraint residuals, the initialized optical flow field (initial horizontal and vertical velocity components), the fractional-order temporal gradient, and the corrected image set, the preprocessed optical flow component of the coarsest layer (the Nth layer of the multi-layer image pyramid) is obtained, as shown in the formula: , , in, The coarsest layer The preprocessed optical flow component, The coarsest layer is the third layer. For example, in a multi-layered image pyramid with three layers, the coarsest layer is the third layer. For Gaussian smoothing kernel, The standard deviation of pixels; The coarsest layer is iteratively optimized until the maximum number of iterations is reached, resulting in a stable optical flow component. This component is then smoothed using a Gaussian kernel function, as shown in the formula: , in, For layer The residual of the iterative optimization This represents the number of layers in the pyramid.

[0028] By integrating the spatial-temporal gradient field with fractional derivatives, the generated fractional-order temporal gradient captures non-integer-order dynamic microvariables through empirically set orders and coefficients. This surpasses the limitations of traditional integer-order gradients in smoothing motion, enabling a more accurate description of the nonlinear temporal changes during lid opening and closing, reducing motion estimation errors. The calculated subgrid stress and scalar transport constraints, based on the Smagorinsky model and Horn-Schunck initial optical flow, introduce hydrodynamically inspired eddy viscosity coefficients and Reynolds number parameters. This cross-disciplinary creative approach enhances the physical constraints of the gradient field and significantly improves robustness in noisy environments. The introduction of initial constraint residuals serves as the starting point for iterative optimization, ensuring the stability of the convergence process. The acquisition of preprocessed optical flow components, combined with Gaussian smoothing kernels, further suppresses pixel-level noise. The iterative optimization mechanism refines stable optical flow components through residual minimization and Gaussian smoothing. The maximum iterative control in this process avoids overfitting risks, improves the detection accuracy of the half-open state, reduces computational complexity, and provides a reliable foundation for subsequent fusion, demonstrating potential advantages in real-time video analysis.

[0029] S3, the optical flow processing and mask generation module, is used to transfer stable optical flow components through bilinear interpolation, fuse multiple stable optical flow components to obtain fused optical flow components, calculate optical flow amplitude, construct an optimization function objective, use the AntLion algorithm for iterative optimization to obtain the optimal threshold, generate a static mask in combination with optical flow amplitude, calculate local consistency weight, adjust the fusion to obtain a multi-scale mask, and generate a background suppression image. Specifically, the local consistency weights are calculated, and the fusion is adjusted to obtain a multi-scale mask, generating a background-suppressed image, including: The stable optical flow component of the coarsest layer is transferred to other layers via bilinear interpolation, using the following formula: , , , , in, and For layer The interpolated optical flow represents the stable optical flow component. and The coordinates of the coarsest layer. For the set of coarse-layer neighborhood pixels, For pixels The interpolation weights are calculated based on the bilinear interpolation distance; The set of coarse-layer neighbor pixels refers to the set obtained by trilinear interpolation of the mapping coordinates of the coarsest layer. By fusing the stable optical flow components of multiple layers, a fused optical flow component is obtained, as shown in the formula: , , in, and To fuse optical flow components, This represents the total number of floors. For layer The fusion weight, For bilinear interpolation, This is the scaling factor for layer 1; The optical flow amplitude is calculated based on the fused optical flow components, using the following formula: , in, For layer optical flux amplitude, and For layer The fused optical flow components; Based on the optical flow amplitude, an optimization function objective is constructed to optimize the motion threshold, and the formula is as follows: , , in, For layer motion threshold The objective function of optimization, For coverage weight, For layer Temporary mask, For target resolution; The objective function is iteratively optimized using the Ant Lion optimization algorithm until the maximum number of iterations is reached, at which point the optimal threshold is obtained. The formula is as follows: , in, layer Time The motion threshold, It is a random number. For layer The motion threshold obtained by minimizing the objective function. For Ant Lion acceleration factor; Based on the optimal threshold and amplitude, a static mask is obtained. Its local consistency weight is calculated, and it is adjusted and fused to obtain a multi-scale mask, as shown in the formula: , , , in, For layer Consistency weight, For neighborhood windows, referring to Define a rectangular window with radius U centered at the specified point. The standard deviation of pixel consistency For layer Adjusting the mask, For multi-scale masks, For layer The fusion weight, Number of floors; Applying a multi-scale mask to the highest resolution layer yields a background-suppressed image, as shown in the formula: , in, Background suppression image, This is the first resolution layer, which is the first layer of the multi-layer image pyramid.

[0030] By efficiently transferring the stable optical flow components of the coarsest layer to other layers through bilinear interpolation, the continuity and consistency of motion information in the multi-scale image pyramid are enhanced, avoiding error accumulation caused by inter-layer breaks in traditional optical flow methods. This transfer mechanism, leveraging mapped coordinates and neighborhood pixel weights, achieves fine interpolation fusion, reducing computational complexity and improving robustness in dynamic environments. This allows the system to better capture subtle motion changes. Weighted fusion of multi-layer stable optical flow components generates a fused optical flow component. This process introduces dynamic adjustment of layer weights and scaling ratios, creatively overcoming the limitations of single-scale analysis and ensuring the smoothness and integrity of the global motion field, thereby improving state detection under complex background interference. Accuracy is improved by further quantifying motion intensity based on the amplitude calculation of fused optical flow components, providing a reliable foundation for subsequent threshold optimization and avoiding subjective bias caused by empirical thresholds. An optimization function objective is constructed and iterative optimization is performed using the antlion algorithm. This creative integration of bio-inspired optimization mechanism and optical flow analysis not only adaptively determines the optimal motion threshold but also minimizes the trade-off between coverage and mask accuracy, significantly improving the system's adaptability. By calculating local consistency weights and adjusting the fusion to generate multi-scale masks, the mask is applied to the highest resolution layer to produce a background-suppressed image, achieving efficient background noise suppression and greatly improving image purity and the reliability of feature extraction. It also exhibits superior performance in scenes with varying illumination or occlusion.

[0031] S4, Corner Detection and Image Correction Module, is used to calculate candidate corners of the background suppression mask using the FAST algorithm, refine them using the Harris algorithm to obtain the cropped image, calculate the local brightness mean of the cropped image, and correct it using the CLAHE algorithm with AOA optimization constraint factor to obtain the optimized image; Specifically, the mean local brightness of the cropped image is calculated, and the CLAHE algorithm with AOA optimization constraint factor is used for correction to obtain the optimized image, including: For the background suppression mask, the FAST algorithm is used to calculate candidate corner points, and the formula is: , in, This refers to the FAST corner response value. A set of pixels For pixels, The threshold value for FAST corner response; The set of pixels refers to Construct a circular region with radius Y centered on a point, and arrange the coordinate points within it vertically to obtain a set of pixels; The candidate corner points are refined using the Harris algorithm to obtain the refined corner point coordinates. The background suppression mask is then cropped to obtain the cropped image, using the following formula: , , in, The corner response values ​​are used to filter and eliminate candidate corners whose response values ​​are less than a threshold, resulting in refined corner coordinates. Let be the gradient covariance matrix. and For the image in direction and The partial derivatives in the direction are calculated through the convolution operation. These are constants of the Harris algorithm. For determinant, The trace represents the sum of the diagonal elements of the gradient covariance matrix; Based on the cropped image, the mean local brightness is calculated using the following formula: , in, To crop the image The local average brightness To crop the image The number of pixels, For pixels The V channel luminance value indicates the conversion of the image from the RGB color space to the HSV color space, representing the luminance component. If the local average brightness is greater than the local average brightness threshold, the CLAHE algorithm is used for correction, and the constraint factor in the CLAHE algorithm is optimized using AOA (Arithmetic Optimization Algorithm) to obtain the optimized image. The formula is as follows: , in, For time Limiting factors, For time Limiting factors, and Let be the optimizer acceleration and optimizer probability, both of which are constants. It is a random number. This is the historically optimal limiting factor.

[0032] By thresholding the pixel response values ​​of circular regions, significant feature points in the image are efficiently captured, overcoming the sensitivity problem of traditional edge detection in noisy environments and improving the initial accuracy of corner point localization. The Harris algorithm is introduced to refine candidate corner points, and the corner response values ​​are calculated using the determinant and trace of the gradient covariance matrix to further eliminate weak response points and obtain refined coordinates, significantly reducing the false detection rate. Furthermore, by cropping the mask to generate focused region images, the computational efficiency of subsequent processing is optimized. Based on the cropped image, the local brightness mean is calculated, and the RGB space is converted to HSV to extract the brightness component. This process... It provides an objective basis for quantifying brightness distribution, avoiding uniformity deviations in global brightness adjustment. If the average brightness exceeds the threshold, the CLAHE algorithm is used for adaptive histogram equalization, and the limiting factor is dynamically adjusted through a manual optimization algorithm. This innovative optimization mechanism integrates acceleration and iterative updates of probability parameters, ensuring fine control of contrast enhancement. It not only preserves image details but also suppresses the risk of over-amplifying noise, ultimately obtaining an optimized image. This improves the system's robustness under low-light or high-contrast conditions, achieves more accurate feature enhancement, and provides a high-quality data foundation for preparing input for deep learning models.

[0033] S5, State Detection and Storage Module, is used to input the optimized image into the YOLOv8n model to predict the state probability and label it. The final state and its confidence level are obtained through the FLNN model optimized by the APSO algorithm, recorded in the local database and uploaded to the cloud platform for storage. Specifically, the FLNN model optimized using the APSO algorithm yields the final state and its confidence level, including: The optimized image is pixel normalized, and its state probability, including open, closed and half open, is obtained through the YOLOv8n model. The state confidence is calculated, sorted in descending order, and the maximum state confidence is selected. If it is less than the state confidence threshold, it is marked as uncertain; otherwise, it is marked as certain. Construct an FLNN model, including an input layer, an FLNN layer, and an output layer; Collect historical bucket lid region coordinates and their state probabilities, and divide them into training set and validation set; A particle swarm is randomly initialized, with each particle representing a weight to be optimized in the FLNN model. The weights to be optimized are assigned to the FLNN model, which is then used to classify the training set. The classification accuracy of the training set is calculated, which is the ratio of the number of correctly classified samples in the training set to the total number of classified samples. This ratio is set as the fitness value of the particles. The APSO algorithm is used for iterative optimization, stopping when the maximum number of iterations is reached. The optimal weights are then output and assigned to the FLNN model. The model is then validated using a validation set. The feature vectors labeled as determined states (obtained by the YOLOv8n model) are input into the FLNN model to obtain the final state and its confidence level. These are then recorded in the local database and uploaded to the cloud platform for storage.

[0034] After performing pixel normalization on the optimized image, the YOLOv8n model is used to extract the probability of the lid state and calculate the confidence score. By sorting in descending order and using a threshold to label uncertain states, the efficiency and reliability of the initial classification are significantly improved, avoiding the fuzzy discrimination of traditional models in the half-open state. A functionally linked neural network model is constructed, including an input layer, a functionally linked layer, and an output layer. Historical data is collected and divided into training and validation sets. This design draws on the principle of nonlinear function mapping to enhance the model's ability to express complex features. Particle swarm optimization is randomly initialized to represent weight parameters, and the classification accuracy is calculated as the fitness value. Then, an accelerated particle swarm optimization algorithm is applied for iteration. This innovative optimization framework, through dynamic speed updates and a global search mechanism, efficiently converges to the optimal weight configuration, which not only improves the model's generalization performance but also reduces the risk of overfitting. After confirming the model's stability through the validation set, the feature vector of the determined state is input into the optimized functionally linked neural network, which outputs the final state and its confidence score and records it to the local database and uploads it to the cloud platform. This final fusion step realizes end-to-end state refinement and data persistence, significantly enhancing the system's traceability and scalability.

[0035] This embodiment also provides a deep learning-based method for detecting the state of a trash can lid, including: Collect RGB images and preprocess them to obtain smooth images. Construct a multi-layer image pyramid based on the smooth images to obtain a pyramid image set. Calculate the scaling ratio of each layer of the image, scale the image width and height using linear interpolation, and fill the image to generate a normalized image set. Perform grayscale conversion and perspective correction to obtain a corrected image set. The spatial and temporal gradients of the corrected image set are calculated to generate a gradient field. The fractional-order temporal gradient is obtained by combining the fractional-order gradients. The sub-grid stress and scalar transport constraints of the corrected image set are calculated to obtain the initial constraint residuals. The preprocessed optical flow components of the coarsest layer are calculated, and the stable optical flow components are obtained through iterative optimization. The stable optical flow component is passed through bilinear interpolation and fused with multiple stable optical flow components to obtain the fused optical flow component. The optical flow amplitude is calculated and an optimization function objective is constructed. The Ant Lion algorithm is used for iterative optimization to obtain the optimal threshold. A static mask is generated by combining the optical flow amplitude, the local consistency weight is calculated, and the fusion is adjusted to obtain a multi-scale mask to generate a background suppression image. The candidate corner points of the background suppression mask are calculated using the FAST algorithm, refined using the Harris algorithm to obtain the cropped image, the local brightness mean of the cropped image is calculated, and the CLAHE algorithm with AOA optimization constraint factor is used for correction to obtain the optimized image. The optimized image is input into the YOLOv8n model to predict the state probability and is labeled. The final state and its confidence are obtained through the FLNN model optimized by the APSO algorithm, recorded in the local database and uploaded to the cloud platform for storage.

[0036] This embodiment also provides a computer device applicable to a deep learning-based trash can lid status detection system, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the deep learning-based trash can lid status detection method proposed in the above embodiment.

[0037] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0038] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the deep learning-based method for detecting the state of a trash can lid as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0039] In summary, this invention constructs a multi-layer image pyramid, calculates scaling ratios and normalizes image sets, generates a corrected image set by combining grayscale conversion and perspective correction, generates a gradient field using spatial and temporal gradients, introduces fractional-order temporal gradients and subgrid stress to calculate optical flow components, extracts and refines corner points using FAST and Harris algorithms, identifies the lid state probability using the YOLOv8n model, and optimizes classification accuracy using the FLNN model; thus improving the accuracy and robustness of trash can lid state detection in fast-moving and complex environments, enhancing the accuracy of state detection and its adaptability to different viewpoints and occlusion scenarios.

[0040] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep learning-based trash can lid status detection system, characterized in that: include, The image preprocessing module is used to collect RGB images and preprocess them to obtain smooth images. Based on the smooth images, a multi-layer image pyramid is constructed to obtain a pyramid image set. The scaling ratio of each layer of the image is calculated, the width and height of the image are scaled using linear interpolation, and then filled. A normalized image set is generated, and grayscale conversion and perspective correction are performed to obtain a corrected image set. The image correction and gradient calculation module is used to calculate the spatial and temporal gradients of the corrected image set, generate the gradient field, obtain the fractional-order temporal gradient by combining the fractional order, calculate the sub-grid stress and scalar transport constraints of the corrected image set, obtain the initial constraint residual, calculate the preprocessed optical flow component of the coarsest layer, and obtain the stable optical flow component through iterative optimization. The optical flow processing and mask generation module is used to transfer stable optical flow components through bilinear interpolation, fuse multiple stable optical flow components to obtain fused optical flow components, calculate optical flow amplitude, construct an optimization function objective, use the Ant Lion algorithm for iterative optimization to obtain the optimal threshold, generate a static mask in combination with optical flow amplitude, calculate local consistency weight, adjust the fusion to obtain a multi-scale mask, and generate a background suppression image. The corner detection and image correction module is used to calculate candidate corners of the background suppression mask using the FAST algorithm, refine them using the Harris algorithm to obtain the cropped image, calculate the local brightness mean of the cropped image, and correct it using the CLAHE algorithm with AOA optimization constraint factor to obtain the optimized image. The state detection and storage module is used to input the optimized image into the YOLOv8n model to predict the state probability and label it. The final state and its confidence level are obtained through the FLNN model optimized by the APSO algorithm, recorded in the local database and uploaded to the cloud platform for storage.

2. The deep learning-based trash can lid status detection system as described in claim 1, characterized in that: The process of generating a standardized image set, performing grayscale conversion and perspective correction to obtain a corrected image set includes: Based on the smoothed image, a multi-layer image pyramid is constructed to obtain the pyramid image set. The scaling ratio of each layer of the image is calculated. The width and height of the image are scaled by linear interpolation, and the width and height of the image are filled to obtain the normalized image set. The normalized image set is then subjected to grayscale conversion and perspective correction to obtain the corrected image set.

3. The deep learning-based trash can lid status detection system as described in claim 2, characterized in that: The calculation of the preprocessed optical flow component of the coarsest layer, and the obtaining of a stable optical flow component through iterative optimization, includes: Spatial and temporal gradients are calculated for the calibrated image set to generate a gradient field. Fractional temporal gradients are obtained by combining fractional gradients. Sub-grid stress and scalar transport constraints of the calibrated image set are calculated to obtain initial constraint residuals. The preprocessed optical flow components of the coarsest layer are calculated and iteratively optimized to obtain stable optical flow components, which are then smoothed using a Gaussian kernel function.

4. The deep learning-based trash can lid status detection system as described in claim 3, characterized in that: The calculation of local consistency weights, adjustment and fusion to obtain a multi-scale mask, and generation of a background-suppressed image include: The stable optical flow component of the coarsest layer is transferred to other layers through bilinear interpolation. The stable optical flow components of multiple layers are fused to obtain the fused optical flow component. The optical flow amplitude is calculated, an optimization function objective is constructed, the motion threshold is optimized to obtain the optimal threshold, a static mask is generated, its local consistency weight is calculated, and it is adjusted and fused to obtain a multi-scale mask and a background suppression image.

5. The deep learning-based trash can lid status detection system as described in claim 4, characterized in that: The process involves calculating the mean local brightness of the cropped image and correcting it using the CLAHE algorithm with AOA optimization constraint factor to obtain an optimized image, including: For the background suppression mask, the FAST algorithm is used to calculate candidate corner points, and the Harris algorithm is used to refine them to obtain the refined corner point coordinates. The background suppression mask is then cropped to obtain the cropped image. The local brightness mean is calculated and corrected using the CLAHE algorithm. Finally, the constraint factor in the CLAHE algorithm is optimized using AOA to obtain the optimized image.

6. The deep learning-based trash can lid status detection system as described in claim 5, characterized in that: The FLNN model optimized by the APSO algorithm yields the final state and its confidence level, including: The optimized image is pixel normalized, and its state probability is obtained through the YOLOv8n model. The determined state is obtained by filtering. The feature vector of the determined state is input into the FLNN model to obtain the final state and its confidence level. This is then recorded in the local database and uploaded to the cloud platform for storage.

7. The deep learning-based trash can lid status detection system as described in claim 6, characterized in that: The process of collecting RGB images and preprocessing them to obtain a smoothed image includes: A high-definition camera is deployed vertically above the top of the trash can to collect RGB images and calculate the image sharpness score. If the image sharpness score is less than the image sharpness score threshold, it is marked as a blurry image, and RGB images are collected again. Gaussian filtering is used to denoise the RGB images to obtain a smooth image.

8. A deep learning-based method for detecting the state of a trash can lid, based on the deep learning-based trash can lid state detection system according to any one of claims 1 to 7, characterized in that: include, Collect RGB images and preprocess them to obtain smooth images. Construct a multi-layer image pyramid based on the smooth images to obtain a pyramid image set. Calculate the scaling ratio of each layer of the image, scale the image width and height using linear interpolation, and fill the image to generate a normalized image set. Perform grayscale conversion and perspective correction to obtain a corrected image set. The spatial and temporal gradients of the corrected image set are calculated to generate a gradient field. The fractional-order temporal gradient is obtained by combining the fractional-order gradients. The sub-grid stress and scalar transport constraints of the corrected image set are calculated to obtain the initial constraint residuals. The preprocessed optical flow components of the coarsest layer are calculated, and the stable optical flow components are obtained through iterative optimization. The stable optical flow component is passed through bilinear interpolation and fused with multiple stable optical flow components to obtain the fused optical flow component. The optical flow amplitude is calculated and an optimization function objective is constructed. The Ant Lion algorithm is used for iterative optimization to obtain the optimal threshold. A static mask is generated by combining the optical flow amplitude, the local consistency weight is calculated, and the fusion is adjusted to obtain a multi-scale mask to generate a background suppression image. The candidate corner points of the background suppression mask are calculated using the FAST algorithm, refined using the Harris algorithm to obtain the cropped image, the local brightness mean of the cropped image is calculated, and the CLAHE algorithm with AOA optimization constraint factor is used for correction to obtain the optimized image. The optimized image is input into the YOLOv8n model to predict the state probability and is labeled. The final state and its confidence are obtained through the FLNN model optimized by the APSO algorithm, recorded in the local database and uploaded to the cloud platform for storage.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the deep learning-based trash can lid state detection method as described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the deep learning-based trash can lid state detection method as described in claim 8.

Images