Family dangerous object identification method and system based on improved YOLOv8

Abstract

The invention discloses a family dangerous object identification method and system based on improved YOLOv8, and belongs to the technical field of computer vision and edge computing. Aiming at the problems of long tail distribution of dangerous objects in a family scene, missing detection of small targets, false alarm of similar objects and limited computing power of edge equipment, three improvements are carried out on a YOLOv8s model: a class prototype adaptive mechanism is introduced, and the long tail class discrimination capability is enhanced; a multi-scale feature enhancement module is added, and the small target feature extraction effect is improved; a coordinate attention mechanism is embedded, and the target positioning precision is enhanced. Furthermore, the detection result is subjected to semantic review by using a lightweight multi-modal large model, so that false alarms are inhibited. And efficient deployment of edge equipment is realized through a model quantification technology. Experiments show that the mAP50: 95 is improved by 2.84% on a self-built data set, and the recognition accuracy and the system reliability are remarkably improved while low delay is kept.

Description

A method and system for identifying hazardous materials in the home based on an improved YOLOv8 Technical Field

[0001] This invention relates to the fields of computer vision and edge computing technology, specifically to a method and system for identifying hazardous materials in the home based on an improved YOLOv8. Background Technology

[0002] With the aging of society and the trend towards smaller family structures, safety hazards in the home environment are receiving increasing attention. Statistics show that accidental injuries occurring in the home account for a significant proportion of all injury-related medical cases, with injuries involving sharp objects, high temperatures, and fragile items being particularly prevalent among vulnerable groups such as the elderly and children. Therefore, developing technologies that can automatically and in real-time identify hazardous materials in the home environment is of significant practical importance for preventing accidents and ensuring home safety.

[0003] In recent years, deep learning-based object detection technologies, especially one-stage detectors represented by the YOLO series, have been widely used in fields such as security monitoring and autonomous driving due to their good balance between accuracy and speed. YOLOv8, as one of the latest evolutions in this series, has achieved excellent performance on general object detection datasets thanks to its efficient network structure and training strategy. However, when directly applied to the specific scenario of identifying hazardous materials in the home, it still faces a series of unique and severe technical challenges: the safety and monitoring of the home environment, especially the protection of vulnerable groups such as the elderly and children, is a focus of social concern. According to publicly available monitoring data, accidental injuries occurring in the home account for a high proportion of all injury cases, with incidents involving sharp objects, burns, falls, and other dangerous factors being common. Most of these accidents are preventable; therefore, using technological means to automatically and in real-time identify and warn of dangerous items in the home has significant social implications and application value.

[0004] In recent years, deep learning-based object detection technology has made significant progress. Among them, the YOLO series of algorithms, with its good balance between speed and accuracy, has become one of the mainstream frameworks in the field of real-time detection. YOLOv8, as the latest representative model of this series, performs excellently on general object detection tasks. However, when directly applied to the specific scenario of complex and dynamic household hazard identification, it still faces a series of severe technical challenges: First, the long-tail distribution of data and intra-class diversity are prominent issues. Household hazard categories are naturally unbalanced. For example, the frequency of dangerous states or items such as "open flame" and "broken glass" is much lower than that of common items such as "knives" and "scissors," resulting in insufficient learning of tail categories by the model, weak feature representation, and blurred decision boundaries, which easily leads to missed detections or excessive sensitivity to background perturbations.

[0005] Secondly, detecting small-scale targets and fine-grained structures is challenging. Many danger signals in home scenes, such as knife tips, glass cracks, and the edges of small flames, often appear as small targets in images. After multiple layers of downsampling by deep neural networks, the semantic and detailed information of these small targets is easily attenuated or lost, resulting in low recall and poor localization accuracy.

[0006] Secondly, environmental interference is severe and highly similar. Home lighting conditions are complex and variable, including low light, backlighting, specular reflection, and smoke / vapor obstruction, leading to degraded image quality and difficulties in feature extraction. More importantly, there are numerous pairs of objects that appear similar but pose vastly different risks, such as toy knives versus real knives, or decorative candlesticks versus open flames. Detection systems relying solely on visual appearance confidence are prone to false alarms in such situations, and their decision-making process lacks interpretable semantic basis, reducing the system's reliability and credibility. Finally, practical applications face strict resource constraints. Terminal devices in home environments typically have limited computing power, small storage space, and are power-sensitive. This necessitates highly lightweight algorithm models while maintaining inference efficiency; traditional complex models or unoptimized deployment schemes are insufficient for practical application needs.

[0007] Currently, despite numerous studies in the field of object detection, most improvements focus on general performance enhancements or optimize only one of the aforementioned challenges (such as small object detection or long-tail recognition). There is a lack of collaborative and systematic solutions to address the multiple challenges (long tail, small objects, similar interference, complex environment, and resource constraints) specific to the scenario of household hazard identification. Especially after the detection stage, there is a lack of mechanisms to utilize high-level semantic information for result verification and interpretation, making it difficult to fundamentally reduce the false alarm rate. Therefore, it is necessary to design a household hazard identification method and system based on an improved YOLOv8 to solve these problems. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for identifying hazardous materials in the home based on an improved YOLOv8, in order to solve at least one technical problem existing in the prior art.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a method for identifying hazardous materials in the home based on improved YOLOv8, comprising the following steps: S1: acquiring real-time images of a home scene; S2: inputting the real-time images into a pre-trained improved YOLOv8s detection model to obtain preliminary detection results including output boxes, categories, and confidence levels; the improved YOLOv8s detection model is obtained by sequentially integrating a class prototype self-adaptation mechanism, a multi-scale feature enhancement module, and a coordinate attention mechanism module on the basis of a baseline YOLOv8s model; S3: performing semantic consistency and scene risk analysis on the preliminary detection results based on a lightweight deployed large language model to generate semantic verification results; S4: performing a fusion judgment based on the confidence level of the preliminary detection results and the semantic verification results to output the final hazard determination result.

[0010] Preferably, in S2, the training method of the improved YOLOv8s detection model includes: S21: constructing a training dataset containing multiple categories of household hazardous materials; S22: introducing a class prototype self-adaptation mechanism in the classification branch of the detection head of the YOLOv8s model, improving the robustness of long-tail category decisions through the fusion of prototype scoring and convolutional logit; wherein, in the class prototype self-adaptation mechanism, classification branch features are extracted at each detection scale P3 / P4 / P5, aligned to dimension d by 1×1 convolution and L2 normalized, using the mean of the features of the training set classification as the initial prototype, and normalized to a unit hypersphere (Equation 3-1). In the formula, This represents the embedding vector of the j-th sample in the i-th class in the classification branch of the detection head. This represents the number of samples in class i; the initial position of class i is obtained by averaging the embedding vectors of all samples in class i. The results are then normalized after summation; normalization is maintained throughout the training period to mitigate the convergence bias caused by the imbalance between the first and last classes; and τ cosine with scale parameter is used for scoring during the inference and training phases (Equation (3-2)). ; ; where τ is used to align the magnitude of the prototype score with the convolution class logit, and then linearly fuse it with the original convolution class logit in the logit domain; S23: Introduce a multi-scale feature enhancement module in the YOLOv8s model, extract and fuse features of different scales through a parallel multi-branch structure; S24: Introduce a coordinate attention mechanism module in the YOLOv8s model, and recalibrate the feature channels through directional attention weights; S25: Use the training dataset to perform end-to-end training on the YOLOv8s model that integrates the above improvements.

[0011] Preferably, S3 specifically includes: S31: selecting candidate hazardous targets to be reviewed from the preliminary detection results according to a preset confidence threshold; S32: constructing a structured input of image and text by combining the cropped image of each candidate target, its category label, and a preset risk assessment prompt template; S33: submitting the structured input to the large language model to obtain its output of a deterministic judgment result on whether the target constitutes an actual risk, as the semantic review result.

[0012] Preferably, before S2, a model deployment preparation step is included: converting the trained improved YOLOv8s detection model into an intermediate representation format; using a calibration dataset covering multiple scenarios, performing post-training quantization on the model to generate a low-precision inference model for deployment on edge devices.

[0013] A home hazard identification system based on improved YOLOv8, the system comprising: an image input module for acquiring real-time images of a home scene; an improved YOLOv8s detection module loaded with a pre-trained improved YOLOv8s detection model for processing the real-time images and outputting preliminary detection results; the improved YOLOv8s detection model integrating a prototype-based self-adaptation unit, a multi-scale feature enhancement unit, and a coordinate attention mechanism unit; a semantic review module with a built-in lightweight large language model for performing semantic-level risk analysis on the preliminary detection results and outputting review opinions; and a decision fusion module for making a comprehensive judgment based on the confidence level of the preliminary detection results and the output of the semantic review module, generating a final hazard alarm or decision instruction.

[0014] Preferably, the class prototype self-adaptation unit in the improved YOLOv8s detection module is used to calculate the similarity between the feature vector and the stored class prototype vectors during model inference, and to perform weighted fusion of the similarity with the original output of the classification convolutional layer.

[0015] Preferably, the multi-scale feature enhancement unit in the improved YOLOv8s detection module is a residual parallel structure, which includes at least four parallel depthwise separable convolutional branches with different receptive fields and a global branch.

[0016] Preferably, the coordinate attention mechanism unit in the improved YOLOv8s detection module encodes spatial information by performing global pooling along the height and width directions respectively, and generates a direction-aware attention map for channel reweighting.

[0017] Preferably, the system further includes a model optimization deployment module for converting the trained improved YOLOv8s detection model into a low-bit-precision optimized format suitable for edge computing devices.

[0018] Compared with existing technologies, the present invention provides a method and system for identifying hazardous objects in the home based on an improved YOLOv8, which has the following beneficial effects: 1. The present invention significantly improves detection accuracy, especially for long-tail categories and small targets: it enhances the detection capability of small targets and long-tail data in the home environment, suppresses interference in different scenarios, reduces the false detection rate, effectively integrates multi-scale feature information, and, with the support of semantic verification, can understand and judge the scene environment, thus reducing the false alarm rate. Experiments have verified the effectiveness of the improved algorithm and architecture, with a 2.84% improvement in mAP50:95. Compared with mainstream target detection algorithms, the algorithm in this paper has fewer parameters, lower model complexity, lower overall system overhead, and higher detection accuracy, making it suitable for the deployment and implementation of edge devices in the home environment.

[0019] 2. This invention effectively suppresses false alarms caused by appearance similarity and complex environments: The invention introduces a semantic verification module based on the lightweight multimodal large model Qwen2.5-VL at the back end of the detection process. This module can perform scene context and semantic consistency analysis on detected hazardous candidate objects (such as "knife"), thereby effectively distinguishing between objects that are similar in appearance but pose different risks, such as "toy knife" and "real knife". This two-level verification mechanism of "detection + semantics" suppresses false alarms that may arise from relying solely on visual confidence at a high-level semantic level, enhancing the interpretability and credibility of the system's decisions. Attached Figure Description

[0020] Figure 1 is a flowchart of the household hazardous materials identification method based on the improved YOLOv8 according to the present invention; Figure 2 is a YOLOv8 structure diagram of the household hazardous materials identification method and system based on the improved YOLOv8 according to the present invention; Figure 3 is a diagram of the CA module combination of the household hazardous materials identification method and system based on the improved YOLOv8 according to the present invention; Figure 4 is a structure diagram of the improved YOLOv8s of the household hazardous materials identification method and system based on the improved YOLOv8 according to the present invention; Figure 5 is a prediction result diagram of the household hazardous materials identification method and system based on the improved YOLOv8 according to the present invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0022] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0023] This embodiment provides a method for identifying hazardous materials in the home based on an improved YOLOv8 model, comprising the following steps: S1: acquiring real-time images of a home scene; S2: inputting the real-time images into a pre-trained improved YOLOv8 detection model to obtain preliminary detection results including output boxes, categories, and confidence levels; the improved YOLOv8 detection model is obtained by sequentially integrating a class prototype self-adaptation mechanism, a multi-scale feature enhancement module, and a coordinate attention mechanism module on the basis of a baseline YOLOv8 model; S3: performing semantic consistency and scene risk analysis on the preliminary detection results based on a lightweight deployment large language model to generate semantic verification results; S4: performing a fusion judgment based on the confidence level of the preliminary detection results and the semantic verification results to output the final hazard determination result.

[0024] In S2, the training method of the improved YOLOv8s detection model includes: S21: constructing a training dataset containing multiple household hazardous material categories; S22: introducing a class prototype self-adaptation mechanism in the classification branch of the detection head of the YOLOv8s model, and improving the robustness of long-tail category decision by fusing prototype scoring with convolutional logit.

[0025] S23: Introduce a multi-scale feature enhancement module into the YOLOv8s model, which extracts and fuses features of different scales through a parallel multi-branch structure; S24: Introduce a coordinate attention mechanism module into the YOLOv8s model, which recalibrates feature channels through directional attention weights; S25: Use the training dataset to perform end-to-end training on the YOLOv8s model that integrates the above improvements.

[0026] S3 specifically includes: S31: Based on a preset confidence threshold, selecting candidate hazardous targets to be reviewed from the preliminary detection results; S32: Constructing a structured input of image and text by cropping the image of each candidate target, its category label, and a preset risk assessment prompt template; S33: Submitting the structured input to the large language model to obtain its output of a deterministic judgment result on whether the target constitutes an actual risk, as the semantic review result.

[0027] Prior to S2, a model deployment preparation step is included: converting the trained improved YOLOv8s detection model into an intermediate representation format; and performing post-training quantization on the model using a calibration dataset covering multiple scenarios to generate a low-precision inference model for deployment on edge devices.

[0028] A home hazard identification system based on improved YOLOv8, the system comprising: an image input module for acquiring real-time images of a home scene; an improved YOLOv8s detection module loaded with a pre-trained improved YOLOv8s detection model for processing the real-time images and outputting preliminary detection results; the improved YOLOv8s detection model integrating a prototype-based self-adaptation unit, a multi-scale feature enhancement unit, and a coordinate attention mechanism unit; a semantic review module with a built-in lightweight large language model for performing semantic-level risk analysis on the preliminary detection results and outputting review opinions; and a decision fusion module for making a comprehensive judgment based on the confidence level of the preliminary detection results and the output of the semantic review module, generating a final hazard alarm or decision instruction.

[0029] The class prototype self-adaptation unit in the improved YOLOv8s detection module is used to calculate the similarity between the feature vector and the stored class prototype vectors during model inference, and to perform weighted fusion of the similarity with the original output of the classification convolutional layer.

[0030] The multi-scale feature enhancement unit in the improved YOLOv8s detection module is a residual parallel structure, which includes at least four parallel depthwise separable convolutional branches with different receptive fields and a global branch.

[0031] The coordinate attention mechanism unit in the improved YOLOv8s detection module encodes spatial information by performing global pooling along the height and width directions respectively, and generates a direction-aware attention map for channel reweighting.

[0032] Referring to Figure 2, the specific implementation of the present invention mainly includes the following stages: data preparation and model training stage: construct and label a household hazardous materials dataset, and train and validate the improved YOLOv8s model based on this dataset.

[0033] Model optimization and conversion stage: The trained model is exported to an intermediate format and quantized and accelerated using an inference optimization framework to generate a deployment model suitable for edge devices.

[0034] Edge deployment and inference phase: The optimized model is loaded on an edge device (such as a micro PC) and real-time images from the camera are received to detect hazardous objects.

[0035] Semantic review and decision-making stage: The detection results are fed into a lightweight multimodal large model for semantic consistency judgment, and the results of the two are combined to generate the final alarm or instruction.

[0036] The implementation phases described above are detailed below: First, a dedicated dataset of hazardous materials for the home needs to be constructed. In one embodiment of this invention, approximately 7000 images were collected and labeled, encompassing the following five categories: broken-glass, hammer, knife, scissors, and person. Single-object / multi-object instance bounding boxes were used for annotation. The data was stratified by category, with training / validation / testing at 70% / 20% / 10%, and a fixed random seed to ensure reproducibility; the validation and test sets contained at least 50 images per category.

[0037] This invention improves upon YOLOv8s as a baseline model, as shown in Figure 4. The integration of the specific improvement modules is as follows: Implementation of the class prototype self-adaptation module: Class prototype self-adaptation is introduced into the classification branch of the YOLOv8s detection head. Without changing the detection head and post-processing interface, the robustness of long-tail category decisions is improved through the fusion of prototype scoring and convolutional logit. Specifically, classification branch features are extracted at each detection scale (P3 / P4 / P5), aligned to dimension d by 1×1 convolution, and L2 normalized. The mean feature value of this class in the training set is used as the initial prototype and normalized to a unit hypersphere (Equation 3-1). In the formula, This represents the embedding vector of the j-th sample in the i-th class in the classification branch of the detection head. This represents the number of samples in class i. The initial position of class i is obtained by averaging the embedding vectors of all samples in class i. The results are then normalized after summation. Normalization is maintained throughout the training period to mitigate the convergence bias caused by head-to-tail class imbalance. Cosine τ with a scale parameter is used for scoring during both the inference and training phases (Equation (3-2)). ; ; where τ is used to align the magnitude of the prototype score with the convolutional class logit, and then linearly fuse it with the original convolutional class logit in the logit domain (weight α is selected in the validation set, Equation (3-3)); the fusion result is directly fed into the existing Focal loss, and the regression, DFL branch and inference post-processing remain unchanged, with no intrusive impact on OpenVINO export and deployment. The calculation increment is approximately one 1×1 convolution and several inner products, and the increase in GMACs and end-to-end latency is negligible at 640×640. Under the experimental settings aligned with the baseline, a set of class prototypes are shared in P3–P5 and fused at the end of the classification. Params and latency p50 / p90 are given to quantify the cost and evaluate the scene recognition performance.

[0038] Implementation of the Multi-Scale Feature Enhancement (MSFE) module: The MSFE module is designed as a pluggable unit with a residual structure. As shown in Figure 4, it is seamlessly integrated before the Backbone output (C3, C4, after SPPF) and the Neck's upsample-concat operation.

[0039] This module contains four parallel branches: Branch 1: Depthwise Separable Convolution (DWConv), with a kernel size of 3×3, is used to capture fine-grained details.

[0040] Branch 2: Depthwise separable convolution with a kernel size of 5×5, used to capture mesoscale features.

[0041] Branch 3: Depthwise separable convolution with a kernel size of 7×7, used to capture large-scale context.

[0042] Branch 4: 1×1 convolution, serving as the global information branch.

[0043] The outputs of the four branches are concatenated along the channel dimension and then fused using a 1x1 convolution. The final output is then residually connected to the module's input. This design significantly enhances the network's feature extraction capabilities for multi-scale targets, especially small targets.

[0044] Implementation of the Coordinate Attention (CA) module: The CA module is mainly integrated into the path responsible for detecting small targets in the neck network. Specifically, as shown in Figure 3, it is inserted into the path that generates the P3 feature map, i.e., after Upsample→Concat(E3)→C2f and before entering the next Conv→Concat→C2f. The workflow of this module is shown in Figure 3: Global average pooling is performed on the P3 feature Y along the height and width directions to obtain C×H×1 and C×1×W directional encodings; after sharing 1×1 embedding and SiLU activation, A_h and A_w are generated, and then broadcast back to the original size and multiplied element-wise with Y, avoiding the position loss caused by spatial dimension compression in traditional channel attention.

[0045] The lightweight modification of "prototype-based self-adaptation + MSFE + CA" improves the discriminability of small targets and fine edges while maintaining the same stride and resolution, and the incremental changes in computation and latency are controllable. The improved YOLOv8s network structure is shown in Figure 4.

[0046] In one specific embodiment, the model training is configured as follows: To verify the effectiveness of the improved YOLOv8s algorithm and the Qwen2.5 VL dual-validation household hazardous materials identification system, this system evaluates the method and deployment scheme in the hazardous materials identification scenario: On a self-built 5-class dataset, with YOLOv8s@640 as the baseline, a consistent training strategy is adopted, the training period is 60, the initial learning rate is set to 0.003, the batch size is set to 32, AMP is enabled, and a fixed random seed is used for comparison; Precision / Recall / F1, mAP@0.5 / 0.5:0.95, AP by class and confusion matrix are evaluated, and the model complexity (Params / GFLOPs) and deployment performance are reported.

[0047] Model Export: After training, use the export function provided by Ultralytics to convert the improved YOLOv8s model (.pt format) to ONNX format. Ensure that the export includes the logic for preprocessing (such as letterboxing) and postprocessing (such as nonmaximum suppression, NMS) or explicitly specifies their parameters.

[0048] Model optimization and quantization: ONNX was converted to IR format using OpenVINO Model Optimizer, and then INT8 quantization was performed using the Post-TrainingQuantization tool. The calibration set was selected from a multi-sample subset of the training set.

[0049] Without altering the detection head and post-processing interface, the improved YOLOv8s was deployed and accelerated on the edge using OpenVINO Runtime: After training, ONNX was exported from Ultralytics, converted to IR via ModelOptimizer, and FP16 was generated; while maintaining accuracy, PTQ (accuracy-aware, weights channel-wise symmetric, activations tensor-wise) was used to obtain INT8. Evaluation used a CPU plugin with a batch size of 1, and preprocessing / postprocessing was consistent with the training end; after sufficient warm-up, end-to-end p50 / p90 (including preprocessing, inference, NMS, excluding I / O) was statistically analyzed and compared horizontally with FP16 and INT8. To suppress false alarms caused by similar appearances and contextual ambiguity, a minimal semantic verification is introduced: After NMS, hazard candidates with a confidence level not lower than t_det (calibrated once by the validation set and fixed during the testing phase) are minimally pruned and normalized. They are then constructed into structured image-text input using a fixed prompt template and candidate labels, and submitted to Qwen2.5-VL for YES / NO judgment under deterministic settings (temperature=0, no sampling, fixed output upper limit). Only the top-k candidates for each image are triggered with linear constraints on the worst-case latency. Both configurations (detection only vs. detection + verification) maintain the same data partitioning and threshold caliber. The above process does not introduce custom operators, maintains the tensor shape and interface unchanged, and requires no additional adaptation for export and edge inference. To verify the effectiveness of the improved YOLOv8s algorithm in identifying hazardous household items, an ablation experiment was conducted on the improved model using YOLOv8s as the baseline model, listing mAP50, mAP50:95, Params, and end-to-end latency.

[0050] MSFECA class prototype self-adaptive PRF1mAP50mAP50:95ParamsLat_p50Lat_p9092.4885.0588.6190.9168.3411.1386.7087.032P93.5084.8788.9791.4469.9611.2877.7768.131PP93.6485.2489.2490.9770.8311.2917.8878.205PPP93.7385.0689.1990.8871.1811.2918.4128.879 Table 1 shows that, in terms of complexity and efficiency, the baseline model has 11.138M parameters (Params). Using the median mAP weights, the end-to-end latency with bs=1 was measured as follows: Lat_p50 = 6.708ms, Lat_p90 = 7.032ms (hardware and thread settings are the same as in the deployment section). The single MSFE significantly improves mAP50:95 by 1.62% compared to the baseline, with a slight increase in latency, but still within the lightweight range. This indicates that, without changing the detector head, multi-scale fusion effectively improves the separability of small targets and weakly textured structures. MSFE+CA further improves mAP50:95 by 0.87% compared to the single MSFE, simultaneously improving P, R, and F1, with manageable latency overhead. Coordinate attention is provided in the P3 layer for alignment geometric priors, suppressing near-background interference. Prototype self-adaptation continues to deliver stable gains on top of MSFE+CA, increasing mAP50:95 by 0.35%, at the cost of a slight latency increase, p50≈+0.52ms. This result demonstrates that prototype self-adaptation can improve overall robustness without fundamental changes to the model through aggregation and scoring recalibration of similar sample clusters, with a manageable impact on throughput. Longitudinal comparison shows that compared to baseline detection only, MSFE+CA+prototype achieves a cumulative improvement of approximately 2.84% in mAP50:95, while maintaining a relatively constant parameter scale and latency within the acceptable range for lightweight models. Overall, this validates the progressive improvement path of "multi-scale enhancement → geometric attention → prototype recalibration" for enhancing the YOLOv8s model, resulting in higher detection accuracy.

[0051] We exported OpenVINO FP16 / INT8 deployment versions for the final model. Without modifying NMS and the output interface, FP16 achieved the same accuracy as the PyTorch version on the validation set; INT8 showed only a negligible decrease in accuracy, verifying the feasibility and robustness of quantization deployment.

[0052] As shown in Table 2, INT8 has an mAP50:95 of 68.5%, while FP16 has an mAP50:95 of 68.8%. FP16 has higher accuracy than INT8 and is more worthwhile to deploy. It maintains accuracy within a controllable engineering cost and significantly improves latency compared to PyTorch, thus meeting deployment requirements.

[0053] PRmAP50mAP50:95p50p90INT893.484.189.468.512.61479.32FP1693.484.689.568.810.49678.02 Table 2 also introduces a simplified semantic review process to suppress false alarms caused by similar appearances and contextual ambiguity: when a hazard candidate is in the NMS and the confidence level is not lower than t_det=0.5, it is sent to Qwen2.5-VL for deterministic binary classification (YES / NO). The review is only triggered for the top-k candidates for each image. Figure 5 shows the results after prediction. Since a hazardous item was identified, Qwen2.5-VL performed a secondary judgment, as shown in Figure 5, outputting a description of the scene. It outputs "yes" for whether it is hazardous and "yes" for whether an alarm should be triggered. The entire engineering architecture is complete and the latency is controlled within a manageable range.

[0054] This invention proposes a YOLOv8s detection algorithm that incorporates MSFE+CA attention mechanism and prototype-based self-adaptation. This algorithm improves the detection capability of small targets and long-tail data in home environments, suppresses interference in different scenarios, reduces false detection rate, effectively integrates multi-scale feature information, and, with the support of semantic verification, can understand and judge the scene environment, further reducing the false alarm rate. Experiments verify the effectiveness of the improved algorithm and architecture, achieving a 2.84% improvement in mAP50:95. Compared with mainstream target detection algorithms, this invention's algorithm has fewer parameters, lower model complexity, lower overall system overhead, and higher detection accuracy, making it suitable for the deployment and implementation of edge devices in home environments.

[0055] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying hazardous materials in the home based on an improved YOLOv8, characterized in that, Includes the following steps: S1: Acquire real-time images of a home scene; S2: Input the real-time images into a pre-trained improved YOLOv8s detection model to obtain preliminary detection results including output boxes, categories, and confidence scores; the improved YOLOv8s detection model is obtained by sequentially integrating a class prototype self-adaptation mechanism, a multi-scale feature enhancement module, and a coordinate attention mechanism module on the basis of a baseline YOLOv8s model; S3: Based on a lightweight deployed large language model, perform semantic consistency and scene risk analysis on the preliminary detection results to generate semantic verification results; S4: Based on the confidence level of the preliminary detection results and the semantic verification results, a final danger determination result is output.

2. The method for identifying hazardous materials in the home based on an improved YOLOv8 according to claim 1, characterized in that: In S2, the training method of the improved YOLOv8s detection model includes: S21: constructing a training dataset containing multiple categories of household hazardous materials; S22: introducing a class prototype self-adaptation mechanism in the classification branch of the detection head of the YOLOv8s model, improving the robustness of long-tail category decisions through the fusion of prototype scoring and convolutional logit; wherein, in the class prototype self-adaptation mechanism, classification branch features are extracted at each detection scale P3 / P4 / P5, aligned to dimension d by 1×1 convolution and L2 normalized, using the mean of the training set classification features as the initial prototype, and normalized to a unit hypersphere (Equation 3-1). In the formula, This represents the embedding vector of the j-th sample in the i-th class in the classification branch of the detection head. This represents the number of samples in class i; the initial position of class i is obtained by averaging the embedding vectors of all samples in class i. The results are then normalized after summation; normalization is maintained throughout the training period to mitigate the convergence bias caused by the imbalance between the first and last classes; and τ cosine with scale parameter is used for scoring during the inference and training phases (Equation (3-2)). ； ; where τ is used to align the magnitude of the prototype score with the convolution class logit, and then linearly fuse it with the original convolution class logit in the logit domain; S23: Introduce a multi-scale feature enhancement module in the YOLOv8s model, extract and fuse features of different scales through a parallel multi-branch structure; S24: Introduce a coordinate attention mechanism module in the YOLOv8s model, and recalibrate the feature channels through directional attention weights; S25: Use the training dataset to perform end-to-end training on the YOLOv8s model that integrates the above improvements.

3. The method for identifying hazardous materials in the home based on an improved YOLOv8 according to claim 1, characterized in that, S3 specifically includes: S31: Based on a preset confidence threshold, selecting candidate hazardous targets to be reviewed from the preliminary detection results; S32: Constructing a structured input of image and text by cropping the image of each candidate target, its category label, and a preset risk assessment prompt template; S33: Submitting the structured input to the large language model to obtain its output of a deterministic judgment result on whether the target constitutes an actual risk, as the semantic review result.

4. The method for identifying hazardous materials in the home based on an improved YOLOv8 according to claim 1, characterized in that: Prior to S2, a model deployment preparation step is included: converting the trained improved YOLOv8s detection model into an intermediate representation format; and performing post-training quantization on the model using a calibration dataset covering multiple scenarios to generate a low-precision inference model for deployment on edge devices.

5. A household hazardous materials identification system based on an improved YOLOv8, characterized in that, The system for implementing the method of any one of claims 1-4 comprises: an image input module for acquiring real-time images of a home scene; an improved YOLOv8s detection module loaded with a pre-trained improved YOLOv8s detection model for processing the real-time image and outputting preliminary detection results; the improved YOLOv8s detection model integrating a prototype-based self-adaptation unit, a multi-scale feature enhancement unit, and a coordinate attention mechanism unit; a semantic review module with a built-in lightweight large language model for performing semantic-level risk analysis on the preliminary detection results and outputting review opinions; and a decision fusion module for making a comprehensive judgment based on the confidence level of the preliminary detection results and the output of the semantic review module, and generating a final danger alarm or decision instruction.

6. A household hazardous materials identification system based on an improved YOLOv8 according to claim 5, characterized in that: The class prototype self-adaptation unit in the improved YOLOv8s detection module is used to calculate the similarity between the feature vector and the stored class prototype vectors during model inference, and to perform weighted fusion of the similarity with the original output of the classification convolutional layer.

7. A household hazardous materials identification system based on an improved YOLOv8 according to claim 5, characterized in that: The multi-scale feature enhancement unit in the improved YOLOv8s detection module is a residual parallel structure, which includes at least four parallel depthwise separable convolutional branches with different receptive fields and a global branch.

8. A household hazardous materials identification system based on an improved YOLOv8 according to claim 5, characterized in that: The coordinate attention mechanism unit in the improved YOLOv8s detection module encodes spatial information by performing global pooling along the height and width directions respectively, and generates a direction-aware attention map for channel reweighting.

9. A household hazardous materials identification system based on an improved YOLOv8 according to claim 5, characterized in that: The system also includes a model optimization and deployment module, which converts the trained improved YOLOv8s detection model into a low-bit-precision optimized format suitable for edge computing devices.

Images