Charging station abnormal article long-term accumulation detection method based on mercuration edge device

Abstract

The invention belongs to the technical field of target detection, and particularly relates to a charging station abnormal article long-term accumulation detection method based on sublimation edge equipment. The method comprises the following steps: constructing a charging station general target detection model, and formulating a charging station target object classification standard; performing adaptive optimization on the charging station general target detection model on the mercuration edge device to obtain a charging station target detection model adaptive to the mercuration edge device; identifying targets by using a charging station target detection model adaptive to the mercuration edge device, and classifying the identified targets according to a formulated charging station target object classification standard; and judging abnormal articles accumulated in the charging station for a long time based on a classification result. According to the invention, accurate detection of long-term accumulation of abnormal articles in the charging station is realized, and powerful support is provided for safety management of the charging station.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and particularly relates to a method for long-term accumulation detection of abnormal items in a charging station based on Ascend edge devices. Background Art

[0002] With the vigorous development of the new energy vehicle industry, the construction scale of charging infrastructure has shown an exponential growth trend. However, the fire hazards and environmental pollution problems caused by long-term accumulation of non-station area items in charging stations have become increasingly prominent, and there is an urgent need to build an intelligent and highly reliable safety detection method. Traditional video surveillance solutions based on cloud centralized processing face significant challenges: the high-frequency transmission of large-scale video data places strict requirements on network bandwidth, and the centralized processing mode leads to low utilization rate of computing resources and limited system scalability.

[0003] In the field of abnormal item target detection, YOLO series and R-CNN (Region-based Convolutional Neural Networks) series algorithms are currently widely used. These algorithms are improved by introducing new modules, replacing or deleting original modules in the network to adapt to different application scenarios and improve detection accuracy and efficiency. However, for the long-term accumulation detection of abnormal items in the charging station scenario, existing target detection technologies mainly focus on the existence of targets and it is difficult to accurately judge the long-term accumulation state of abnormal items. Summary of the Invention

[0004] In order to overcome the problems in the prior art, the present invention proposes a method for long-term accumulation detection of abnormal items in a charging station based on Ascend edge devices.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] The present invention provides a method for long-term accumulation detection of abnormal items in a charging station based on Ascend edge devices, including the following steps:

[0007] Construct a general target detection model for the charging station and formulate a classification standard for the target objects in the charging station;

[0008] Adapt and optimize the general target detection model for the charging station on the Ascend edge device to obtain a charging station target detection model adapted to the Ascend edge device;

[0009] Use the charging station target detection model adapted to the Ascend edge device to identify targets, classify the identified targets according to the formulated classification standard for the target objects in the charging station; and discriminate the abnormal items that have been long-term accumulated in the charging station based on the classification results.

[0010] Further, the construction of the general object detection model for charging stations includes:

[0011] Collect the real-time video stream of the charging station, obtain image frames through video processing technology, pre-label the collected image frames of the charging station based on the pre-trained YOLOv8 object detection model, and then use the annotation tool to fine-tune and supplement the pre-labeled regions of interest to construct a private general object detection dataset for charging stations.

[0012] Based on the constructed private general object detection dataset for charging stations, adapt and iteratively upgrade the pre-trained YOLOv8 general object detection model under the charging station scenario to obtain the general object detection model for charging stations.

[0013] Further, the formulation of the classification standard for objects in the charging station includes: normal object categories, abnormal object categories, and other categories, where the other categories are other detected objects and objects with a confidence level lower than the preset threshold.

[0014] Further, the adaptation and optimization of the general object detection model for charging stations on the Ascend edge device includes:

[0015] Convert the weight file of the general object detection model for charging stations to ONNX, and use the tensor compiler under the Ascend CANN system to convert ONNX to the OM format to adapt to the hardware architecture of the Ascend edge device.

[0016] Use InferSession in the ais_bench inference toolkit for model loading and inference to achieve offline inference on the Ascend edge device.

[0017] Further, it also includes: preprocessing the collected image frames of the charging station on the Ascend edge device and saving the preprocessed image frames on the Ascend edge device, where the preprocessing includes image blur judgment and image data format processing.

[0018] Further, the image blur judgment uses the Laplacian operator method based on gradients to evaluate the clarity by calculating the second derivative of the image grayscale value.

[0019] Further, the image data format processing is to format the names of the collected image frames of the charging station, and the naming method is the device number and the timestamp of the intercepted moment.

[0020] Further, classify the identified objects according to the formulated classification standard for objects in the charging station, and discriminate the abnormal objects that have been accumulated in the charging station for a long time based on the classification results, including:

[0021] Eliminate normal item categories according to the formulated classification standard for target objects in charging stations, screen the prediction boxes in the previous image frame with the same target object label category as the current prediction box, and perform secondary screening on whether there is an intersection between the prediction boxes with the same target object category and the current prediction box to screen out the prediction boxes with intersections; verify whether the intersection over union (IoU) of the prediction boxes with intersections is greater than the preset IoU threshold. If it is greater than the IoU threshold, it is an abnormal item of the same category and record the occurrence times.

[0022] Determine whether it is an abnormally piled item for a long time according to the threshold of the occurrence times of abnormally piled items of the same category. If it is higher than the threshold of the occurrence times of abnormally piled items of the same category, it is an abnormally piled item for a long time.

[0023] Furthermore, if it is an abnormally piled item for a long time, mark the abnormally piled items of this category in the latest image frame, push the alarm content to the cloud at the same time, and then clean up the image frame; otherwise, directly clean up the image frame.

[0024] Furthermore, after eliminating normal item categories according to the formulated classification standard for target objects in charging stations, it also includes: for each newly detected target object, assign a unique prediction box ID to it. If a target object matching the previous image frame is detected in subsequent image frames, keep the prediction box ID of the target object unchanged.

[0025] Compared with the prior art, the present invention has the following technical effects:

[0026] (1) By designing a method for judging the long-term accumulation of abnormal items in charging stations for Ascend edge devices, the present invention realizes the accurate detection of the long-term accumulation of abnormal items in charging stations, and proposes a solution for detecting the long-term accumulation of abnormal items in the charging station scenario under Ascend edge devices, providing strong support for the safety management of charging stations. At the same time, this method performs image processing and analysis on Ascend edge devices, reducing the dependence on cloud computing resources and lowering the bandwidth and network requirements. In addition, by setting a deletion period to clean up the saved images of the long-term accumulation of abnormal items, the effective utilization of the device storage space is ensured, and the resource utilization efficiency is further optimized.

[0027] (2) By formulating a classification standard for target objects in charging stations, the problem that the definition of abnormal items in charging stations varies from scenario to scenario and is difficult to enumerate is solved. This method not only simplifies the definition process of abnormal items, but also improves the accuracy and efficiency of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 It is a schematic diagram of the process of the present invention;

[0030] Figure 2 This is a flow chart of the abnormal object detection and long-term accumulation discrimination method of the present invention. DETAILED DESCRIPTION

[0031] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. The specific features, structures or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention.

[0032] Aiming at the problem of long-term accumulation detection of abnormal objects in charging stations, the present invention proposes a long-term accumulation detection method for abnormal objects in charging stations based on Ascend edge devices. The method first constructs a general target detection model for charging stations adapted to Huawei Ascend edge devices, classifies the identified targets according to the established charging station target classification standards, and performs long-term accumulation logic judgment on the abnormal objects in the classified results, thereby determining whether there is a long-term accumulation of abnormal objects in the charging station.

[0033] In one embodiment of the present invention, referring to Figure 1 , provides a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device, comprising the following steps:

[0034] Construct a general target detection model for charging stations and formulate a classification standard for charging station targets;

[0035] Adapt and optimize the general target detection model of charging stations on Ascend edge devices to obtain a charging station target detection model adapted to Ascend edge devices;

[0036] The charging station target detection model adapted to Ascend edge devices is used to identify normal item categories, and normal item categories are eliminated according to the established charging station target classification standards to screen out abnormal items that have been accumulated for a long time.

[0037] The following is a detailed explanation of each of the above steps:

[0038] Step 100: Build a general object detection model for charging stations.

[0039] As an example, this step includes the following sub-steps:

[0040] Step 1001: Build a general object detection dataset for private charging stations.

[0041] As an example, this step may include the following steps:

[0042] Step 10011: Collect the real-time video stream of the charging station and obtain image frames through video processing technology..

[0043] To build a general object detection dataset for private charging stations, first collect the image frames of the charging station.

[0044] Step 10012: Use the pre-trained YOLOv8 object detection model to pre-label the collected image frames of the charging station, and use annotation tools to fine-tune and supplement the pre-labeled regions of interest to build a general object detection dataset for private charging stations.

[0045] Subsequently, select the pre-trained YOLOv8 object detection model in MMYOLO (OpenMMLab YOLO series toolbox and benchmark, the YOLO series algorithm toolbox and benchmark platform of OpenMMLab). Among them, the pre-trained YOLOv8 object detection model is trained on the COCO (Microsoft Common Objects in Context) dataset. The pre-trained YOLOv8 object detection model can identify 80 object categories in the COCO dataset. The pre-trained model weight file contains all the parameters learned by the model, and these parameters can be used to predict new data or further fine-tune.

[0046] Use the pre-trained YOLOv8 object detection model to pre-label the collected image frames of the charging station, which significantly reduces the workload of initial annotation. Then, to further improve the accuracy and applicability of the general object detection dataset for private charging stations, use labelimg or labelme annotation tools to fine-tune and supplement the pre-labeled regions of interest to build a general object detection dataset for private charging stations.

[0047] By combining model pre-labeling and manual fine-tuning, build a general object detection dataset for private charging stations. This process greatly reduces the cost of image annotation while ensuring the quality and practicality of the dataset.

[0048] Step 1002: Based on the constructed private charging station general object detection dataset, adapt and iteratively upgrade the pre-trained YOLOv8 general object detection model for the charging station scenario to obtain the charging station general object detection model.

[0049] Based on the constructed private charging station general object detection dataset, adapt the pre-trained YOLOv8 general object detection model to the charging station scenario to enable more accurate detection of key objects in the charging station. Subsequently, through continuous iterative upgrades and combined with actual application feedback, optimize the private charging station general object detection dataset and the YOLOv8 general object detection model to continuously improve the detection accuracy and generalization ability of the YOLOv8 general object detection model.

[0050] Step 200: Formulate the classification criteria for the target objects in the charging station.

[0051] Normal object categories: cars, people, charging piles, charging guns, fire extinguishers, fire hydrants, wheel blocks, parking signs, safety warning signs;

[0052] Abnormal object categories: fireworks, non-motor vehicles, cardboard boxes, plastic bags, building materials, gas cylinders;

[0053] Other categories: other detected target objects and target objects with a confidence level lower than 60%.

[0054] Step 300: Adapt and optimize the charging station general object detection model on the Ascend edge device to obtain the charging station object detection model adapted to the Ascend edge device.

[0055] As an example, this step may include the following sub-steps:

[0056] Step 3001: Adapt and optimize the charging station general object detection model on the Ascend edge device.

[0057] The detection process is as Figure 2As shown in the figure, after obtaining the general object detection model for charging stations, the weight file of the general object detection model for charging stations is stored in the PTH format by default and cannot be directly used for offline inference on Ascend edge devices. Therefore, first use the export method to convert the weight file of the general object detection model for charging stations into ONNX (Open Neural Network Exchange), and use the Ascend Tensor Compiler (ATC) under the Ascend CANN system to convert ONNX into the OM format to adapt to the Ascend hardware architecture. Finally, use the InferSession in the ais_bench inference toolkit to load and infer the model, and achieve efficient offline inference on the Ascend edge device to accelerate the execution of the object detection task for charging stations. Among them, the export method specifically uses torch.onnx.export() provided in the pytorch framework to directly export the weight file in the ONNX format.

[0058] Step 3001: Obtain the on-site monitoring image data of the charging station, and preprocess the image frames on the Ascend edge device, reducing the dependence on cloud computing resources and lowering the bandwidth and network requirements.

[0059] During the process of collecting monitoring data for the charging station, obtain the on-site monitoring image data of the charging station from the video monitoring device according to the frame capture frequency. For example, capture image frames regularly at a frame capture frequency of 0.5 hours. And the captured images need to be preprocessed, and the preprocessing includes image blur judgment and image data format processing. Among them, the image blur judgment uses the Laplacian operator method based on gradients to evaluate the clarity by calculating the second derivative of the image grayscale value.

[0060] The Laplacian operator is defined as follows:

[0061]

[0062] In actual calculations, a 3×3 Laplacian kernel is usually used for convolution calculations:

[0063]

[0064] Among them, * represents the convolution operation; represents the Laplacian operator; I represents the image frame matrix obtained from the video monitoring device; x, y represent the Cartesian coordinates on the x-y plane.

[0065] Calculate the variance of the Laplacian operator and compare it with the threshold T = 100. If the variance of the Laplacian operator is less than the threshold, it means that the image is blurred and the image frame needs to be re-obtained. The specific steps are as follows:

[0066] First, convert the intercepted image frame into a grayscale image I gray After that, use the Laplacian operator to convolve the grayscale image to obtain the Laplacian image L:

[0067]

[0068] Secondly, calculate the variance Variance(L) of the Laplacian image;

[0069]

[0070] In the above formula, N represents the number of pixels in the image; μ represents the pixel mean of the image frame L; L i represents the i-th pixel value of the Laplacian image L.

[0071] Finally, compare the calculated variance Variance(L) with the preset threshold T = 100 to obtain the result. If the variance of the Laplacian operator is less than the threshold, it means that the image is blurred and the image frame needs to be acquired again.

[0072] Image data format processing is to format the image name. The naming method is the device number and the timestamp of the interception moment, for the subsequent discrimination of long-term accumulation of abnormal items and alarm processing. Input the image frames intercepted within the set fixed time period into the general target detection model of the charging station adapted to the Ascend edge device. Here, the JSON format is used for model parameter transmission, and the parameters include the list of image storage addresses and the timestamp of the latest intercepted image. The set fixed time period is set according to business experience and is set to 12 hours here.

[0073] Step 400: Use the target detection model of the charging station adapted to the Ascend edge device for recognition, and eliminate the normal item categories according to the formulated classification standard of the charging station target objects to screen out the abnormal items with long-term accumulation.

[0074] As an example, this step may include:

[0075] Step 4001: Use the target detection model of the charging station adapted to the Ascend edge device for target recognition, and eliminate the normal item categories according to the formulated classification standard of the charging station target objects.

[0076] Step 4002: Intersection screening of prediction boxes.

[0077] After excluding normal item categories according to the established charging station target classification standard, filter out the prediction boxes in the previous image frame with the same target label category as the current prediction box, and perform a secondary screening on whether there is an intersection between the prediction boxes with the same target category and the current prediction box, and filter out the prediction boxes with an intersection; verify whether the intersection over union (IoU) of the prediction boxes with an intersection is greater than the preset IoU threshold. If it is greater than the IoU threshold, it is an abnormal item of the same category and record the occurrence times.

[0078] Specifically, for each newly detected target, assign it a unique prediction box ID for tracking the target in subsequent frames; if a target matching the previous image frame is detected in a subsequent image frame, keep the target prediction box ID unchanged.

[0079] Further filter the prediction boxes to remove the prediction boxes that have no intersection with other prediction boxes in the current image frame; calculate the IoU between the current prediction box and the remaining prediction boxes. If there is a prediction box among the remaining prediction boxes whose IoU with the current prediction box is greater than the set IoU threshold (80%), then a matching prediction box is found, enter "1" into the image matching queue of the current prediction box, and update the ID of the matching prediction box to be the same as the current prediction box; if there is a prediction box among the remaining prediction boxes whose IoU with the current prediction box is less than or equal to the set IoU threshold (80%), then no matching prediction box is found, and enter "0" into the image matching queue of the current prediction box.

[0080] The formula is as follows:

[0081]

[0082] In the formula, Queue (i,j) represents the image prediction box matching queue of the j-th prediction box in the i-th image; Intersection(P (i.j) ,P (i+1,k) ) represents determining whether there is an intersection between the j-th prediction box in the i-th image and the j-th prediction box in the (i + 1)-th image; IoU(P (i.j) ,P (i+1,k) ) represents calculating the IoU between the j-th prediction box in the i-th image and the j-th prediction box in the (i + 1)-th image; IoUThreshold represents the set IoU threshold.

[0083] Specifically, let the coordinates of box A in image image1 be [x A1 ,y A1 ,x A2 ,y A2 , and the coordinates of box B in image image2 be [x B1 ,y B1 ,x B2 ,y B2.

[0084] Determine whether there is an intersection between judgment boxes A and B. The formula is as follows:

[0085]

[0086] The operation process of the predicted box intersection over union IoU(A, B) is as follows:

[0087] Intersection Width = max(0, min(x A2 , x B2 ) - max(x A1 , x B1 ));

[0088] Intersection Height = max(0, min(y A2 , y B2 ) - max(y A1 , y B1 ));

[0089] Area Intersection = Intersection Width × Intersection Height

[0090] Area Union = (x A2 - x A1 ) × (y A2 - y A1 ) + (x B2 - x B1 ) × (y B2 - y B1 ) - Area Intersection ;

[0091]

[0092] In the above formula, Intersection Width represents the width of the intersection area; Intersection Height represents the height of the intersection area; Area Intersection represents the area of the intersection area, and Area Union represents the area of the merged area.

[0093] Step 4003: Determine long-term accumulation of abnormal items at the charging station.

[0094] After all image frames have been judged, count the total number of frames in which the predicted box ID in the abnormal items appears according to the predicted box ID in the abnormal items, and perform long-term accumulation detection of abnormal items according to the designed long-term accumulation judgment logic of abnormal items, and store the result in the Ascend edge device.

[0095] The logic for judging long-term accumulation of abnormal items is as follows: calculate the number of images matched in the image matching queue saved in each prediction box ID, that is, calculate the image prediction box matching queue Queue n to determine whether the number of "1"s in it meets the set threshold of the number of matching images. For example, if it exists continuously for 12 times or the number of existence times exceeds 20 times, it is determined that the item has long-term accumulation, so as to screen out the prediction boxes that meet the conditions. If there are prediction boxes that meet the conditions, long-term accumulation annotation of abnormal items is performed in the latest image frame. At the same time, warning content is generated according to the prediction box ID (device number) and timestamp and pushed to the cloud, and the image frame intercepted in the Ascend edge device is synchronously deleted; otherwise, it directly ends after deleting the image frame.

[0096] The formula is as follows:

[0097]

[0098] In the above formula, Warning represents the warning content; NumberThreshold represents the set threshold of the number of matching images; Count(Queue n ) represents the number of images in the image prediction box matching queue Queue n.

[0099] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting long-term accumulation of abnormal items in charging stations based on Ascend edge devices, characterized in that: The following steps are involved: Construct a general target detection model for charging stations and formulate a classification standard for charging station targets; Adapt and optimize the general target detection model of charging stations on Ascend edge devices to obtain a charging station target detection model adapted to Ascend edge devices; The charging station target detection model adapted to the Ascend edge device is used to identify targets, and the identified targets are classified according to the established charging station target classification standards. Based on the classification results, abnormal objects that have been piled up in the charging station for a long time are identified.

2. According to claim 1, a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device is characterized in that: The construction of a universal target detection model for a charging station includes: Collect real-time video streams of charging stations, obtain image frames through video processing technology, pre-label the collected image frames of charging stations based on the pre-trained YOLOv8 target detection model, and then use the labeling tool to fine-tune and supplement the pre-labeled focus areas to build a general target detection dataset for private charging stations; Based on the constructed private charging station general object detection dataset, the pre-trained YOLOv8 general object detection model is trained to obtain the charging station general object detection model.

3. According to claim 1, a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device is characterized in that: The charging station target classification standard includes: normal object category, abnormal object category and other categories, and the other categories are other detected targets and targets with confidence levels lower than a preset threshold.

4. According to claim 1 or 2, a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device is characterized in that: Adaptation and optimization of the general object detection model for charging stations on Ascend edge devices include: Convert the weight file of the general object detection model of the charging station to ONNX, and use the tensor compiler under the Ascend CANN system to convert ONNX to OM format to adapt to the Ascend edge device hardware architecture; Use InferSession in the ais_bench inference toolkit to load models and perform inference, and implement offline inference on Ascend edge devices.

5. According to claim 1, a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device is characterized in that: Also includes: The collected image frames of the charging station are preprocessed on the Ascend edge device, and the preprocessed image frames are saved on the Ascend edge device, wherein the preprocessing includes image blur judgment and image data format processing.

6. According to claim 5, a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device is characterized in that: The image blur judgment adopts a gradient-based Laplace operator method to evaluate the clarity by calculating the second-order derivative of the image grayscale value.

7. According to claim 6, a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device is characterized in that: The image data format processing is to format the name of the image frame of the collected charging station, and the naming method is the device number and the timestamp of the interception time.

8. According to claim 7, a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device is characterized in that: The identified targets are classified according to the established charging station target classification standards, and the abnormal objects that have been accumulated in the charging station for a long time are identified based on the classification results, including: According to the charging station target classification standard, normal items are eliminated, and the prediction box with the same target label category as the current prediction box in the previous image frame is screened. A secondary screening is performed to determine whether there is an intersection between the prediction box with the same target category and the current prediction box, and the prediction boxes with intersections are screened out; verify whether the intersection-over-union ratio of the intersection is greater than the preset intersection-over-union ratio threshold. If it is greater than the intersection-over-union ratio threshold, it is an abnormal item of the same category and the number of occurrences is recorded; Whether it is a long-term accumulated abnormal item is determined based on the number of times abnormal items of the same category appear. If it is higher than the number of times abnormal items of the same category appear, it is a long-term accumulated abnormal item.

9. According to claim 8, a method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device is characterized in that: If it is an abnormal object that has been piled up for a long time, the target object will be marked as an abnormal object with long-term accumulation in the latest image frame, and the alarm content will be pushed to the cloud, and then the image frame will be cleaned; otherwise, the image frame will be cleaned directly.

10. The method for detecting long-term accumulation of abnormal items in a charging station based on an Ascend edge device according to claim 8, characterized in that: After eliminating normal object categories according to the established charging station target classification standards, it also includes: for each newly detected target object, a unique prediction box ID is assigned to it. If a target object matching the previous image frame is detected in a subsequent image frame, the target object prediction box ID remains unchanged.

Images