A management system and method for preventing electric bicycles from entering elevators based on object detection.

Abstract

This invention relates to a management system and method for preventing electric bicycles from entering elevators based on object detection. It trains an improved YOLOv4-tiny object detection model, obtains a lightweight student model through knowledge distillation, and deploys this lightweight student model on an edge device. The system analyzes and detects objects in the acquired images to determine if an electric bicycle has entered the elevator. If so, a pulse signal is transmitted to the elevator control module associated with the elevator's overload detection module, triggering an overload control command in the elevator overload detection module to stop the elevator. If the image analysis module does not detect the electric bicycle, it determines that the bicycle has left the elevator, stops the elevator control module, and the elevator can be used normally. This invention employs an improved YOLOv4-tiny lightweight structure, suitable for edge devices, and can more accurately and quickly identify electric bicycles about to enter the elevator in dimly lit elevator environments and issue warnings, thereby preventing electric bicycles from entering the elevator and avoiding safety hazards.

Description

Technical Field

[0001] This invention belongs to the field of elevator safety technology, and in particular relates to a management system and method for preventing electric bicycles from entering elevators based on target detection. Background Technology

[0002] Due to their low price and convenient parking, electric bicycles have become a popular mode of transportation. However, with the increasing use of electric bicycles, some safety hazards have gradually emerged. For example, some residents, for their own convenience, privately carry electric bicycles upstairs via elevators and charge them indoors or in public areas such as hallways. After a period of use, the wiring inside the electric bicycle is prone to aging and short-circuiting. When a short circuit occurs and the external temperature is high, it can easily ignite and cause a fire. Furthermore, due to insufficient market supervision and inconsistent quality of electric bicycles, some people illegally modify them by installing substandard batteries, which also creates safety hazards. In the enclosed space of an elevator, there is almost no escape route. If an electric bicycle explodes inside, it can easily cause widespread fire and damage, with a higher risk of injury or death than in an open area. To prevent accidents, many residential property management departments have implemented regulations prohibiting electric bicycles from being brought upstairs; and measures are being taken to effectively prevent residents from bringing electric bicycles upstairs and to manage electric bicycle use in public areas.

[0003] Existing technologies include several control systems that prohibit electric bicycles from being brought upstairs. These systems detect electric bicycles by laying inductive loop detectors or by using traditional image recognition methods (such as manually designed features), and then prevent residents from bringing the bicycles upstairs. However, these methods all suffer from high error rates during the identification process, affecting residents' normal use of the bicycles.

[0004] Object detection is one of the most important branches of computer vision. It has been widely applied in people's lives, such as in security surveillance and autonomous driving. With the rapid development of deep learning networks for detection tasks, the performance of object detectors has been greatly improved. While large object detection networks offer high accuracy, they also suffer from high computational cost and resource consumption. A lightweight detection network based on YOLOv4-tiny, with only 6 million parameters (one-tenth of the original), requires less memory, achieving a perfect balance between speed and accuracy. This makes it suitable for real-time object detection in embedded systems. However, when running existing YOLOv4-tiny networks on edge devices for object detection, limited computing power makes it difficult to achieve the expected accuracy and speed in dimly lit environments like elevators.

[0005] Purpose of the invention

[0006] To address the aforementioned problems, the present invention aims to provide a management system and method for preventing electric bicycles from entering elevators based on target detection. Employing an improved YOLOv4-tiny lightweight structure, suitable for edge devices, it can more accurately and quickly identify electric bicycles about to enter the elevator and issue warnings in dimly lit elevator environments, thereby preventing electric bicycles from entering the elevator and avoiding safety hazards.

[0007] The present invention provides a method for preventing electric bicycles from entering elevators based on target detection, comprising the following steps:

[0008] Step 1: Construct the initial dataset: Collect N days of data from elevator monitoring cameras, and collect a preset number of images from the monitoring videos of various electric vehicles to obtain the initial dataset;

[0009] Step 2: Perform data augmentation on the initial dataset;

[0010] Step 3: Label the dataset: Label the electric bikes in the dataset;

[0011] Step 4: Build the improved YOLOv4-tiny object detection model:

[0012] Step 5: Train the improved YOLOv4-tiny object detection model and obtain the lightweight student model through knowledge distillation.

[0013] The dataset is trained using YOLOv5 to obtain a trained teacher model. Knowledge distillation is used to obtain a trained student model. If the highest logit in the teacher model represents an inaccurate class, the highest logit of the inaccurate class is replaced with the logit of the accurate class to generate an accurate class pointing to the highest logit. This corrects the incorrectly predicted logit before the teacher model passes it to the student model, so that the student model always learns the correct knowledge.

[0014] Step 6: Deploy the lightweight student model to an edge device to obtain the image analysis module;

[0015] Step 7: The start control module is triggered in response to the opening of the elevator door, controlling all modules to leave standby and enter the working state, and controlling the image acquisition module to collect images of people and objects entering the elevator from the elevator monitoring camera;

[0016] Step 8: The image acquisition module performs data enhancement on the images of people and objects entering the elevator collected by the elevator monitoring camera and then transmits them to the image analysis module on the edge device.

[0017] Step 9: The image analysis module uses the improved YOLOv4-tiny target detection model to analyze and detect targets in the acquired images to determine whether an electric vehicle has entered the elevator. If so, it transmits a pulse signal to the elevator control module. The elevator control module, which is associated with the elevator overload detection module, receives the pulse signal from the image analysis module and triggers the elevator overload control command in the elevator overload detection module to stop the elevator from running.

[0018] Once the elevator control module is triggered, if the image analysis module fails to detect the electric vehicle and determines that the electric vehicle has left the elevator, it will send a pulse signal to the elevator control module again to stop the elevator control module from operating, and the elevator can be used normally.

[0019] The data augmentation step in step 2 involves denoising and enhancing the contrast of the images in the initial dataset to make the detected target more identifiable. Specifically, it includes the following steps:

[0020] Median filtering is used to filter out noise abrupt changes in the image.

[0021] f out =median(f1,f2,f3,…,f9) (1)

[0022] In equation (1), a 3×3 computational domain is used, f out f1, f2, f3, ..., f9 are the pixel values ​​obtained after filtering. f1, f2, f3, ..., f9 are the sorted pixel values ​​in the operation area. The median() function is used to extract the intermediate values ​​and assign them, which effectively eliminates noise abrupt points.

[0023] To enhance the contrast of an image and reveal details of targets not displayed in that specific environment, we first need to count the number of pixels at different gray levels in the image:

[0024] Count(k) = m k (2)

[0025] In equation (2), 0 ≤ k ≤ 255, m k Count(k) represents the total number of pixels with gray level k in different channels.

[0026] Next, calculate the probability distribution of gray levels:

[0027]

[0028] Further accumulate the gray level distribution probabilities:

[0029]

[0030] In equation (4), 0 ≤ n ≤ 255;

[0031] The new gray level is calculated using the cumulative gray level distribution probability, and then the number of pixels for each gray level is redistributed to obtain the new gray level with increased contrast.

[0032]

[0033] Perform grayscale k pixel and h k The pixel mapping is completed, and the image with enhanced contrast is obtained.

[0034] The improved YOLOv4-tiny target detection model in step 4 includes a backbone feature extraction network, a neck enhancement feature extraction network, and a YOLO Head module;

[0035] The backbone feature extraction network is a lightweight Ghost convolutional feature extraction network improved based on a multi-scale dilated convolutional module. It includes a first depthwise separable convolutional module, a second depthwise separable convolutional module, a first CSP_Ghost multi-scale dilated convolutional feature extraction network module, a second CSP_Ghost multi-scale dilated convolutional feature extraction network module, a third CSP_Ghost multi-scale dilated convolutional feature extraction network module, and a third depthwise separable convolutional module connected in sequence. Among them, one branch of the second CSP_Ghost multi-scale dilated convolutional feature extraction network module performs a downward convolution and connects to the third CSP_Ghost multi-scale dilated convolutional feature extraction network module, while the other branch is directly connected to the connection module of the neck reinforcement feature extraction network.

[0036] The multi-scale dilated convolutional feature extraction network module includes a first to a third Ghost dilated convolutional module connected in sequence. The first Ghost dilated convolutional module is concatenated to a connection module, the second and third Ghost dilated convolutional modules are concatenated to a connection module, and then a 1×1 convolutional module is connected to obtain features and adjust the number of channels. Features are then concatenated with the first Ghost spatial convolutional module through the connection module, and finally connected to a max pooling module to complete the downsampling of the image. The first Ghost dilated convolutional module maintains the original number of channels, the second and third Ghost dilated convolutional modules compress the number of channels, the 1×1 convolutional module adjusts the number of output channels to obtain features, and then concatenates with the first Ghost dilated convolutional module for max pooling.

[0037] The Ghost dilated convolution module introduces dilated convolutions with different dilation rates to obtain features at different scales.

[0038] The neck enhancement feature extraction network includes a connection module connected to the third CSP_Ghost multi-scale dilated convolution feature extraction network module and a Conv convolution module connected to the third depth separable convolution module. Finally, it fuses features from two different scales and outputs them to the YOLO Head module.

[0039] The Yolo Head module is connected to the neck enhancement feature extraction network's connection module and the Conv convolution module, respectively, and converts the features extracted by the neck enhancement feature extraction network into prediction results.

[0040] Step 5 uses knowledge distillation to obtain the trained student model. The loss function used in knowledge distillation is:

[0041]

[0042] Where CE represents the cross-entropy, (σO) S ,1) represents the student model O S Softening is performed at a temperature of 1, where 'a' represents the hyperparameter, 'KL' represents the divergence, and 'Y' is a one-bit thermally encoded label; A k In this context, k represents the most likely predicted category, and A... k Let m represent a specialist model that includes k in all prediction categories, m represent a specialist model that includes k in a specific prediction category, q represent the prediction probability value for each category, and p represent the prediction probability value for each category. m This represents the probability distribution of the generalized full model.

[0043] This invention relates to a management system for preventing electric bicycles from entering elevators based on target detection, comprising a start-up control module, an image acquisition module, an image analysis module, and an elevator control module.

[0044] The start-up control module is triggered in response to the opening of the elevator door, controlling all modules to leave standby and enter the working state, and controlling the image acquisition module to collect images of people and objects entering the elevator from the elevator monitoring camera;

[0045] The image acquisition module performs data enhancement on the images of people and objects entering the elevator captured by the elevator monitoring camera and then transmits them to the image analysis module.

[0046] The image analysis module is obtained by building and training an improved YOLOv4-tiny target detection model, obtaining a lightweight student model through knowledge distillation, and then deploying it on an edge device. The improved YOLOv4-tiny target detection model is used to analyze image information and detect targets to determine whether an electric scooter has entered the elevator. If so, a pulse signal is transmitted to the elevator control module. After the elevator control module is triggered, if the image analysis module fails to detect the electric scooter, it means that the electric scooter has left the elevator. In this case, a pulse signal is sent to the elevator control module again to stop the elevator control module and the elevator can be used normally.

[0047] The improved YOLOv4-tiny object detection model is trained using YOLOv5 on the dataset to obtain a trained teacher model, and then a trained student model is obtained using knowledge distillation. If the highest logit in the teacher model represents an inaccurate class, the highest logit of that inaccurate class is replaced with the logit of the accurate class, thereby generating an accurate class that points to the highest logit. This corrects the incorrectly predicted logit before the teacher model passes it to the student model, ensuring that the student model always learns the correct knowledge.

[0048] The elevator control module is associated with the elevator overload detection module. After receiving a pulse signal from the image analysis module, it triggers the elevator overload control command in the elevator overload detection module, controlling the elevator to stop running. After the elevator control module is triggered, it receives a pulse signal sent by the image analysis module again and stops running, so the elevator can be used normally.

[0049] The data augmentation in the image acquisition module involves denoising and contrast enhancement of the images in the initial dataset to make the detected target more identifiable. Specifically, it includes the following steps:

[0050] Median filtering is used to filter out noise abrupt changes in the image.

[0051] f out =median(f1, f2, f3,..., f9) (1)

[0052] In equation (1), a 3×3 computational domain is used, f out f1, f2, f3, ..., f9 are the pixel values ​​obtained after filtering. f1, f2, f3, ..., f9 are the sorted pixel values ​​in the operation area. The median() function is used to extract the intermediate values ​​and perform the assignment operation, which effectively eliminates noise abrupt points.

[0053] To enhance the contrast of an image and bring out the details of targets that are not visible in this specific environment, first count the number of pixels at different gray levels in the image:

[0054] Count(k) = m k (2)

[0055] In equation (2), 0 ≤ k ≤ 255, m k Count(k) represents the total number of pixels with gray level k in different channels.

[0056] Next, calculate the probability distribution of gray levels:

[0057]

[0058] Further accumulate the gray level distribution probabilities:

[0059]

[0060] In equation (4), 0 ≤ n ≤ 255;

[0061] The new gray level is calculated using the cumulative gray level distribution probability, and then the number of pixels for each gray level is redistributed to obtain the new gray level with increased contrast.

[0062]

[0063] Perform grayscale k pixel and h k The pixel mapping is completed, and the image with enhanced contrast is obtained.

[0064] The improved YOLOv4-tiny target detection model includes a backbone feature extraction network, a neck enhancement feature extraction network, and a YOLO Head module;

[0065] The backbone feature extraction network is a lightweight Ghost convolutional feature extraction network improved based on a multi-scale dilated convolutional module. It includes a first depthwise separable convolutional module, a second depthwise separable convolutional module, a first CSP_Ghost multi-scale dilated convolutional feature extraction network module, a second CSP_Ghost multi-scale dilated convolutional feature extraction network module, a third CSP_Ghost multi-scale dilated convolutional feature extraction network module, and a third depthwise separable convolutional module connected in sequence. Among them, one branch of the second CSP_Ghost multi-scale dilated convolutional feature extraction network module performs a downward convolution and connects to the third CSP_Ghost multi-scale dilated convolutional feature extraction network module, while the other branch is directly connected to the connection module of the neck reinforcement feature extraction network.

[0066] The multi-scale dilated convolutional feature extraction network module includes a first to a third Ghost dilated convolutional module connected in sequence. The first Ghost dilated convolutional module is concatenated to a connection module, the second and third Ghost dilated convolutional modules are concatenated to a connection module, and then a 1×1 convolutional module is connected to obtain features and adjust the number of channels. Features are then concatenated with the first Ghost spatial convolutional module through the connection module, and finally connected to a max pooling module to complete the downsampling of the image. The first Ghost dilated convolutional module maintains the original number of channels, the second and third Ghost dilated convolutional modules compress the number of channels, the 1×1 convolutional module adjusts the number of output channels to obtain features, and then concatenates with the first Ghost dilated convolutional module for max pooling.

[0067] The Ghost dilated convolution module introduces dilated convolutions with different dilation rates to obtain features at different scales.

[0068] The neck enhancement feature extraction network includes a connection module connected to the third CSP_Ghost multi-scale dilated convolution feature extraction network module and a Conv convolution module connected to the third depth separable convolution module. Finally, it fuses features from two different scales and outputs them to the YOLO Head module.

[0069] The Yolo Head module is connected to the neck enhancement feature extraction network's connection module and the Conv convolution module, respectively, and converts the features extracted by the neck enhancement feature extraction network into prediction results.

[0070] The model compression using knowledge distillation yields a trained student model. The loss function used in this knowledge distillation is:

[0071]

[0072] Where CE represents the cross-entropy, (σO) s ,1) represents the student model O S Softening is performed at a temperature of 1, where 'a' represents the hyperparameter, 'KL' represents the divergence, and 'Y' is a one-bit thermally encoded label; A k In this context, k represents the most likely predicted category, and A... k Let m represent a specialist model that includes k in all prediction categories, m represent a specialist model that includes k in a specific prediction category, q represent the prediction probability value for each category, and p represent the prediction probability value for each category. m This represents the probability distribution of the generalized full model.

[0073] The improved YOLOv4-tiny object detection model's backbone feature extraction network in this invention is based on a CSP structure (based on a Ghost multi-scale spatial convolutional feature extraction module). This CSP structure divides the original input feature map into two branches, halving the number of channels. One branch is directly connected to the output, while the other performs multiple repeated ordinary convolution operations. This invention retains the CSP structure, but considering the concentration of parameters there, the branch performing repeated ordinary convolution operations is optimized. The number of branches is increased by borrowing the idea of ​​Inception. However, according to the inverted residual structure, the decrease in the number of channels weakens the network's feature extraction ability. To address this issue, Ghost convolution is introduced. The characteristic of Ghost convolution is that it consumes fewer computational resources to obtain more channels. Therefore, the first Ghost convolution module is used to expand the number of channels, the second Ghost convolution module is used to compress the number of channels, and finally, a 1×1 convolution module is used to adjust the number of output channels. However, if the convolution kernel size is the same in the Inception module, Inception will lose the advantages of extracting multi-scale fusion and reducing redundant features. To avoid increasing the number of parameters, this invention introduces dilated convolutions with different dilation rates to obtain features at different scales. When the dilation rate D = 2 or 3, the corresponding receptive fields are 5 or 7. Compared to a regular 3×3 convolution module, this backbone feature extraction network structure solves the problem of "large and dense" features during feature extraction, speeds up training, reduces the number of parameters, and improves feature representation capabilities.

[0074] Training the detection module requires complex models and significant computational resources to extract information from very large, highly redundant datasets. In experiments, the best-performing models are often large, sometimes even ensembles of multiple models. Large models are inconvenient to deploy on edge devices, leading to slow inference speeds and high resource requirements. This invention employs knowledge distillation to compress the model, reducing memory overhead and improving running speed. The distilled, smaller model is more suitable for deployment on edge devices. To improve the accuracy of the smaller model, this invention corrects incorrectly predicted logits from the teacher model before passing them to the student model, ensuring the student model always learns the correct knowledge. This means that if the highest logit in the teacher model represents an inaccurate class, the highest logit of that inaccurate class will replace the logit of the accurate class, resulting in an accurate class pointing to the highest logit. This ensures the teacher model is always a better teacher, effectively extracting knowledge, while simultaneously enabling the student model to learn more effectively. Attached Figure Description

[0075] Figure 1 This is a diagram of the YOLOv4-tiny network structure.

[0076] Figure 2 This is a diagram of the improved yolov4-tiny network structure in this invention;

[0077] Figure 3 This is a structural diagram of the CSP_Ghost multi-scale dilated convolutional feature extraction network module in this invention;

[0078] Figure 4 Here is a structural diagram of the Ghost dilated convolution module in this invention;

[0079] Figure 5 This is a flowchart of the present invention.

[0080] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0081] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0082] Numerous specific details are set forth in the following description to provide a thorough understanding of the invention. The described embodiments are merely some, not all, of the embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0084] The terms "first," "second," and "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects and not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, is intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0085] like Figure 1As shown, assuming the input of the feature extraction network is 416x416x3, the YOLOv4-tiny network structure consists of three parts: the CSPDarkNet53 (Cross Stage Paritial Network) feature extraction network, the enhanced feature extraction network composed of SPP (Spatial Pyramid Pooling) and PANet (Path Aggregation Network), and the YOLO Head that transforms the extracted features into prediction results.

[0086] Example 1

[0087] like Figure 2-5 As shown, the method for preventing electric bicycles from entering elevators based on target detection according to the present invention includes the following steps:

[0088] Step 1: Construct the initial dataset: Collect data for several days using elevator monitoring cameras. Take one frame out of every 5 frames of the monitoring video of various electric vehicles, and collect about 5,000 images to obtain the initial dataset.

[0089] Step 2: Data augmentation of the initial dataset: In order to adapt to the dim detection environment, data augmentation is performed on the images in the dataset in the Python 3 environment to remove noise and improve contrast, thereby further expanding the initial dataset;

[0090] Step 3: Label the dataset: Use the labeling program to label the electric bikes in the dataset;

[0091] Step 4: Build the improved YOLOv4-tiny object detection model:

[0092] Step 5: Train the improved YOLOv4-tiny object detection model and obtain the lightweight student model through knowledge distillation.

[0093] The dataset is trained using YOLOv5 to obtain a trained teacher model. Knowledge distillation is then used to obtain a trained student model. If the highest logit in the teacher model represents an inaccurate class, the highest logit of that inaccurate class is replaced with the logit of the accurate class, thus generating an accurate class that points to the highest logit. This corrects the incorrectly predicted logit before the teacher model passes it to the student model, ensuring that the student model always learns the correct knowledge.

[0094] Step 6: Deploy the lightweight student model to the edge device: Use kflash as the burning tool to burn the Micropython firmware to the edge device via the command line, copy the files used in the training above to the SD card, and then insert it into the edge device to obtain the image analysis module;

[0095] Step 7: The start control module is triggered in response to the opening of the elevator door, controlling all modules to leave standby and enter the working state, and controlling the image acquisition module to collect images of people and objects entering the elevator from the elevator monitoring camera;

[0096] Step 8: The image acquisition module performs data enhancement on the images of people and objects entering the elevator collected by the elevator monitoring camera and then transmits them to the image analysis module on the edge device.

[0097] Step 9: The image analysis module uses an improved YOLOv4-tiny target detection model to analyze and detect targets in the acquired images, determining whether an electric vehicle has entered the elevator. If so, it transmits a pulse signal to the warning module and the elevator control module. The elevator control module, which is associated with the elevator overload detection module, receives the pulse signal from the image analysis module and triggers the elevator overload control command in the elevator overload detection module, controlling the elevator to stop running. The warning module issues an alarm signal.

[0098] Once the warning module and elevator control module are triggered, if the image analysis module fails to detect the electric vehicle and determines that the electric vehicle has left the elevator, it will send a pulse signal to the warning module and elevator control module again to stop their operation, and the elevator can be used normally.

[0099] The data augmentation step in step 2 involves denoising and contrast enhancement of the images in the dataset to make the detected targets more identifiable, helping the target detection model to better identify targets and improving the robustness and practicality of the system. Specifically, it includes the following steps:

[0100] Median filtering is used to filter out noise abrupt changes in the image.

[0101] f out =median(f1, f2, f3,..., f9) (1)

[0102] In equation (1), a 3×3 computational domain is used, f out f1, f2, f3, ..., f9 are the pixel values ​​obtained after filtering. f1, f2, f3, ..., f9 are the sorted pixel values ​​in the operation area. The median() function is used to extract the intermediate values ​​and perform the assignment operation, which effectively eliminates noise abrupt points.

[0103] To enhance the contrast of an image and bring out the details of targets that are not visible in this specific environment, first count the number of pixels at different gray levels in the image:

[0104] Count(k) = m k (2)

[0105] In equation (2), 0 ≤ k ≤ 255, m k Count(k) represents the total number of pixels with gray level k in different channels.

[0106] Next, calculate the probability distribution of gray levels:

[0107]

[0108] Further accumulate the gray level distribution probabilities:

[0109]

[0110] In equation (4), 0 ≤ n ≤ 255;

[0111] The new gray level is calculated using the cumulative gray level distribution probability, and then the number of pixels for each gray level is redistributed to obtain the new gray level with increased contrast.

[0112]

[0113] Perform grayscale k pixel and h k The pixel mapping is completed, and the image with enhanced contrast is obtained.

[0114] The improved YOLOv4-tiny target detection model in step 4 includes a backbone feature extraction network, a neck enhancement feature extraction network, and a YOLO Head module;

[0115] like Figure 2 As shown, the backbone feature extraction network is a lightweight Ghost convolutional feature extraction network improved based on the multi-scale dilated convolution module. It includes a first depthwise separable convolution module, a second depthwise separable convolution module, a first CSP_Ghost multi-scale dilated convolutional feature extraction network module, a second CSP_Ghost multi-scale dilated convolutional feature extraction network module, a third CSP_Ghost multi-scale dilated convolutional feature extraction network module, and a third depthwise separable convolution module connected in sequence. Among them, one branch of the second CSP_Ghost multi-scale dilated convolutional feature extraction network module performs a downward convolution and connects to the third CSP_Ghost multi-scale dilated convolutional feature extraction network module, while the other branch is directly connected to the connection module of the neck reinforcement feature extraction network.

[0116] like Figure 3 As shown, the multi-scale dilated convolutional feature extraction network module includes a first to a third Ghost dilated convolutional module connected in sequence. The first Ghost dilated convolutional module is concatenated to a connection module, the second and third Ghost dilated convolutional modules are concatenated to a connection module, and then a 1×1 convolutional module is connected to obtain features and adjust the number of channels. Features are then concatenated with the first Ghost spatial convolutional module through the connection module, and finally connected to a max pooling module to complete the downsampling of the image. The first Ghost dilated convolutional module is used to maintain the original number of channels, the second and third Ghost dilated convolutional modules are used to compress the number of channels, the number of output channels is adjusted through the 1×1 convolutional module to obtain features, and then concatenated with the first Ghost dilated convolutional module for max pooling.

[0117] like Figure 4 As shown, the Ghost dilated convolution module introduces dilated convolutions with different dilation rates to acquire features at different scales without increasing the number of parameters. When the dilation rate D = 2 or 3, the corresponding receptive fields are 5 or 7. Compared to a regular 3×3 convolution, this structure solves the problem of "large and dense" features during feature extraction, speeds up training, reduces the number of parameters, and improves feature representation capabilities.

[0118] The neck enhancement feature extraction network includes a connection module connected to the third CSP_Ghost multi-scale dilated convolution feature extraction network module and a Conv convolution module connected to the third depth separable convolution module. Finally, it fuses features from two different scales and outputs them to the YOLO Head module.

[0119] The Yolo Head module is connected to the neck enhancement feature extraction network's connection module and the Conv convolution module, respectively, and converts the features extracted by the neck enhancement feature extraction network into prediction results.

[0120] Step 5 uses knowledge distillation to obtain the trained student model. The loss function used in knowledge distillation is:

[0121]

[0122] Where CE represents the cross-entropy, (σO) S ,1) represents the student model O S Softening is performed at a temperature of 1, where 'a' represents the hyperparameter, 'KL' represents the divergence, and 'Y' is a one-bit thermally encoded label; A k In this context, k represents the most likely predicted category, and A... k Let m represent a specialist model that includes k in all prediction categories, m represent a specialist model that includes k in a specific prediction category, q represent the prediction probability value for each category, and p represent the prediction probability value for each category. m This represents the probability distribution of the generalized full model.

[0123] Example 2

[0124] like Figure 5 As shown, the present invention provides a management system for preventing electric bicycles from entering elevators based on target detection, comprising a start control module, an image acquisition module, an image analysis module, a warning module, and an elevator control module.

[0125] The start-up control module is triggered in response to the opening of the elevator door, controlling all modules to leave standby and enter the working state, and controlling the image acquisition module to collect images of people and objects entering the elevator from the elevator monitoring camera;

[0126] The image acquisition module performs data augmentation on images of people and objects entering the elevator captured by the elevator monitoring camera and then transmits them to the image analysis module. The data augmentation involves denoising and contrast enhancement of the images in the dataset to make the detected targets more identifiable, helping the target detection model to better identify targets and improving the system's robustness and practicality. Specifically, it includes the following steps:

[0127] Median filtering is used to filter out noise abrupt changes in the image.

[0128] f out =median(f1, f2, f3,..., f9) (1)

[0129] In equation (1), a 3×3 computational domain is used, f out f1, f2, f3, ..., f9 are the pixel values ​​obtained after filtering. f1, f2, f3, ..., f9 are the sorted pixel values ​​in the operation area. The median() function is used to extract the intermediate values ​​and perform the assignment operation, which effectively eliminates noise abrupt points.

[0130] To enhance the contrast of an image and bring out the details of targets that are not visible in this specific environment, first count the number of pixels at different gray levels in the image:

[0131] Count(k) = m k (2)

[0132] In equation (2), 0 ≤ k ≤ 255, m k Count(k) represents the total number of pixels with gray level k in different channels.

[0133] Next, calculate the probability distribution of gray levels:

[0134]

[0135] Further accumulate the gray level distribution probabilities:

[0136]

[0137] In equation (4), 0 ≤ n ≤ 255;

[0138] The new gray level is calculated using the cumulative gray level distribution probability, and then the number of pixels for each gray level is redistributed to obtain the new gray level with increased contrast.

[0139]

[0140] Perform grayscale k pixel and h k Pixel mapping is performed, and the image with enhanced contrast is obtained after the mapping is completed;

[0141] The image analysis module is obtained by building and training an improved YOLOv4-tiny target detection model, obtaining a lightweight student model through knowledge distillation, and then deploying it on an edge device. The improved YOLOv4-tiny target detection model is used to analyze image information and detect targets to determine whether an electric scooter has entered the elevator. If so, a pulse signal is sent to the warning module and the elevator control module. After the warning module and the elevator control module are triggered, if the image analysis module fails to detect the electric scooter, it means that the electric scooter has left the elevator. In this case, a pulse signal is sent to the warning module and the elevator control module again to stop their operation, and the elevator can be used normally.

[0142] The improved YOLOv4-tiny object detection model is trained using YOLOv5 on the dataset to obtain a trained teacher model, and then a trained student model is obtained using knowledge distillation. If the highest logit in the teacher model represents an inaccurate class, the highest logit of that inaccurate class is replaced with the logit of the accurate class, thereby generating an accurate class that points to the highest logit. This corrects the incorrectly predicted logit before the teacher model passes it to the student model, ensuring that the student model always learns the correct knowledge.

[0143] The improved YOLOv4-tiny target detection model includes a backbone feature extraction network, a neck enhancement feature extraction network, and a YOLO Head module;

[0144] like Figure 2As shown, the backbone feature extraction network is a lightweight Ghost convolutional feature extraction network improved based on the multi-scale dilated convolution module. It includes a first depthwise separable convolution module, a second depthwise separable convolution module, a first CSP_Ghost multi-scale dilated convolutional feature extraction network module, a second CSP_Ghost multi-scale dilated convolutional feature extraction network module, a third CSP_Ghost multi-scale dilated convolutional feature extraction network module, and a third depthwise separable convolution module connected in sequence. Among them, one branch of the second CSP_Ghost multi-scale dilated convolutional feature extraction network module performs a downward convolution and connects to the third CSP_Ghost multi-scale dilated convolutional feature extraction network module, while the other branch is directly connected to the connection module of the neck reinforcement feature extraction network.

[0145] like Figure 3 As shown, the CSP_Ghost multi-scale dilated convolutional feature extraction network module includes a first to a third Ghost dilated convolutional module connected in sequence. The first Ghost dilated convolutional module is concatenated to a connection module, the second and third Ghost dilated convolutional modules are concatenated to a connection module, and then a 1×1 convolutional module is connected to obtain features and adjust the number of channels. Features are then concatenated with the first Ghost spatial convolutional module through the connection module, and finally connected to a max pooling module to complete the downsampling of the image. The first Ghost dilated convolutional module is used to maintain the original number of channels, the second and third Ghost dilated convolutional modules are used to compress the number of channels, the number of output channels is adjusted through the 1×1 convolutional module to obtain features, and then concatenated with the first Ghost dilated convolutional module for max pooling.

[0146] like Figure 4 As shown, the Ghost dilated convolution module introduces dilated convolutions with different dilation rates to obtain features at different scales without increasing the number of parameters. When the dilation rate D = 2 or 3, the corresponding receptive fields are 5 or 7. Compared to the ordinary 3×3 convolution, this structure solves the problem of "large and dense" features during feature extraction, speeds up training, reduces the number of parameters, and improves feature representation capabilities.

[0147] The neck enhancement feature extraction network includes a connection module connected to the third CSP_Ghost multi-scale dilated convolution feature extraction network module and a Conv convolution module connected to the third depth separable convolution module. Finally, it fuses features from two different scales and outputs them to the YOLO Head module.

[0148] The YOLO Head module is connected to the connection module of the neck enhancement feature extraction network and the Conv convolution module, respectively, and converts the features extracted by the neck enhancement feature extraction network into prediction results.

[0149] Knowledge distillation is used to compress the model, resulting in a trained student model. The loss function used in knowledge distillation is:

[0150]

[0151] Where CE represents the cross-entropy, (σO) S ,1) represents the student model O S Softening is performed at a temperature of 1, where 'a' represents the hyperparameter, 'KL' represents the divergence, and 'Y' is a one-bit thermally encoded label; A k In this context, k represents the most likely predicted category, and A... k Let m represent a specialist model that includes k in all prediction categories, m represent a specialist model that includes k in a specific prediction category, q represent the prediction probability value for each category, and p represent the prediction probability value for each category. m Represents the probability distribution of the generalized full model;

[0152] The elevator control module is associated with the elevator overload detection module. After receiving a pulse signal from the image analysis module, it triggers the elevator overload control command in the elevator overload detection module, controlling the elevator to stop running. After the elevator control module is triggered, it receives a pulse signal sent by the image analysis module again and stops running, so the elevator can be used normally.

[0153] The warning module can be a common buzzer used to issue alarm signals.

[0154] This invention can be applied to any terminal device, such as a mobile phone, tablet, server, etc. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. The non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0155] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units or modules as needed, that is, the internal structure of the storage device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application.

[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for preventing a battery car from entering an elevator based on target detection, characterized by Comprising the following steps: Step 1, constructing an initial data set: collecting N-day data through an elevator monitoring camera, collecting a preset number of images from monitoring videos of various electric vehicles, and obtaining an initial data set; Step 2, data augmentation step on the initial data set; Step 3, marking the data set: marking the electric vehicles in the data set; Step 4, build improved YOLOv4 Tiny target detection model: improved YOLOv4 The tiny target detection model comprises a backbone feature extraction network, a neck reinforced feature extraction network and a Yolo Head module. The backbone feature extraction network is a lightweight Ghost convolution feature extraction network improved on the basis of a multi-scale hollow convolution module, comprising a first depth separable convolution module, a second depth separable convolution module, a first CSP_Ghost multi-scale hollow convolution feature extraction network module, a second CSP_Ghost multi-scale hollow convolution feature extraction network module, a third CSP_Ghost multi-scale hollow convolution feature extraction network module and a third depth separable convolution module connected in turn; wherein one branch of the second CSP_Ghost multi-scale hollow convolution feature extraction network module is connected downward, connecting the third CSP_Ghost multi-scale hollow convolution feature extraction network module, and the other branch is directly connected with the connecting module of the neck enhanced feature extraction network; The multi-scale hollow convolution feature extraction network module comprises first to third Ghost hollow convolution modules connected in turn, wherein the first Ghost hollow convolution module is spliced to the connecting module, the second and third Ghost hollow convolution modules are spliced to the connecting module, and then connected to a 1×1 convolution module to obtain features and adjust the number of channels, then connected to the connecting module and the first Ghost spatial convolution module for splicing, and finally connected to a max pooling module to complete the down-sampling of the image; wherein the first Ghost hollow convolution module is used to maintain the original number of channels, the second and third Ghost hollow convolution modules are used to compress the number of channels, the number of output channels is adjusted through the 1×1 convolution module to obtain features, and then spliced with the first Ghost hollow convolution module for max pooling; The Ghost hollow convolution module introduces hollow convolutions with different dilation rates to obtain features of different scales; The neck enhanced feature extraction network comprises a connecting module connected with the third CSP_Ghost multi-scale hollow convolution feature extraction network module, and a Conv convolution module connected with the third depth separable convolution module, finally fusing two features of different scales and outputting to a Yolo Head module; The Yolo Head module is connected with the connecting module and the Conv convolution module of the neck enhanced feature extraction network, respectively, and converts the features extracted by the neck enhanced feature extraction network into prediction results; Step 5, training improved YOLOv4 tiny target detection model, through knowledge distillation to get light weight child model YOLOv5 is used to train the data set to obtain a trained teacher model, and knowledge distillation is used to obtain a trained student model; if the highest logit in the teacher model represents an inaccurate class, the maximum logit of the inaccurate class is replaced with the logit of an accurate class, thereby generating an accurate class and pointing to the highest logit, and the student model always learns correct knowledge before the teacher model passes the incorrectly predicted logit to the student model. Step 6, deploy the light-weighted student model to the edge device to obtain an image analysis module; Step 7, the starting control module is triggered in response to the opening of the elevator door, controls all modules to leave standby and enter a working state, and controls the image acquisition module to collect images of people and objects entering the elevator from the elevator surveillance camera; Step 8, the image acquisition module transmits the images of people and objects entering the elevator collected by the elevator surveillance camera to the image analysis module on the edge device after data enhancement; Step 9, the image analysis module adopts improved YOLOv4 The tiny target detection model analyzes and detects the collected image, judges whether the electric vehicle enters the elevator, and if so, transmits a pulse signal to the elevator control module; the elevator control module associated with the overload detection module of the elevator triggers the control instruction of the elevator overload in the elevator overload detection module after receiving the pulse signal of the image analysis module, and controls the elevator to suspend operation; After the elevator control module is triggered, once the image analysis module does not identify the electric vehicle, it is judged that the electric vehicle has left the elevator, and a pulse signal is sent to the elevator control module again to control the elevator control module to stop running, and the elevator can be normally used.

2. The method for preventing electric vehicles from entering elevators based on target detection according to claim 1, characterized in that: The data enhancement step in step 2 is to denoise and enhance the contrast of the images in the initial data set to make the detection target more recognizable, and specifically includes the following steps: The median filter method is used to filter the noise mutation points of the image: (1) In formula (1), a 3x3 operation region is used, f out In formula (1), a 3x3 operation region is used, f out In formula (1), a 3x3 operation region is used, f out In formula (1), a 3x3 operation region is used, f out In formula (1), a 3x3 operation region is used, f out In formula (1), a 3x3 operation region is used, f out In formula (1), a 3x3 operation region is used, f out In formula The contrast of the image is enhanced to make the target detail information not displayed in a specific environment appear, and the number of pixels of different gray levels in the image is first counted: In formula (2), 0≤k≤255, m k Count(k) represents the total number of pixels with the gray level k counted. Then the gray level distribution probability is calculated: The gray level distribution probability is further accumulated: In formula (4), 0≤n≤255; The accumulated gray level distribution probability is used to calculate the new gray level, and then the gray level pixel number is redistributed to obtain the new gray level after the contrast is increased: （5） The pixel of gray level k is mapped with the pixel of and the mapping is completed to obtain the image after contrast enhancement.

3. The method for preventing electric vehicles from entering elevators based on target detection according to claim 1, characterized in that: The trained student model is obtained by using knowledge distillation in step 5, and the loss function used in knowledge distillation is: wherein represents cross-entropy, represents a student model is softened with temperature 1, represents a hyperparameter, represents divergence, is a one-hot encoded label; wherein k represents the most likely predicted class, represents a specialist model that contains k in all predicted classes, represents a specialist model that contains k in a certain predicted class, represents the predicted probability value for each class, represents the generalized full model distribution probability.

4. A management system for preventing a battery car from entering an elevator based on target detection, characterized by: It comprises a starting control module, an image acquisition module, an image analysis module, and an elevator control module, wherein The starting control module is triggered in response to the opening of the elevator door, controls all modules to leave standby and enter a working state, and controls the image acquisition module to collect images of people and objects entering the elevator from the elevator surveillance camera; The image acquisition module transmits the images of people and objects entering the elevator collected by the elevator surveillance camera to the image analysis module after data enhancement; The image analysis module is built and trained by using an improved YOLOv4 The tiny target detection model is obtained by deploying a light-weighted student model obtained through knowledge distillation to an edge device by using an improved YOLOv4 The tiny target detection model analyzes image information and detects a target, judges whether an electric vehicle enters an elevator, and if yes, transmits a pulse signal to an elevator control module; and after the elevator control module is triggered, once the image analysis module does not identify the electric vehicle, it means that the electric vehicle has left the elevator, and then a pulse signal is transmitted to the elevator control module again to control the elevator control module to stop running, so that the elevator can be normally used. The improved YOLOv4 The tiny target detection model comprises a backbone feature extraction network, a neck reinforced feature extraction network and a Yolo Head module. The backbone feature extraction network is a light-weighted Ghost convolution feature extraction network improved based on a multi-scale hollow convolution module, and comprises a first depth separable convolution module, a second depth separable convolution module, a first CSP_Ghost multi-scale hollow convolution feature extraction network module, a second CSP_Ghost multi-scale hollow convolution feature extraction network module, a third CSP_Ghost multi-scale hollow convolution feature extraction network module, and a third depth separable convolution module connected in sequence; wherein one branch of the second CSP_Ghost multi-scale hollow convolution feature extraction network module is connected to the third CSP_Ghost multi-scale hollow convolution feature extraction network module through down convolution, and the other branch is directly connected to the connection module of the neck reinforced feature extraction network. The multi-scale hollow convolution feature extraction network module comprises first to third Ghost hollow convolution modules connected in sequence, wherein the first Ghost hollow convolution module is spliced to a connection module, the second and third Ghost hollow convolution modules are spliced to the connection module, and then a 1*1 convolution module is connected to obtain features and adjust the number of channels, and then connected to the connection module and the first Ghost spatial convolution module for splicing, and finally connected to a maximum pooling module to complete the down-sampling of the image; wherein the first Ghost hollow convolution module is used to maintain the original number of channels, the second and third Ghost hollow convolution modules are used to compress the number of channels, the number of output channels is adjusted through the 1*1 convolution module to obtain features, and then spliced with the first Ghost hollow convolution module for maximum pooling; The Ghost hollow convolution module introduces hollow convolutions with different dilation rates to obtain features of different scales. The neck strengthening feature extraction network comprises a connection module connected with the third CSP_Ghost multi-scale hollow convolution feature extraction network module, and a Conv convolution module connected with the third depth separable convolution module, and finally fuses the features of two different scales and outputs to the Yolo Head module. The Yolo Head module is connected with the connection module and the Conv convolution module of the neck strengthening feature extraction network, respectively, and converts the features extracted by the neck strengthening feature extraction network into prediction results. The improved YOLOv4 is trained Tiny target detection model, using YOLOv5 to train the data set, get the trained teacher model, use knowledge distillation to obtain the trained student model; if the highest logit in the teacher model represents an inaccurate class, replace the highest logit with the logit of the accurate class, so as to produce an accurate class and point to the highest logit, correct the logit of the teacher model before it is transmitted to the student model, so that the student model always learns the correct knowledge; The elevator control module is associated with the overload detection module of the elevator, and after receiving the pulse signal of the image analysis module, triggers the control instruction of the elevator overload in the elevator overload detection module to control the elevator to pause running; after the elevator control module is triggered, the pulse signal sent by the image analysis module is received again to stop running, and then the elevator can be normally used.

5. The management system for preventing the entry of a battery car into an elevator based on target detection according to claim 4, characterized by: The data enhancement in the image acquisition module is to denoise and enhance the contrast of the images in the initial data set, so that the detection target is more recognizable, and the specific steps include: The median filter method is used to filter the noise mutation points of the image: (1) In formula (1), a 3x3 operation region is adopted, f out In formula (1), a 3x3 operation region is adopted, f out In formula (1), a 3x3 operation region is adopted, f out In formula (1), a 3x3 operation region is adopted, f out In formula (1), a 3x3 operation region is adopted, f out In formula (1), a 3x3 operation region is adopted, f out In formula (1), a 3x3 operation region is adopted, f out In formula The contrast of the image is enhanced to make the target detail information not displayed in a specific environment appear, and the number of pixels of different gray levels in the image is counted first: In formula (2), 0≤k≤255, m k Count(k) represents the total number of pixels with the gray level k counted. Then the gray level distribution probability is calculated: The gray level distribution probability is further accumulated: In formula (4), 0≤n≤255; The accumulated gray level distribution probability is used to calculate the new gray level, and then the gray level pixel number is redistributed to obtain the new gray level after the contrast is increased: （5） The pixel of gray level k is mapped with the pixel of and the mapping is completed to obtain the image with enhanced contrast.

6. The management system for preventing the entry of a battery car into an elevator based on target detection according to claim 4, characterized by The model compression is performed using knowledge distillation to obtain the trained student model, and the loss function used in the knowledge distillation is: wherein represents cross-entropy, represents a student model is softened with temperature 1, represents a hyperparameter, represents divergence, is a one-hot encoded label; wherein k represents the most likely predicted class, represents a specialist model that contains k in all predicted classes, represents a specialist model that contains k in a certain predicted class, represents the predicted probability value for each class, represents the generalized full model distribution probability.

Images