Pedestrian detection method and system for marine vessels

Abstract

A method and system for pedestrian detection on maritime vessels, comprising the following steps: Step S1, Image Acquisition: First, frames are captured from real-time maritime video to obtain original images; Step S2, Detection Area Marking: Detection areas are manually marked on the obtained original images, and calibration cloths are placed in these areas; Step S3, Detection Area Classification: The marked detection areas are sent to a first classification algorithm for classification. Only when the hull portion is present will the entire image be sent to the detection algorithm; Step S4, Full Image Detection: The detection algorithm detects the entire image, identifying the positions of people and calibration cloths, and checking the neatness of the calibration cloths. If a pedestrian obstructs the calibration cloth, a report is sent, indicating that the pedestrian should leave the area covered by the calibration cloth. This invention reduces image inference time by adjusting the model input to decrease the size of the black border during adaptive scaling of the letterbox.

Description

Technical Field

[0001] This invention discloses a pedestrian detection method, particularly a pedestrian detection method and system for marine vessels, which relates to the field of CV algorithm technology. Background Technology

[0002] Pedestrian detection is widely used in many scenarios, but there is no existing technology that can detect pedestrians on ships while they are anchored at sea.

[0003] Currently, most commonly used pedestrian detection methods rely on deep learning to detect pedestrians in image or video data. Leveraging the powerful detection and recognition capabilities of AI, these methods can effectively replace and save on manual labor. Conventional pedestrian detection typically uses video or image capture cameras positioned at or slightly above the human body, combining this with facial recognition technology. However, this method is not suitable for pedestrian detection on ships. Summary of the Invention

[0004] To address the shortcomings of existing pedestrian detection methods that are not applicable to pedestrian detection on ships, this invention provides a pedestrian detection method and system for marine vessels. The detection algorithm used is the YOLOv5s algorithm from the YOLO series, and the classification algorithm is the CLSsification algorithm. By combining the classification algorithm and the detection algorithm, high detection and recognition accuracy is achieved.

[0005] The technical solution adopted by this invention to solve its technical problem is: a method for pedestrian detection on marine vessels, which includes the following steps:

[0006] Step S1, Image Acquisition: First, frames are extracted from the real-time maritime video to obtain the original images;

[0007] Step S2, Detection Area Marking: Manually mark the detection area on the obtained original image, and place the calibration cloth in the area for calibration;

[0008] Step S3, Detection Area Classification: The marked detection area is sent to the first classification algorithm to classify the area. Only when the hull part appears will the entire image be sent to the detection algorithm.

[0009] Step S4, Full Image Detection: The detection algorithm will detect the entire image, identify the positions of people and the calibration cloth, and whether the calibration cloth is clean. If a pedestrian is blocking the calibration cloth, it will be reported and the person will be prompted to leave the calibration cloth area.

[0010] A pedestrian detection system for marine vessels, the system comprising:

[0011] Image acquisition unit: used to extract frames from real-time maritime video to obtain original images;

[0012] Detection area marking unit: used to manually mark the detection area on the obtained original image, and a calibration cloth for calibration is placed in the area;

[0013] Detection region classification unit: used to send the obtained marked detection region to the first classification algorithm to classify the region. Only when the hull part is found will the entire image be sent to the detection algorithm.

[0014] Full-map detection unit: Used to detect the entire map, including the positions of people and the calibration cloth, and whether the calibration cloth is clean. If a pedestrian is blocking the calibration cloth, it will be reported and the person will be prompted to leave the calibration cloth area.

[0015] The technical solution adopted by the present invention to solve its technical problem further includes:

[0016] In step S1, the image obtained by frame extraction is image data of size 1920×1080 pixels.

[0017] In step S3, a classification-then-detection approach is adopted, and the classification algorithm used is the clssification algorithm. The core feature extraction part of the entire algorithm is the lightweight algorithm MobilenetV2. During model training, the modification to the algorithm is that a softmax layer is directly inserted into the final output of the neural network structure.

[0018] In step S4, the detection algorithm uses a pruned version of the YOLOv5s algorithm model. The size of the YOLOv5s training input model, the adjustment of the weights of each class, and the loss function have all been optimized. Specifically, the first part is the optimization of the rectangular input size, the second part is the manual adjustment of the weights of each class, and the third part is the tuning of the loss function.

[0019] The first part of the optimization of the rectangular input size changes from the conventional 1:1 square input to a rectangular input with a non-1:1 aspect ratio for training, with the input image having a width and height of 1920 × 1080.

[0020] The specific method for manually adjusting the weights of each category in the second part is as follows:

[0021] (1) Count the number of samples in each category;

[0022] (2) Calculate the mean of the total sample categories based on the obtained total sample size and number of categories;

[0023] (3) Use the average of the total sample categories to divide the value obtained by the number of samples in each category as the weight ratio of each category. When the weight value of a category is greater than 2, the weight ratio of that category is directly adopted as 2.0. When the weight value of a category is less than 0.6, the weight ratio of that category is directly adopted as 0.6.

[0024] (4) Add the weight ratio of each category to a one-dimensional list, convert the list into a tensor and assign weights to BCEWithLogitsLoss. When assigning weights, only adjust the loss of BCEcls, and do not add weights to the loss of BCEobj.

[0025] The third part, optimizing the loss function, involves replacing the Focal loss with VFloss to improve the detection rate. The calculation formula for VFloss is as follows:

[0026]

[0027] The target score for the predicted IACS is set as follows: for foreground points, its ground truth class is set to the IOU between the generated bounding box and its ground truth (gt_iou), otherwise it is 0. For background points, the target is 0 for all classes. In the above formula, p represents the model prediction value, q represents the label true value, α represents the balancing factor of the negative part of zoom loss, and γ represents the modulation factor between the foreground class and the background class. In step S4, when the detection module performs full-image detection, the presence of ships in the calibration area obtained by the classification model indicates that the ships are entering the calibration area. At this time, the entire image needs to be detected to identify the position of the calibration cloth. It is then determined whether the entire calibration cloth is already within the calibration area. Based on the assumption that the calibration cloth is within the calibration area, it is determined whether the calibration cloth is clean, i.e., without any objects or people obstructing it. The area of ​​the calibration cloth frame detected in the image is used as the target area. It is determined whether the frames of other categories intersect with this area to determine whether the calibration cloth is obstructed. The detection object is the pedestrian category. Pedestrians are detected, and their positions are bounded to prevent pedestrians from obstructing the calibration cloth. If a pedestrian is detected obstructing the calibration cloth, a report is sent to remind them to leave the area where the calibration cloth is located.

[0028] The beneficial effects of this invention are: by adjusting the model input to reduce the size of the black border during adaptive scaling of the letterbox, the image inference time is reduced; the weight of each category can be manually balanced; the detection rate of the algorithm is improved; and the cleanliness of the calibration cloth is effectively determined by AI for pedestrian detection.

[0029] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0030] Figure 1 This is a flowchart of the algorithm of the present invention.

[0031] Figure 2 This is a screenshot of the actual detection process of the present invention. Figure 1 .

[0032] Figure 3 This is a screenshot of the actual detection process of the present invention. Figure 2 .

[0033] Figure 4 This is a screenshot of the actual detection process of the present invention. Figure 3 .

[0034] Figure 5 This is a screenshot of the actual detection process of the present invention. Figure 4 . Detailed Implementation

[0035] This embodiment is a preferred embodiment of the present invention. All other embodiments that are the same as or similar to this embodiment in principle and basic structure are within the protection scope of the present invention.

[0036] This invention relates to a method for pedestrian detection on ships at sea. Its main purpose is to detect pedestrians on board ships returning from offshore operations and to ensure that there are no pedestrians obstructing the calibration cloth within the calibration range.

[0037] Please refer to the appendix. Figure 1 The present invention mainly includes the following steps:

[0038] Step S1, Image Acquisition: First, frames are extracted from the real-time maritime video to obtain the original images;

[0039] In this embodiment, image acquisition is a preliminary preparatory step, mainly involving the extraction of images from the real-time recording. In this embodiment, the images obtained by frame extraction are 1920×1080 pixel images. In actual implementation, other image sizes can be selected according to actual needs.

[0040] Step S2, Detection Area Marking: The detection area is manually marked on the obtained original image. A calibration cloth for calibration will be placed in this area. In practice, the manually marked detection area is not fixed and is determined by the port's import and export location.

[0041] Due to differences in port locations, lighthouse locations, and other factors, the angles at which ships anchor may vary. Therefore, it is necessary to calibrate the detection areas for different port video recordings. The calibrated areas then serve as input data for the classification algorithm to classify those areas.

[0042] Step S3, Detection Area Classification: The obtained calibrated detection area is sent to the first classification algorithm to classify the area. Only when the hull part appears will the entire image be sent to the detection algorithm.

[0043] Most of the time, there are no ships anchored at sea, and the most intuitive representation in the image is that the entire picture is seawater. Directly detecting such data presents a significant challenge: the computing power of the development board is insufficient. With only 0.8T of computing power, achieving real-time detection around the clock is simply impossible. Therefore, a strategy adjustment is necessary. In this embodiment, a classification-then-detection approach is adopted. Since classification algorithms are much faster than detection algorithms, the classification algorithm used in this embodiment is CLSCIMENT. The core feature extraction part of the algorithm is the classic lightweight algorithm MobileNetV2. During model training, the modified part of the algorithm in this embodiment is the insertion of a softmax layer directly into the final output of the neural network structure. After the softmax function, the model output vector values ​​highlight the largest value and suppress other components far below the maximum value, making it easier to directly obtain the predicted image category and score from the maximum value in the vector. In this embodiment, a 112×112 image is used for training, and the resulting algorithm model can achieve an inference speed of 5 milliseconds per frame, demonstrating perfect real-time performance and effectively reducing the computing power burden on the development board.

[0044] In this embodiment, the entire classification only applies to the calibrated detection area, and is mainly divided into two categories: seawater is classified into one category, and areas containing part or all of a ship's hull are classified into another category. Whenever a ship is detected within a detection area, the entire image is fed into the detection algorithm for full-image detection.

[0045] Step S4, Full Image Detection: The detection algorithm will detect the entire image, mainly to detect the position of people and the calibration cloth, and whether the calibration cloth is clean. If a pedestrian is blocking the calibration cloth, it will be reported and the person will be prompted to leave the calibration cloth area.

[0046] In this embodiment, the detection algorithm uses YOLOv5s, specifically a pruned version. Unlike conventional methods, this embodiment optimizes the size of the YOLOv5s training input model, the weights of each class, and the loss function. The optimization is divided into three parts: first, optimizing the size of the rectangular input; second, manually adjusting the weights of each class; and third, fine-tuning the loss function.

[0047] In this embodiment, the first part will be changed from the conventional 1:1 square input to a rectangular input with a non-1:1 aspect ratio for training. The purpose of this is to reduce the excessive black borders added when scaling the original image with a width-to-height ratio close to 2:1, which would cause the model to spend more time inference in actual use. In this embodiment, the input image is 1920 × 1080 in size, with the height being significantly smaller than the width, and does not reach a 1:1 ratio.

[0048] The second part involves manually assigning weights to each category. Since the project focuses primarily on pedestrians and marker cloths, and data on ship berthing at sea is difficult to obtain, manually assigning bias weights to each category yields better results, especially given the limited data and the significantly larger number of data points for other categories compared to the categories emphasized in this invention. The specific implementation of the second part is as follows:

[0049] (1) Count the number of samples in each category;

[0050] (2) Calculate the mean of the total sample categories based on the obtained total sample size and number of categories;

[0051] (3) The average of the total sample categories is divided by the number of samples in each category as the weight ratio of each category in this invention. However, it should be noted that when the weight value of a certain category is greater than 2, the weight ratio of that category is directly adopted as 2.0 in this embodiment. When the weight value of a certain category is less than 0.6, the weight ratio of that category is directly adopted as 0.6 in this embodiment. It is limited that the weight ratio of each category cannot be much greater than or much less than that of other categories due to the large difference in the amount of data.

[0052] (4) Add the weight ratio of each category to a one-dimensional list, convert the list into a tensor and assign weights to BCEWithLogitsLoss. It should be noted that this embodiment only affects the classification decoupling head in the entire detection algorithm, so only the loss of BCEcls is adjusted, and no weights are added to the loss of BCEobj.

[0053] The third part is the optimization of the loss function. In this embodiment, without this third part, it was found that the detection rate of the model optimized only by the first and second parts was significantly reduced. The main problem was severe missed detection when multiple targets appeared. To solve this problem, optimization of the loss function was attempted. In this embodiment, replacing Focal loss with VFloss can improve the detection rate, but it has a drawback: it greatly extends the model training time. VFloss, short for Variational Loss, is an IACS that can simultaneously represent the confidence of object presence and the accuracy of localization. It is used to predict dense object detectors. This embodiment also tried adding OTA, but it did not yield good results and was significantly worse than the results of VFloss. Therefore, it was ultimately decided to use VFloss to replace Focal loss. The calculation formula for VFloss is as follows:

[0054]

[0055] The VFloss calculation formula is a publicly available formula, not the formula developed in this invention. In this embodiment, it is merely reproduced and applied in the algorithm. Here, is the predicted IACS, which is the target score. For foreground points, its ground truth class is set to the IOU between the generated bounding box and its ground truth (gt_iou), otherwise it is 0. For background points, the target for all classes is 0. In the above formula, p represents the model's predicted value, q represents the true label value, α represents the balancing factor for the negative part of the zoom loss, and γ represents the modulation factor between the foreground and background classes.

[0056] When the detection module performs full-image detection, the presence of ships within the calibration area obtained by the classification model indicates that ships are entering the calibration area. At this point, the entire image needs to be detected to pinpoint the location of the calibration cloth. It's then determined whether the entire calibration cloth is within the calibration area. If the cloth is within the calibration area, it's further assessed for cleanliness and the absence of objects or people obstructing it. The detected area of ​​the calibration cloth's bounding box is used as the target region, and other categories' bounding boxes are compared to this region to determine if the cloth is occluded. The primary detection target is pedestrians. Pedestrians are detected and their locations are bounded to prevent them from obstructing the calibration cloth. If a pedestrian is detected obstructing the calibration cloth, a warning is issued, prompting the pedestrian to leave the area where the calibration cloth is located.

[0057] This invention also protects a pedestrian detection system for marine vessels, mainly comprising:

[0058] Image acquisition unit: used to extract frames from real-time maritime video to obtain original images;

[0059] In this embodiment, image acquisition is a preliminary preparatory step, mainly involving the extraction of images from the real-time recording. In this embodiment, the images obtained by frame extraction are 1920×1080 pixel images. In actual implementation, other image sizes can be selected according to actual needs.

[0060] Detection area marking unit: used to manually mark the detection area on the obtained original image. A calibration cloth for calibration will be placed in this area. In practice, the manually marked detection area is not fixed and is determined by the port's import and export location.

[0061] Due to differences in port locations, lighthouse locations, and other factors, the angles at which ships anchor may vary. Therefore, it is necessary to calibrate the detection areas for different port video recordings. The calibrated areas then serve as input data for the classification algorithm to classify those areas.

[0062] Detection area classification unit: used to send the obtained calibrated detection area to the first classification algorithm to classify the area. Only when the hull part is found will the entire image be sent to the detection algorithm.

[0063] Most of the time, there are no ships anchored at sea, and the most intuitive representation in the image is that the entire picture is seawater. Directly detecting such data presents a significant challenge: the computing power of the development board is insufficient. With only 0.8T of computing power, achieving real-time detection around the clock is simply impossible. Therefore, a strategy adjustment is necessary. In this embodiment, a classification-then-detection approach is adopted. Since classification algorithms are much faster than detection algorithms, the classification algorithm used in this embodiment is CLSCIMENT. The core feature extraction part of the algorithm is the classic lightweight algorithm MobileNetV2. During model training, the modified part of the algorithm in this embodiment is the insertion of a softmax layer directly into the final output of the neural network structure. After the softmax function, the model output vector values ​​highlight the largest value and suppress other components far below the maximum value, making it easier to directly obtain the predicted image category and score from the maximum value in the vector. In this embodiment, a 112×112 image is used for training, and the resulting algorithm model can achieve an inference speed of 5 milliseconds per frame, demonstrating perfect real-time performance and effectively reducing the computing power burden on the development board.

[0064] In this embodiment, the entire classification only applies to the calibrated detection area, and is mainly divided into two categories: seawater is classified into one category, and areas containing part or all of a ship's hull are classified into another category. Whenever a ship is detected within a detection area, the entire image is fed into the detection algorithm for full-image detection.

[0065] Full-map detection unit: Used to detect the entire map. Its main functions are to detect the position of people and the calibration cloth, and to check whether the calibration cloth is clean. If a pedestrian is blocking the calibration cloth, it will be reported and the person will be prompted to leave the calibration cloth area.

[0066] In this embodiment, the detection algorithm uses YOLOv5s, specifically a pruned version. Unlike conventional methods, this embodiment optimizes the size of the YOLOv5s training input model, the weights of each class, and the loss function. The optimization is divided into three parts: first, optimizing the size of the rectangular input; second, manually adjusting the weights of each class; and third, fine-tuning the loss function.

[0067] In this embodiment, the first part will be changed from the conventional 1:1 square input to a rectangular input with a non-1:1 aspect ratio for training. The purpose of this is to reduce the excessive black borders added when scaling the original image with a width-to-height ratio close to 2:1, which would cause the model to spend more time inference in actual use. In this embodiment, the input image is 1920 × 1080 in size, with the height being significantly smaller than the width, and does not reach a 1:1 ratio.

[0068] The second part involves manually assigning weights to each category. Since the project focuses primarily on pedestrians and marker cloths, and data on ship berthing at sea is difficult to obtain, manually assigning bias weights to each category yields better results, especially given the limited data and the significantly larger number of data points for other categories compared to the categories emphasized in this invention. The specific implementation of the second part is as follows:

[0069] (1) Count the number of samples in each category;

[0070] (2) Calculate the mean of the total sample categories based on the obtained total sample size and number of categories;

[0071] (3) The average of the total sample categories is divided by the number of samples in each category as the weight ratio of each category in this invention. However, it should be noted that when the weight value of a certain category is greater than 2, the weight ratio of that category is directly adopted as 2.0 in this embodiment. When the weight value of a certain category is less than 0.6, the weight ratio of that category is directly adopted as 0.6 in this embodiment. It is limited that the weight ratio of each category cannot be much greater than or much less than that of other categories due to the large difference in the amount of data.

[0072] (4) Add the weight ratio of each category to a one-dimensional list, convert the list into a tensor and assign weights to BCEWithLogitsLoss. It should be noted that this embodiment only affects the classification decoupling head in the entire detection algorithm, so only the loss of BCEcls is adjusted, and no weights are added to the loss of BCEobj.

[0073] The third part is the optimization of the loss function. In this embodiment, without this third part, it was found that the detection rate of the model optimized only by the first and second parts was significantly reduced. The main problem was severe missed detection when multiple targets appeared. To solve this problem, optimization of the loss function was attempted. In this embodiment, replacing Focal loss with VFloss can improve the detection rate, but it has a drawback: it greatly extends the model training time. VFloss, short for Variational Loss, is an IACS that can simultaneously represent the confidence of object presence and the accuracy of localization. It is used to predict dense object detectors. This embodiment also tried adding OTA, but it did not yield good results and was significantly worse than the results of VFloss. Therefore, it was ultimately decided to use VFloss to replace Focal loss. The calculation formula for VFloss is as follows:

[0074]

[0075] The VFloss calculation formula is a publicly available formula, not the formula developed in this invention. In this embodiment, it is merely reproduced and applied in the algorithm. Here, is the predicted IACS, which is the target score. For foreground points, its ground truth class is set to the IOU between the generated bounding box and its ground truth (gt_iou), otherwise it is 0. For background points, the target for all classes is 0. In the above formula, p represents the model's predicted value, q represents the true label value, α represents the balancing factor for the negative part of the zoom loss, and γ represents the modulation factor between the foreground and background classes.

[0076] When the detection module performs full-image detection, the presence of ships within the calibration area obtained by the classification model indicates that ships are entering the calibration area. At this point, the entire image needs to be detected to pinpoint the location of the calibration cloth. It's then determined whether the entire calibration cloth is within the calibration area. If the cloth is within the calibration area, it's further assessed for cleanliness and the absence of objects or people obstructing it. The detected area of ​​the calibration cloth's bounding box is used as the target region, and other categories' bounding boxes are compared to this region to determine if the cloth is occluded. The primary detection target is pedestrians. Pedestrians are detected and their locations are bounded to prevent them from obstructing the calibration cloth. If a pedestrian is detected obstructing the calibration cloth, a warning is issued, prompting the pedestrian to leave the area where the calibration cloth is located.

[0077] This invention reduces image inference time by adjusting the model input to decrease the size of the black border during adaptive scaling of the letterbox; it allows for manual balancing of the weights of each category; it improves the algorithm's detection rate; and it effectively uses AI to detect pedestrians and determine whether the calibration cloth is clean.

Claims

1. A method for pedestrian detection on maritime vessels, characterized by: The detection method includes the following steps: Step S1, Image Acquisition: First, frames are extracted from the real-time maritime video to obtain the original images; Step S2, Detection Area Marking: Manually mark the detection area on the obtained original image, and place the calibration cloth in the area for calibration; Step S3, Detection Region Classification: The identified detection regions are sent to the first classification algorithm to classify the regions. Only when the hull part appears will the entire image be sent to the detection algorithm. The method of classification before detection is adopted. The first classification algorithm used is the clssification algorithm. The main feature extraction part of the entire algorithm is the lightweight algorithm MobilenetV2. During model training, the modification to the algorithm is that a softmax layer is directly inserted into the final output of the neural network structure. Step S4, Full Image Detection: The detection algorithm will detect the entire image, identify the positions of people and the calibration cloth, and whether the calibration cloth is clean. If a pedestrian is blocking the calibration cloth, it will be reported, prompting the pedestrian to leave the calibration cloth area. The detection algorithm uses a pruned version of the YOLOv5s algorithm model. The size of the YOLOv5s training input model, the adjustment of the weights of each category, and the loss function have all been optimized. Specifically, the first part is the optimization of the rectangular input size, the second part is the manual adjustment of the weights of each category, and the third part is the tuning of the loss function.

2. The pedestrian detection method for ships at sea according to claim 1, characterized in that: In step S1, the image obtained by frame extraction is image data of size 1920×1080 pixels.

3. The pedestrian detection method for maritime vessels according to claim 1, characterized in that: The first part of the optimization of the rectangular input size changes from the conventional 1:1 square input to a rectangular input with a non-1:1 aspect ratio for training, with the input image having a width and height of 1920 × 1080.

4. The pedestrian detection method for maritime vessels according to claim 1, characterized in that: The specific method for manually adjusting the weights of each category in the second part is as follows: (1) Count the number of samples in each category; (2) Calculate the mean of the total sample categories based on the obtained total sample size and number of categories; (3) Use the average of the total sample categories to divide the value obtained by the number of samples in each category as the weight ratio of each category. When the weight value of a category is greater than 2, the weight ratio of that category is directly adopted as 2.

0. When the weight value of a category is less than 0.6, the weight ratio of that category is directly adopted as 0.

6. (4) Add the weight ratio of each category to a one-dimensional list, convert the list into a tensor and assign weights to BCEWithLogitsLoss. When assigning weights, only adjust the loss of BCEcls, and do not add weights to the loss of BCEobj.

5. The pedestrian detection method for maritime vessels according to claim 1, characterized in that: In the third part, when optimizing the loss function, VFloss is used to replace Focal loss to improve the detection rate. The calculation formula for VFloss is as follows: ， In the above formula, p represents the model prediction value, q represents the true label value, α represents the balancing factor of the negative part of zoom loss, and γ represents the modulation factor between the foreground and background classes.

6. The pedestrian detection method for maritime vessels according to claim 1, characterized in that: In step S4, when the detection module performs full-image detection, the presence of ships in the calibration area obtained by the classification model indicates that the ships are entering the calibration area. At this time, the entire image needs to be detected to identify the position of the calibration cloth. It is then determined whether the entire calibration cloth is already within the calibration area. Based on the assumption that the calibration cloth is within the calibration area, it is determined whether the calibration cloth is clean, i.e., without any objects or people obstructing it. The area of ​​the calibration cloth frame detected in the image is used as the target area. It is determined whether the frames of other categories intersect with this area to determine whether the calibration cloth is obstructed. The detection object is the pedestrian category. Pedestrians are detected, and their positions are bounded to prevent pedestrians from obstructing the calibration cloth. If a pedestrian is detected obstructing the calibration cloth, a report is sent to remind them to leave the area where the calibration cloth is located.

7. A pedestrian detection system for marine vessels, characterized in that: The detection system includes: Image acquisition unit: used to extract frames from real-time maritime video to obtain original images; Detection area marking unit: used to manually mark the detection area on the obtained original image, and a calibration cloth for calibration is placed in the area; The detection region classification unit is used to send the identified detection regions to the first classification algorithm to classify the regions. Only when the hull part is present will the entire image be sent to the detection algorithm. The method of classification before detection is adopted. The first classification algorithm used is the clssification algorithm. The main feature extraction part of the entire algorithm is the lightweight algorithm MobilenetV2. During model training, the modification to the algorithm is to directly insert a softmax layer into the final output of the neural network structure. The full-image detection unit is used to detect the entire image, including the positions of people and the calibration cloth, and whether the calibration cloth is clean. If a pedestrian is found to be obscuring the calibration cloth, it is reported, indicating that the pedestrian should leave the area covered by the calibration cloth. The detection algorithm uses a pruned version of the YOLOv5s algorithm model. The size of the YOLOv5s training input model, the weights of each class, and the loss function have all been optimized. Specifically, the first part optimizes the size of the rectangular input, the second part manually adjusts the weights of each class, and the third part fine-tunes the loss function.

Images