A collision warning method, device and electronic equipment for a suspended fuel dispenser

Abstract

The present application provides a collision warning method, device and electronic device for a suspended fuel dispenser, which inputs the acquired target scene image into a pre-trained object annotation model, outputs the target scene image with an object annotation box annotated with the target object; detects whether there is an overlapping area between the object annotation box and the dangerous area; if there is an overlapping area and the area between the overlapping areas is greater than a preset area threshold, generates a warning message. In this way, the target scene image is acquired in real time and input into the object annotation model, and the pruned object annotation model quickly outputs the target scene image annotated with the target object. According to the area of ​​the overlapping area between the object annotation box annotated with the target object in the target scene image and the pre-set dangerous area, it is determined whether to generate a warning message, and the collision risk at the fuel dispenser can be detected in real time and quickly, which helps to improve the efficiency and timeliness of the collision warning of the suspended fuel dispenser.

Description

Technical Field

[0001] The present application relates to the technical field of risk warning, and in particular to a collision warning method, device and electronic equipment for a suspended refueling machine. Background Art

[0002] With the development of science and technology, the vehicle ownership rate has increased sharply. In this case, traditional gas pumps are still set up on the refueling island. In order to prevent vehicle collisions, anti-collision fences are set up around the refueling island. Vehicles need to be parked at a certain distance from the fences, which occupies a large area. Vehicle traffic is not smooth, and it is impossible to provide refueling services to vehicles effectively and conveniently. It may even cause vehicle congestion at the gas station. Based on the above situation, hanging refueling pumps came into being. Since the hanging refueling pumps are set on the roof of the gas station, they do not occupy the ground area of ​​the gas station. This can reduce the congestion risk at the gas station to a certain extent. However, the hanging refueling pumps may collide with moving vehicles or pedestrians during the descent of the refueling gun, which poses a certain risk.

[0003] At present, the risk warning for the hanging fuel dispenser during the lowering process of the fuel gun is still provided by on-site guidance of the staff, or by manual screening through obtaining scene images near the hanging fuel dispenser. However, regardless of the above methods, the workload of the staff is relatively large and there is no guarantee that the risk can be detected in real time. The efficiency and timeliness of the warning are relatively low. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a collision warning method, device and electronic device for a suspended refueling machine, which inputs the target scene image within a preset range of the suspended refueling machine acquired in real time into an object annotation model, and the pruned object annotation model quickly outputs the target scene image annotated with the target object, and determines whether to generate warning information based on the area of ​​the overlapping area between the object annotation box annotated with the target object in the target scene image and the pre-set danger zone. The collision risk at the refueling machine can be detected in real time and quickly, which helps to improve the efficiency and timeliness of the collision warning of the suspended refueling machine.

[0005] In a first aspect, an embodiment of the present application provides a collision warning method for a suspended fuel dispenser, the collision warning method comprising:

[0006] Acquire a target scene image within a preset range of the suspended refueling machine;

[0007] The target scene image is input into a pre-trained object annotation model, and an annotated target scene image is output; the annotated target scene image displays an object annotation box indicating the area where the target object in the image is located; the object annotation model is a model obtained by pruning the backbone network in the YOLOv5s model to remove the feature extraction layer whose feature extraction efficiency is lower than a preset efficiency threshold;

[0008] Detecting whether there is an overlapping area between the object annotation box and a pre-set dangerous area in the target scene image;

[0009] If so, when the area of ​​the overlapping region is greater than a preset area threshold, a warning message is generated.

[0010] In a possible implementation, the object annotation model includes an input network, a backbone network, a Neck network, and an output network; inputting the target scene image into a pre-trained object annotation model and outputting the annotated target scene image includes:

[0011] Inputting the target scene image into the input network of the object annotation model, scaling the scene image according to a preset image size, and outputting a scaled filling image;

[0012] Inputting the filled image into the backbone network, segmenting and splicing the filled image according to a preset segmentation threshold to obtain a plurality of spliced ​​images, performing feature extraction on each spliced ​​image, and outputting a feature mapping image;

[0013] Input the feature map image into the Neck network, perform feature upsampling fusion, and output a feature fusion image;

[0014] The feature fusion image is input into the output network, and after filtering multiple overlapping anchor frames according to the anchor frames in a preset range, the feature fusion image is annotated using the target anchor frame, and the annotated target scene image is output.

[0015] In a possible implementation, the backbone network includes an image segmentation layer and a first feature extraction layer; the filling image is input into the backbone network, the filling image is segmented and spliced ​​according to a preset segmentation threshold to obtain a plurality of spliced ​​images, and features are extracted from each spliced ​​image to output a feature mapping image, including:

[0016] The filled image is input to the image segmentation layer, and after the image segmentation layer segments the filled image according to a preset segmentation threshold, adjacent pixels in the filled image that meet the preset segmentation threshold are determined as a slice, and pixels at the same position in each slice are spliced ​​to output multiple spliced ​​images;

[0017] The plurality of stitched images are input into the first feature extraction layer, features are extracted from the plurality of stitched images, and the feature mapping image is output.

[0018] In a possible implementation, the Neck network includes a plurality of upsampling feature extraction layers connected in sequence; the step of inputting the feature mapping image into the Neck network, performing feature upsampling fusion, and outputting a feature fusion image includes:

[0019] The feature mapping image is input into a plurality of continuous upsampling feature extraction layers, multiple feature upsampling is performed, and after the preset features after sampling are fused, the feature fusion image is output.

[0020] In a possible implementation, the object annotation model is trained by the following steps:

[0021] Acquire multiple initial sample images, and annotate the area where the target object is located in each initial sample image to obtain multiple annotated sample images;

[0022] The multiple initial sample images and the multiple labeled sample images are input into a pre-constructed YOLOv5s model, so that the YOLOv5s model adjusts the position and size of a pre-set initial anchor frame through each initial sample image and the corresponding labeled sample image. When the loss value of the YOLOv5s model is less than a preset loss threshold, it is determined that the training of the YOLOv5s model is completed, and the object labeling model is obtained; the initial anchor frame is determined after clustering the labeled areas in each labeled sample image.

[0023] In a possible implementation manner, after the position and size of the preset initial anchor frame are adjusted, when the loss of the YOLOv5s model is less than a preset loss threshold, it is determined that the training of the YOLOv5s model is completed, the collision warning method further includes:

[0024] Determine a target pruning channel and / or a target pruning layer in the YOLOv5s model, and a target network structure including the target pruning channel and / or the target pruning layer in the YOLOv5s model;

[0025] Imposing constraints on the scaling coefficient of the BN layer in the target pruning channel and / or the target pruning layer in the target network structure, and determining the output coefficient in the target pruning channel and / or the target pruning layer;

[0026] The target pruned channel and / or target pruned layer in the target network structure has an output coefficient less than the convolution layer and / or convolution channel of a preset coefficient threshold, to obtain a pruned YOLOv5s model, and the pruned YOLOv5s model is determined as the object labeling model.

[0027] In one possible implementation, the loss value of the YOLOv5s model is determined by the following steps:

[0028] Determine the category loss value, target confidence loss value, and bounding box regression loss value of the YOLOv5s model during training;

[0029] The category loss value, target confidence loss value, and border regression loss value are weightedly calculated to determine the loss value of the YOLOv5s model.

[0030] In a possible implementation, when the area of ​​the overlapping region is greater than a preset area threshold, generating warning information includes:

[0031] When the area of ​​the overlapping region is greater than a preset area threshold, the color of the object marking box is filled with a preset warning color to obtain the warning information.

[0032] In a second aspect, an embodiment of the present application further provides a collision warning device for a suspended fuel dispenser, the collision warning device comprising:

[0033] An image acquisition module, used to acquire a target scene image within a preset range of the suspended refueling machine;

[0034] An image annotation module is used to input the target scene image into a pre-trained object annotation model and output an annotated target scene image; the annotated target scene image displays an object annotation box indicating the area where the target object in the image is located; the object annotation model is a model obtained by pruning the backbone network in the YOLOv5s model to remove feature extraction layers whose feature extraction efficiency is lower than a preset efficiency threshold;

[0035] A region detection module, used to detect whether there is an overlapping region between the object annotation frame and a pre-set dangerous region in the target scene image;

[0036] The warning information generating module is used to generate warning information when the area of ​​the overlapping area is greater than a preset area threshold, if any.

[0037] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the collision warning method for a suspended refueling machine as described in any one of the first aspects.

[0038] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the collision warning method for a suspended refueling machine as described in any one of the first aspects are executed.

[0039] The collision warning method, device and electronic device of the hanging tanker provided in the embodiment of the present application obtain the target scene image within the preset range of the hanging tanker, and input the target scene image into the pre-trained object annotation model, and output the target scene image with the object annotation box annotated with the target object area; wherein the object annotation model is a model after pruning the backbone network in the YOLOv5s model, removing the feature extraction layer whose feature extraction efficiency is lower than the preset efficiency threshold; detecting whether there is an overlapping area between the object annotation box and the pre-set dangerous area; if there is an overlapping area between the two and the area between the overlapping areas is greater than the preset area threshold, generating warning information. In the embodiment of the present application, the target scene image within the preset range of the hanging tanker obtained in real time is input into the object annotation model, and the pruned object annotation model quickly outputs the target scene image annotated with the target object, and determines whether to generate warning information according to the area of ​​the overlapping area between the object annotation box annotated with the target object in the target scene image and the pre-set dangerous area, so that the collision risk at the tanker can be detected in real time and quickly, which helps to improve the efficiency and timeliness of the collision warning of the hanging tanker.

[0040] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1A flowchart of a collision warning method for a suspended fuel dispenser provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of another collision warning method for a suspended fuel dispenser provided in an embodiment of the present application;

[0044] Figure 3 One of the structural schematic diagrams of a collision warning device for a suspended fuel dispenser provided in an embodiment of the present application;

[0045] Figure 4 The second structural schematic diagram of a collision warning device for a suspended fuel dispenser provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0048] First, the application scenarios to which this application can be applied are introduced. This application can be applied to the field of risk warning technology. With the development of science and technology, the vehicle ownership rate has increased sharply. In this case, the traditional refueling machine is still set on the refueling island. In order to prevent vehicle collisions, anti-collision fences are set around the refueling island. The vehicle needs to be parked at a certain distance from the fence, which occupies a large area, and the vehicle traffic is not smooth. It is impossible to provide refueling services to the vehicle effectively and conveniently, and it may even cause vehicle congestion at the gas station. Based on the above situation, a suspended refueling machine came into being. The suspended refueling machine includes a refueling metering unit, an electrical control unit, a suspended refueling device, a handheld terminal, a display device, an operating switch, an oil and gas recovery device and other parts. The traditional refueling machine is divided into several modules according to its function, and the refueling device is hung on the canopy of the gas station to achieve efficient and safe refueling operations, empowering gas stations to improve quality and efficiency.

[0049] However, since the suspended fuel dispenser is installed on the roof of the gas station, it does not occupy the ground area of ​​the gas station. This can reduce the risk of congestion at the gas station to a certain extent. However, when the fuel gun of the suspended fuel dispenser is lowered, it may collide with moving vehicles or pedestrians, which poses a certain risk.

[0050] At present, the risk warning for the hanging fuel dispenser during the lowering process of the fuel gun is still provided by on-site guidance of the staff, or by manual screening through obtaining scene images near the hanging fuel dispenser. However, regardless of the above methods, the workload of the staff is relatively large and there is no guarantee that the risk can be detected in real time. The efficiency and timeliness of the warning are relatively low.

[0051] Based on this, an embodiment of the present application provides a collision warning method for a suspended fuel dispenser to improve the efficiency and timeliness of the collision warning of the suspended fuel dispenser.

[0052] See also Figure 1 , Figure 1 This is a flow chart of a collision warning method for a suspended fuel dispenser provided in an embodiment of the present application. Figure 1 As shown in the figure, the collision warning method of the suspended fuel dispenser provided in the embodiment of the present application includes:

[0053] S101, acquiring a target scene image within a preset range of the suspended refueling machine.

[0054] S102, inputting the target scene image into a pre-trained object annotation model, and outputting an annotated target scene image; the annotated target scene image displays an object annotation box indicating the area where the target object in the image is located; the object annotation model is a model obtained by pruning the YOLOv5s model to remove feature extraction layers whose feature extraction efficiency is lower than a preset efficiency threshold.

[0055] S103: Detect whether there is an overlapping area between the object annotation box and a preset dangerous area in the target scene image.

[0056] S104: If so, when the area of ​​the overlapping region is greater than a preset area threshold, generate a warning message.

[0057] A collision warning method for a suspended refueling machine provided in an embodiment of the present application inputs a target scene image within a preset range of the suspended refueling machine acquired in real time into an object annotation model, and the pruned object annotation model quickly outputs a target scene image annotated with a target object, and determines whether to generate warning information based on the area of ​​an overlapping area between an object annotation box annotating the target object in the target scene image and a pre-set danger zone. This method can detect the collision risk at the refueling machine in real time and quickly, and helps to improve the efficiency and timeliness of the collision warning of the suspended refueling machine.

[0058] The following is a description of the exemplary steps of the present application embodiment:

[0059] S101, acquiring a target scene image within a preset range of the suspended refueling machine.

[0060] In the embodiment of the present application, it is aimed at the scenario of collision warning during the falling process of the refueling gun of a suspended refueling pump. The suspended refueling pump is a device that divides a traditional refueling pump into several modules according to its functions, and suspends the refueling device on the canopy of the gas station to realize efficient and safe refueling operations.

[0061] In a possible implementation, a camera may be disposed at the bottom of the suspended fuel dispenser to capture scene images within a certain range of the suspended fuel dispenser.

[0062] In a possible implementation, the camera can shoot a scene video within a preset range of the suspended refueling machine in real time, capture each frame of the scene video, and obtain a target scene image to achieve a technical effect of obtaining a scene image in real time.

[0063] S102, inputting the target scene image into a pre-trained object annotation model, and outputting an annotated target scene image; the annotated target scene image displays an object annotation box indicating the area where the target object in the image is located; the object annotation model is a model obtained by pruning the backbone network in the YOLOv5s model to remove the feature extraction layer whose feature extraction efficiency is lower than a preset efficiency threshold.

[0064] In an embodiment of the present application, after acquiring the target scene image, the target scene image is input into a pre-trained object annotation model, and the object annotation model annotates the area where the target object is located on the target scene image through an object annotation box, and outputs the annotated target scene image.

[0065] In one possible implementation, the target object may be a pedestrian, vehicle, or obstacle (for example, a stone on the ground, etc.) near the suspended fuel dispenser. When the target object is a pedestrian or a vehicle, the purpose of performing a collision warning in the embodiment of the present application is to prevent the suspended fuel dispenser from colliding with the pedestrian or vehicle during the descent of the fuel dispenser, thereby causing danger (injuring pedestrians or scratching vehicles, etc.); when the target object is an obstacle, the purpose of performing a collision warning in the embodiment of the present application is to prevent the suspended fuel dispenser from colliding with the obstacle during the descent of the fuel dispenser, thereby affecting the descent of the fuel gun of the suspended fuel dispenser, or damaging the fuel gun.

[0066] In one possible implementation, the object labeling model in the embodiment of the present application uses a YOLOv5s model, and in order to reduce the size of the YOLOv5s model and improve the computational efficiency of the YOLOv5s model, the feature extraction layer (convolutional layer) of the YOLOv5s model is pruned during the training of the YOLOv5s model, and a feature extraction layer in the YOLOv5s model whose feature extraction efficiency is lower than a preset efficiency threshold is proposed to reduce the data processing steps of the YOLOv5s model and improve the efficiency of the trained object labeling model in labeling target objects in target scene images.

[0067] Specifically, the network structure of Yolov5 can be divided into four parts: input structure, backbone network, Neck network, and output network. The following will combine the network structure of the object annotation model to explain the process of the object annotation model processing the target scene image:

[0068] Specifically, the step of “inputting the target scene image into a pre-trained object annotation model and outputting the annotated target scene image” includes:

[0069] a1: Input the target scene image into the input network of the object annotation model, scale the scene image according to a preset image size, and output the scaled filling image.

[0070] In an embodiment of the present application, the target scene image is input into the input network of the object annotation model. Since the default input image size is 640×640×3 for the object annotation model, after the target scene image is input into the object annotation model, adaptive image scaling is performed. For target scene images with an image size less than 640×640×3, a filled image can be obtained by filling the image with a preset color (which can be black (0, 0, 0), but for the YOLOv5s model, the image is filled with gray (114, 114, 114)).

[0071] In a possible implementation, the target scene image may be downsampled by 32 times, 16 times, and 8 times in the input network, and a filled image may be obtained after feature extraction.

[0072] a2: Input the filled image into the backbone network, segment and splice the filled image according to a preset segmentation threshold to obtain multiple spliced ​​images, perform feature extraction on each spliced ​​image, and output a feature mapping image.

[0073] In an embodiment of the present application, after obtaining the filled image through the input end, the backbone network will perform feature extraction on the filled image. Specifically, the filled image is first segmented according to a preset segmentation threshold, and then the image slices obtained after the segmentation are spliced ​​to obtain multiple spliced ​​images. After feature extraction of the multiple spliced ​​images, the backbone network outputs a feature mapping image.

[0074] Among them, for the backbone network, the backbone network includes an image segmentation layer and a first feature extraction layer. The image processing process can be roughly divided into two parts. One part is the segmentation and splicing of the image, and the other part is feature extraction of the spliced ​​image and output of the feature mapping image.

[0075] Specifically, the step of "inputting the filled image into the backbone network, segmenting and splicing the filled image according to a preset segmentation threshold to obtain a plurality of spliced ​​images, and extracting features from each spliced ​​image to output a feature mapping image" includes:

[0076] b1: Input the filled image to the image segmentation layer. After the image segmentation layer segments the filled image according to a preset segmentation threshold, the adjacent pixels in the filled image that meet the preset segmentation threshold are determined as a slice, and the pixels at the same position in each slice are spliced ​​to output multiple spliced ​​images.

[0077] In an embodiment of the present application, the filled image is input into the image segmentation layer, and the image segmentation layer segments the filled image according to a preset segmentation threshold, and adjacent pixels in the filled image that meet the preset segmentation threshold are determined as a slice. Among the multiple slices determined, the pixels at the same position in each slice are spliced ​​to obtain multiple spliced ​​images.

[0078] Specifically, each 2x2 adjacent pixel in the padded image is divided into a slice, and then the pixels at the same position in each slice are pieced together to obtain four feature maps.

[0079] b2: inputting the plurality of stitched images into the first feature extraction layer, performing feature extraction on the plurality of stitched images, and outputting the feature mapping image.

[0080] In the embodiment of the present application, the first feature extraction layer is a convolution layer. Specifically, the first feature extraction layer can be a 32-channel convolution layer. After feature extraction by the first feature extraction layer, a 304×304×32 feature mapping image can be obtained.

[0081] a3: Input the feature mapping image into the Neck network, perform feature upsampling fusion, and output a feature fusion image.

[0082] In an embodiment of the present application, after the feature mapping image is output through the backbone network, the obtained feature mapping image is input into the Neck network, the feature mapping image is upsampled, feature extraction and feature fusion are continued, and then a feature fusion image is output.

[0083] In a possible real-time method, the Neck network includes a plurality of sequentially connected upsampling feature extraction layers, and the step of "inputting the feature mapping image into the Neck network, performing feature upsampling fusion, and outputting a feature fusion image" includes:

[0084] c1: inputting the feature mapping image into a plurality of consecutive upsampling feature extraction layers, performing multiple feature upsampling, fusing the sampled preset features, and outputting the feature fusion image.

[0085] In an embodiment of the present application, the obtained feature mapping image is input into a plurality of consecutive upsampling feature extraction layers, multiple feature upsampling is performed, and the preset features after sampling are fused to obtain a feature fusion image.

[0086] In a possible implementation, the Neck network plays a connecting role in the object labeling model, reprocessing and fusing the features extracted from the backbone network, which is beneficial for the output network to perform further data processing.

[0087] For example, the Neck network uses a more efficient SPPF network structure to serially pass the input feature map image through multiple 5x5 size maximum pooling layers, perform multiple feature upsampling, and further perform feature extraction.

[0088] At the same time, the Neck network can adopt the FPN+PAN network structure of CSPNet to improve the ability of network feature fusion, using the method of up-sampling to transmit high-level feature information and perform fusion.

[0089] a4: Input the feature fusion image into the output network, filter multiple overlapping anchor frames according to the anchor frames in a preset range, annotate the feature fusion image using the target anchor frame, and output the annotated target scene image.

[0090] In an embodiment of the present application, the feature fusion image obtained by the Neck network is input into the output network, multiple overlapping anchor frames are filtered according to the anchor frames in a preset range to obtain the target anchor frame, the target object in the feature fusion image is labeled, and the labeled target scene image is output.

[0091] Among them, the target anchor box is an anchor box whose position and size are most suitable for marking the target object among multiple overlapping anchor boxes.

[0092] In a possible implementation, for identifying the target object contained in the target scene image and being able to annotate the target object in the target scene image, the object annotation model needs to be trained based on the YOLOv5s model. The training process of the object annotation model in the embodiment of the present application will be specifically described below:

[0093] Specifically, see Figure 2 , Figure 2 This is a flow chart of another collision warning method for a suspended fuel dispenser provided in an embodiment of the present application. Figure 2 As shown in , the object annotation model is trained by the following steps:

[0094] S201 , obtaining a plurality of initial sample images, and marking the area where the target object is located in each initial sample image to obtain a plurality of marked sample images.

[0095] In an embodiment of the present application, the initial sample image may also be obtained by frame extraction from a plurality of scene videos captured by a camera disposed at the bottom of the suspended refueling pump, wherein the plurality of scene videos may be a plurality of scene videos captured at preset time intervals within a historical time period.

[0096] Herein, image extraction may be performed every 20 frames for the collected scene video to obtain a plurality of initial sample images.

[0097] In a possible implementation, after a plurality of initial sample images are acquired, for each initial sample image, it is necessary to annotate a target object in the initial sample image to obtain a plurality of annotated sample images.

[0098] Among them, for each sample annotated image, in order to ensure the accuracy and efficiency of image recognition, for each annotated sample image, it is necessary to control the number of target objects included in each sample annotated image within a certain range (for example, the number of target objects included in each annotated sample image is in the range of 1-5).

[0099] In a possible implementation, after labeling the target object in each initial sample image acquired in real time to obtain multiple labeled sample images, images containing the target object can be extracted again from the VOC data set corresponding to the YOLOv5s model itself to expand the labeled sample images.

[0100] In a possible implementation, the processed initial sample images may be labeled using open source software labelImg, and the labeled file format is an XML file. For the labeled multiple sample images, they are randomly assigned to training sets, test sets, and validation sets to train the YOLOv5s model according to different sets.

[0101] Among them, the proportions of the training set, test set, and validation set can be 80%, 10%, and 10% respectively.

[0102] S202. Input the multiple initial sample images and the multiple annotated sample images into a pre-constructed YOLOv5s model, so that the YOLOv5s model adjusts the position and size of a pre-set initial anchor frame through each initial sample image and the corresponding annotated sample image. When the loss value of the YOLOv5s model is less than a preset loss threshold, it is determined that the training of the YOLOv5s model is completed, and the object annotation model is obtained; the initial anchor frame is determined after clustering the annotated area in each annotated sample image.

[0103] In an embodiment of the present application, a plurality of annotated initial sample images and a plurality of annotated sample images are input into a pre-constructed YOLOv5s model, and the YOLOv5s model is controlled to adjust the position and size of a pre-set initial anchor frame through each initial sample image and the corresponding annotated sample image, so that the trained YOLOv5s model can more accurately annotate the target object in the scene image through the screened target anchor frame. When the loss value of the YOLOv5s model is less than the preset loss threshold, it is determined that the YOLOv5s model training is completed, and an object annotation model is obtained.

[0104] In an embodiment of the present application, in order to ensure that the YOLOv5s model converges as quickly as possible during the training process, the sizes (width and height) of the true value boxes in different labeled sample images are counted before training, and the true value boxes are clustered using the k-means algorithm to obtain new anchor boxes as the initial anchor boxes of the YOLOv5s model, which can improve the speed of model training.

[0105] It is worth noting that since the annotated data file is in XML format, before inputting the data into the YOLOv5s model for training, the data file is first converted from XML format to YOLO format, and then input into the YOLOv5s model for training.

[0106] In a possible implementation, it is necessary to determine whether the YOLOv5s model is trained based on the loss value of the YOLOv5s model. In an embodiment of the present application, three different losses need to be considered, and the loss of the entire YOLOv5s model is calculated using the three different losses.

[0107] Specifically, the YOLOv5s model loss mainly includes category loss, bounding box regression loss and target confidence loss. The category loss uses the cross entropy loss, which only calculates the classification loss of positive samples. The target confidence loss still uses the cross entropy loss, which calculates the loss of all samples. The bounding box regression loss uses the CIoU loss, which only calculates the bounding box regression loss for positive samples.

[0108] Specifically, the loss value of the YOLOv5s model is determined by the following steps:

[0109] d1: Determine the category loss value, object confidence loss value, and bounding box regression loss value of the YOLOv5s model during training.

[0110] d2: Perform weighted calculation on the category loss value, target confidence loss value, and bounding box regression loss value to determine the loss value of the YOLOv5s model.

[0111] In the embodiment of the present application, the loss value of the YOLOv5s model can be determined by the following formula:

[0112] Loss = λ1L cls +λ2L obj +λ3L loc ;

[0113] Among them, Loss is the loss value of the YOLOv5s model; L cls Refers to the category loss value; L obj Refers to the border regression loss value; L loc Refers to the target confidence loss value; λ1, λ2, λ3 are loss balance coefficients (weighted coefficients).

[0114] In a possible implementation, in the subsequent processing of the YOLOv5s model, NMS can be used to filter the target box and improve some overlapping targets.

[0115] It is worth noting that the activation function used by the backbone network in the YOLOv5s model is the SiLU function. The SiLU function can improve the generalization ability of the algorithm, and also improve the detection effect of the algorithm, further improving the efficiency and accuracy of training the YOLOv5s model.

[0116] In a possible implementation, in order to improve the computational efficiency of the YOLOv5s model, it is necessary to prune the YOLOv5s model during training, pruning the convolutional layers with smaller responses to reduce the size of the YOLOv5s model and improve the annotation speed of the trained object annotation model.

[0117] Specifically, when the position and size of the preset initial anchor frame are adjusted, when the loss of the YOLOv5s model is less than a preset loss threshold, after determining that the YOLOv5s model training is completed, the collision warning method further includes:

[0118] e1: Determine a target pruning channel and / or a target pruning layer in the YOLOv5s model, and a target network structure including the target pruning channel and / or the target pruning layer in the YOLOv5s model.

[0119] e2: impose constraints on the scaling coefficient of the BN layer in the target pruning channel and / or the target pruning layer in the target network structure, and determine the output coefficient in the target pruning channel and / or the target pruning layer.

[0120] e3: Prune the convolutional layers and / or convolutional channels whose output coefficients of the target pruned channels and / or target pruned layers in the target network structure are less than a preset coefficient threshold to obtain a pruned YOLOv5s model, and determine the pruned YOLOv5s model as the object labeling model.

[0121] In an embodiment of the present application, a target pruned channel and / or target pruned layer that needs to be pruned in the YOLOv5s model is determined, and a target network structure (generally a backbone network in the YOLOv5s model in the embodiment of the present application) containing the target pruned channel and / or target pruned layer is determined, a scaling coefficient constraint is applied to the BN layer of the target pruned channel and / or target pruned layer, and the convolutional layers and / or convolutional channels whose output coefficients of the target pruned channel and / or target pruned layer in the target network structure are less than a preset coefficient threshold are pruned to obtain a pruned YOLOv5s model, and then the object labeling model is determined.

[0122] Specifically, there are two parts for the specific operations of the BN layer:

[0123]

[0124]

[0125] Among them, g(s) = |s|; λ is the regularization coefficient, which is adjusted according to the data set. During the training process, the gradient descent algorithm is used to optimize the above formula, and the optimization result is:

[0126] L'=∑l'+λ∑g'(γ)=∑l'+λ∑|γ|'=∑l'+λ∑γ*sign(γ);

[0127] After normalization, a linear transformation is performed. When the coefficient gamma is very small, the corresponding activation (Zout) response will be very small. The output with a very small response is pruned, thus completing the channel pruning of the BN layer. By adding the L1 regularization constraint of gamma in the loss function, the sparseness of gamma can be achieved.

[0128] Specifically, during BP propagation, the BN layer weights are multiplied by the sign function output and coefficients of the weights for processing.

[0129] In another possible implementation, in order to allow tensors to be added together, gamma constraints are not applied to all BN layers.

[0130] It is worth noting that the pruning operation of the YOLOv5s model and the training process of the YOLOv5s model are carried out simultaneously. After pruning the YOLOv5s model, the parameters of the YOLOv5s model need to be adjusted again so that the pruned YOLOv5s model can more accurately mark the target objects in the scene image.

[0131] In a possible implementation, after the YOLOv5s model is trained to obtain the object labeling model, the trained object labeling model needs to be deployed on the device to perform object detection and labeling on the scene where the suspended refueling machine is located in real time.

[0132] Specifically, the algorithm corresponding to the trained object annotation model can be deployed to the Jetson AGX Xavier device (AI edge device server, Jetson AGX Xavier device is often used in visual ranging, sensor fusion, positioning and mapping, obstacle detection and other fields). After being deployed to the Jetson AGX Xavier device, the labeled target scene image output by the object annotation model is detected and then whether to issue a collision warning is selected.

[0133] S103: Detect whether there is an overlapping area between the object annotation box and a preset dangerous area in the target scene image.

[0134] S104: If so, when the area of ​​the overlapping region is greater than a preset area threshold, generate a warning message.

[0135] In an embodiment of the present application, an electronic fence can be set in advance within a preset range of the suspended refueling machine as a dangerous area, and it is detected whether there is an overlapping area between the marked box of the target object in the target scene image and the dangerous area. If there is an overlapping area and the area of ​​the overlapping area is greater than a preset threshold, a warning message is generated to warn of the above-mentioned collision situation through the warning message.

[0136] Specifically, the step of “generating warning information when the area of ​​the overlapping region is greater than a preset area threshold” includes:

[0137] f1: When the area of ​​the overlapping region is greater than a preset area threshold, the color of the object marking box is filled with a preset warning color to obtain the warning information.

[0138] In an embodiment of the present application, when it is determined that the overlapping area between the object marking box of the target object and the dangerous area is greater than a preset threshold, it is determined that there is a risk of collision between the target object and the suspended refueling machine. At this time, the color of the object marking box marking the target object can be changed to red to provide a collision warning.

[0139] Specifically, when it is determined that there is a risk of collision between the target object and the suspended refueling machine, the color of the object annotation box that marks the target object can be displayed in red, indicating that the target object is currently in danger and the target object needs to be prompted to leave the current position; when it is determined that there is no risk of collision between the target object and the suspended refueling machine, the color of the object annotation box that marks the target object can be displayed in green, indicating that the target object is currently safe.

[0140] In another possible implementation, the generated warning information can also be a voice prompt information. When the target object is a pedestrian or a vehicle driven by a person, a voice prompt information is used to remind the pedestrian or vehicle driven by a person in the danger area to leave the current position to ensure their own safety; when the target object is an obstacle, the staff is prompted by semantics to stop the descent of the refueling gun, and after removing the obstacle, the refueling gun is controlled to continue to descend.

[0141] The collision warning method of the hanging tanker provided in the embodiment of the present application obtains the target scene image within the preset range of the hanging tanker, and inputs the target scene image into the pre-trained object annotation model, and outputs the target scene image with the object annotation box annotated with the target object area; wherein the object annotation model is a model after pruning the backbone network in the YOLOv5s model, removing the feature extraction layer whose feature extraction efficiency is lower than the preset efficiency threshold; detects whether there is an overlapping area between the object annotation box and the pre-set dangerous area; if there is an overlapping area between the two and the area between the overlapping areas is greater than the preset area threshold, generates warning information. In the embodiment of the present application, the target scene image within the preset range of the hanging tanker acquired in real time is input into the object annotation model, and the pruned object annotation model quickly outputs the target scene image annotated with the target object, and determines whether to generate warning information according to the area of ​​the overlapping area between the object annotation box annotating the target object in the target scene image and the pre-set dangerous area, so that the collision risk at the tanker can be detected in real time and quickly, which helps to improve the efficiency and timeliness of the collision warning of the hanging tanker.

[0142] Based on the same inventive concept, the embodiment of the present application also provides a collision warning device for a suspended refueling machine corresponding to the collision warning method for a suspended refueling machine. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the collision warning method for the suspended refueling machine in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0143] See also Figure 3 , Figure 4 , Figure 3 This is one of the structural schematic diagrams of a collision warning device for a suspended fuel dispenser provided in an embodiment of the present application. Figure 4 This is a second structural diagram of a collision warning device for a suspended fuel dispenser provided in an embodiment of the present application. Figure 3 As shown in , the collision warning device 300 includes:

[0144] An image acquisition module 310 is used to acquire a target scene image within a preset range of the suspended refueling machine;

[0145] An image annotation module 320 is used to input the target scene image into a pre-trained object annotation model and output an annotated target scene image; the annotated target scene image displays an object annotation box indicating the area where the target object in the image is located; the object annotation model is a model obtained by pruning the backbone network in the YOLOv5s model to remove feature extraction layers whose feature extraction efficiency is lower than a preset efficiency threshold;

[0146] A region detection module 330 is used to detect whether there is an overlapping region between the object annotation box and a pre-set dangerous region in the target scene image;

[0147] The warning information generating module 340 is used to generate warning information when the area of ​​the overlapping area is greater than a preset area threshold, if any.

[0148] In a possible implementation, Figure 4 As shown, the collision warning device 300 further includes a model training module 350, and the model training module 350 is used to:

[0149] Acquire multiple initial sample images, and annotate the area where the target object is located in each initial sample image to obtain multiple annotated sample images;

[0150] The multiple initial sample images and the multiple labeled sample images are input into a pre-constructed YOLOv5s model, so that the YOLOv5s model adjusts the position and size of a pre-set initial anchor frame through each initial sample image and the corresponding labeled sample image. When the loss value of the YOLOv5s model is less than a preset loss threshold, it is determined that the training of the YOLOv5s model is completed, and the object labeling model is obtained; the initial anchor frame is determined after clustering the labeled areas in each labeled sample image.

[0151] In a possible implementation, Figure 4 As shown, the collision warning device 300 further includes a model pruning module 360, and the model pruning module 360 ​​is used to:

[0152] Determine a target pruning channel and / or a target pruning layer in the YOLOv5s model, and a target network structure including the target pruning channel and / or the target pruning layer in the YOLOv5s model;

[0153] Imposing constraints on the scaling coefficient of the BN layer in the target pruning channel and / or the target pruning layer in the target network structure, and determining the output coefficient in the target pruning channel and / or the target pruning layer;

[0154] The target pruned channel and / or target pruned layer in the target network structure has an output coefficient less than the convolution layer and / or convolution channel of a preset coefficient threshold, to obtain a pruned YOLOv5s model, and the pruned YOLOv5s model is determined as the object labeling model.

[0155] In a possible implementation, the object annotation model includes an input network, a backbone network, a Neck network, and an output network; when the image annotation module 320 is used to input the target scene image into a pre-trained object annotation model and output the annotated target scene image, the image annotation module 320 is used to:

[0156] Inputting the target scene image into the input network of the object annotation model, scaling the scene image according to a preset image size, and outputting a scaled filling image;

[0157] Inputting the filled image into the backbone network, segmenting and splicing the filled image according to a preset segmentation threshold to obtain a plurality of spliced ​​images, performing feature extraction on each spliced ​​image, and outputting a feature mapping image;

[0158] Input the feature map image into the Neck network, perform feature upsampling fusion, and output a feature fusion image;

[0159] The feature fusion image is input into the output network, and after filtering multiple overlapping anchor frames according to the anchor frames in a preset range, the feature fusion image is annotated using the target anchor frame, and the annotated target scene image is output.

[0160] In a possible implementation, the backbone network includes an image segmentation layer and a first feature extraction layer; the image annotation module 320 is used to input the filled image into the backbone network, segment and splice the filled image according to a preset segmentation threshold, obtain multiple spliced ​​images, and extract features from each spliced ​​image. When outputting a feature mapping image, the image annotation module 320 is used to:

[0161] The filled image is input to the image segmentation layer, and after the image segmentation layer segments the filled image according to a preset segmentation threshold, adjacent pixels in the filled image that meet the preset segmentation threshold are determined as a slice, and pixels at the same position in each slice are spliced ​​to output multiple spliced ​​images;

[0162] The plurality of stitched images are input into the first feature extraction layer, features are extracted from the plurality of stitched images, and the feature mapping image is output.

[0163] In a possible implementation, the Neck network includes a plurality of upsampling feature extraction layers connected in sequence; when the image annotation module 320 is used to input the feature mapping image into the Neck network, perform feature upsampling fusion, and output a feature fusion image, the image annotation module 320 is used to:

[0164] The feature mapping image is input into a plurality of continuous upsampling feature extraction layers, multiple feature upsampling is performed, and after the preset features after sampling are fused, the feature fusion image is output.

[0165] In a possible implementation, the model training module 350 is used to determine the loss value of the YOLOv5s model through the following steps:

[0166] Determine the category loss value, target confidence loss value, and bounding box regression loss value of the YOLOv5s model during training;

[0167] The category loss value, target confidence loss value, and border regression loss value are weightedly calculated to determine the loss value of the YOLOv5s model.

[0168] In a possible implementation, when the warning information generation module 340 is used to generate warning information when the area of ​​the overlapping region is greater than a preset area threshold, the warning information generation module 340 is used to:

[0169] When the area of ​​the overlapping region is greater than a preset area threshold, the color of the object marking box is filled with a preset warning color to obtain the warning information.

[0170] The collision warning device of the hanging tanker provided in the embodiment of the present application obtains the target scene image within the preset range of the hanging tanker, and inputs the target scene image into the pre-trained object annotation model, and outputs the target scene image with the object annotation box annotated with the target object area; wherein the object annotation model is a model after pruning the backbone network in the YOLOv5s model, removing the feature extraction layer whose feature extraction efficiency is lower than the preset efficiency threshold; detects whether there is an overlapping area between the object annotation box and the pre-set dangerous area; if there is an overlapping area between the two and the area between the overlapping areas is greater than the preset area threshold, generates warning information. In the embodiment of the present application, the target scene image within the preset range of the hanging tanker acquired in real time is input into the object annotation model, and the pruned object annotation model quickly outputs the target scene image annotated with the target object, and determines whether to generate warning information according to the area of ​​the overlapping area between the object annotation box annotating the target object in the target scene image and the pre-set dangerous area, so that the collision risk at the tanker can be detected in real time and quickly, which helps to improve the efficiency and timeliness of the collision warning of the hanging tanker.

[0171] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5As shown in , the electronic device 500 includes a processor 510 , a memory 520 and a bus 530 .

[0172] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 as well as Figure 2 The steps of the collision warning method for the suspended fuel dispenser in the method embodiment shown, and the specific implementation method can be found in the method embodiment, which will not be repeated here.

[0173] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The steps of the collision warning method for the suspended fuel dispenser in the method embodiment shown, and the specific implementation method can be found in the method embodiment, which will not be repeated here.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0175] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0176] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0178] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0179] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A collision warning method for a suspended fuel dispenser, characterized in that: The collision warning method comprises: Acquire a target scene image within a preset range of the suspended refueling machine; The target scene image is input into a pre-trained object annotation model, and an annotated target scene image is output; the annotated target scene image displays an object annotation box indicating the area where the target object in the image is located; the object annotation model is a model obtained by pruning the backbone network in the YOLOv5s model to remove the feature extraction layer whose feature extraction efficiency is lower than a preset efficiency threshold; Detecting whether there is an overlapping area between the object annotation box and a pre-set dangerous area in the target scene image; If so, when the area of ​​the overlapping region is greater than a preset area threshold, a warning message is generated; The object annotation model includes an input network, a backbone network, a Neck network, and an output network; the target scene image is input into a pre-trained object annotation model, and the annotated target scene image is output, including: Inputting the target scene image into the input network of the object annotation model, scaling the scene image according to a preset image size, and outputting a scaled filling image; Inputting the filled image into the backbone network, segmenting and splicing the filled image according to a preset segmentation threshold to obtain a plurality of spliced ​​images, performing feature extraction on each spliced ​​image, and outputting a feature mapping image; Input the feature map image into the Neck network, perform feature upsampling fusion, and output a feature fusion image; Inputting the feature fusion image into the output network, filtering multiple overlapping anchor frames according to the anchor frames in a preset range, annotating the feature fusion image using the target anchor frame, and outputting the annotated target scene image; In the adjusting of the position and size of the preset initial anchor frame, when the loss of the YOLOv5s model is less than a preset loss threshold, after determining that the YOLOv5s model training is completed, the collision warning method further includes: Determine a target pruning channel and / or a target pruning layer in the YOLOv5s model, and a target network structure including the target pruning channel and / or the target pruning layer in the YOLOv5s model; Imposing constraints on the scaling coefficient of the BN layer in the target pruning channel and / or the target pruning layer in the target network structure, and determining the output coefficient in the target pruning channel and / or the target pruning layer; Pruning the convolutional layers and / or convolutional channels whose output coefficients of the target pruning channels and / or target pruning layers in the target network structure are less than a preset coefficient threshold, obtaining a pruned YOLOv5s model, and determining the pruned YOLOv5s model as the object labeling model; Among them, the specific operations for the BN layer are divided into two parts: L=∑ (x,y) l(f(x,W),y)+λ∑ γ∈Г g(γ); Among them, g(s) = |s|, λ is the regularization coefficient, which is adjusted according to the data set; during the training process, the gradient descent algorithm is used to optimize the above formula, and the optimization result is: L'=∑l'+λΣg'(γ)=Σl'+λΣ|γ|'=Σl'+λΣγ*sign(γ); After normalization, a linear transformation is performed, and gamma is sparsified by adding the L1 regularization constraint of gamma to the loss function; that is, during BP propagation, the weights in the BN layer are multiplied by the sign function output and coefficients of the weights for processing; The pruning operation of the YOLOv5s model and the training process of the YOLOv5s model are performed simultaneously. After the YOLOv5s model is pruned, the parameters of the YOLOv5s model are adjusted again so that the pruned YOLOv5s model can mark the target object in the scene image.

2. The collision warning method according to claim 1, characterized in that: The backbone network includes an image segmentation layer and a first feature extraction layer; the filling image is input into the backbone network, the filling image is segmented and spliced ​​according to a preset segmentation threshold, a plurality of spliced ​​images are obtained, and features are extracted from each spliced ​​image to output a feature mapping image, including: The filled image is input to the image segmentation layer, and after the image segmentation layer segments the filled image according to a preset segmentation threshold, adjacent pixels in the filled image that meet the preset segmentation threshold are determined as a slice, and pixels at the same position in each slice are spliced ​​to output multiple spliced ​​images; The plurality of stitched images are input into the first feature extraction layer, features are extracted from the plurality of stitched images, and the feature mapping image is output.

3. The collision warning method according to claim 1, characterized in that: The Neck network includes a plurality of upsampling feature extraction layers connected in sequence; the feature mapping image is input into the Neck network, feature upsampling fusion is performed, and a feature fusion image is output, including: The feature mapping image is input into a plurality of continuous upsampling feature extraction layers, multiple feature upsampling is performed, and after the preset features after sampling are fused, the feature fusion image is output.

4. The collision warning method according to claim 1, characterized in that: The object annotation model is trained by the following steps: Acquire multiple initial sample images, and annotate the area where the target object is located in each initial sample image to obtain multiple annotated sample images; The multiple initial sample images and the multiple labeled sample images are input into a pre-constructed YOLOv5s model, so that the YOLOv5s model adjusts the position and size of a pre-set initial anchor frame through each initial sample image and the corresponding labeled sample image. When the loss value of the YOLOv5s model is less than a preset loss threshold, it is determined that the training of the YOLOv5s model is completed, and the object labeling model is obtained; the initial anchor frame is determined after clustering the labeled areas in each labeled sample image.

5. The collision warning method according to claim 4, characterized in that: The loss value of the YOLOv5s model is determined by the following steps: Determine the category loss value, target confidence loss value, and bounding box regression loss value of the YOLOv5s model during training; The category loss value, target confidence loss value, and bounding box regression loss value are weightedly calculated to determine the loss value of the YOLOv5s model.

6. The collision warning method according to claim 1, characterized in that: When the area of ​​the overlapping region is greater than a preset area threshold, generating warning information includes: When the area of ​​the overlapping region is greater than a preset area threshold, the color of the object marking box is filled with a preset warning color to obtain the warning information.

7. A collision warning device for a suspended fuel dispenser, characterized in that: The collision warning device comprises: An image acquisition module, used to acquire a target scene image within a preset range of the suspended refueling machine; An image annotation module is used to input the target scene image into a pre-trained object annotation model and output an annotated target scene image; the annotated target scene image displays an object annotation box indicating the area where the target object in the image is located; the object annotation model is a model obtained by pruning the backbone network in the YOLOv5s model to remove feature extraction layers whose feature extraction efficiency is lower than a preset efficiency threshold; A region detection module, used to detect whether there is an overlapping region between the object annotation frame and a pre-set dangerous region in the target scene image; A warning information generating module, used for generating warning information when the area of ​​the overlapping area is greater than a preset area threshold, if any; The object annotation model includes an input network, a backbone network, a Neck network, and an output network; the target scene image is input into a pre-trained object annotation model, and the annotated target scene image is output, including: Inputting the target scene image into the input network of the object annotation model, scaling the scene image according to a preset image size, and outputting a scaled filling image; Inputting the filled image into the backbone network, segmenting and splicing the filled image according to a preset segmentation threshold to obtain a plurality of spliced ​​images, performing feature extraction on each spliced ​​image, and outputting a feature mapping image; Input the feature map image into the Neck network, perform feature upsampling fusion, and output a feature fusion image; Inputting the feature fusion image into the output network, filtering multiple overlapping anchor frames according to the anchor frames in a preset range, annotating the feature fusion image using the target anchor frame, and outputting the annotated target scene image; In the adjusting of the position and size of the preset initial anchor frame, when the loss of the YOLOv5s model is less than a preset loss threshold, after determining that the YOLOv5s model training is completed, the collision warning method further includes: Determine a target pruning channel and / or a target pruning layer in the YOLOv5s model, and a target network structure including the target pruning channel and / or the target pruning layer in the YOLOv5s model; Imposing constraints on the scaling coefficient of the BN layer in the target pruning channel and / or the target pruning layer in the target network structure, and determining the output coefficient in the target pruning channel and / or the target pruning layer; Pruning the convolutional layers and / or convolutional channels whose output coefficients of the target pruning channels and / or target pruning layers in the target network structure are less than a preset coefficient threshold, obtaining a pruned YOLOv5s model, and determining the pruned YOLOv5s model as the object labeling model; Among them, the specific operations for the BN layer are divided into two parts: L=∑ (x,y) l(f(x,W),y)+λ∑ γ∈Г g(γ); Among them, g(s) = |s|, λ is the regularization coefficient, which is adjusted according to the data set; during the training process, the gradient descent algorithm is used to optimize the above formula, and the optimization result is: L'=∑l'+λ∑g'(γ)=∑l'+λ∑|γ|'=∑l'+λ∑γ*sign(γ); After normalization, a linear transformation is performed, and gamma is sparsified by adding the L1 regularization constraint of gamma to the loss function; that is, during BP propagation, the weights in the BN layer are multiplied by the sign function output and coefficients of the weights for processing; The pruning operation of the YOLOv5s model and the training process of the YOLOv5s model are performed simultaneously. After the YOLOv5s model is pruned, the parameters of the YOLOv5s model are adjusted again so that the pruned YOLOv5s model can mark the target object in the scene image.

8. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the collision warning method for a suspended refueling machine as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the collision warning method for a suspended fuel dispenser as described in any one of claims 1 to 6 are executed.

Images