Expressway thrown object detection method and system based on open vocabulary detection

Abstract

The invention provides an expressway thrown object detection method and system based on open vocabulary detection. The method belongs to the field of thrown object detection, and comprises the following steps: carrying out local histogram equalization on each monitoring image block; extracting a potential target area of a throwing object in the image after local histogram equalization; calling a visual encoder and a text encoder in a visual language pre-training model CLIP to respectively extract regional features of the potential target region and text features of categories in the open vocabulary detection list, and then inputting the regional features and the text features into a plurality of visual language fusion modules to obtain regional features of fused text information; and finally, calculating the similarity between the regional features of the fusion text information and the text features to align the potential target region with the category in the open vocabulary detection list, so as to identify the category of the thrown object in the potential target region, and recording the throwing time. The method can accurately capture and identify the thrown objects on the expressway.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of litter detection, and in particular, to a highway litter detection method and system based on open vocabulary detection. BACKGROUND

[0002] Highway litter can easily cause traffic accidents and threaten road safety. Currently, the detection of highway litter mainly relies on manual inspection and fixed camera monitoring, which has the problems of low efficiency, poor real-time performance, and being easily affected by human factors.

[0003] In the development of intelligent transportation systems, video monitoring technology has been widely applied in highway management, but existing highway litter detection methods often rely on specific rules or pre-set object models, which cannot effectively identify a variety of litter with different shapes. To solve this problem, some researchers have tried to use the visual language pre-training model CLIP (Contrastive Language-Image Pre-training) for litter detection. However, the CLIP model performs poorly in solving the open vocabulary classification task at the region level. This is because the CLIP model only performs contrastive learning of vision and language in the last stage to achieve alignment of the two. This design makes it difficult for CLIP to accurately capture and identify key information in images when dealing with complex and variable litter detection tasks. SUMMARY

[0004] The purpose of the present application is to provide a highway litter detection method and system based on open vocabulary detection, which can accurately capture and identify highway litter.

[0005] The present application is implemented as follows:

[0006] In a first aspect, the present application provides a highway litter detection method based on open vocabulary detection, comprising the following steps:

[0007] S101: Obtain common highway litter, and construct an open vocabulary detection list for highway litter based on the common litter;

[0008] S102: Obtain a monitoring image, segment the lanes in the vehicle driving area of the monitoring image to obtain an image of lane line positions, perform binaryzation processing on the image of lane line positions to generate a lane line mask and store it, and when the angle and height of the monitoring camera change, re-segment the lanes and store the segmented mask;

[0009] S103: Perform block processing on the monitoring image, and perform local histogram equalization on each monitoring image block;

[0010] S104: extracting a potential target region of the litter in the locally histogram equalized image using a region proposal network;

[0011] S105: calling a visual encoder and a text encoder in a visual language pre-training model CLIP to respectively extract a region feature of the potential target region and a text feature of a category in an open vocabulary detection list, then inputting the region feature and the text feature into a plurality of visual language fusion modules to obtain a region feature fused with text information, and finally calculating a similarity between the region feature fused with text information and the text feature to align the potential target region and the category in the open vocabulary detection list, for identifying a category of the litter in the potential target region and recording a littering time;

[0012] S106: obtaining a position of the litter according to a mask of a lane where the litter is located, recording position, category and littering time information of the litter, and submitting the information to a road supervision system.

[0013] In a second aspect, the application provides a highway litter detection system based on open vocabulary detection, comprising the following steps:

[0014] An open vocabulary detection list module is configured to obtain common highway litter, and construct an open vocabulary detection list for highway litter according to the common highway litter;

[0015] A mask module is configured to obtain a monitoring image, segment lanes in a vehicle driving area in the monitoring image to obtain an image of lane line positions, perform binaryzation processing on the image of lane line positions to generate a lane line mask and store the lane line mask, and when the angle and height of a monitoring camera change, re-segment the lanes and store the segmented mask;

[0016] An equalization module is configured to perform block processing on the monitoring image, and perform local histogram equalization on each monitoring image block;

[0017] A potential target region module is configured to extract a potential target region of the litter in the locally histogram equalized image using a region proposal network;

[0018] An identification module is configured to call a visual encoder and a text encoder in a visual language pre-training model CLIP to respectively extract a region feature of the potential target region and a text feature of a category in an open vocabulary detection list, then input the region feature and the text feature into a plurality of visual language fusion modules to obtain a region feature fused with text information, and finally calculate a similarity between the region feature fused with text information and the text feature to align the potential target region and the category in the open vocabulary detection list, for identifying a category of the litter in the potential target region and recording a littering time;

[0019] Information submission module: used for obtaining the position of the litter according to the mask of the lane where the litter is located, recording the position, category and litter time information of the litter, and submitting to the road supervision system.

[0020] In a third aspect, the present application provides an electronic device, comprising:

[0021] a memory for storing one or more programs;

[0022] a processor;

[0023] When the one or more programs are executed by the processor, the above-mentioned method is implemented.

[0024] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned method.

[0025] Compared with the prior art, the present application has at least the following advantages or beneficial effects:

[0026] The present application provides a highway litter detection method and system based on open vocabulary detection, which performs histogram equalization on the image to make the brightness distribution of the image more uniform, and alleviates the problem of poor night lighting conditions and serious glare. By using a region proposal network to extract the potential target region of the litter in the image after local histogram equalization, the litter recognition speed and accuracy are improved. By inputting the region features and text features into a plurality of visual language fusion modules, the region features with fused text information are obtained, and finally the similarity of the region features with fused text information and the text features is calculated to align the potential target region and the categories in the open vocabulary detection list, which is used to identify the category of the litter in the potential target region. The multi-modal feature fusion is performed in advance before contrast learning, which improves the open vocabulary detection capability of the visual language pre-training model CLIP in the region-level task, so that the invention can adapt to the changing litter types, and the flexibility and accuracy of the detection are improved. The present application can effectively improve the operation efficiency of the highway, ensure the driving safety, reduce the traffic congestion and potential economic losses caused by the litter, and provide important support for the improvement of the intelligent traffic management system and the social public safety. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0028] Figure 1 Flow chart of an embodiment of a highway litter detection method based on open vocabulary detection according to the present application;

[0029] Figure 2 Flow chart of another embodiment of a highway litter detection method based on open vocabulary detection according to the present application;

[0030] Figure 3 Flow chart of yet another embodiment of a highway litter detection method based on open vocabulary detection according to the present application;

[0031] Figure 4 Schematic diagram of a histogram equalization method in an embodiment of a highway litter detection method based on open vocabulary detection according to the present application;

[0032] Figure 5 Flow chart of yet another embodiment of a highway litter detection method based on open vocabulary detection according to the present application;

[0033] Figure 6 Framework diagram of a target region classification method based on a visual language pre-training model CLIP and a visual language fusion module in an embodiment of a highway litter detection method based on open vocabulary detection according to the present application;

[0034] Figure 7 Structural block diagram of a highway litter detection system based on open vocabulary detection according to the present application;

[0035] Figure 8 Structural block diagram of an electronic device according to the present application.

[0036] Icon:

[0037] 1, open vocabulary detection list module; 2, mask module; 3, equalization module; 4, potential target region module; 5, recognition module; 6, information submission module; 7, processor; 8, memory; 9, communication interface. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0039] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. In the case of no conflict, each of the following embodiments and each feature in the embodiments can be combined with each other.

[0040] Embodiments

[0041] The embodiments of the present application provide a highway litter detection method based on open vocabulary detection, which can accurately capture and identify highway litter.

[0042] Please refer to Figure 1 The highway litter detection method based on open vocabulary detection comprises the following steps:

[0043] S101: Obtain common highway litter, and construct an open vocabulary detection list for highway litter according to the common highway litter;

[0044] Specifically, the types of common highway litter are investigated, including but not limited to plastic bags, bottles, paper, stones, wood blocks, etc., and image samples of each type of litter are collected. Further, in order to ensure sample diversity, litter image samples under different angles, sizes and lighting conditions are collected, the collected litter image samples are labeled, an open vocabulary detection list is constructed, and the list contains all labeled litter category names. Further, the list is ensured to be comprehensive to avoid missing common litter categories, and the list is updated regularly to include new litter types.

[0045] S102: Obtain a monitoring image, segment the lane in the vehicle driving area in the monitoring image to obtain an image of lane line position, perform binaryzation processing on the image of lane line position to generate a lane line mask and store the lane line mask, and when the angle and height of the monitoring camera change, re-segment the lane and store the segmented mask;

[0046] Specifically, a lane segmentation model is used to segment the lane in the vehicle driving area in the image. The predicted feature map is binaryzation processed to generate a lane line mask. The segmented lane mask is persistently stored to provide lane position information in the subsequent litter potential area screening link. When the angle, height and other parameters of the monitoring camera beside the highway change, lane segmentation is performed in time.

[0047] Further, please refer to Figure 2 The lane line mask can be generated by the following steps:

[0048] S201: Apply morphological operations such as dilation and erosion to remove noise and small holes, and fill gaps; the dilation operation will increase the white area (or foreground object) in the image, which is used to fill small holes in the object or connect adjacent objects. The erosion operation will reduce the white area in the image, which is used to remove boundary points. By combining the use of dilation and erosion operations, noise (small, irrelevant white or black spots) in the image can be removed, gaps on the lane line can be filled, and the edges of the lane line can be smoothed.

[0049] S201: Using curve fitting (such as polynomial fitting) method to fit the segmented lane line points, to get the accurate representation of the lane line. In lane line detection, after the points on the lane line are segmented, the curve fitting method is used to get the best fitting curve of these points, so as to accurately represent the position and shape of the lane line.

[0050] S103: Block processing is performed on the monitoring image, and local histogram equalization is performed on each monitoring image block;

[0051] Specifically, in the monitoring image, due to the change of light condition, some parts of the image may be darker or brighter than other parts. Through block processing and local histogram equalization, the local contrast of the image can be improved, and the features such as lane lines can be more obvious, so as to improve the accuracy of lane line detection.

[0052] Please refer to Figure 3 and Figure 4 , further, the step of performing local histogram equalization on each monitoring image block comprises

[0053] S301: The picture is divided into 23x13 picture blocks, and histogram equalization operation is performed on each picture block separately, and different blocks do not affect each other;

[0054] In this step, the night glare caused by car lights and street lights only affects the local part of the picture. In order to adapt to the local light change, reduce the noise enhancement caused by global equalization, and reduce the influence of glare on the global, the picture is divided into 23x13 picture blocks, and histogram equalization operation is performed on each picture block separately, and different blocks do not affect each other.

[0055] S302: According to the characteristics of the distribution of highway litter, only the image blocks containing the lane are processed, by traversing each pixel in the image block, the number of occurrences of each gray level in the image block is obtained, the cumulative distribution function CDF of the histogram is calculated, and the calculation formula of the cumulative distribution function CDF of the histogram is: Where h(j) is the number of pixels with gray level j in the histogram; the cumulative distribution function CDF value of the histogram is mapped to a new gray level, which can be calculated by the following formula: new_evel = round((L-1)·CDF(i)), where L is the total number of gray levels in the image block, and the round function is used to round to the nearest integer; using the calculated new gray level, each pixel in the original image block is transformed, that is, each pixel value i in the original image block is replaced by the corresponding new gray level new_evel.

[0056] In this step, only the image blocks containing the lane are processed, thereby targetedly processing the image and improving the efficiency of image processing. The histogram of the image block is first calculated. The histogram is a discrete function representing the number of pixels of each gray level. The number of occurrences of each gray level in the image block is calculated, which can be completed by traversing each pixel in the image block. Then, the cumulative distribution function CDF of the histogram is calculated. The cumulative distribution function is the cumulative sum of the histogram, representing the proportion of the total number of pixels of each gray level and below to the total number of pixels. The calculation formula of the cumulative distribution function CDF of the histogram is: In order to equalize the histogram, the CDF value needs to be mapped to a new gray level, so that the histogram of the output image can be as flat as possible. The new gray level can be calculated by the following formula: new_evel=round((L-1)·CDF(i)), wherein L is the total number of gray levels in the image block, and the round function is used to round to the nearest integer. The gray level of each pixel is transformed, and the new gray level calculated in the previous step is used to transform each pixel in the original image block. That is, each pixel value i in the original image block is replaced by the corresponding new gray level new_evel. After the transformation of each target image block, the new image obtained will have a more uniform histogram, and the problems of insufficient light and glare will be improved.

[0057] S104: Extracting the potential target region of the scatterer in the image after local histogram equalization using the region proposal network;

[0058] Specifically, the potential target region of the scatterer is extracted. When judging whether the region coincides with the lane, only whether the boundary of the region coincides with the lane needs to be traversed, without traversing the entire region, thereby saving computing resources and improving speed.

[0059] Please refer to Figure 5 Further, the step of extracting the potential target region of the scatterer in the image after local histogram equalization includes:

[0060] S401: Removing the regions with a confidence lower than the threshold according to the confidence threshold;

[0061] Specifically, the confidence threshold is a preset value for determining which regions have a high enough confidence to be retained as potential target regions. All regions with a confidence lower than the preset threshold are removed. The confidence of these regions is too low, and the model considers that they are unlikely to be target objects.

[0062] S402: Calculating the area of each region and screening out the regions smaller than the set maximum area;

[0063] Specifically, when processing the image, each candidate region can be regarded as a two-dimensional shape, and the area of the two-dimensional shape can be calculated. The maximum area is set to determine which regions are too small to be the target object. All regions with an area smaller than the preset maximum area are removed. This is because too small regions are likely to be noise or small details in the image, and are unlikely to be the target object.

[0064] S403: According to the mask of the lane line, set the region overlapping with the lane as a potential target region.

[0065] Specifically, all regions overlapping with the lane line are marked as potential target regions. This is because these regions may be related to the litter on the lane line and need further analysis, so they are set as potential target regions.

[0066] S105: Call the visual encoder and text encoder in the visual language pre-training model CLIP to extract the region features of the potential target region and the text features of the categories in the open vocabulary detection list, respectively, then input the region features and text features into several visual language fusion modules to obtain the region features with fused text information, and finally calculate the similarity between the region features with fused text information and the text features to align the potential target region and the categories in the open vocabulary detection list, for identifying the category of the litter in the potential target region and recording the littering time;

[0067] Please refer to Figure 6 The target region classification method based on CLIP and visual language fusion module includes an image encoder of CLIP, a text encoder of CLIP, and a visual language fusion module. First, the categories in the open vocabulary detection list of highway litter are input into the text encoder of the CLIP model to encode into text features. Specifically, given an open vocabulary detection list

[0068] [Plastic bag, bottle, paper, stone, ……, wood block]

[0069] The following text prompt is used to convert each category in the list into a sentence for input into the text encoder of CLIP:

[0070] Prompt=“A picture of a{object}spill"

[0071] Wherein, the categories in the open vocabulary detection list are filled into {object}, and the sentence obtained by filling the categories in the open vocabulary detection list into the text prompt is input into the CLIP text encoder to obtain the text features.

[0072] Secondly, the target region is input into an image encoder in the CLIP model to encode into a region feature.

[0073] Thirdly, the text feature and the region feature are input into N visual language fusion modules to obtain a region feature fused with text information, and N=6 in this embodiment; each visual language fusion module comprises a self-attention layer, a cross-modal cross-attention layer and a feedforward neural network; in each visual language fusion module, the region feature is first input into the self-attention layer, Q, K and V in the self-attention layer are all from the region feature, then the output of the self-attention layer and the text feature are input into the cross-modal cross-attention layer for multi-modal feature fusion, wherein Q in the cross-modal cross-attention layer is from the output of the self-attention layer, K and V are from the text feature, finally the output of the cross-modal cross-attention layer is input into the feedforward neural network, and the feedforward neural network comprises an output layer and a hidden layer, wherein the activation function uses a Sigmoid function; the meanings of Q, K and V in the attention layer are taken from the paper Attention Is All You Need.

[0074] Finally, the similarity between the region feature fused with the text information and the text feature is calculated, the target region and the categories in the open vocabulary detection list are aligned to identify the category of the litter in the target region; given n target regions and m detection categories, let the region feature of the i-th target region be O i , the text feature of the j-th detection category be T j , then the similarity between O i and T j is their inner product O i ·T j , and the category C i of the litter in the i-th target region is the detection category with the highest similarity, i.e., C i =argmax j O i ·T j .

[0075] S106: Obtain the position of the litter according to the mask of the lane where the litter is located, record the position, category and littering time information of the litter, and submit to the road supervision system.

[0076] Specifically, the recorded information is integrated into a report and submitted to the road supervision system. The system issues an alarm according to the submitted information and notifies relevant personnel to handle.

[0077] Please refer to Figure 7 , in a second aspect, the application provides a highway litter detection system based on open vocabulary detection, comprising the following steps:

[0078] An open vocabulary detection list module 1 is configured to obtain common litter on the expressway, and construct an open vocabulary detection list for litter on the expressway according to the common litter.

[0079] A mask module 2 is configured to obtain a monitoring image, segment a lane in a vehicle driving area in the monitoring image to obtain a lane line position image, perform binaryzation processing on the lane line position image to generate and store a lane line mask, and re-perform lane segmentation and store the segmented mask when the angle and height of the monitoring camera change.

[0080] An equalization module 3 is configured to perform block processing on the monitoring image, and perform local histogram equalization on each monitoring image block.

[0081] A potential target area module 4 is configured to extract a potential target area of the litter in the image subjected to the local histogram equalization using a region proposal network.

[0082] An identification module 5 is configured to call a visual encoder and a text encoder in a visual language pre-training model CLIP to respectively extract a region feature of the potential target area and a text feature of a category in the open vocabulary detection list, then input the region feature and the text feature into a plurality of visual language fusion modules to obtain a region feature fused with text information, and finally calculate a similarity between the region feature fused with the text information and the text feature to align the potential target area and the category in the open vocabulary detection list, so as to identify a category of the litter in the potential target area and record a littering time.

[0083] An information submission module 6 is configured to obtain a position of the litter according to a mask of a lane where the litter is located, record position, category and littering time information of the litter, and submit the information to a road supervision system.

[0084] Specifically, the specific implementation of the expressway litter detection system based on the open vocabulary detection can refer to the specific implementation of the expressway litter detection method based on the open vocabulary detection, which will not be described in detail here.

[0085] Please refer to Figure 8 , in a third aspect, the application provides an electronic device, comprising:

[0086] a memory 8 for storing one or more programs;

[0087] a processor 7; the processor 7 and the memory 8 are connected through a communication interface 9;

[0088] When the one or more programs are executed by the processor 7, all or part of the above methods are realized.

[0089] In a fourth aspect, the present application provides a computer readable storage medium, having stored thereon a computer program, wherein the computer program is executed by a processor 7 to implement all or part of the method described above.

[0090] It will be apparent to those skilled in the art that the application is not limited to the details of the above-exemplified embodiments and that the application can be implemented in other particular forms without departing from the spirit or essential characteristics of the application. The embodiments should therefore be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the above description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No reference signs in the claims should be considered as limiting the scope of the claims with respect to the figures of the application.

Claims

1. A method for detecting spilled objects on highways based on open vocabulary detection, characterized in that: The following steps are involved: S101: Obtain common objects scattered on highways, and construct an open vocabulary detection list for objects scattered on highways based on the common objects scattered; S102: Acquire a surveillance image, segment the lanes of the vehicle's driving area in the surveillance image to obtain an image of the lane line positions, perform binarization on the lane line position image to generate a lane line mask, and store the mask. When the surveillance camera angle or height changes, re-segment the lanes and store the segmented mask. S103: Divide the monitoring image into blocks and perform local histogram equalization on each monitoring image block; S104: Using a region proposal network to extract potential target regions of the spilled objects in the image after local histogram equalization; S105: calling the visual encoder and text encoder in the visual language pre-training model CLIP to extract the regional features of the potential target area and the text features of the categories in the open vocabulary detection list respectively, then inputting the regional features and text features into several visual language fusion modules to obtain the regional features of the fused text information, and finally calculating the similarity between the regional features and text features of the fused text information to align the potential target area with the categories in the open vocabulary detection list, so as to identify the category of the scattered objects in the potential target area and record the time of the scattered objects; S106: Obtain the location of the spilled object based on the mask of the lane where the spilled object is identified, record the location, category, and time of spilling of the spilled object, and submit it to the road supervision system.

2. The method for detecting spilled objects on highways based on open vocabulary detection according to claim 1, characterized in that: The step of binarizing the image of the lane line position to generate a lane line mask in step S102 includes: S201: Apply dilation and erosion morphological operations to remove noise and small holes and fill gaps; S201: Use a curve fitting method to fit the segmented lane line points to obtain an accurate lane line mask.

3. The method for detecting spilled objects on highways based on open vocabulary detection according to claim 1, characterized in that: Step S103 divides the monitoring image into blocks and performs local histogram equalization on each monitoring image block, including: S301: Divide the image into 23×13 blocks and perform histogram equalization on each block independently, ensuring that different blocks do not affect each other. S302: Based on the distribution characteristics of spilled objects on the highway, only the image blocks containing the lanes are processed. By traversing each pixel in the image block, the number of occurrences of each grayscale level in the image block is obtained, and the cumulative distribution function (CDF) of the histogram is calculated. The calculation formula of the cumulative distribution function (CDF) of the histogram is: Where h(j) is the number of pixels with gray level j in the histogram; the CDF value of the histogram is mapped to a new gray level, which can be calculated by the following formula: new_evel = round((L-1)·CDF(i)), where L is the total number of gray levels in the image block, and the round function is used to round to the nearest integer; using the calculated new gray level, each pixel in the original image block is transformed, that is, each pixel value i in the original image block is replaced by the corresponding new gray level new_evel.

4. The method for detecting spilled objects on highways based on open vocabulary detection according to claim 3, characterized in that: Step S104 of extracting potential target areas of spilled objects from the image after local histogram equalization using a region proposal network includes: S401: Screening out regions with confidence levels lower than the threshold according to the confidence threshold; S402: Calculate the area of ​​each region and filter out regions with a smaller area than a set maximum area; S403: According to the lane line mask, the area overlapping with the lane is set as a potential target area.

5. The method for detecting spilled objects on highways based on open vocabulary detection according to claim 4, characterized in that: Step S105 includes: Given an open vocabulary detection list [Plastic bag, bottle, paper, stone,..., wood block] The following text prompt is used to convert each category in the list into a sentence for input to the text encoder in CLIP: Prompt="A picture of a{object}spill" Among them, the categories in the open vocabulary detection list are filled into {object}, and the sentence obtained by filling the categories in the open vocabulary detection list into the text prompt is input into the CLIP text encoder to obtain text features; the target area is input into the image encoder of the CLIP model and encoded into regional features; The text features and regional features are input into N visual language fusion modules to obtain regional features that are fused with text information; each visual language fusion module contains a self-attention layer, a cross-modal cross-attention layer and a feedforward neural network; in each visual language fusion module, the regional features are first input into the self-attention layer, and Q, K, and V in the self-attention layer all come from regional features. Then the output of the self-attention layer and the text features are input into the cross-modal cross-attention layer for multimodal feature fusion, where Q in the cross-modal cross-attention layer comes from the output of the self-attention layer, and K and V come from text features. Finally, the output of the cross-modal cross-attention layer is input into the feedforward neural network, which contains an output layer and a hidden layer, where the activation function uses the Sigmoid function; finally, the similarity between the regional features and text features that are fused with text information is calculated, and the categories in the target area and the open vocabulary detection list are aligned to identify the category of the scattered objects in the target area; given n target areas and m detection categories, the regional features of the i-th target area are recorded as O i , the text feature of the jth detection category is T j , then O i and T j The similarity is their inner product O i ·T j , the category C of the scattered objects in the i-th target area i Take the detection category with the highest similarity, that is, C i =argmax j O i ·T j .

6. The method for detecting spilled objects on highways based on open vocabulary detection according to claim 1, characterized in that: In step S101, the open vocabulary detection list for highway littering is constructed based on common littering objects, including: Collect image samples of scattered objects at different angles, sizes, and lighting conditions, annotate the collected image samples of scattered objects, and build an open vocabulary detection list, which contains the names of all annotated categories of scattered objects.

7. The method for detecting spilled objects on highways based on open vocabulary detection according to claim 6, characterized in that: Also includes: The list is updated regularly with new types of spills.

8. A highway spilled object detection system based on open vocabulary detection, characterized in that: The following steps are involved: Open vocabulary detection list module: used to obtain common spilled objects on highways and build an open vocabulary detection list for highway spilled objects based on these common spilled objects; Mask module: This module is used to obtain surveillance images, segment the lanes in the vehicle driving area in the surveillance images to obtain an image of the lane line positions, perform binarization on the lane line positions, generate a lane mask, and store it. When the surveillance camera angle or height changes, the lanes are re-segmented and the segmented mask is stored. Equalization module: used to process the monitoring image in blocks and perform local histogram equalization on each monitoring image block; Potential target region module: used to extract potential target regions of scattered objects in images after local histogram equalization using a region proposal network; Recognition module: This module calls the visual encoder and text encoder in the visual language pre-training model CLIP to extract the regional features of the potential target area and the text features of the categories in the open vocabulary detection list. The regional features and text features are then input into several visual language fusion modules to obtain the regional features of the fused text information. Finally, the similarity between the regional features and text features of the fused text information is calculated to align the potential target area with the categories in the open vocabulary detection list. This is used to identify the category of the scattered objects in the potential target area and record the time of the spillage. Information submission module: used to obtain the location of the spilled objects based on the mask of the lane where the spilled objects are identified, record the location, category, and time of the spilled objects, and submit them to the road supervision system.

9. An electronic device, characterized in that: include: a memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images