Deep learning-based illegal advertising board detection method, device, equipment and medium

Abstract

The application discloses a method, device and equipment for detecting illegal advertising boards based on deep learning and a medium, wherein the method comprises the following steps: constructing a sample set and a test set; inputting sample images in the sample set into a YOLOv7 model for detection, and outputting detected targets; performing character recognition on the detected targets, comparing all words in the recognized text with a pre-created word library, searching for words with the highest similarity in the word library as keywords and corresponding probability values; using a loss function to optimize the model training process; using sample images in the test set to test the optimized YOLOv7 model, and detecting illegal advertising boards for to-be-detected images after the test is completed. The application enhances the feature expression capability, uses keywords obtained through character recognition to assist in training, is beneficial to distinguishing illegal advertising boards from regular advertising boards, and improves the detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, equipment and medium for detecting illegal billboards based on deep learning. Background Technology

[0002] With the development of new technologies such as big data, the Internet of Things and artificial intelligence, digital city technology has been promoted nationwide. Refinement, informatization and networking have become the major trends in urban management. The new model of "smart city management" has brought new opportunities and challenges to urban governance.

[0003] Illegal outdoor advertising has always been a common problem in urban governance, not only affecting the aesthetics of the city but also posing certain safety hazards, requiring further regulation and rectification. Previously, enforcement relied mainly on manual patrols, which were costly in terms of manpower. With the reform of the "smart city management" model, AI technology has begun to be used to automatically detect illegal billboards during patrol vehicle inspections. However, the diverse styles and colors of billboards present significant challenges to AI detection, resulting in inaccurate results. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for detecting illegal billboards based on deep learning, aiming to solve the problem that existing AI detection technologies are not accurate enough when detecting illegal billboards.

[0005] In a first aspect, the present invention provides a method for detecting illegal billboards based on deep learning, comprising:

[0006] The detection video is acquired and frames are extracted. The extracted image frames are labeled to obtain sample images with illegal billboards and sample images with legitimate billboards. A sample set and a test set are then constructed.

[0007] The sample images in the sample set are input into the YOLOv7 model for detection, and the detected targets are output; wherein, each ELAN module in the neck network of the YOLOv7 model is connected to an attention module;

[0008] The detected target is subjected to text recognition, the recognized text is saved, all words in the recognized text are compared with a pre-created lexicon, and the words with the highest similarity in the lexicon are retrieved and used as keywords and their corresponding probability values.

[0009] The model training process is optimized using a loss function, which includes a classification loss for associated keywords.

[0010] The sample images in the test set are used to test the optimized YOLOv7 model, and after the test is completed, the to-be-detected image is detected for the illegal advertising board.

[0011] In a second aspect, the present application further provides a deep learning-based illegal advertising board detection device, comprising:

[0012] The acquisition unit is configured to acquire a detection video, perform frame extraction, label the extracted image frames, obtain sample images with illegal advertising boards and sample images with regular advertising boards, and construct a sample set and a test set.

[0013] The training unit is configured to input the sample images in the sample set into the YOLOv7 model for detection, and output the detected targets.

[0014] The recognition unit is configured to perform text recognition on the detected targets, save the recognized text, compare all the words in the recognized text with a pre-created word library, retrieve the word with the highest similarity in the word library as a keyword and a corresponding probability value.

[0015] The optimization unit is configured to optimize the model training process by using a loss function, wherein the loss function comprises a classification loss associated with the keyword.

[0016] The detection unit is configured to test the optimized YOLOv7 model by using the sample images in the test set, and perform illegal advertising board detection on the to-be-detected image after the test is completed.

[0017] In a third aspect, the present application further provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the deep learning-based illegal advertising board detection method as described above when executing the computer program.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program causes the processor to execute the deep learning-based illegal advertising board detection method as described above when executed by the processor.

[0019] The embodiment of the present application provides a deep learning-based illegal advertising board detection method, device, equipment and medium, the embodiment of the present application mainly uses a YOLOv7 model as a target detection network, meanwhile, the model is specifically modified for the advertising board scene, and CA attention mechanism, character recognition and keyword search technology are added. Specifically, a CA (attention module) is connected behind each ELAN module in the YOLOv7 model, so as to enhance the feature expression capability; the detected target is subjected to character recognition, and the character is converted into a vector for keyword search, and the classification loss function of the detection network is dynamically adjusted according to the search result, and the adaptive weight is adjusted. The present application uses the keyword of character recognition for assistance during training, which is beneficial to distinguish illegal advertisements and regular advertising boards, and improves the detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 The flowchart of the deep learning-based illegal advertising board detection method provided by the embodiment of the present application is shown.

[0022] Figure 2 The sub-flowchart of the deep learning-based illegal advertising board detection method provided by the embodiment of the present application is shown.

[0023] Figure 3 The schematic block diagram of the deep learning-based illegal advertising board detection device provided by the embodiment of the present application is shown.

[0024] Figure 4 The schematic block diagram of the recognition unit in the deep learning-based illegal advertising board detection device provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0026] It should be understood that the terms "comprises" and "comprising," when used in this specification and the following claims, indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0027] It should also be understood that the terms used in the specification of the application herein are used for the purpose of describing particular embodiments only and are not intended to limit the application. As used in the specification and the appended claims of the application, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0028] It should further be understood that the term "and / or" used in the specification of the application and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0029] Please refer to Figure 1 , Figure 1 A flowchart of a deep learning-based illegal advertising board detection method provided for an embodiment of the application, comprising steps S101-S105:

[0030] S101, acquire a detection video and perform frame extraction, label the extracted image frames to obtain sample images with illegal advertising boards and sample images with regular advertising boards, and construct a sample set and a test set;

[0031] S102, input the sample images in the sample set into a YOLOv7 model for detection, and output the detected targets; wherein each ELAN module in the neck network in the YOLOv7 model is connected with an attention module;

[0032] S103, perform text recognition on the detected targets, save the recognized text, compare all the words in the recognized text with a pre-created word library, retrieve the word with the highest similarity in the word library as a keyword and a corresponding probability value;

[0033] S104, use a loss function to optimize the model training process, wherein the loss function includes a classification loss associated with the keyword;

[0034] S105, test the optimized YOLOv7 model using sample images in the test set, and perform illegal advertising board detection on the images to be detected after the test is completed.

[0035] The embodiment of the application mainly uses a YOLOv7 model as a target detection network, and simultaneously performs specific modification on the model for a billboard scene, and adds a CA attention mechanism, character recognition and keyword retrieval technology. Specifically, a CA (attention module) is connected behind each ELAN module in the YOLOv7 model, so as to enhance the feature expression capability; the detected target is subjected to character recognition, and the character is converted into a vector for keyword retrieval, and the classification loss function of the detection network is dynamically adjusted according to the retrieval result, and the adaptive weight is adjusted. The application utilizes the keywords of character recognition for assistance during training, which is beneficial to distinguish illegal advertisements and regular billboards, and improves the detection accuracy.

[0036] In the step S101, the pictures shot by the patrol vehicle can be collected, the images are screened by a labeling expert, target detection labeling is performed, and a data set is obtained.

[0037] Firstly, when driving the patrol vehicle to conduct inspection on the street, the images of different streets in the city are collected by the method of video frame extraction, and then the images are subjected to data cleaning, and the images of the billboards standing in various scenes are retained, including different types of illegal billboards, such as catering advertisements, recruitment advertisements, clothing store advertisements, and regular billboards, such as traffic, warning, and civilization propaganda.

[0038] Secondly, the labeling expert uses the Labelme tool to label the street pictures, mainly labels the billboards in the images, which are mainly divided into two categories of illegal billboards and regular billboards, records the rectangular coordinates and labels, and then stores the labeling information in a json format file, and the labeling information includes (z, cx, cy, w, h), z represents the category, (cx, cy) represents the coordinates of the center point of the labeling box, and w, h represent the width and height.

[0039] Finally, all the data are divided in equal proportion according to the labels, 70% of which are used as the training set, 20% of which are used as the verification set, and 10% of which are used as the test set.

[0040] Before training, the data can be subjected to data preprocessing, for example, the training set is subjected to data preprocessing by using the methods of random cropping and normalization, and the sample number is balanced by using data enhancement.

[0041] In step S102, the YOLOv7 model is used as the target detection network. Since the model of the embodiment of the application is applied to the edge inference scene, real-time inference is required during the patrol vehicle inspection. Therefore, the model needs to be lightweight. The embodiment of the application is based on the public YOLOv7 model and a layer of CA (Coordinate attention) attention mechanism (attention module) is connected behind each ELAN module of the Backbone (neck network) to further refine the features before pyramid feature fusion, so that the model detection is more accurate.

[0042] wherein the ELAN module is composed of a plurality of CBS, and the input and output feature sizes remain unchanged.

[0043] The processing procedure of the attention module is as follows:

[0044] The input features are respectively subjected to global average pooling in the width and height directions to obtain a first feature in the width direction and a second feature in the height direction;

[0045] The first feature in the width direction and the second feature in the height direction are spliced, and then subjected to convolution to realize dimension reduction, and then subjected to batch normalization processing and activation in sequence;

[0046] The features after activation are respectively subjected to convolution in the width and height directions to obtain a third feature in the width direction and a fourth feature in the height direction, which have the same number of original channels;

[0047] The third feature and the fourth feature are respectively activated by using an activation function to obtain corresponding attention weights, and then the input features are subjected to multiplication weighting calculation to obtain output features with corresponding attention weights.

[0048] The CA attention mechanism encodes the channel relationship and long-term dependency through accurate position information, which can enhance the expression ability of the mobile network to learn features. It can transform and change any intermediate feature tensor in the network to output a tensor of the same size.

[0049] In step S103, the open source PP-OCRv2 algorithm is used for character recognition, which is a super lightweight Chinese OCR model. Then, keyword retrieval is performed, and the classification loss is dynamically adjusted through the retrieval result to form adaptive weights, control the label weights, accelerate the convergence of model training, and improve the discrimination ability of regular billboards and illegal billboards.

[0050] Specifically, as shown in Figure 2 step S103 includes steps S201-S205:

[0051] S201, performing character recognition on the detected target using a PP-OCRv2 algorithm, and converting the recognized text into an array L containing at least one word;

[0052] S202, constructing a word library M containing words in the plurality of regular billboards in advance;

[0053] S203, converting the array L and all words in the word library M into word vectors using a Skip-gram algorithm;

[0054] S204, for each word in the array L, calculating the vector distance between the corresponding word vector and the word vectors of all words in the word library M, obtaining each corresponding vector distance and sorting, returning the word with the highest similarity in the word library and the probability value, and storing the corresponding probability value in a probability value array;

[0055] S205, sorting the probability values in the probability value array to obtain the maximum probability value, and taking the word in the word library corresponding to the maximum probability value as the keyword.

[0056] In step S201, the open source PP-OCRv2 algorithm is used to recognize the characters of the target detected by the YOLOv7 model, and the recognition result is converted into an array L, so that the detected characters are saved in the form of an array. For example, array L = ["Please scan the code", "QR code",...].

[0057] In step S202, a keyword library is established to serve as a corpus for word vector comparison and retrieval. Since the labels of the billboard detection model mainly have two categories: regular billboards and illegal billboards, the regular billboards mainly include traffic, warning, civilization propaganda, etc., and the characters on the regular billboards are relatively fixed and generally do not repeat with the characters on the illegal billboards, so the present embodiment can establish a keyword library M of regular billboards. M = ["Prohibit", "Traffic", "Promote",...].

[0058] The word library M exhaustively lists the characters that may appear in the regular billboards and do not repeat with the illegal billboards, including but not limited to the above words.

[0059] In step S203, to compare the similarity of two Chinese words or phrases and compare their semantic information, it is necessary to convert the words into word vectors. Since the vocabulary in this embodiment is not large, a simple word embedding algorithm, word2vec, can be used. Skip-gram is an algorithm within word2vec; in this embodiment, the Skip-gram algorithm is used to convert all words in the vocabulary M into word vectors. The vectorized representation of the vocabulary M is M_vec. Similarly, each word in L is traversed, and the Skip-gram algorithm is used to convert the word into a word vector, resulting in the vectorized representation l_vec.

[0060] In step S204, for each word in the array L, the corresponding word vector l_vec is used to calculate the vector distance, i.e., the cosine distance, with the word vector M_vec corresponding to all words in the word library M. The vector distances of each pair are obtained and sorted. The words with the highest similarity in the word library and their probability values ​​are returned, and the corresponding probability values ​​are stored in the probability value array.

[0061] In this way, for each word in array L, we will obtain the word with the highest similarity in the dictionary and its probability value, so that the final probability value array contains multiple probability values.

[0062] In step S205, the probability values ​​in the probability value array are sorted from largest to smallest to obtain the largest probability value, and the word in the word library corresponding to it is taken as the keyword.

[0063] For example, array L contains two words: "attention" and "safety". The word with the highest similarity for "attention" is "prevention", with a probability value of "0.935613587746". The word with the highest similarity for "safety" is "life", with a probability value of "0.585613587746". Therefore, the final maximum probability value is "0.935613587746", and the keyword is "prevention".

[0064] In step S104, a loss function is used to optimize the model training process.

[0065] The loss function includes class classification loss for associated keywords, confidence loss, and bounding box regression loss.

[0066] The confidence loss uses the BCEWithLogitsLoss (binary cross-entropy loss function), as shown in the following formula:

[0067] Loss = {l1, ..., l N}, l n =-[y n·log(σ(x n ))+(1-y n )·log(1-σ(x n ))]

[0068]

[0069] In the formula: N represents the training set batch, and each batch contains n targets. σ(x) n () is the Sigmoid function, which can convert x... n Mapped to the interval (0,1). x n For the prediction result of the nth target, y n Let be the true label of the nth target, and calculate the loss using both.

[0070] The loss function for class classification is also BCEWithLogitsLoss (Binary Cross Entropy Loss), but because this embodiment of the invention adds text recognition and keyword retrieval technology, this embodiment redefines and modifies BCEWithLogitsLoss, associating the vector retrieval results with the classification loss to form adaptive weights. The calculation formula for class classification loss is as follows:

[0071] Loss = {l1, ..., l N}, l n =-w n ·[y n ·log(σ(x n ))+(1-y n )·log(1-σ(x n ))]

[0072] in,

[0073]

[0074] Where N is the training set batch, and each batch has n targets, σ(x) n ) is the Sigmoid function, x n For the prediction result of the nth target, y n Let x be the true label of the nth target, l_max be the maximum probability value corresponding to a certain detected target, α be the control coefficient, and the true labels are divided into three categories: illegal billboards (0), legitimate billboards (1), and background (2), where x n The probability values ​​are in the (1, 3) dimension, representing the probabilities of the three categories, w. n It has (1,3) dimensions and is used to control the weights of the three types of labels.

[0075] The embodiment can control the size of the increased weight by controlling the coefficient a. When l_max is greater than or equal to 0.5, the 0 label is focused on, and when l_max is less than 0.5, the 1 label is focused on.

[0076] The adaptive weight mentioned in the embodiment refers to w n , which can be automatically adjusted by l_max n , and further adjusts the class classification loss loss. And a is a coefficient in w n , mainly controls the amplitude of the weight, which can be controlled artificially.

[0077] The (1, 3) dimension here refers to: because the model classification label is 3 categories, the model will output 3 category probability values, that is, (1, 3), and finally the label with the largest probability value will be selected as the prediction result. For example, the output probability value is (0.1, 0.75, 0.15). Finally, the label with the largest probability value 0.75 is selected as the prediction result.

[0078] In an embodiment, the bbox bounding box regression loss uses GIOUloss. The calculation formula of the bbox bounding box regression loss is as follows:

[0079] L GIOU =1-GIOU(A,B)

[0080]

[0081]

[0082] Where A represents the real boundary box, B represents the predicted boundary box, IOU(A,B) represents the area intersection ratio of the real boundary box and the predicted boundary box, and C represents the smallest rectangle that can cover the predicted boundary box and the real boundary box.

[0083] When the model training starts, the initial training parameters are set first, and the sample image is sent to the modified YOLOv7 model for training. In each iteration process of training, each target recognized by the YOLOv7 model is sent to the recognition unit for character recognition, the recognized characters are converted into word vectors, and the word vectors are used to search the keyword library M. According to the search result, the classification loss is automatically adjusted to form an adaptive weight.

[0084] In an embodiment, the optimized YOLOv7 model is tested using sample images in a test set, and after the test is completed, the image to be detected is detected for illegal billboards, comprising:

[0085] When testing or detecting, the test image or the image to be detected is input into the optimized YOLOv7 model, and the output result is taken as the final detection result.

[0086] The embodiment of the present application stops training after 300 epochs of training iteration, saves the best model of training, and needs to delete the subsequent identification process and retrieval process when saving. The module units corresponding to the two processes are only used to assist the training of the YOLOv7 model and do not act on inference, so as to realize light weight and ensure the real-time performance of edge inference.

[0087] In actual application, the patrol vehicle performs real-time frame extraction on the camera video when patrolling on the street, and sends the extracted image to the trained model for inference. The model detects whether there is a rule violation billboard in the image and returns the detection result in real time. If the detection result has a rule violation billboard, the alarm information is sent to the background.

[0088] As shown in Figure 3 The embodiment of the present application also provides a rule violation billboard detection device 300 based on deep learning, which comprises:

[0089] The acquisition unit 301 is configured to acquire a detection video and perform frame extraction, label the extracted image frames, obtain sample images with rule violation billboards and sample images with regular billboards, and construct a sample set and a test set;

[0090] The training unit 302 is configured to input the sample images in the sample set into the YOLOv7 model for detection and output the detected target. Each ELAN module in the neck network of the YOLOv7 model is connected with an attention module;

[0091] The identification unit 303 is configured to perform text recognition on the detected target, save the recognized text, compare all words in the recognized text with a pre-created word library, retrieve the word with the highest similarity in the word library as a keyword and a corresponding probability value;

[0092] The optimization unit 304 is configured to optimize the model training process by using a loss function, wherein the loss function comprises a classification loss associated with the keyword;

[0093] The detection unit 305 is configured to test the optimized YOLOv7 model by using the sample images in the test set, and perform rule violation billboard detection on the to-be-detected image after the test is completed.

[0094] In an embodiment, the attention module comprises:

[0095] The pooling unit is configured to perform global average pooling in the width and height directions on the input features respectively, and obtain a first feature in the width direction and a second feature in the height direction respectively;

[0096] The first convolution unit is configured to concatenate the first feature in the width direction and the second feature in the height direction, and then perform convolution to realize dimension reduction, and then sequentially perform batch normalization processing and activation;

[0097] The second convolution unit is configured to perform convolution on the features obtained after the activation in the width direction and the height direction respectively, to obtain a third feature in the width direction and a fourth feature in the height direction with the same number of original channels.

[0098] The weighting unit is configured to activate the third feature and the fourth feature respectively by using an activation function to obtain corresponding attention weights, and then perform multiplication weighting calculation on the input features to obtain output features with corresponding attention weights.

[0099] In an embodiment, as shown in FIG. 3, the recognition unit 303 includes: Figure 4

[0100] The recognition subunit 401 is configured to perform text recognition on the detected target by using a PP-OCRv2 algorithm, and convert the recognized text into an array L containing at least one word.

[0101] The construction unit 402 is configured to pre-construct a word library M containing words in the plurality of normal billboards.

[0102] The conversion unit 403 is configured to convert the array L and all words in the word library M into word vectors by using a Skip-gram algorithm.

[0103] The calculation unit 404 is configured to, for each word in the array L, perform vector distance calculation on the corresponding word vector and the word vectors corresponding to all words in the word library M, obtain each group of corresponding vector distances and sort them, return the word with the highest similarity in the word library and a probability value, and store the corresponding probability value in a probability value array.

[0104] The sorting unit 405 is configured to sort the probability values in the probability value array to obtain the maximum probability value, and take the word in the word library corresponding to the maximum probability value as the keyword.

[0105] In an embodiment, the loss function includes a class classification loss, a confidence loss, and a bbox bounding box regression loss. For specific technical details of the above losses, refer to the foregoing content.

[0106] The embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the deep learning-based illegal billboard detection method as above when executing the computer program.

[0107] ​The embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program causes a processor to execute the deep learning based illegal billboard detection method as above when the processor executes the computer program.

[0108] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described here. Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in general terms in the foregoing description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0109] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of the units is merely logical function division, and actual implementation can have another division mode, or units with the same function can be combined into one unit, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other form of connection.

[0110] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0111] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software functional unit.

[0112] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art that contributes to the present application, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a mechanical hard disk, a solid state disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0113] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for detecting illegal billboards based on deep learning, characterized in that, The application relates to the technical field of image processing, and discloses a method for detecting illegal advertising boards. The method comprises the following steps: acquiring a detection video and performing frame extraction, labeling the extracted image frames to obtain sample images with illegal advertising boards and sample images with regular advertising boards, and constructing a sample set and a test set; inputting the sample images in the sample set into a YOLOv7 model for detection, and outputting the detected targets; wherein an attention module is connected behind each ELAN module in the neck network of the YOLOv7 model; performing character recognition on the detected targets, saving the recognized text, comparing all the words in the recognized text with a pre-created word library, searching for the word with the highest similarity in the word library as a keyword and a corresponding probability value; using a loss function to optimize the model training process, wherein the loss function comprises a classification loss of the associated keyword; using the sample images in the test set to test the optimized YOLOv7 model, and detecting illegal advertising boards in the to-be-detected images after the test is completed; the loss function comprises a class classification loss, a confidence loss and a bbox bounding box regression loss; Loss = {li, · · ·, ln}, i = 1, · · ·, n N}, i = 1, · · ·, n n = -w n · [y n · log(σ(x n )) + (1 - y n ) · log(1 - σ(x n )) wherein, where N is the training set batch, each batch has n targets, σ(x n ) is a Sigmoid function, x n is the prediction result of the nth target, y n is the true label of the nth target, l_max is the maximum probability value corresponding to a certain detection target, l_max represents the semantic matching degree of the recognized text and a certain word in the word library, α is a control coefficient, the true label is divided into three categories: illegal billboards, regular billboards and background, wherein x n is a probability value of (1, 3) dimensions, respectively representing the probabilities of the three categories, w n is (1, 3) dimension, used to control the weights of the three labels. 2.The deep learning-based illegal billboard detection method of claim 1, wherein, the calculation formula of the class classification loss is as follows: the processing procedure of the attention module is as follows: performing global average pooling on the input features in the width and height directions respectively to obtain a first feature in the width direction and a second feature in the height direction; splicing the first feature in the width direction and the second feature in the height direction, then performing convolution to realize dimension reduction, and then sequentially performing batch normalization processing and activation; convolving the features obtained after activation in the width and height directions respectively to obtain a third feature in the width direction and a fourth feature in the height direction with the same number of original channels; 3.The deep learning-based illegal billboard detection method of claim 1, wherein, using an activation function to activate the third feature and the fourth feature respectively to obtain corresponding attention weights, and then performing multiplication weighting calculation on the input features to obtain output features with corresponding attention weights. The method comprises the following steps: using a PP-OCRv2 algorithm to perform character recognition on the detected targets, and converting the recognized text into an array L containing at least one word; pre-building a word library M containing the words in the plurality of regular advertising boards; using a Skip-gram algorithm to convert the array L and all the words in the word library M into word vectors; for each word in the array L, performing vector distance calculation on the corresponding word vector and the word vectors corresponding to all the words in the word library M to obtain each group of corresponding vector distances and perform sorting, returning the word with the highest similarity in the word library and a probability value, and storing the corresponding probability value in a probability value array; 4.The deep learning-based illegal billboard detection method of claim 1, wherein, sorting the probability values in the probability value array to obtain the maximum probability value, and taking the word in the word library corresponding to the maximum probability value as a keyword. The calculation formula of the bbox bounding box regression loss is as follows: L GIOU = 1 - GIOU(A, B) Wherein, A represents a real bounding box, B represents a predicted bounding box, IOU(A, B) represents the area intersection over union of the real bounding box and the predicted bounding box, and C represents the smallest rectangle that can cover the predicted bounding box and the real bounding box. 5.The deep learning-based illegal billboard detection method of claim 1, wherein, The optimized YOLOv7 model is tested by using sample images in the test set, and after the test is completed, the image to be detected is detected for illegal billboards. When testing or detecting, the test image or the image to be detected is input into the optimized YOLOv7 model, and the output result is taken as the final detection result. 6.A device for detecting illegal billboards based on deep learning, characterized in that, It comprises: The acquisition unit is used for acquiring a detection video and performing frame extraction, labeling the extracted image frames, obtaining sample images with illegal billboards and sample images with regular billboards, and constructing a sample set and a test set; The training unit is used for inputting sample images in the sample set into the YOLOv7 model for detection, and outputting the detected target; wherein each ELAN module in the neck network of the YOLOv7 model is connected with an attention module; The recognition unit is used for performing text recognition on the detected target, saving the recognized text, comparing all words in the recognized text with a pre-created word library, retrieving the word with the highest similarity in the word library as a keyword and a corresponding probability value; The optimization unit is used for optimizing the model training process by using a loss function, wherein the loss function comprises a classification loss associated with the keyword; The detection unit is used for testing the optimized YOLOv7 model by using sample images in the test set, and detecting the image to be detected for illegal billboards after the test is completed. The loss function comprises a class classification loss, a confidence loss and a bbox bounding box regression loss. The calculation formula of the class classification loss is as follows: Loss = {li, · · ·, ln}, i = 1, · · ·, n N}, i = 1, · · ·, n n = -w n · [y n · log(σ(x n )) + (1 - y n ) · log(1 - σ(x n ))] wherein, where N is the training set batch, each batch has n targets, σ(x n ) is a Sigmoid function, x n is the prediction result of the nth target, y n is the true label of the nth target, l_max is the maximum probability value corresponding to a certain detection target, l_max represents the semantic matching degree of the recognized text and a certain word in the word library, α is a control coefficient, the true label is divided into three categories: illegal billboards, regular billboards and background, where x n is a probability value of (1, 3) dimensions, representing the probabilities of the three categories respectively, w n is (1, 3) dimension, used to control the weight of the three labels.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the deep learning-based illegal billboard detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program causes the processor to execute the deep learning-based illegal billboard detection method according to any one of claims 1 to 5 when the processor executes the computer program.

Images